Methods for studying public administration

Public Administration is a hybrid sector where a number of dynamics overlap and intersect: the rationales based on efficiency and effectiveness that typify the business world, a highly politicized context which profoundly impacts decision-making mechanisms, and the need to consider the multi-faceted demands of stakeholders, who often coincide with the entire community the public administration serves. Such complexity inevitably reverberates onto studies in the field of PA. But in the search for optimal, practical solutions for solving concrete problems, we can’t overlook administrative dynamics and the influence of politics.

To fully grasp the unique features of public administration, it is imperative to utilize appropriate methods of analysis, if possible without letting theoretical postulates take precedence over efforts to comprehend the reality of PA. This is why a pragmatic approach is particularly well-suited to the study of public administration. In other words, an approach that does not hold researchers prisoner in this or that epistemological paradigm, but instead allows them to exploit the potentialities and insights from a variety of different methodologies and perspectives.

Here is where mixed methods (MMs) come into play, merging techniques and viewpoints from quantitative research with the characteristics of qualitative research. This combination of different tools and perspectives obviously mustn’t be improvised, but should be the direct outcome of an explicit and consequential line of reasoning, based on an analysis of the problem at hand. But how far have PA studies gone until now in effectively exploiting the potentialities of mixed methods?

To understand the extent to which mixed methods are currently being utilized in PA studies, we conducted research on seven of the leading international, peer-reviewed scientific journals in the sector. Our analysis was not limited to articles based explicitly on MMs; instead we extended our investigation to all research with a hybrid quali-quantitative methodology, even if it was not overtly described as ‘mixed.’

For our investigation, we identified 104 studies classifiable as MM, although out of these, only a third made explicit reference to this methodology. Most of the studies (75%) used qualitative interviews combined with statistics gleaned from surveys (in 43% of the cases) or archives (the remaining 33%). Less common were both qualitative and quantitative analyses via interviews or archival data, or the use of ‘mixed’ quali-quantitative surveys (including both open and closed questions), or case studies.

The main motivations for adopting MMs primarily center on the effort to obtain more solid results and to test their validity. For example, when surveys are used, respondents might be asked for qualitative explanations for their responses. A second motivation has to do with the possibility of emerging multidimensional explanations for a given phenomenon, such is the case for instance with the perceived impact of technological change on performance. Lastly, MMs can be a contributing factor in explaining the causal mechanisms that underlie different variables.

With regard to the connection between various methodologies employed, in 31% of the studies we examined, they were applied in parallel. In other words, quantitative studies were conducted independently of qualitative studies. Generally speaking, the results of the two study streams were later analyzed together to verify whether the results were actually consistent.

In the remaining 69% of the papers, the two methods were used sequentially, in other words, the results from the first phase were further explained and explored in the second phase. The most common example of this were studies on survey or archival data accompanied by interviews (generally conducted subsequently). The aim here was to obtain more precise interpretations of the findings, for example in one case investigating forcible stops by police in New York City.

Instead when researchers started with qualitative interviews, their aim was generally to develop research hypotheses to be later corroborated via surveys, as with a study run on the bureaucratization of administrative procedures. Here the surveys systematically explored individual perceptions of a given aspect in light of its relevance, as emerged in preliminary interviews.

When using MMs, the most crucial phase – and also the most problematic – is the connection between the different methods. When the two were conducted in parallel, this connection proved to be effective when researchers actually integrated their findings, instead of simply underscoring that the quantitative and qualitative studies centered on the same context or data base.

When studies were done sequentially, with the quantitative phase preceding the qualitative phase, the critical step was to proceed in the analysis from a representative sample to a significant sample. For instance, when moving to the second qualitative phase, researchers would choose interviewees who, thanks to their profiles and experience, could offer useful insights on the findings from the previous phase.

Lastly, in sequential studies where the qualitative phase came before the quantitative phase, the determinant was the ability to discern themes and research questions during the interviews and effectively integrate them in subsequent data collection. An example here is a study on the importance of personal networks among mayors. In this case certain key phrases were extracted from the interviews done in the preliminary phase and later incorporated verbatim in the subsequent quantitative survey.

In the past ten years, studies on public administration seem to be increasingly receptive to the use of mixed methods. Depending on the specific research design, MMs can contribute to corroborating hypotheses, providing multidimensional interpretations of existing phenomena, and shining a light on underlying causal mechanisms. Fundamental, in any case, is the connection between the qualitative and quantitative phases, which must be explicit and well-designed.

Particularly interesting is the case where mixed methods produce contradictory results, or MMs are utilized to explain why findings did not bear out an initial research hypothesis. As we found in some of the studies in our research, this does not represent a failure, but instead an opportunity to develop more accurate, refined insights.

By encouraging interpretations that are more nuanced, intellectually honest and well thought out, MMs can help us come to a clearer understanding of the complexity of the phenomena underpinning public administration. All this contributes not only to advancing research, but also to making the ‘public machine’ function more smoothly and efficiently.