Methods of Policy Analysis

Dror (1984) calls policymaking “a very presumptuous activity” (p.13). Though the same might be said about research. In comparing the two activities, Edwards (2005) describes the “uneasy relationship” between research and policymaking. Whereas policymakers believe that research doesn’t concern itself with issues that are relevant to the lived realities of the studied populations and is often “driven by ideology” masked as intellectual inquiry, researchers believe that there is a lack of government interest in research and that there are roadblocks put in place by policymakers to make research more difficult to carry out. Furthermore, Edwards describes a general anti-intellectualism embraced by policymakers, in that they are wary about critical analysis because it could make certain policies embarrassing or irrelevant. Similarly, researchers note that there is a lack of incentives for researchers to create policy-relevant research. To Edwards (2005), all of this bickering illustrates two distinct cultures, but more significantly, two distinct ways of communicating.

Stone (as cited in Edwards, 2005) outlines twelve perspectives that encompass the many reasons that there is a gap between research and policy. Though all of Stone’s supply side, demand side, and socio-cultural factors are insightful and accurate, the ones that I see as being the most striking are:

• Comprehension: Researchers don’t understand what the policy process is and how research fits into the policy process.
• Communication: Researchers don’t have the adequate tools to convey messages about their research to policymakers.
• Anti-intellectualism in government: Government ideology is driven by an inherent distrust in using pure science to base policy decisions.
• Politicization of research: Researchers and policymakers are not viewed as being objective, but rather present information and/or make decisions based on ideology.

Dror’s (1984) concept of “fuzzy gambling” challenges some of these above factors, with policymaking being a process of unknowable, and constantly changing “rules of the game” (p.15). There is an assumption that research is a yes/no or true/false process. But, Dror (1984) suggests that it is more of a “multi-valued” logic. By viewing research and policymaking through this lens, research and policymaking become more open to dialogue between one another, because they understand that it’s not an all-or-nothing process.

Building upon the policy gambling perspective, Dror (1984) suggests that policy scientists broaden their methodologies, including methodologies “capable of handling irreducible uncertainty, including qualitative uncertainty with inability to specify qualitatively main alternative possible futures” (p.16). Social workers work with qualitative data all the time, in listening to clients’ situations and when conveying these situations through advocacy efforts in attempts to persuade colleagues and government bodies. Edwards (2005) mentions a valuable qualitative methodology, which bridges research and policy: the case study. Edwards notes: “To determine more systematically what works when, there and how, ideally calls for case studies designed to illustrate the diverse ways in which research can connect to policy” (p. 71). Qualitative research methods are an important addition to the policymakers’ toolbox, for they can persuade and convince in a way that quantitative data alone cannot. For example, the United Nation’s Office of the Special Representative of the Secretary General for Children in Armed Conflict has been consistently monitoring the six grave violations (killing and maiming of children; recruitment and use of child soldiers; rape and other forms of sexual violence against children; abduction of children; attacks against schools and hospitals; denial of humanitarian access to children) against children during armed conflict. In the gathering of evidence, the UN has solely relied upon qualitative data, which has made quite an impact on the depth of the problem. Nevertheless, the scope of the problem has yet to be understood, and therefore, efforts are being made to supplement the qualitative data with quantitative methods, both of which provide breadth and depth.

Perhaps the divide between research and policy is not so dichotomous. Aren’t researchers always asking themselves, How can our work be relevant and useful? Or is this an idealistic or naïve perspective of research? As a humanist science, social work researchers should be constantly thinking about their work’s impact on practice and policy. For example, I worked as the program manager for the Care and Protection of Children in Crisis-Affected Countries (CPC) Initiative (www.cpclearningnetwork.org), which sought to create a global network of researchers, practitioners, and policymakers to collaborate on research projects to develop new methodologies for child protection in crisis-affected settings. This culminated in the Child Protection Action Summit, which gathered these figures for a four-day meeting to discuss ways that international child protection research could be relevant to policy, and vice versa. Edwards (2005) notes that even though the literature may suggest that if researchers and policymakers work closely together then there should be good policy outcomes, this might not be true in all circumstances, because there are multiple factors at play. For what didn’t work with CPC was what Edwards noted was most important for the success of bridging the divide. She notes: “The degree to which research influences policy often depends on individuals building relationships of mutual trust and respect, rather than on an ongoing and sustained discourse between governments and researchers” (p.73). CPC conducted the Child Protection Summit, holding conversations with the goal of creating common research priorities. However, over one year later, these conversations are no longer taking place. Meetings are currently happening with researchers and practitioners determining the research agenda, with policymakers at the receiving line of research several years after the major decisions have been made. Why can’t we include policymakers in the “ongoing and sustained discourse”, starting from the research proposal and moving forward? We know that it shouldn’t be such a dichotomy, but rather a partnership. Nevertheless, we still don’t know how to effectively bridge the divide for effective partnerships between research and policy.

Sources:

Dror, Y. (1984). Perspectives on public policy: On becoming more of a policy scientist. Policy Studies Review, 4(1), 13-21.

Edwards, M. (2005). Social science research and public policy: Narrowing the divide. Australian Journal of Public Administration, 64(1), 68-74.

Phenomenology and Social Construction

As learners, ‘experts’ tell us what the truth is and we understand this explanation as our own. Yet as researchers, how do we really know what we know? We often call upon our knowledge, beliefs, and ethics to explore and interpret phenomena. This speaks to the world of research. For example, Schutz (1967) explains the process of how we grapple with our understanding of others:

On the one hand, what is understood is the sign itself, then again what the other person means by using this sign, and finally the significances of the fact that he is using the sign, here, now, and in this particular context (p.269).

In grappling with the meanings of others and their actions, we contribute to the accumulation of scientific inquiries.

Schutz’s work testifies to his support of Weber. He emphasizes Weber’s value of freedom in social scientific research, with science asserting itself through other areas of life. As opposed to its association with the natural sciences, Schutz frames social science as a science in its own right, with its own meaning-making strategies, (i.e., research methodologies). The main tenet of science is the same across disciplines: science offers an approach to discover and know reality through experience.

Schutz specifically describes knowledge-gathering through actions without communication, which draws parallels to the role of the complete observer in qualitative methodologies. Even if what we study is very abstract and difficult to explain, there is something in our lived reality that we can use in the interpretation. In raising awareness of others’ experiences, we create our own meaning-context. At the same time, others have arranged their own meaning-contexts. Yet there is a danger in completely adopting the point of view of another. By abandoning objectivity in favor of empathy, there is the risk of losing an understanding of the phenomenon from alternate frames of reference.

What are the labels that we use to give meaning to others? Labels for concepts are merely devices used to organize phenomena and communicate with others. This premise was first described by Berger and Luckmann (1966) as mental representations of other’s actions. These typifications eventually become a part of the social dialogue of meaning-making. As time moves forward, the developed meaning is embedded into the fabric and structure of society, thereby becoming a social construct. Quantitative research methodologies force us to choose constructs and fit people into those constructs. Individuals are viewed, labeled, and analyzed in regards to social constructs, such as gender, age, and socioeconomic status. Some may argue that the shared experience may be lost in quantitative methods, as the individual is often reduced to his constructs.

There is a great need for social researchers to treat the beliefs they study as worthy of respect rather than as objects of condemnation. This can be done by allowing participants to determine the meanings attributed to the phenomena being described in the study. A promising methodology is mixed methods research, which combines qualitative and quantitative approaches to provide a better understanding of research problems than using either single approach alone. In utilizing aspects of both methods, the researcher acknowledges alternate frames of reference for interpreting meaning in others’ lives.

References:
Berger, P. and Luckmann, T. (1966). The social construction of reality: A treatise in the sociology of knowledge. New York, NY: Anchor Books.
Schutz, A. (1967). The phenomenology of the social world. In S. Appelrouth and L. Dasfor Eldes (Eds.), Social theory in the contemporary era: Text and readings. Thousand Oaks, CA: Sage.