making the case for case studies in empirical legal research

• Law and Method

• Submission of articles
• Notifications
• Methods courses
• Editorial team
• E-mail alerts
• October, 2016
• Stumbling Blocks in Empirical Legal Research: Case Study Research
• October 2016
• Artikel Stumbling Blocks in Empirical Legal Research: Case Study Research
• Artikel Statistical Analyses of Court Decisions: An Example of Multilevel Models of Sentencing
• Redactioneel Introduction Special Issue Stumbling Blocks in Empirical Legal Research

Citeerwijze van dit artikel: Lisa Webley, ‘Stumbling Blocks in Empirical Legal Research: Case Study Research’, 2016, oktober-december, DOI: 10.5553/REM/.000020

Dit artikel wordt geciteerd in

• Introduction

Such legal research employs an empirical method to draw inferences from observations of phenomena extrinsic to the researcher. Putting it simply, legal researchers often collect and then analyse material (data) that they have read, heard or watched and subsequently make claims about how what they have learned may apply in similar situations that they have not observed (by inference). 1 x For insight into the extent to which legal researchers undertake empirical research and the lack of clarity around empirical methods in law see Epstein, L. and King, G. (2002) ‘The Rules of Inference’ Vol. 69 No. 1 The University of Chicago Law Review 1-133 at 3-6, Part I. One such form of empirical method is the case study, a methodological term which has been used by some researchers to describe studies that employ a combination of data sources to derive in-depth insight into a particular situation, by others to denote a particular ideological approach to research recognizing that the study is situated within its real-world context. 2 x Yin, R. K. (2014) Case Study Research Design and Methods (5th edn.) Sage Publications, 12-14. It is consequently a flexible definition encompassing approaches to the data and the stance of the researcher, 3 x See Hamel, J. with Dufour, S. and Fortin, D. (1993) Qualitative Research Methods Volume 2 , Sage Publications, ch 1. but its malleability has led some researchers incorrectly to stretch the term to encompass any study that focuses on one or a restricted number of situations. 4 x See Gerring, J. Case Study Research: Principles and Practices , Cambridge, Cambridge University Press, 2007 at 6. For a further discussion see Gerring, J. ‘What is a case study and what is it good for?’ (2004) Vol. 98 No 2 American Political Science Review 341-354. This looseness in definition in a legal context may perhaps be linked to confusion as between teaching and research case studies; some traditions in legal education employ a teaching method known also as ‘case study method’ which operates quite differently from its research counterpart. For a discussion of the differences between teaching and research case studies see Yin, 2014, 20 and for a discussion of teaching case studies see Ellet, W., (2007) The Case Study Handbook: How to Read, Discuss, and Write Persuasively About Cases , Boston MA, Harvard Business Review Press; Garvin, D. A. (2003) ‘Making the Case: Professional Education for the World of Practice.’ (Sept–Oct) Harvard Magazine 56-65. This has resulted in concerns that the definition has been co-opted as a means to explain any small n 5 x ‘n’ (number) is used to denote the number of observations in the study, N is used to describe the total number within the population when n denotes the sample observed. empirical study that has a focus on a particular subject, time-frame or location, and further that this has led to poor quality empirical research in law. 6 x For a discussion of the state of empirical research in law see Epstein and King, 2002. This is perhaps unsurprising, as law programmes tend to be very strong at teaching lawyers how to source, interrogate and then draw valid inferences from legal data sources such as cases and legislation, but less adept in the context of other types of data (for example survey data, interviews, non-legal documents and/or observation). 7 x See Webley, L. (2010) ‘Part III Doing Empirical Legal Studies Research Chapter 38 - Qualitative Approaches to ELS’ in Cane, P. and Kritzer, H. (eds.) Oxford Handbook of Empirical Legal Studies , Oxford, Oxford University Press. Case study method usually involves an array of research methods to generate a spectrum of numerical and non-numerical data that when triangulated provide a means through which to draw robust, reliable, valid inferences about law in the real world. 8 x For a discussion about the differences between numerical (quantitative) and non-numerical (qualitative) data see Webley, id; Epstein and King, 2002, at 2-3; King,G., Keohane, R. O., and Verba, S ., Designing Social Inquiry: Scientific Inference in Qualitative Research , Princeton NJ, Princeton University Press, 1994 at 6. It is relatively underused in empirical legal research. This article aims to make a contribution to those new to the case study method. It will examine the purpose of and why one may wish to undertake a case study, and work through the key elements of case study method including the main assumptions and theoretical underpinnings of this method. It will then turn to the importance of research design, including the crucial roles of the academic literature review, the research question and the use of rival theories to develop hypotheses in case study method. It will touch upon the relevance of identifying the observable implications of those hypotheses, and thus the selection of data sources and modes of analysis to allow for valid analytical inferences to be drawn in respect of them. In doing so it will consider, in brief, the importance of case study selection and variations like single or multi case approaches. Finally, it will conclude with some thoughts about the strengths and weaknesses associated with undertaking research via a case study method. It will address frequent stumbling blocks encountered by researchers, as well as ways so as to militate against common problems that researchers encounter. The discussion is necessarily cursory given the length of this article, but the footnotes provide much more detailed sources of guidance on each of the points raised here. This article is an introduction to a case study method rather than an analytical work on the method.

• 1. Case Study Method: Purpose of a Case Study, Why Undertake One?

Case study method falls within the social science discipline and as such has scientific underpinnings. The case study examines phenomena in context, where context and findings cannot be separated. Case study design is also sometimes used to investigate how actors consider, interpret and understand phenomena (e.g., law, procedure, policy) and therefore allows the researcher to study perceptions of processes and how they influence behaviour, for example to understand judges’ sentencing choices in a Dutch police court. 9 x Mascini, P., van Oorschot, I., Weenink, D. and Schippers, G., (2016) ‘Understanding judges’ choices of sentence types as interpretative work: An explorative study in a Dutch police court’, (37) (1) Recht der Werkelijkheid 32-49. This may help to understand how laws are understood, and how and why they are applied and misapplied, subverted, complied with or rejected. This can flow back into the legal and policy making processes, court procedure, sentencing, punishment, diversion of offenders etc., and may have a high impact as a result. The conditions precedent for case study method have been succinctly explained by Yin as follows:

‘doing a case study would be the preferred method, compared to the others, in situations when (1) the main research questions are “how” or “why” questions; (2) a researcher has little or no control over behavioural events; and (3) the focus of study is a contemporary (as opposed to entirely historical) phenomenon.’ 10 x Yin, 2014: xxxi and further 16-17.

The key points to note here are that a case study is a real-world in-depth investigation of a current complex phenomenon. The research will take place in situ (rather than in the library or moot court room) where the researcher cannot control the behaviour of research participants.

The purpose of the study is to learn how or why something happens or is the way it is, and this is achieved by collecting and triangulating a range of data sources to test or explore hypotheses. 11 x Triangulation is the term used to explain that a research question is considered from as many different standpoints as possible, using as many different data types as possible to permit a holistic examination of the question to see which explanations, if any, remain consistent across all data sources. It caters for a wide range of modes of enquiry: the investigation may be exploratory (explore why or how something is the way it is), descriptive (describe why or how something is the way it is) or explanatory (determine which of a range of rival hypotheses, theories etc. explain why or how X is the way it is). 12 x Yin, 2014: 5-6. Some categorise case studies as those designed to be theory orientated, and those designed to be practice orientated. 13 x See Dul, J. and Hak, T. (2008) Case Study Methodology in Business Research , Oxford: Elsevier 8-11, 30-59. Thereafter the design scope is very broad; the data collected may be qualitative and/or quantitative, collected via a variety of methods, and the case study may be a single case or be made up of a small number of cases. The breadth of data collected may be illustrated by Latour’s ethnography of the Conseil d’Etat in France, which studied the connections between human and non-human actors to explore their relationship with ‘the legal’ and ‘the Law’ is assembled in that court context. 14 x Latour, B. (2010) The Making of Law: An Ethnography of the Conseil D’Etat , Cambridge: Polity Press. Case study method is a way of thinking about research and a process through which one seeks to produce reliable, fair findings. It can provide deep insight into a particular situation, whether particular in time, in location or in subject-matter. 15 x For a discussion of ethnomethodological aims to study practical life as experienced in context as an end in itself, as experience is subjective and situational, see Small, M.L. ‘‘How many cases do I need?’ On science and the logic of case selection in field-based research’ (2009) Vol. 10 (1) Ethnography 5, 18. It may allow for transferable findings in respect of the theoretical propositions/hypotheses being examined if not to a population as would often be the situation in much quantitative research. 16 x For greater insight on this point see Lipset, S. M., Trow, M. and Coleman, J.S. (1956) Union Democracy: The Internal Politics of the International Typographical Union , New York: New York Free Press at 419-420; Yin, 2014, 21. For a discussion of the problems inherent in aping quantitative terminology in qualitative work see, Small, 2009, 10, and at 19 for further reading on the logic of case study selection and further reading on extended case method. It aims to examine rival hypotheses, propositions, potential explanations previously advanced (exploratory study), or to test findings from a previous case study examining similar phenomena in a new instance (a replication or confirmation study). 17 x Gerring, 2007, 346.

As described so far it is a research method that appears to have a lot in common with experiments and tests of statistical significance. But case study method differs markedly from a big data survey or double-blind experiment in that it seeks explicitly a phenomenon in its natural environment and (in most instances) without means to control for variables, including the behaviour of any participants. 18 x Although note that there are some scholars who believe that case study method can include elements of experimental testing, for example, Gerring, J. and McDermott, R. (2007) ‘An Experimental Template for Case Study Research’ Vol. 51 No. 3 American Journal of Political Science 688-701. One such study in law that has been described by some, if not by the researchers themselves, as a case study did include an experimental design within the battery of methods employed see: Moorhead, R., Sherr, A., Webley, L., Rogers, S., Sherr, L., Paterson, A. & Domberger, S. (2001) Quality and Cost: Final Report on the Contracting of Civil Non-Family Advice and Assistance Pilot (Norwich: The Stationery Office). Experiments aim to control some factors so as to test hypotheses under different conditions, quantitative studies attempt to control for environmental factors through sampling techniques and data collection instrument design so as to minimise their biasing effects, but case study method does not involve control of the environment, or control for the environment, instead it aims to harness context and work within it. It examines in great detail one situation (referred to as a case or unit) or a very small number of situations, to use context as a means to particularise the findings. It also seeks to explain which elements of context may mean that some of the findings are applicable to other situations and if so under what conditions. A case study tells the researcher about the case and the extent to which previous explanations are sustained, in some instances it may also allow the researcher to make claims that some of the findings can be applied to another case or cases too, although this is heavily dependent on the research design and its execution. 19 x Campbell, D.T. Foreword in Yin, 2014 xviii. But it is rarely, if ever, a method that can be used by one to want to make universal claims. A case is not a proxy for a sample of a population in a survey, for example, it is a study of a phenomenon in itself rather than a means through which to view the whole world. Having said that, samples can be used to help select cases in a sound manner. 20 x Seawright, J. and Gerring, J. (2008) ‘Case Selection Techniques in Case Study Research: A Menu of Qualitative and Quantitative Options’ Vol. 61 No. 2 Political Research Quarterly 294-308.

Case studies are only one of a number of ways to undertake socio-legal or criminological research and it is important to give proper consideration to the full range of research methods prior to making a final decision to adopt a case study method. 21 x Yin, 2014: chapter 1. It may be better to employ a different one: legal history; doctrinal legal study (legal cases, legislation, regulatory documents); a policy study (policy documents, communiqués etc.); a statistical analysis (an analysis of the number of different types of legal cases that go before the courts, their key features and what role these play in chances of success for the plaintiff); a large-scale survey; stand-alone interviews; or an experiment in a simulated setting (asking lawyers to read through some scenarios and explain what advice they would give to a client in those situations). But a case study could employ a number of these methods in combination, so how then does one determine whether case study method is right for one’s study? It will largely depend on the nature of the research question to be answered and one’s appetite for undertaking in-depth research aimed at achieving thick description (detailed description of how or why something is as it is) 22 x For a discussion see: Ryle, G. (1949). The Concept of Mind . London: Hutchinson; Lincoln, Y.S. and Guba, E.G. (1985). Naturalistic Inquiry . Newbury Park, CA: Sage Publications; Holloway, I. (1997). Basic Concepts for Qualitative Research . London: Blackwell Science. and/or triangulated findings derived from a range of data sources that develop a new theory or test existing rival theories. It is an intensive study, it requires extremely good planning and design and a robust approach to data analysis too.

• 2. Case Study Method: Research Design

Research design is of paramount importance in achieving a successful case study, especially so given that the study focuses on one or a small number of situations and the researcher’s in-depth knowledge of and immersion within that case may lead more readily to confirmation bias than in some other forms of study. 23 x Confirmation or interpretive bias of data is something we all have grapple with, as the natural human tendency is to place more weight on evidence that confirms our view than on evidence that contradicts it. Strong research design can assist with counter-balancing this to some extent, including the transformation of any expected finding into a hypothesis that one then seeks to falsify rather than to confirm. Research design begins with the choice of research topic, usually then followed by a review of any relevant academic literatures (perhaps beyond the boundaries of one’s own discipline, for example sociology, criminology, political science) to determine an appropriate research question, noting all possible answers to the research question that are posited in the literature. 24 x See Yin, 2014, chapter 1 for more information on the role of the literature review. During this iterative phase the research question will be further refined, so that it may be articulated with precision, which is particularly important for much case study research as the link between the question and the chosen case or cases is usually explicit and explained, consequently a clear research question is considered by many to be an essential starting point to aid the selection of cases to be examined. The rival hypotheses, theories or propositions that may answer the question should, normally, be similarly delineated and clarified, those that remain plausible answers to the research question should be retained and be supplemented with any others that the researcher considers to be alternative viable explanations. Other approaches may be used that are more inductive than deductive, as in the case of many ethnographic case studies such as Latour’s, for example. This phase is an intellectually demanding one, but it sets the foundation for a strong study that is easier to execute at the point of data collection. The literature review also helps to ensure one is up-to-date, that one does not make the same mistakes that earlier researchers have reported as hazards, and to add a theoretical depth to one’s study that aids sophisticated analysis. It may also help to identify useful data collection methods and instruments too. And so time spent on the literature review may be very profitable. So far the discussion has been very general and therefore a little abstract. At this stage it may help to consider a hypothetical research proposal for case study research and work though it as the article progresses. The researcher in our hypothetical scenario is interested in undertaking research on recent reforms to the use of family mediation in the family justice system in England and Wales. She knows that it is now compulsory in most instances for the person who is initiating any court proceedings in a divorce to have participated in at least one mediation information and advice session with the aim of negotiating an outcome in relation to children, money and/or property prior to initiating proceedings in court. She is clear on the law and the procedural issues but not clear on how effective have been these changes, and this is her broad area of interest. After completing her literature review she understands that the key aims of the reforms were to reduce the number of cases going to court by increasing the number of cases that result in negotiated agreements between the divorcing spouses and in doing so to reduce the cost and the time involved in reaching outcomes in divorce cases, reduce the need for people to use lawyers in the negotiating process, to reduce acrimony between the divorcing spouses and to reduce the negative effects on children. Further, the reforms were intended to promote more durable outcomes between divorcing spouses that could be renegotiated effectively if the arrangements for the children needed to be updated to meet changing circumstances. But the researcher still needs to work these insights into a research question before making a final decision on whether a case study is the best method by which to conduct the research. The next sub-section will consider the framing of the research question, and will include examples of how our researcher may draft her question to maximise her chances of undertaking a great study on her area of interest.

A. The Research Question

The process of defining the research question may be a painful, frustrating one but it could also be creative too. It may be necessary to spend a considerable period of time reading the literature so as to narrow down the research topic or statement to a manageable, novel and/or important and scholarly question. 25 x For more assistance with legal research questions refer to Epstein, L. (1995) Studying Law and the Courts in Lee Epstein (ed) Contemplating Courts , Cong Q, 1, 3-5. Some argue it should also seek to address a real world problem, although that is a controversial component and suggests that knowledge for knowledge’s sake is not a legitimate aim. 26 x See King, Keohane and Verba, 1994, 15. The development of a research topic into a research question with reference to the academic literature is sometimes described as the phase in which the researcher has a conversation or dialogue with the literature. This dialogue grounds the study, it also informs the study design, including the case selection and data to be collected. The research question (a statement that ends with a question mark) is made up of two key elements: its substance, the topic or issue that you wish to address and the form of the question ‘who, what, where, how, why’. 27 x See Yin, 2014, at 11, and see further Campbell, J.P., Daft, R.L., Hulin, C. L. (1982) What to Study: Generating and Developing Research Questions (Studying Organizations) , Sage Publications, for further thoughts on research questions The substance of the research question is not simply the topic but the specifics of the topic – is your study to be a contemporary one or a historical one? In what context are you operating? What precisely are you endeavouring to study? The form of the question is also important: as indicated previously, case study method is considered to be better suited to research questions framed in ‘how’ or ‘why’ terms. Single case studies are considered to be an excellent means by which to uncover and understand the processes or mechanisms that influence particular variables (known as process tracing 28 x On process tracing see: Collier, D. ‘Understanding Process Tracing’ Vol. 44 Political Science and Politics 823-830 and George, A. L. and Bennett, A. Case Studies and Theory Development in the Social Sciences , Cambridge, MA, MIT Press, 2005. For an excellent insight into how this has been used in a legal and policy context with reference to changes in Georgia’s tax laws see Ulriksen, M.S, and Dadalauri, N. ‘Single Studies and Theory-testing: The Knots and Dots of the Process-tracing Method’ (2014) International Journal of Social Research Methodology 1- 17. ), why or how different variables are related to each other, for example what influences legislative change or policy formation on a given topic in a given country at a particular time. They are also a sophisticated means through which to test empirically and deductively the congruence of rival explanations (theories or propositions), ‘to what extent’ or ‘how’ and ‘why’ different theories are borne out by the data. 29 x See Blatter, J. and Haverland, M. Designing Case Studies , Basingstoke, Palgrave MacMillan, 2012 at 145 who consider process tracing involves inductive reasoning to build theory and congruence testing involves deductive reasoning to test theories. The form of the initial research question can confound some researchers who initially phrase their question as a ‘what’ question and as a result unnecessarily rule out case study method. Questions can often be reframed, for example: ‘what have the prosecutorial authorities in England and Wales done to integrate victims of domestic violence into the criminal justice process?’ may be rephrased as ‘how have the prosecutorial authorities in England and Wales integrated victims of domestic violence into the criminal justice process?’. At this stage our hypothetical researcher is faced with some choices: should she consider ‘how have the reforms to family mediation used in the divorce context affected the durability and suitability of post-divorce arrangements in England and Wales?’ This would focus on the agreements whether agreed or adjudicated, their longevity, the extent to which they could be made to work after the divorce and how any amendments to arrangements were sought. Alternatively, she could ask ‘how have the reforms to family mediation affected the way in which divorces are conducted in England and Wales?’ This would examine the steps people took so as to get divorced but may also consider the divorcing couples’ perceptions about the process to assist with examining the policy to reduce acrimony, it could also address how much time and money they spent in the process and it could also elicit data on how constructive was their relationship and negotiations subsequent to the initial agreement or adjudication. It could also bring in the role of lawyers and/or consider the children’s experience of the divorce process too. The research design would then follow the focus of the question. Case studies are much easier to design when the research question is expressed clearly, the theory is used to provide possible answers that may be explored or tested and the boundaries of the study are articulated. Some people find it helps to break down a draft research question into its substance and its form, and describe the purpose of the study in a couple of lines too, 30 x Epstein and King suggest a range of possible purposes, at 59, including: to explore something that has not previously been studied; to attempt to settle a debate that has been ongoing within the literature; to examine a well-considered question but in a new way; to collect and analyse new data to seek to confirm or refute previous findings; to analyse an existing data set in a new or better way to seek to confirm or refute previous findings or to develop new ones. and then compare the extent to which all three are congruent and precise before moving on to the next phase of the design process. We shall consider the important role of theory in the next sub-section.

B. The Theory

Case studies afford the opportunity to observe a sequence of events or factors, to evaluate which produce an outcome and why, 31 x Peters, B.G. Comparative Politics: Theory and Method , Basingstoke, Palgrave, 1998 at 14. and to do so in their natural environment. One of the challenges for legal researchers, less so for criminological or sociological researchers who are often trained more fully in this regard, is the need to engage with theory before moving on to the next stage of research design. By theory I mean the explanations that have been posited in the academic literature for why or how something is the way that it is, or claims that suggest relationships between certain things. 32 x But interestingly, stance, or more accurately epistemology is of less significance to this research method than to many others. Case study method links the research question, research design, analysis and logic of inference to such an extent that is can accommodate a range of epistemological traditions from the realist to relativist/interpretivist. And thus scholars who consider that there are facts independent of our interpretation of them (in essence, hard facts operating in an objective reality) and scholars who consider all ‘facts’ to be local interpretations constructed through our own lenses, are able to operate within a case study framework. Having said that their choice of data sources, and their approach to data generation and analysis may well vary considerably. In this context a theory is a relatively precise speculative answer to a research question, which may have been developed by undertaking a study or by analysing others’ studies (a meta-analysis). And theories can be converted into hypotheses when considered in the light of a new research question. The use of theory is exemplified by Uriksen and Dadalauri’s case study on tax policy reform in Georgia which aimed to answer why and how Georgia initiated and managed to implement quite radical and substantial tax reforms between 1991 and 2005 and in doing so sought to interrogate theoretical explanations about the nature of policy reform in developing countries and further to develop a model that could be tested in other post-Soviet states. 33 x Ulriksen and Dadalauri, ibid. See further Dadalauri, N. Tax Policy Formation and the Transnationalizationof the Public Policy Arena; A Case Study of Georgia , Aarhus, Politica, 2011) Georgia was selected as a crucial case. The reasoning for this and also for the methods employed in this study are elegantly set out in the article cited above. In our hypothetical case study it may be possible, for example, to test the theory that mediated agreements lead to less acrimonious relationships between the divorced couple than do lawyer negotiated agreements. One could examine the theory that family mediation is a cheaper and faster alternative to lawyer negotiated settlements and that those mediated agreements are more durable and better suited to family circumstances. In doing so, one may test existing theories and/or to develop a new theory. Or one could test in the chosen context a single theory that is dominant or particularly novel. Our researcher could undertake further reading of the literature to add to these hypotheses and to refine them and eliminate those that are no longer plausible in the light of more detailed investigation. This is known as setting out ‘priors’, prior explanations raised in the academic literature. 34 x On the importance of the identification of priors see Beach, D. and Pedersen, R.B., ‘What is process tracing actually tracing? The three variants of process tracing methods and their uses and limitations’ Sept 1-4 2011 The American Political Science Association Annual Meeting , Seattle, WA as cited by Ulriksen and Dadalauri, ibid; and further Gerring, 2007 ibid. The researcher’s next task would be to consider what she would expect to observe in the study, were any of these hypotheses true (the observable implications of the hypotheses), for example, our researcher would expect to see that divorces conducted using family mediation would be settled through mediation, that the settlement process would cost less and be concluded quicker than in lawyer-led divorces, and that those that used family mediation would be more able to engage in constructive dialogue post-divorce and to renegotiate arrangements in respect of children without the need to resort to lawyers or to the courts. The researcher may also draw up a hypothesis that the agreements would be more durable and the outcomes for children more positive. This pre-emptive delineation of as many possible observable implications, and how they could be measured, would allow the researcher to plan how to conduct the study and to adopt an appropriate design more likely to lead to a robust answer to the research question. 35 x For more assistance on extracting observable implications and considering their measurement see Epstein and King, 2002, 70-76. But one needs to be able to articulate the theory converted into hypotheses with clarity, in order for an observation protocol to be developed. Further, it helps the research design if the researcher is able to pose rival theories or explanations so as to design the study to test for plausible alternative explanations too. For example, one of the rival theories in the family mediation study is that the kind of people who use mediation through to conclusion and the kind of people who either refuse to do so or who drop out without reaching a settlement are different, and those drawn to mediation are more consensus driven and better able to communicate with their spouse than are those who do not. This rival hypothesis would alert the researcher to the need to design the study to examine those who do conclude mediated agreements and those who do not in order to analyse this rival claim. A case study allows for the examination of complex interrelationships between variables in situ, and the theory helps to identify what those variables are.

C. The Selection of the ‘Case’ or ‘Cases’

Definitionally this is when it gets somewhat complicated, as ‘case’ can easily become confused with a ‘legal case’ and further a ‘case‘ can sometimes be confused with the same word used in a different context in quantitative research, a case meaning a single observation or a single data point. This has led some, such as Gerring, to suggest that it may be more accurate to refer to a case study as a unit study so as to underline that this type of study examines multiple things within one unit rather than examining one data source in one context. 36 x See Gerring, 2004, 342. See further King, Keohane, and Verba, 1994, 76-77. The selection of the case or cases is a profound one in any study of this kind. A case may be selected because it is critical to the research question, it is typical, atypical, it provides a longitudinal opportunity (study over time), or it is revelatory meaning that it allows insight where previously this has not be possible. 37 x See Yin, 2014, 51. The nature of the case, its boundaries and features and why it was selected should be set out clearly. 38 x Gerring, 2004, 344. There are often difficulties in establishing the boundaries of the case, the phenomenon under study, and the context that provides a background to the phenomenon but is not itself the object of enquiry. Gerring articulates this as the formal case (the phenomenon) and the informal cases (the penumbra of phenomena which are the context but which will need to be explored in a less formal way so as to distinguish the boundaries of the formal case). The informal units are peripheral, but may have bearing on the formal unit or case, and by considering these informal units at the beginning of the study, and close to its conclusion it will help the researcher to work out what is particular about the unit, and what is transferable to other units. The selection of the case should be guided by the extent to which this location in space, focus and/or time lends itself to construct validity, internal validity, external validity and reliability of design in respect to the research question. 39 x See Yin, 2014, chapter 2. Case studies are particularly prone to selection bias, meaning that the case is selected on the basis of the dependent variable rather than on the basis of the independent variable – selected because of an effect that has been noted rather than its cause when the nature of the cause is the real object of many ‘how’ or ‘why’ research study questions. 40 x See Geddes, B. Paradigms and Sand Castle: Theory Building and Research Design in Comparative Politics , Ann Arbor, University of Michigan Press, 2005. For example, in a legal context if we wanted to examine rival explanations for how a particular legislative reform, for example the introduction of same sex marriage in England and Wales, has had an impact on community cohesion between different religious and community groups, it may seem, on the face of it, a good idea to select a town like Brighton with a vibrant LGBT community as the case to be studied. After-all, the uptake of same-sex marriage has been very high in Brighton and so it could be considered to be a key site of study. However, Brighton is well known as a LGBT friendly town and people drawn to live there would tend to be very positive about the introduction of same-sex marriage. If the reason for the study was to consider whether tensions have emerged between community and religious groups with different views on marriage, then Brighton would not likely give much opportunity to examine these issues. It was LGBT friendly before the reform and it continues to be. And community and religious groups have worked well together before and after the change in the law. By selecting the case on the basis of the effect of the changed legal landscape, the high numbers of gay and lesbian marriages, we may have selected a case that is atypical or simply a poor unit within which to view the causes or the influences that led to the legislative reform. Researchers are prone to make this mistake when undertaking a deductive study to test the congruence of rival hypotheses in a context where they have insufficient knowledge about the independent variable (the causes) that gave rise to, say, the change in the law. There may be other factors to consider too: in our family mediation study, the researcher may choose to steer away from London as the case study location, if she is interested in ‘typical’ divorces given that London has a much greater than average number of high net worth divorces that include very large sums of money and property portfolios, in addition to many divorces involving non-British couples who married abroad. A solid grasp of the literature can help to alleviate this possibility of incorrect case selection along with detailed consideration of the relevant features of a range of possible case studies prior to final selection. Does selecting a multiple-case study limit the likelihood of such problems, further should the study be at one point in time or a repeated measure at different periods of time? A case study may be designed so as to allow for cross sectional analysis between two or more cases, and further a temporal variation may be introduced into this form of analysis too. Our researcher could study family mediation over time: the same divorces pre, during and post settlement and then later again to examine durability. Single, multiple or cross-sectional case studies often serve different purposes. Multiple case studies are more likely to be used when the causal relationship between an independent and dependent variables are being analysed, so that the interaction of the variables in different environments can be examined comparatively in different contexts (in an experimental protocol one would be able to manipulate the conditions so as to test the variables and thus the different hypotheses). For example, if the study was examining the relationship between violent crime rates and criminal justice sentencing policy to examine whether tougher criminal penalties for violent crime lead to a reduction in violent crime rates, and if tougher criminal penalties for lesser offences led to greater imprisonment levels and greater recidivism including an escalation of violent crime, then a multi-jurisdictional case study may allow for a better assessment of those by permitting different combinations of variables to be compared as against each other. In our family mediation study, the researcher may choose to use London as a crucial case study (with its unusual profile of divorcing spouses with a very wide range of asset values) alongside a more typical rural and a more typical urban location to consider the hypotheses under different conditions. However, it may also be possible to test the hypotheses in a single case study by charting the relationship between the variables over time, with particular attention being paid to the points in time when sentencing policy changed or crime rates dropped or raised, or when family mediation was first introduced, when it became established as a compulsory part of the system. 41 x But the difficulty with multi case studies is that specified conditions or features within individual cases may have more influence on the variables being studied than the variables that one are analysing across the studies. This may lead one to draw erroneous conclusions about causality. This single case study also illustrates the independent variable problem: to what extent are violent crime rates and/or sentencing policy more likely attributable to other societal changes evident at different points in time than each other? Without knowledge of this it is difficult to proceed. Sometimes the extent to which a case study is referred to as single or multiple is a matter of nomenclature, for example, Elliott and Kling’s study 42 x Elliott, M. and Kling, R. ‘Organizational Usability of Digital Libraries: Case Study of Legal Research in Civil and Criminal Courts’ (1997) Vol 48 (11) Journal of the American Society for Information Science 1023-1035. on the organisational usability of digital libraries, a case study of legal research in civil and criminal courts, could be described as a single case study (as in their study) because it addresses digital libraries in one context – legal research in courts – it is also geographically bounded to the Los Angeles County, but data is collected from a number of courts and thus it could be argued to be a multiple case study if each court were considered to be a case. The important distinction, however, is how that data are treated: if the data are pooled and analysed as a single unit then the case study is generally considered to be a single unit or single case study, if the data are analysed comparatively as between the sites of collection then it would generally be considered to be a multiple unit or multiple case study. Where comparisons are being made over time but within a unit then the terminology is often that of a single unit as data is both compared and pooled too. Single case study research is considered to be an excellent vehicle for exploratory and developmental research (as evidenced by the Georgia tax policy study and Dnes analysis of the nature of a particular type of contract – franchise contracts in the UK 43 x See Dnes, A.W. ‘A Case-Study Analysis of Franchise Contracts’ (1993) Vol. 28 Journal of Legal Studies 367-393. or Latour’s Conseil d’Etat study mentioned above 44 x See Latour, B. (2010) above. ), confirmatory research necessitates a design that allows the researcher robustly to test a small number of hypotheses forensically and it may be advisable to consider a multiple case study method to achieve this aim. 45 x Although note that Gerring, 2004, at 347 indicates that a single case study may credibly make causal claims, if, for example, the case has been selected as it is particularly representative of others or it is a critical or crucial case, see further: Eckstein, H. (1975) ‘Case Studies and Theory in Political Science’ in Regarding Politics: Essays on Political Theory, Stability, and Change, Berkeley: University of California Press, 1992). The family mediation case study conducted in London and a more typical urban and rural area is a good example of this, given that the types of divorce cases are likely to be quite different, the context is also different too, and so cost, duration, durability and acrimony could be tested under different conditions to see whether they held true in all conditions or were context or divorce type dependent. Sometimes researchers are inclined to use a multiple case study approach in the expectation that more cases (units) will provide more data and more comparable data that can be used to derive robust findings. However, data collected across multiple case studies is less rather than more likely to be comparable as the conditions within the case study cannot be manipulated or controlled by the researcher and yet environment is expected to have an impact on the data. 46 x Gerring indicates that researchers tend to face the choice between knowing more about less or less about more, 2014, 348. Where more than one case is selected, each subsequent addition should provide a more complete and accurate picture in respect of the research question, instead of attempting to provide greater representativeness (as indicated above, this is not the purpose of case study method). 47 x See Small, 2009, 24-26. If one wishes to make comparisons between case studies it is important to adhere closely to comparative methodology in the definition of the cases to be selected, the analysis of the relevant similarities and differences between those cases, the data to be collected, compared and why, and the likely limitations of the cross unit comparison. 48 x See Gerring, 2004 at 348. For guidance on comparative methodology in a legal context see: Van Hoecke, M. (2004) Epistemology and Methodology of Comparative Law , Portland, Oregon: Hart Publishing. Multiple cases studies are more difficult to accomplish successfully, and it is advisable to work out clearly what each of the case studies will contribute to answering the research question before finalising those to be the subject of the enquiry. The most appropriate design will be dependent on the research question selected and the hypotheses or propositions under investigation through the case study method. 49 x For a detailed discussion see Gerring, 2004 at 343.

D. The Selection of Data Sources, Data Generation and Collection

Strong research design logically links the research question(s) with the hypotheses, with the data generation and collection methods, which in turn should be logically linked with the data analysis methods employed too. By now the researcher is likely to have a very good idea of the types of data that may be relevant to the study (derived from documents, people, extant statistics, other artefacts like images), 50 x Yin sets out six sources of evidence: documents, archival records, interviews, direct observations, participant observation, physical artefacts, Yin, 2014, 105-118, and four principles of data collection: multiple sources of data; creation of a case study database; maintain the chain of evidence; exercise care when using data from electronic sources, at 118-129. having identified the substance, form and purpose of the question, the rival hypotheses that may contribute to answering the question, and the observable implications of those hypotheses and how they may be measured. Our researcher having drawn her broad question as ‘how have the reforms to family mediation affected the way in which divorces are conducted?’ and narrowed down the case study to a geographic location(s) or a type of divorcing couples or divorce context, will have considered the possible data sources as including the spouses, their children, the family mediators and lawyers, court files, mediation and lawyer negotiated agreements, official statistics and more. And so it should be possible to chart how the data collection and data analysis methods all fit together so as to allow the observable implications to be explored, the hypotheses proved, amended, or disproved and a rounded, reliable answer to the research question be achieved. But this design phase may also go beyond the identification and selection of data sources, it requires choices to be made about how data will be collected and measured. 51 x See Yin, 2014, chapter 3 for more information on what one needs to do before data collection begins, and chapter 4 on data collection itself. Each choice that is made, consciously, or unconsciously, will have an effect on the data that is captured and the reliability and validity of that data. And this in turn will affect the outcome of the study. Consequently, the design should be scrutinised to uncover the biases that may become entrenched within it, the study redesigned where necessary to eliminate or limit bias and any remaining biases be taken into account during the later analysis and reporting phases. 52 x See Small, 2009, 12-15. This will require a degree of reflection on data type (strengths and weaknesses in allowing observable implications to be explored), data selection (all data, if not all then what process is being used to select it and how may that skew the findings, known as selection bias); data collection (how is the data being derived, and is it raw data or is it material that requires a judgement to be made, for example how will we measure ‘satisfaction or ‘acrimony’ or ‘durability of an agreement’, how reliable and valid is the data collection instrument 53 x For a discussion of reliability and validity in measurement see Webley, 2010 and further Epstein and King, 2002, 80-99. ) and later too data analysis. It is also worth piloting each data collection exercise with a small number of observations so as to allow defects to be worked out, and experience in the field to allow for redesign too. And then one should collect as much data on each of the possible observable implications as is practicable, including data of different types generated or collected via different methods so as to allow for triangulation in respect of each hypothesis. For example, in Elliott and King’s study they collected data via observation, participant observation and interviews, analysed court documents and legal technology documents; 54 x Elliott and Kling, ibid at 1025. in Dnes’ study the data included franchise agreements and contracts, financial accounts and other financial data, interviews; 55 x Dnes, ibid at 369-370. and in Dadalauri’s study data were derived from primary sources (policy proposals, experts’ recommendations and the minutes of parliamentary sessions) and secondary sources (reports, media briefs, statistical sources) plus from semi-structured interviews with key actors in the policy process. 56 x Ulrikesn and Dadalauri, ibid at 13. Case study method necessitates a measure of flexibility in research design to allow for new knowledge to shape and improve the starting design, but that does not reduce the need for a robust design plan at the inception of the study. The design needs to be scrupulously documented, including challenges faced and amendments made so as to aid others to analyse the validity of the research design and to assess the extent to which the study findings are reliable and robust. 57 x For a discussion see King, Keohane and Verba, 1994, 12. Epstein and King suggest that legal scholars give the same attention to the recording, storage and analysis of data as they would expect of the police and prosecutors when securing the chain of evidence in a criminal case. 58 x Epstein and King, 2002, 24. And it is to data analysis that we now turn.

• 3. Case Study Method: Use of the Data, Inferences and Finding Meaning

Case study findings are reached through a process of logical valid inferences regardless of whether the data collected and analysed is qualitative, quantitative or both. 59 x For a discussion on this point see King, Keohane and Verba, 1994, chapter 1 ‘The Science in Social Science’. But first the data must usually be described in summary form, before being subjected to further analysis to consider what the data indicates about the various hypotheses and their observable implications in this case context. Subsequently, it is possible to attempt to derive descriptive inferences that suggest what these data on observable instances indicate about non-observables ones, in other words what findings one considers to be transferable to a non-observed context. The analysis may also allow for causal inferences to be made, that explain what effects would be expected to occur if certain conditions were fulfilled in this or another context. This is not dissimilar to data analysis in other types of empirical legal research and therefore it is considered only briefly here. However, case study method is structured with triangulation of data at the fore, allowing the researcher to reach robust findings reached by integrating analysis from multiple data points gathered using different methods. This section will briefly address data analysis, the drawing of inferences and the importance of demonstrating one’s working out, in turn.

A. Data Analysis

The first stage of data analysis is often validly to summarise the data collected in the light of the research question and hypotheses and anticipated observable implications, to summarise the numbers (mean, median, mode, standard deviation, range) and to summarise the text (for example, categorise and consider relationships between categories, or code and consider the frequency or codes). 60 x See Epstein and King, 2002, 25-29 for more information on quantitative data description. Different types of data will often be analysed using different methods or traditions, as illustrated by the way in which legal cases are analysed according to traditions accepted by lawyers, which is distinct from legal analysis of legislation, and policy analysis of policy documents: survey data would be analysed statistically, text based data (interviews, documents etc.) via the mode of analysis selected to interrogate and derive meaning from language, for example via grounded theory method, thematic coding, content analysis, hermeneutics etc. 61 x For a discussion of the different methods of text based data analysis see Yin, 2014, chapter 5 or Webley, 2010. There are a range of general strategies open to the researchers, some of which focus on the theoretical propositions, others aim to develop thick description, others still examine plausible rival explanations. 62 x See Yin, 2014, 136-142. Findings are considered robust where they are evidenced via multiple stands of data and its analysis. The use of multiple data sources to test each hypothesis allows the researcher to build up a thoroughly nuanced picture of the extent to which each hypothesis is sustained, needs to be refined, or rejected. The analysis will be conducted in the light of the research question parameters and also the hypotheses being examined by the research, as exemplified by the discussion in the Georgia tax policy case study. 63 x Ulriksen and Dadalauri, ibid. This process is likely to be iterative, in that data will often be analysed as one phase of data collection is complete and any lessons learned from that may lead to some reframing of the research question, reconsideration of the hypotheses, and amendments to the next phase of data collection yet to begin. The key is that, as with all social science methods, amendments to the question and methods, the analysis of the data and the inferences drawn from the data should be publicly explained and in sufficient detail so that they are replicable by others on the basis of the information provided in the write-up of the study; King, Keohane and Verba remind us that inferences lead to uncertain conclusions – inferences are not proven facts, they are propositions being advanced that are available to be tested by others. 64 x King, Keohane and Verba, 1994, 8. Conclusions remain tentative until replicated validly and consistently. The science and the rules of inference are important in allowing us to judge the validity and reliability of the findings, and these are closely interwoven with the research design and execution of the study.

B. Inferences

Case studies are often considered to be more useful when seeking to derive descriptive rather than causal inferences, as the researcher is not able to manipulate the environment so as to test propositions in such a way as to be sure that causal relationships have satisfactorily been established. Descriptive inferences are ‘the process of using the facts we know to learn about facts we do not know’ , by describing something that has been observed and inferring under what circumstances a similar pattern or occurrence may occur in a carefully defined unobserved situation. 65 x Epstein and King, 2002, 29. For example, if in our hypothetical family mediation study we learned that greater numbers of the divorcing clients who we interviewed/observed before the introduction of the compulsory mediation information and assessment meetings were aware that there was state funding available for family mediation, compared with the divorcing clients who we interviewed/observed after the introduction of these meetings then we may infer that this finding was likely to apply to divorcing clients outside our observed group too (all other things being equal). We do not know for certain that is accurate, as we only have data from our study participants, but our description of our findings has led us to infer something about those outside our observation group. Many doctoral candidates and early career academics baulk at the suggestion that descriptive inference is a valuable mode of analysis, as they associate ‘descriptive’ with the less positive feedback that they may have received in earlier work. But the pursuit of descriptive inferences is not a low-level aspiration in a context in which little is known about the case under scrutiny. Descriptive inferences allow for categorisation of findings which may lead on to further theory building and theory testing, categorisation goes to the heart of analysis development. So our finding above begs the question ‘why is this so?’ and we could either extend our study to answer this sub-question, or leave that for a later study. In some instances the inferences a researcher wishes to draw may be causal ones that infer an effect that will be caused by a set of defined factors occurring together. As an example, in our family mediation study we may wish to examine whether family mediation is more likely to be successful for couples with relatively similar educational backgrounds, medium to high incomes with both spouses in full-time employment, when compared with those who have unequal educational backgrounds, incomes and job-statutes and with low incomes. Where causal inferences are the point of the study, it may be possible to develop these with a well-chosen cross-case multi-case case study design. However, a causal inference first requires the identification of a causal mechanism (the process by which dependent variable A is affected by independent variable B, for example the causal mechanism for a defendant in the UK to be released from pre-trial detention (variable A) is a bail hearing in court (variable B)). 66 x See Epstein and King, 2002, 34-37 for help distinguishing between causal mechanisms and variables and causal effects. Case studies are often a really good means by which these mechanisms, or processes, may be uncovered – known as ‘process tracing’ whereby the researcher charts in detail the relationships between two or more variables and explores these connections to deduce those that are causal and those more likely to be coincidental. 67 x See Gerring, 2004, 348 and further Roberts, C., The Logic of Historical Explanation, Pennsylvania State University Press 1996, 66. Further, a single case study may allow a researcher to interrogate extant explanations that suggest causal implications, in other words to test predictions about what will happen in particular situations (assuming those situations are observable as part of the case study). This is known as ‘pattern-matching’. 68 x See Gerring, 2004, 348 and further Campbell, D. T. [1975] “‘Degrees of Freedom’ and the Case Study” in E. Samuel Overnman (ed) Methodology and Epistemology in Social Science , Chicago, University of Chicago Press: 1988, 380. This is where clarity about the purpose of the study becomes particularly important, as certain conditions will need to have been built into the research design for some analytical techniques. 69 x Yin, 2014, 142-168, provides five different analytical techniques: pattern-matching; explanation building; time-series analysis; logic models; and cross-case synthesis and suggests that after this phase the researcher will likely move on to work through all plausible alternative conclusions to examine whether the most likely conclusion is the only conclusion. The study will need to be designed with a very clear and narrow focus to achieve its aims. As indicated above, case studies may be entirely self-contained studies that provide in-depth knowledge of a single unit of analysis, but more often than not the researcher will wish for those findings to be considered applicable to situations that she/he has not observed. The challenge is to explain which findings are particular to the case study and which elements of the findings are relevant beyond the case study’s boundaries. 70 x Gerring, 2004, 345. It may be difficult to define this with precision, but where there is ambiguity it is safer to over explain and to over report the ambiguity and the possible range of inferences and their limitations rather than to over simplify and obfuscate the difficulty in reaching definitive findings. 71 x See Gerring, 2004, 346. Legal researchers are sometimes criticised for being vague in their explanations of the target of their inferences (to which other unobserved situations do these findings apply, and why?), or worse still their claims in the absence of evidence to prove that their inference is generalizable to a wide variety of situations. 72 x Epstein and King, 2002, 31. This may be a function of lawyers’ professional training as advocates, who in presentation would seek to persuade others to accept their position and who would gloss over inconvenient precedents. But lawyers are also trained to be forensic in seeking out the weaknesses of their arguments as well as those of their opponents and by harnessing these skills in the presentation of their case study findings; they should be able to display the highest standards of scientific reporting. Some of the ambiguities associated with inferences may be avoided if, as Gerring suggests, the scholar specifies clearly which propositions apply to which novel circumstances and exhibit and explain the evidence upon which this contention is based. 73 x Gerring, J. (2001) Social Science Methodology: A Criterial Framework , Cambridge, Cambridge University Press, 90-99. In other words, do the findings relate only to this case, are they intended to relate more broadly to similar cases and if so what marks out other situations as similar? Is similarity about time frame, location, a certain set of markers such types of participants, socio-economic, legal or political factors? And what is one’s evidence in support of this? The burden of proof always rests with the researcher. We shall turn to this in the next sub-section.

C. Reporting Findings

It can be challenging to know how to report one’s findings in an article or thesis, which is unsurprising when one considers that little attention is paid to this aspect of scholarship on doctoral legal programmes in many jurisdictions. 74 x See further Epstein and King, 2002. For assistance in how to report on cases studies and writing up and presentational considerations, including audience and purpose, see Yin, 2014, chapter 6. The rule is that one must provide as much detail as possible, at least enough to allow someone else to be able to replicate the study using only the information provided. Further, there needs to be sufficient discussion of the decisions taken, challenges faced and the consequent limitations of the findings so as to allow others to evaluate the reliability of one’s findings. As King et al. note: report uncertainty, be sceptical about causality, and consider rival hypotheses. 75 x King et al, 1996, 30-33. The process of interrogating one’s own decisions and inferences and reporting on them in full in the article or thesis may allow one to avoid the invidious charge made of many other legal scholars’ work. 76 x Epstein and King, 2002, 6-7. One suggestion is that legal scholars may wish to look for the weakest link in their chain of reasoning, something which lawyers are trained to do in a legal context, and then estimate how certain they are of their findings taking that weakness into account. 77 x Epstein and King, 2002, 50. However, other aspects of our professional training sometimes come into conflict with this approach: a research study is not an act of advocacy, and training as a lawyer may derail the process of empirical enquiry when lawyers unconsciously act for the client in their head and seek to persuade the outside world that their client’s view is a valid one, rather than to act as a legal social-scientist and demonstrate to other social scientists the extent to which their findings are valid, robust, reliable, and subject to limitation. 78 x See further Miller, A. S. ‘The Myth of Objectivity in Legal Research and Writing’ (1969) Vol. 18 Catholic University Law Review 290. For example, a researcher who is more in favour or less in favour of family mediation may inadvertently confirm their stance and steps need to be taken to lessen this risk. This role conflict is particularly problematic given that empirical legal research may lead to legal reform affecting large sections of the population and the findings confidently exhorted in the literature may be used to justify policy changes. 79 x Epstein and King, 2002, 8-10. Further, even if the research were not to be read outside of an academic environment, it is incumbent on all academics to produce research that is reliable and robust, lawyers are quick to critique legal scholarship that had been poorly executed and socio-legal scholarship should be treated no differently.

• 4. Conclusions: Why (Not to) Use Case Study Method?

Case study method is a powerful and engaging approach to research that has real utility in socio-legal and criminological research even if it has to-date been relatively little used. Our reticence to use it may be explained by the need for a researcher to be sufficiently adept with a range of social science research methods; (non-legal) empirical methods have historically had little treatment within undergraduate legal courses and relatively little attention even at a postgraduate level. 80 x Genn, H., Partington, M, and Wheeler, S. (2006) Law in the Real World: Improving Our Understanding of How Law Works Final Report and Recommendations , London: The Nuffield Foundation. Further, doctoral supervisors may feel inadequate to the task of supervising doctoral students proposing to undertake research through case study method and steer them towards a more standard mixed method approach such as a survey coupled with some interviews, or away from non-legal empirical methods altogether. But with some training, and a high degree of planning it is perfectly possible to undertake a good quality case study in a legal context and we can learn much from them. They are also an ideal means to focus on the particular and yet to draw analytical inferences to similar contexts too, something which lawyers are trained to do throughout their studies and a skill which they can bring to bear on a broader range of data than they otherwise often do. However, case study method is far more than focusing on a single situation, or ‘case’, it is far more than providing a temporal or physical boundary to our research endeavour. It requires us to adopt a structured and reflective approach to research design in many instances, to consider pre-emptively possible explanations (hypotheses) and rival propositions and to engage with theory at an early stage in a study. In a legal context case studies are generally, if not exclusively, more effective when: seeking to make descriptive rather than causal inferences; examining issues in depth rather than broadly and when the researcher is seeking to examine multiple sources of data so as to make comparisons within a case rather than between multiple cases. Further, they are also often more effective for seeking causal mechanisms rather than causal effects; for research that is exploratory rather than confirmatory; and when variations within the case selected are important for the study of the phenomenon. 81 x Gerring, 2004, 352. They are extremely useful when analysing how those involved in law and policy-making, the application of legal rules and procedures perceive these processes, how they react to them and how this influences the effectiveness of those rules, processes and procedures. The research process is an iterative and creative one that engages lawyers’ considerable analytical skills. As such case study method is worthy of a larger presence within the legal academic empirical tool-kit.

1 For insight into the extent to which legal researchers undertake empirical research and the lack of clarity around empirical methods in law see Epstein, L. and King, G. (2002) ‘The Rules of Inference’ Vol. 69 No. 1 The University of Chicago Law Review 1-133 at 3-6, Part I.

2 Yin, R. K. (2014) Case Study Research Design and Methods (5th edn.) Sage Publications, 12-14.

3 See Hamel, J. with Dufour, S. and Fortin, D. (1993) Qualitative Research Methods Volume 2 , Sage Publications, ch 1.

4 See Gerring, J. Case Study Research: Principles and Practices , Cambridge, Cambridge University Press, 2007 at 6. For a further discussion see Gerring, J. ‘What is a case study and what is it good for?’ (2004) Vol. 98 No 2 American Political Science Review 341-354. This looseness in definition in a legal context may perhaps be linked to confusion as between teaching and research case studies; some traditions in legal education employ a teaching method known also as ‘case study method’ which operates quite differently from its research counterpart. For a discussion of the differences between teaching and research case studies see Yin, 2014, 20 and for a discussion of teaching case studies see Ellet, W., (2007) The Case Study Handbook: How to Read, Discuss, and Write Persuasively About Cases , Boston MA, Harvard Business Review Press; Garvin, D. A. (2003) ‘Making the Case: Professional Education for the World of Practice.’ (Sept–Oct) Harvard Magazine 56-65.

5 ‘n’ (number) is used to denote the number of observations in the study, N is used to describe the total number within the population when n denotes the sample observed.

6 For a discussion of the state of empirical research in law see Epstein and King, 2002.

7 See Webley, L. (2010) ‘Part III Doing Empirical Legal Studies Research Chapter 38 - Qualitative Approaches to ELS’ in Cane, P. and Kritzer, H. (eds.) Oxford Handbook of Empirical Legal Studies , Oxford, Oxford University Press.

8 For a discussion about the differences between numerical (quantitative) and non-numerical (qualitative) data see Webley, id; Epstein and King, 2002, at 2-3; King,G., Keohane, R. O., and Verba, S ., Designing Social Inquiry: Scientific Inference in Qualitative Research , Princeton NJ, Princeton University Press, 1994 at 6.

9 Mascini, P., van Oorschot, I., Weenink, D. and Schippers, G., (2016) ‘Understanding judges’ choices of sentence types as interpretative work: An explorative study in a Dutch police court’, (37) (1) Recht der Werkelijkheid 32-49.

10 Yin, 2014: xxxi and further 16-17.

11 Triangulation is the term used to explain that a research question is considered from as many different standpoints as possible, using as many different data types as possible to permit a holistic examination of the question to see which explanations, if any, remain consistent across all data sources.

12 Yin, 2014: 5-6.

13 See Dul, J. and Hak, T. (2008) Case Study Methodology in Business Research , Oxford: Elsevier 8-11, 30-59.

14 Latour, B. (2010) The Making of Law: An Ethnography of the Conseil D’Etat , Cambridge: Polity Press.

15 For a discussion of ethnomethodological aims to study practical life as experienced in context as an end in itself, as experience is subjective and situational, see Small, M.L. ‘‘How many cases do I need?’ On science and the logic of case selection in field-based research’ (2009) Vol. 10 (1) Ethnography 5, 18.

16 For greater insight on this point see Lipset, S. M., Trow, M. and Coleman, J.S. (1956) Union Democracy: The Internal Politics of the International Typographical Union , New York: New York Free Press at 419-420; Yin, 2014, 21. For a discussion of the problems inherent in aping quantitative terminology in qualitative work see, Small, 2009, 10, and at 19 for further reading on the logic of case study selection and further reading on extended case method.

17 Gerring, 2007, 346.

18 Although note that there are some scholars who believe that case study method can include elements of experimental testing, for example, Gerring, J. and McDermott, R. (2007) ‘An Experimental Template for Case Study Research’ Vol. 51 No. 3 American Journal of Political Science 688-701. One such study in law that has been described by some, if not by the researchers themselves, as a case study did include an experimental design within the battery of methods employed see: Moorhead, R., Sherr, A., Webley, L., Rogers, S., Sherr, L., Paterson, A. & Domberger, S. (2001) Quality and Cost: Final Report on the Contracting of Civil Non-Family Advice and Assistance Pilot (Norwich: The Stationery Office).

19 Campbell, D.T. Foreword in Yin, 2014 xviii.

20 Seawright, J. and Gerring, J. (2008) ‘Case Selection Techniques in Case Study Research: A Menu of Qualitative and Quantitative Options’ Vol. 61 No. 2 Political Research Quarterly 294-308.

21 Yin, 2014: chapter 1.

22 For a discussion see: Ryle, G. (1949). The Concept of Mind . London: Hutchinson; Lincoln, Y.S. and Guba, E.G. (1985). Naturalistic Inquiry . Newbury Park, CA: Sage Publications; Holloway, I. (1997). Basic Concepts for Qualitative Research . London: Blackwell Science.

23 Confirmation or interpretive bias of data is something we all have grapple with, as the natural human tendency is to place more weight on evidence that confirms our view than on evidence that contradicts it. Strong research design can assist with counter-balancing this to some extent, including the transformation of any expected finding into a hypothesis that one then seeks to falsify rather than to confirm.

24 See Yin, 2014, chapter 1 for more information on the role of the literature review.

25 For more assistance with legal research questions refer to Epstein, L. (1995) Studying Law and the Courts in Lee Epstein (ed) Contemplating Courts , Cong Q, 1, 3-5.

26 See King, Keohane and Verba, 1994, 15.

27 See Yin, 2014, at 11, and see further Campbell, J.P., Daft, R.L., Hulin, C. L. (1982) What to Study: Generating and Developing Research Questions (Studying Organizations) , Sage Publications, for further thoughts on research questions

28 On process tracing see: Collier, D. ‘Understanding Process Tracing’ Vol. 44 Political Science and Politics 823-830 and George, A. L. and Bennett, A. Case Studies and Theory Development in the Social Sciences , Cambridge, MA, MIT Press, 2005. For an excellent insight into how this has been used in a legal and policy context with reference to changes in Georgia’s tax laws see Ulriksen, M.S, and Dadalauri, N. ‘Single Studies and Theory-testing: The Knots and Dots of the Process-tracing Method’ (2014) International Journal of Social Research Methodology 1- 17.

29 See Blatter, J. and Haverland, M. Designing Case Studies , Basingstoke, Palgrave MacMillan, 2012 at 145 who consider process tracing involves inductive reasoning to build theory and congruence testing involves deductive reasoning to test theories.

30 Epstein and King suggest a range of possible purposes, at 59, including: to explore something that has not previously been studied; to attempt to settle a debate that has been ongoing within the literature; to examine a well-considered question but in a new way; to collect and analyse new data to seek to confirm or refute previous findings; to analyse an existing data set in a new or better way to seek to confirm or refute previous findings or to develop new ones.

31 Peters, B.G. Comparative Politics: Theory and Method , Basingstoke, Palgrave, 1998 at 14.

32 But interestingly, stance, or more accurately epistemology is of less significance to this research method than to many others. Case study method links the research question, research design, analysis and logic of inference to such an extent that is can accommodate a range of epistemological traditions from the realist to relativist/interpretivist. And thus scholars who consider that there are facts independent of our interpretation of them (in essence, hard facts operating in an objective reality) and scholars who consider all ‘facts’ to be local interpretations constructed through our own lenses, are able to operate within a case study framework. Having said that their choice of data sources, and their approach to data generation and analysis may well vary considerably.

33 Ulriksen and Dadalauri, ibid. See further Dadalauri, N. Tax Policy Formation and the Transnationalizationof the Public Policy Arena; A Case Study of Georgia , Aarhus, Politica, 2011) Georgia was selected as a crucial case. The reasoning for this and also for the methods employed in this study are elegantly set out in the article cited above.

34 On the importance of the identification of priors see Beach, D. and Pedersen, R.B., ‘What is process tracing actually tracing? The three variants of process tracing methods and their uses and limitations’ Sept 1-4 2011 The American Political Science Association Annual Meeting , Seattle, WA as cited by Ulriksen and Dadalauri, ibid; and further Gerring, 2007 ibid.

35 For more assistance on extracting observable implications and considering their measurement see Epstein and King, 2002, 70-76.

36 See Gerring, 2004, 342. See further King, Keohane, and Verba, 1994, 76-77.

37 See Yin, 2014, 51.

38 Gerring, 2004, 344. There are often difficulties in establishing the boundaries of the case, the phenomenon under study, and the context that provides a background to the phenomenon but is not itself the object of enquiry. Gerring articulates this as the formal case (the phenomenon) and the informal cases (the penumbra of phenomena which are the context but which will need to be explored in a less formal way so as to distinguish the boundaries of the formal case). The informal units are peripheral, but may have bearing on the formal unit or case, and by considering these informal units at the beginning of the study, and close to its conclusion it will help the researcher to work out what is particular about the unit, and what is transferable to other units.

39 See Yin, 2014, chapter 2.

40 See Geddes, B. Paradigms and Sand Castle: Theory Building and Research Design in Comparative Politics , Ann Arbor, University of Michigan Press, 2005.

41 But the difficulty with multi case studies is that specified conditions or features within individual cases may have more influence on the variables being studied than the variables that one are analysing across the studies. This may lead one to draw erroneous conclusions about causality. This single case study also illustrates the independent variable problem: to what extent are violent crime rates and/or sentencing policy more likely attributable to other societal changes evident at different points in time than each other? Without knowledge of this it is difficult to proceed.

42 Elliott, M. and Kling, R. ‘Organizational Usability of Digital Libraries: Case Study of Legal Research in Civil and Criminal Courts’ (1997) Vol 48 (11) Journal of the American Society for Information Science 1023-1035.

43 See Dnes, A.W. ‘A Case-Study Analysis of Franchise Contracts’ (1993) Vol. 28 Journal of Legal Studies 367-393.

44 See Latour, B. (2010) above.

45 Although note that Gerring, 2004, at 347 indicates that a single case study may credibly make causal claims, if, for example, the case has been selected as it is particularly representative of others or it is a critical or crucial case, see further: Eckstein, H. (1975) ‘Case Studies and Theory in Political Science’ in Regarding Politics: Essays on Political Theory, Stability, and Change, Berkeley: University of California Press, 1992).

46 Gerring indicates that researchers tend to face the choice between knowing more about less or less about more, 2014, 348.

47 See Small, 2009, 24-26.

48 See Gerring, 2004 at 348. For guidance on comparative methodology in a legal context see: Van Hoecke, M. (2004) Epistemology and Methodology of Comparative Law , Portland, Oregon: Hart Publishing.

49 For a detailed discussion see Gerring, 2004 at 343.

50 Yin sets out six sources of evidence: documents, archival records, interviews, direct observations, participant observation, physical artefacts, Yin, 2014, 105-118, and four principles of data collection: multiple sources of data; creation of a case study database; maintain the chain of evidence; exercise care when using data from electronic sources, at 118-129.

51 See Yin, 2014, chapter 3 for more information on what one needs to do before data collection begins, and chapter 4 on data collection itself.

52 See Small, 2009, 12-15.

53 For a discussion of reliability and validity in measurement see Webley, 2010 and further Epstein and King, 2002, 80-99.

54 Elliott and Kling, ibid at 1025.

55 Dnes, ibid at 369-370.

56 Ulrikesn and Dadalauri, ibid at 13.

57 For a discussion see King, Keohane and Verba, 1994, 12.

58 Epstein and King, 2002, 24.

59 For a discussion on this point see King, Keohane and Verba, 1994, chapter 1 ‘The Science in Social Science’.

60 See Epstein and King, 2002, 25-29 for more information on quantitative data description.

61 For a discussion of the different methods of text based data analysis see Yin, 2014, chapter 5 or Webley, 2010.

62 See Yin, 2014, 136-142.

63 Ulriksen and Dadalauri, ibid.

64 King, Keohane and Verba, 1994, 8.

65 Epstein and King, 2002, 29.

66 See Epstein and King, 2002, 34-37 for help distinguishing between causal mechanisms and variables and causal effects.

67 See Gerring, 2004, 348 and further Roberts, C., The Logic of Historical Explanation, Pennsylvania State University Press 1996, 66.

68 See Gerring, 2004, 348 and further Campbell, D. T. [1975] “‘Degrees of Freedom’ and the Case Study” in E. Samuel Overnman (ed) Methodology and Epistemology in Social Science , Chicago, University of Chicago Press: 1988, 380.

69 Yin, 2014, 142-168, provides five different analytical techniques: pattern-matching; explanation building; time-series analysis; logic models; and cross-case synthesis and suggests that after this phase the researcher will likely move on to work through all plausible alternative conclusions to examine whether the most likely conclusion is the only conclusion.

70 Gerring, 2004, 345.

71 See Gerring, 2004, 346.

72 Epstein and King, 2002, 31.

73 Gerring, J. (2001) Social Science Methodology: A Criterial Framework , Cambridge, Cambridge University Press, 90-99.

74 See further Epstein and King, 2002. For assistance in how to report on cases studies and writing up and presentational considerations, including audience and purpose, see Yin, 2014, chapter 6.

75 King et al, 1996, 30-33.

76 Epstein and King, 2002, 6-7.

77 Epstein and King, 2002, 50.

78 See further Miller, A. S. ‘The Myth of Objectivity in Legal Research and Writing’ (1969) Vol. 18 Catholic University Law Review 290.

79 Epstein and King, 2002, 8-10.

80 Genn, H., Partington, M, and Wheeler, S. (2006) Law in the Real World: Improving Our Understanding of How Law Works Final Report and Recommendations , London: The Nuffield Foundation.

81 Gerring, 2004, 352.

• ↑ Back to top

Journal code: 23527927

Journal title: Law and Method

Journal id: 114364

Report an error Suggestions? Disclaimer © Boom juridisch

Making the Case for Case Studies in Empirical Legal Research

Dspace/manakin repository.

Harvard Empirical Legal Studies Series

5005 Wasserstein Hall (WCC) 1585 Massachusetts Avenue Cambridge, MA02138

Contact the Graduate Program

The  Harvard Empirical Legal Studies (HELS) Series  explores a range of empirical methods, both qualitative and quantitative, and their application in legal scholarship in different areas of the law. It is a platform for engaging with current empirical research, hearing from leading scholars working in a variety of fields, and developing ideas and empirical projects.

HELS is open to all students and scholars with an interest in empirical research. No prior background in empirical legal research is necessary. If you would like to join HELS and receive information about our sessions, please subscribe to our mailing list by completing the HELS mailing list form .

If you have any questions, do not hesitate to contact the current HELS coordinator,  Tiran Bajgiran.

All times are provided in U.S. Eastern Time (UTC/GMT-0400).

Spring 2024 Sessions

Empire and the shaping of american constitutional law.

Aziz Rana, BC Law

Monday, Mar. 25, 12:15 PM Lewis 202

This talk will explore how US imperial practice has influenced the methods and boundaries of American constitutional study.

Historical Approaches to Neoliberal Legality

Quinn Slobodian, Boston University

Thursday, Mar. 28, 12:15 PM Lewis 202

Fall 2023 Sessions

On critical quantitative methods.

Hendrik Theine , WU, Vienna/Univ. of Pennsylvania Monday, Nov. 6, 12:30 PM Lewis 202

Economic inequality is a profound challenge in the United States. Both income and wealth inequality increased remarkably since the 1980s. This growing concentration of economic inequality creates real-world political and societal problems which are increasingly reflected by social science scholarship. Among those detriments is for instance the increasing economic and political power of the super-rich. The research at hand takes a new radical look at media discourses of economic inequality over four decades in various elite US newspapers by way of quantitative critical discourse analysis. It shows that up until recently, there was minimal media coverage of economic inequality, but interest has steadily increased since then. Initially, the focus was primarily on income inequality, but over time, it has expanded to encompass broader issues of inequality. Notably, the discourse on economic inequality is significantly influenced by party politics and elections. The study also highlights certain limitations in the discourse. Critiques of inequality tend to remain at a general level, discussing concepts like capitalist and racial inequality. There is relatively less focus on policy-related discussions, such as tax reform, or discussions centered around specific actors, like the wealthy and their charitable contributions.

Spring 2023 Sessions

How to conduct qualitative empirical legal scholarship.

Jessica Silbey , Professor of Law at Boston University Yanakakis Faculty Research Scholar

Friday, March 31, 12:30 PM WCC 3034

This session explores the benefits and some limitations of qualitative research methods to study intellectual property law. It compares quantitative research methods and the economic analysis of law in the same field as other kinds of empirical inquiry that are helpful in collaboration but limited in isolation. Creativity and innovation, the practices intellectual property law purports to regulate, are not amenable to quantification without identifying qualitative variables. The lessons from this session apply across fields of legal research.

Fall 2022 Sessions

How to read quantitative empirical legal scholarship.

Holger Spamann , Lawrence R. Grove Professor of Law

Friday, September 13, 12:30 PM WCC 3007

As legal scholars, what tools do we need to read critically and engage productively with quantitative empirical scholarship? In the first session of the 2022-2023 Harvard Empirical Legal Studies Series, Harvard Law School Professor Holger Spamann will compare and discuss different quantitative studies. This session will be a first approximation to be able to understand and eventually produce empirical legal scholarship. All students and scholars interested in empirical research are welcome and encouraged to attend.

How do People Learn from Not Being Caught? An Experimental Investigation of a “Non-Occurrence Bias”

Tom Zur , John M. Olin Fellow and SJD candidate, HLS

Friday, November 4, 2:00 PM WCC 3007

The law and economics literature on specific deterrence has long theorized that offenders rationally learn from being caught and sanctioned. This paper presents evidence from a randomized controlled trial showing that offenders learn differently when not being caught as compared to being caught, which we call a “non-occurrence bias.” This implies that the socially optimal level of investment in law enforcement should be lower than stipulated by rational choice theory, even on grounds of deterrence alone.

Empirical Legal Research: Using Data and Methodology to Craft a Research Agenda

Florencia Marotta-Wurgler , NYU Boxer Family Professor of Law Faculty Director, NYU Law in Buenos Aires

Monday, November 14, 12:30 PM Lewis 202

Using a series of examples, this discussion will focus on strategies to conduct empirical legal research and develop a robust research agenda. Topics will include creating a data set and leveraging to answer unexplored questions, developing meaningful methodologies to address legal questions, building on existing work to develop a robust research agenda, and engaging the process of automation and scaling up to develop large scale data sets using machine learning approaches. 

Resources for Empirical Research

• HLS Library Empirical Research Service
• Harvard Institute for Quantitative Social Research (IQSS)
• Harvard Committee on the Use of Human Subjects
• Qualtrics Harvard
• Harvard Kennedy School Behavioral Insights Group

Past HELS Sessions

Holger Spamann (Lawrence R. Grove Professor of Law) – How to Read Quantitative Empirical Legal Scholarship?

Katerina Linos (Professor of Law at UC Berkeley School of Law) – Qualitative Methods for Law Review Writing

Aziza Ahmed (Professor of Law at UC Irvine School of Law) – Risk and Rage: How Feminists Transformed the Law and Science of AIDS

Amy Kapczynski and Yochai Benkler –(Professor of Law at Yale; Professor of Law at Harvard) Law & Political Economy and the Question of Method

Jessica Silbey – (Boston University School of Law) Ethnography in Legal Scholarship

Roberto Tallarita – (Lecturer on Law, and Associate Director of the Program on Corporate Governance at Harvard) The Limits of Portfolio Primacy

Susan S. Silbey – (Leon and Anne Goldberg Professor of Humanities, Sociology and Anthropology at MIT) HELS with Susan Silbey: Analyzing Ethnographic Data and Producting New Theory

Cass R. Sunstein  (University Professor at Harvard) – Optimal Sludge? The Price of Program Integrity

Scott L. Cummings  (Professor of Legal Ethics and Professor of Law at UCLA School of Law) – The Making of Public Interest Lawyers

Elliot Ash  (Assistant Professor of Law, Economics, and Data Science at ETH Zürich) – Gender Attitudes in the Judiciary: Evidence from U.S. Circuit Courts

Kathleen Thelen  (Ford Professor of Political Science at MIT) – Employer Organization in the United States: Historical Legacies and the Long Shadow of the American Courts

Omer Kimhi  (Associate Professor at Haifa University Law School) – Caught In a Circle of Debt – Consumer Bankruptcy Discharge and Its Aftereffects

Suresh Naidu  (Professor in Economics and International and Public Affairs, Columbia School of International and Public Affairs) – Ideas Have Consequences: The Impact of Law and Economics on American Justice

Vardit Ravitsky  (Full Professor at the Bioethics Program, School of Public Health, University of Montreal) – Empirical Bioethics: The Example of Research on Prenatal Testing

Johnnie Lotesta  (Postdoctoral Democracy Fellow at the Ash Center for Democratic Governance and Innovation at the Harvard Kennedy School) – Opinion Crafting and the Making of U.S. Labor Law in the States

David Hagmann  (Harvard Kennedy School) – The Agent-Selection Dilemma in Distributive Bargaining

Cass R. Sunstein  (Harvard Law School) – Rear Visibility and Some Problems for Economic Analysis (with Particular Reference to Experience Goods)

Talia Gillis  (Ph.D. Candidate and S.J.D. Candidate, Harvard Business School and Graduate School of Arts and Sciences and Harvard Law School) – False Dreams of Algorithmic Fairness: The Case of Credit Pricing

Tzachi Raz (Ph.D. Candidate in Economics at Harvard University) – There’s No Such Thing as Free Land: The Homestead Act and Economic Development

Crystal Yang (Harvard Law School) – Fear and the Safety Net: Evidence from Secure Communities

Adaner Usmani (Harvard Sociology) – The Origins of Mass Incarceration

Jim Greiner (Harvard Law School) – Randomized Control Trials in the Legal Profession

Talia Shiff  (Postdoctoral Fellow, Weatherhead Center for International Affairs and Department of Sociology, Harvard University) – Legal Standards and Moral Worth in Frontline Decision-Making: Evaluations of Victimization in US Asylum Determinations

Francesca Gino (Harvard Business School) – Rebel Talent

Joscha Legewie (Department of Sociology, Harvard University) – The Effects of Policing on Educational Outcomes and Health of Minority Youth

Ryan D. Enos (Department of Government, Harvard University) – The Space Between Us: Social Geography and Politics

Katerina Linos (Berkeley Law, University of California) – How Technology Transforms Refugee Law

Roie Hauser (Visiting Researcher at the Program on Corporate Governance, Harvard Law School) – Term Length and the Role of Independent Directors in Acquisitions

Anina Schwarzenbach (Fellow, National Security Program, the Belfer Center for Science and International Affairs, Harvard Kennedy School) – A Challenge to Legitimacy: Effects of Stop-and-Search Police Contacts on Young People’s Relations with the Police

Cass R. Sunstein (Harvard Law School) – Willingness to Pay to Use Facebook, Twitter, Youtube, Instagram, Snapchat, and More: A National Survey

Netta Barak-Corren (Hebrew University of Jerusalem) – The War Within

James Greiner & Holger Spamann (Harvard Law School) – Panel: Why​ ​Does​ ​the​ ​Legal​ ​Profession​ ​Resist​ ​Rigorous​ ​Empiricism?

Mila Versteeg (University of Virginia School of Law) (with Adam Chilton) – Do Constitutional Rights Make a Difference?

Susan S. Silbey (MIT Department of Anthropology) (with Patricia Ewick) – The Common Place of Law

Holger Spamann (Harvard Law School) – Empirical Legal Studies: What They Are and How NOT to Do Them

Arevik Avedian (Harvard Law School) – How to Read an Empirical Paper in Law

James Greiner (Harvard Law School) – Randomized Experiments in the Law

Robert MacCoun (Stanford Law School) – Coping with Rapidly Changing Standards and Practices in the Empirical Sciences (including ELS)

Mario Small (Harvard Department of Sociology) – Qualitative Research in the Big Data Era

Adam Chilton (University of Chicago Law School) – Trade Openness and Antitrust Law

Jennifer Lerner (Harvard Kennedy School and Department of Psychology) – Anger in Legal Decision Making

Sarah Dryden-Peterson (Harvard Graduate School of Education) – Respect, Reciprocity, and Relationships in Interview-Based Research

Charles Wang (Harvard Business School) – Natural Experiments and Court Rulings

Guhan Subramanian (Harvard Law School) – Determining Fair Value

James Greiner (Harvard Law School) – Randomized Control Trials and the Impact of Legal Aid

Maya Sen (Harvard Kennedy School) – The Political Ideologies of Law Clerks and their Judges

Daria Roithmayr (University of Southern California Law School) – The Dynamics of Police Violence

Crystal Yang (Harvard Law School) – Empiricism in the Service of Criminal Law and Theory

Oren Bar-Gill (Harvard Law School) – Is Empirical Legal Studies Changing Law and Economics?

Elizabeth Linos (Harvard Kennedy School; VP, Head of Research and Evaluation, North America, Behavioral Insights Team) – Behavioral Law and Economics in Action: BIT, BIG, and the policymaking of choice architecture

Meira Levinson (Harvard School of Education) – Justice in Schools: Qualitative Sociological Research and Normative Ethics in Schools

Howell Jackson (HLS) – Cost-Benefit Analysis

Michael Heise (Cornell Law School) – Quantitative Research in Law: An Introductory Workshop

Susan Silbey (MIT) – Interviews: An Introductory Workshop

Kevin Quinn (UC Berkeley) – Quantifying Judicial Decisions

Holger Spamman (Harvard Law School) – Comparative Empirical Research

James Greiner (Harvard Law School) – Randomized Controlled Trials in the Research of Legal Problems

Michael Heise (Cornell Law School) – Quantitative Research in Law

James Greiner (Harvard Law School) – A Typology of Empirical Methods in Law

David Wilkins (Harvard Law School) – Mixed Methods Work and the Legal Profession

Tom Tyler (Yale Law School) – Fairness and Policing

Modal Gallery

Gallery block modal gallery.

Making the Case for Case Studies in Empirical Legal Research

Title: Making the Case for Case Studies in Empirical Legal Research

Author: aikaterini argyrou, description: article, 19 pages., subject: legal research, qualitative research, publish date: - -, dois: https://doi.org/10.18352/ulr.409, language: english, location: online | click here.

Your browser does not support PDFs. Download the PDF

Empirical Legal Research: Getting Started

• Getting Started
• Articles and Journals
• Data Sets and Other Sources for Statistical Information
• Other Resources

Research Guides

If you are not familiar with the topic, you may find that a research guide or tutorial is a useful tool to assist you with your research. We hope that you will find this guide helpful.

You should review the date the research guide was last updated. Using a research guides updated in the last year will help ensure you are relying on current information. However, if you are conducting historical research, an older guide may be helpful.

Some of the most popular guides include:

1)  Georgetown Law Library, Statistics & Empirical Legal Studies Research Guide , (Updated June 2010)

2)  Chicago-Kent School of Law,   Empirical and Non-Legal Research Resources Guide , (Updated August 10, 2010)

3)  While not specifically providing a research guide, the Empirical Research Support site   of Goodson Law Library at Duke University provides links to training tools, data-sets, and other information to assist empirical legal researchers.

Current Awareness and News - Empirical Legal Studies Blog

The ELS (Empirical Legal Studies) Blog is one of the premier sources of news and comment about empirical legal research, publications and training. Below is a feed from the blog.

Empirical Research in Law, Empirical Legal Studies or Scholarship? Why is empirical research in the law important?

While there has been some debate regarding the proper name for and definition of empirical research in law, for purposes of introduction, this guide accepts the explanation put forth by John Baldwin and Gwynn Davis in Chapter 39 of the Oxford Handbook of Legal Studies: 

"...empirical research in law involves the study, through direct methods rather than secondary sources, of the institutions rules, procedures, and personnel of the law, with a view to understanding how they operate and what effects they have.  It is not a synonym for 'statistical' or 'factual', and its intellectual depth and significance are not determined by the empirical label ... 

Pauline Kim, University of Washington School of Law, put it similarly: 

Empirical legal scholarship involves methods developed in the social sciences and is different from traditional legal research in that it "systematically explores facts about the operation of the law and legal institutions."

Empirical research is important because "there are important questions in the law and about legal institutions that can’t be answered" through the traditional textual analysis methods of research.   For example, if a researcher is interested in researching the impact of selecting a particular rule of law on the decision making of individuals and businesses. Textual analysis would not shed light on a topic. However, we can certainly understand how an argument for a judgment accepting a particular rule of law would be strengthened by including evidence on the likely effect on "actors in the real world."

Pauline Kim, Do We Have the Numbers? Empirical Research in Law – International Law as a Case Study, Program at the American Association of Law Libraries Annual Meeting (July 10, 2006)

Funding for Empirical Legal Research

Conducting empirical research often involves significant costs, including the costs associated with collection or accessing data. As a result, empirical researchers may need to seek funds from grants awarding organizations and other funding sources. Academic researchers may find the institutional Office of Research can provide assistance in locating information about grant awarding institutions as well as assistance in preparing grant proposals. If the services of a Research Office are not available, some of the resources below will assist the researcher in seeking funding.

• American Bar Association Section of Litigation Research Fund The Litigation Research Fund was established by the section to support original and practical scholarly work that significantly advances the understanding of civil litigation in the United States. Projects addressing issues of low-income individuals' access to civil justice are of particular interest to the section and legal academics as well as social scientists and scholars from other disciplines are invited to apply for funding of their research.
• Grants.gov All discretionary grants offered by federal grant-making agencies can be found on Grants.gov. Federal grant applications can also be submitted online via this site.

Subject Guide

Insitutional Review Boards

Empirical researchers are often required, before they begin the research, to obtain approval of the research from an Institutional Review Board (IRB). They may also be required to submit periodic progress reports to the IRB. Academic researchers may find that the Office of Research can provide useful information or guide the researcher through the IRB Process. 

• Code of Federal Regulations Title 45 Part 46 Address the Protection of Human Subjects and the Requirements for IRB Review.
• OCU Office of Institutional Research and Assessment
• University of Oklahoma Office of Research Services
• Next: Books >>
• Last Updated: Sep 24, 2020 2:41 PM
• URL: https://libguides.okcu.edu/empiricallegalresearch

Empirical Legal Research Resources

• Statistics and Datasets
• Journals and Law Reviews

Selected Treatises: How to Conduct Empirical Legal Research

Representative empirical legal research works.

• Data Analytics Help and Resources at Stanford
• Current Awareness and Training Resources
• Bulk Download Availability

Ask us anything!

The law librarians and library staff at Robert Crown Law Library are here to help you! You can find the latest information about our hours and services on the library homepage . There are several ways to contact us:

Zoom : Drop in to our virtual reference room  during our Reference Desk hours to ask any questions.  

Make an appointment:  Online and in-person appointments with the reference staff can be made from the library homepage . 

• [email protected]
• Recommend a Resource
• Borrowing Services Desk: (650) 723-2477
• Reference Desk: (650) 725-0800

This page contains a selected list of major treatises on conducting empirical legal research, as well as a selection of books that demonstrate how some scholars use empirical legal research in their writing. You may find more works on the topic using subject headings, such as "legal research -- methodology," in Searchworks , the Stanford Library catalog.

• << Previous: Journals and Law Reviews
• Next: Data Analytics Help and Resources at Stanford >>
• Last Updated: May 14, 2024 4:23 PM
• URL: https://guides.law.stanford.edu/empiricallegalresearch

• Stanford Home
• Maps & Directions
• Search Stanford
• Emergency Info
• Terms of Use
• Non-Discrimination
• Accessibility

© Stanford University , Stanford , California 94305 .

leiden lawmethods portal

Writing an empirical legal article.

Last update: April 12, 2022

In comparison to a typical article in a law journal that you are perhaps used to, empirical articles – especially those published in social science journals – have a more standardized structure. Such articles typically start with an introduction, then discuss the methods used, subsequently report the findings in a results section, and ultimately discuss the findings and implications in a discussion section (see Figure 1). Below, we will briefly discuss some key points to take into consideration when writing an empirical legal article.

Writing the introduction

In several respects, the introduction of an empirical article mimics that of a typical law journal article. For example, the introduction is used to stress the importance of the research and to entice the reader to continue reading. However, whereas the introductions of articles in law journals are relatively short, the introduction of empirical articles comprises everything up until the methods section. Hence, the introduction then also includes a literature review, sometimes a theoretical framework, perhaps a conceptual framework, the hypotheses and justification/motivation thereof, etc.

Some pointers for the introduction are:

• Start broadly and in easy language when making your opening statements. Take the reader by the hand and guide them towards the key focus of your article. refrain from using jargon and other complex terminology in the first few lines of your introduction.
• Make it clear relatively early in the paper why the problem discussed in your article is important. Answer the “Who cares?” question. This will help make the reader enthusiastic to continue reading. For an empirical legal article, it is also important to explain why the empirical perspective is of added value for your legal research.
• Discuss what we already know about the topic in a carefully curated literature review. It is not always necessary to give a comprehensive overview of everything that has ever been said on the topic, but make sure to discuss/cite the key articles in the field and the most recent insights.
• Make it clear how your research helps the literature (and ultimately society). Are there some pressing questions that are yet to be resolved? Are there contradicting findings that you are hoping to resolve? Are you shedding new light on the assumptions underlying certain legal rules or mechanisms?
• When your goal is to test certain hypotheses, make your way towards the hypotheses and make sure it is clear to the reader why you expect a certain relationship or effect. In other words, ground them in the literature. In case your research is more exploratory in nature, therefore making it hard to formulate specific hypotheses, there is no need to arbitrarily define hypotheses. Just be transparent about the exploratory nature of your research.
• Finally, end your introduction with a short ‘bridge’ to the methods section. What are you going to do you in your study and how does that logically follow from the introduction?

Writing the methods section

The ultimate goal of this section is to make it clear to the reader what you have done. By being transparent, you allow other researchers to verify your findings through conducting the exact same study.

The method section typically discusses:

• The sample that you used, including all relevant demographic information. Take the reader by the hand and explain who (or what) your population is, how you chose your sample and why you made certain sampling decisions (e.g., random sampling, theoretical sampling, saturation).
• The design of the study. Explain what method you used. If you used a quantitative method, explain the variables (independent and dependent) and explain how you measured certain relationships. If you used a qualitative method, explain how you analyzed the data by for example explaining your coding methods.
• Make sure to clarify the procedure you followed, so that your reader knows how you’ve conducted your research. Say something about the set-up of the interview, the structure of the survey, or the design of the experiments, so the reader can understand what it was like to participate in your study.

If you want to publish your empirical research in a law journal or interdisciplinary journal, it is important to explain your methods in such a way that it is understandable for researchers that may not have any experience in empirical research. Some law journals might be a bit hesitant to include an extensive methods section and might want you to move most methodological details to an appendix, or describe them very succinctly in the text. Look for a recent edition of your target journal to see what is common for that journal, and make sure to write with your target audience in mind.

Writing the results section

In this section, you report your analyses and key findings. Regarding the text you use to describe your findings in words, make sure to first state your findings in words, and only then add the statistics. For example: “We hypothesized that judges would dole out harsher punishment if the unintended harm caused was higher. In line with this hypothesis, we found that judges awarded higher damages in the condition where they read the version of the case with the more significant collateral damage versus the cases with only minor side effects. Specifically, in the former condition the average damages awarded was USD 340.345 (SD = 23.763) versus USD 234.567 (SD = 17.345).”

You can already mention whether certain findings were in line with your hypotheses or not, but refrain from elaborately interpreting your findings in this section, as that will be done it the final part of your paper (the discussion section).

If possible, try to visualize your results in graphs and tables. Ideally, one is able to get a full picture of your findings by either looking at the visuals, or by reading the text and ignoring the visuals. It is therefore important to make clear graphs and tables with informative titles.

When publishing your empirical findings in a law journal, it is again important to write with your target audience in mind. Some journals may be hesitant to publish articles that contain a lot of statistical information. It could be that you want to focus on your key findings, and keep most of the statistical tests and number-heavy sections in your footnotes or an appendix. Similar to the methods section, we would say that the more data and information you manage to keep in the main text, the better.

Writing the discussion section

The main goal of this section is to interpret your data and explain what your results may imply. Start off by briefly reiterating the main goals of your research as well as your key findings. Then go on by critically evaluating the methods used and the robustness of your findings. What are some limitations imposed by your chosen method? How valid are your findings? To what extent might they generalize to the real world and other contexts?

Explain how your results fit in the context of your research topic as you explained in your introduction. What knowledge did you add to the existing literature? Explain what your findings mean for debates that are going in into the literature. If you find different results than other scholars, what might explain those differences? Do your findings say anything about how the law works in practice or do they confirm underlying assumptions of legal rules? Also, depending on whether your research has direct implications for (legal) practitioners, you may want to devote a subsection to the practical implications of your findings. In case any normative claims can (and should?) be made based on your findings, you can consider doing so in your article. For more information on this so-called ‘fact-value gap’ (i.e., going from data to normative implications), please see the entry on this topic in the methods portal.

Bem, D. J. (2000). Writing an empirical article. Guide to publishing in psychology journals , 3-16.

1 Historical Roots of Empirical Legal Studies

Empirical Legal Studies is the product of several histories. Mid-nineteenth century U.S. legal instruction and practice paved the way for law as an academic discipline. Late nineteenth century development of social science departments gave law faculty access to university infrastructure and interdisciplinary colleagues. Early twentieth century experiments in law and social science paved the way for research subfields such as law and economics, empirical legal studies, new legal realism, and law and social science. Many refer to these recent traditions collectively, as Empirical Legal Studies, empirical legal studies, or ELS, though this labelling is a source of debate.

Understanding these histories is important for three reasons. First, the histories point us to primary source research material such as the Litchfield notebooks, which were written by law students at one of the nation’s early law schools and digitized for research by twenty-first century law librarians (Eiseman, Bagnall, Kellett, & Lam, 2016). Second, the histories explain the dominance of certain social scientific disciplines and methods in past and current empirical legal studies. Finally, the histories of empirical legal studies enable us to predict the future of the field.

Chapter learning objectives

• Understand the early history of U.S. legal practice and instruction
• Analyze the interdisciplinary roots of empirical legal research
• Evaluate the influence of historical practices on current legal research and teaching
• Create historical research aids

Abbreviations and specialized terms

American Association of Law Libraries (AALL), Annual Law and Economics Review , Annual Review of Law and Social Science , case law, Conference on Empirical Legal Studies (CELS), clerk, content analysis, de facto , de jure , empirical legal studies, Inns of Court, IPUMS, Jurimetrics , Journal of Empirical Legal Studies (JELS), Law and Human Behavior , law and social science, Law & Society Review , Law Library Journal , legal realism, natural sciences, new legal realism, pragmatism, qualitative coding, Society for Empirical Legal Studies (SELS), social sciences, statutes

Early Legal Education and U.S. Law

United States legal instruction developed slowly. The colonial colleges offered no practical training in law (Klafter, 1993). So, wealthy young men went to Britain, attended Inns of Court, and applied to become English barristers (Lucas, 1974; Klafter, 1993). As trainees, they observed court proceedings and read barristers’ books and legal notes (Lucas, 1974). This pathway to practice was reserved for affluent, well-connected families. Following the U.S. revolution, these students began to train closer to home.

By the late 1700s, most new attorneys were self-taught practitioners that studied state statutes, or laws created by legislatures, in preparation for admission to the state bar (Klafter, 1993). Many served as apprentices, or law clerks, to practicing attorneys and judges. Clerkships were often unstructured, menial, and inadequate (Klafter, 1993). To fill the gap, prominent attorneys and judges began to launch private training programs.

Beginning in the late 1700s in Connecticut and surrounding states, aspiring attorneys could enroll in private classes (Forgeus, 1939). Many of these classes took place in law offices. The Litchfield law school, founded by Tapping Reeve and known for decades as Judge Reeve’s School, is widely considered the earliest formal training program, though a number of others opened soon after (Forgeus, 1939, 1942).

The early law schools typically had classes of a dozen or fewer students. Judge Reeve’s school was especially popular, though, and had 50 students around 1810 (Forgeus, 1942). Law students often hailed from prominent families, attended Ivy League undergraduate institutions, and went on to notable positions in government and industry (Forgeus, 1939, 1942, 1946). They kept notebooks that they recited from and corrected to master their assigned subjects.

By the early 1800s, the private training programs began developing formal curricula. A student notebook, circa 1811-13, from Sylvester Gilbert’s law school in Hebron, Connecticut lists a program of study focused on common legal issues such as mortgages, legal writing (e.g., pleadings), and legal practice in Connecticut state (Forgeus, 1942; Figure 1.1).

 1.1 Edited excerpt of Gilbert curriculum (Forgeus, 1942, p. 201)

Gilbert and other instructors utilized legal treatises, such as Kirby, Root, and Day’s collection of Connecticut cases or Swift’s treatise on Evidence (Fernandez, 2012; Forgeus, 1939, 1942). Still, their lectures focused on English cases (Forgeus, 1942) and the old country principles the instructors knew best (Klafter, 1993). As a result, the private training programs offered a regressive legal education that failed to fully prepare students in emerging U.S. laws and legal customs (Klafter, 1993).

 1.2 Law Librarian Spotlight: Elizabeth Forgeus

From the mid-1930s to late 1940s, Elizabeth Forgeus published numerous articles on law school history. Rich with passages and lists extracted from student letters and notebooks, her pithy articles humanized the early law schools. She also assisted with longer research works on the history of Yale Law School and legal practice in New England. Serving as Assistant Law Librarian at the Yale Law School for more than a decade, Ms. Forgeus was also active in the American Association of Law Libraries (AALL). Past issues of AALL’s flagship publication, Law Library Journal , list Ms. Forgeus as a member of the Committee on Cooperation with Latin-American Law Libraries and the Advisory Committee on Education for Law Librarianship—Editorial Board on Monographs. Her work demonstrated the value of reference librarianship, archival collecting, international outreach, and educational advocacy in law libraries.

By the mid-nineteenth century, private law programs gave way to academic law schools. The College of William and Mary and Transylvania College offered the first vocational legal instruction (Klafter, 1993). Harvard, the University of Maryland, Washington and Lee, Yale and others followed (Klafter, 1993). Despite the growing number of academic law programs, university legal instruction remained in flux for decades, in part due to a dearth of textbooks.

In 1870, Christopher Columbus Langdell joined the Harvard Law faculty. The next year, he published a Contracts casebook for use in his classes. In the preface, Langdell explained that most law students accessed books through the library and had no lasting copies of the cases they read (1871). He felt that reported cases were the best evidence of how the law was developing and that students needed to rigorously analyze case law to learn the principles and doctrines, or science as he called it, of U.S. law (1871).

Langdell’s approach was treatise-based and deductive. This view of legal education—and legal practice—was a hallmark of the era known as the classical period, or era of legal formalism (Hackney, 2014). Langdell’s Contracts casebook encouraged students to deduce general rules from existing cases. This case law method is still the most common form of teaching in first-year law school courses.

Langdell’s casebook is an artifact of a larger debate over legal formalism in the early twentieth century. Legal formalism posited law as apolitical or a priori to social concerns (Hackney, 2014). Believing that law should exist outside of messy social situations, many experts recommended contracts as a way to deal with complex social issues such as unfair working conditions and environmental pollution (Hackney, 2014). The Lochner case is now taught as quintessential formalism.

 1.3 Legal Touchstone: Lochner v. New York

In 1905, the Supreme Court issued its opinion in Lochner v. New York , finding that a New York state employment law was unconstitutional. The law prohibited bakeries from allowing their employees to work more than 60 hours a week or 10 hours a day, with some exceptions. In a five-justice majority opinion written by Justice Peckham, the Supreme Court found that the state statute interfered with employees’ contract rights under the 14th Amendment to the U.S. Constitution. Justice Harlan wrote a dissent, which was joined by Justices White and Day, that discussed workers’ health and argued that the government could legislate to protect the health of citizens. Justice Holmes also dissented and argued that Lochner was “decided upon an economic theory [laissez-faire] which a large part of the country [did] not entertain” (p. 75) The Lochner majority opinion represented the formalist position that the law is generally set apart from concerns of health and economics (Hackney, 2014). The dissents reflected a growing trend toward viewing the law as embedded in social relations.

Today, scholars and practitioners actively debate the law’s relation to society (Suchman & Mertz, 2010). In fact, empirical legal studies can be understood in opposition to formalism because ELS explores how the law can cause or ameliorate social problems. But despite the widespread acceptance of empirical legal studies as a valid research field, the empirical approach has been slow to infuse law school curriculum. First year law students still largely dissect cases presented in Langdell-style casebooks; they do not study how real people interpret liability waivers or rental contracts, etc. So, law school instruction focuses on formal rules while empirical legal studies research explores law in social context.

Research that examines how the law treats various classes of plaintiffs or improves the function of government, etc. requires more than a social orientation. It requires social science theories and research methodologies. It is no surprise, then, that empirical legal research followed the rise of modern social science departments in the United States.

Development of U.S. Social Science Departments

Prior to the 1840s, U.S. colleges generally taught a classics curriculum—Latin, Greek, mathematics, literature, and rhetoric—derived from leading Western European universities (Geiger, 2000; Thomas, 2015). By the mid-1800s, this standard curriculum began to vary across institutions and regions of the country (Geiger, 2000). Institutions such as the Lander College for women in South Carolina featured the classical A.B. degree into the twentieth century, whereas colleges throughout the state of Ohio began moving away from the curriculum decades earlier (Bondurant, 1909; Geiger, 2000). Overall, the traditional classics degree declined nationally as multipurpose colleges began to take shape (Geiger, 2000). Multipurpose colleges enrolled a more diverse student body, including women and lower class students, and responded to “emerging markets for practical, vocational skills” (Geiger, 2000, p. 148).

 1.4 Understanding early U.S. college admissions trends through scholarly literature and government data

The multipurpose colleges were more diverse than the colleges that preceded them, but their student bodies were still largely White and upper class. Most nineteenth century colleges did not admit Black students and many had de jure (by policy or law) or de facto (in practice) admissions policies that limited the attendance of Asian, Latino, Jewish, and Native American students. Empirical researchers can explore scholarly literature on college admissions via a literature search. For instance, the following search string should produce useful results in EBSCOhost: ((college OR university) AND admiss* AND (historic* OR “nineteenth century”) AND (race OR gender OR Black OR “African American” OR Asian OR Hispanic OR Jewish OR Latino OR “Native American”)). For demographic context, the U.S. Census Bureau maintains historical publication series. The Minnesota Population Center lists the Census questionnaires from 1850 forward on its IPUMS, or Census microdata, site at https://usa.ipums.org/usa/voliii/tEnumForm.shtml. The Census questionnaires offer a glimpse into the evolving language of racial and ethnic identity—and exclusion—in the U.S.

In 1862 and 1890, Congress passed the Morrill Acts (Geiger, 2000). The Morrill Acts gave federal lands to states for the creation of local colleges focused on “agriculture and the mechanic arts” (Morrill Act of 1862). The new laws furthered an existing system of publicly-supported colleges—including tax-supported private colleges like Harvard and Yale (Thomas, 2015)—but emphasized the vocational role of land-grant institutions. As a result, U.S. colleges became more attuned to local industries and interests than their Western European counterparts (Thomas, 2015). But though U.S. colleges were becoming more pragmatically focused, they were not yet incubators of scientific discovery or rigorous social science research.

A number of factors slowed the development of scientific and social scientific research in U.S. colleges. Most U.S. colleges had “serious religious commitments,” and formalistic theological instruction permeated the curricula until well after the Civil War (Veysey, 1965). For instance, more than 40% of Yale University’s 1900 budget supported the Theological Department (Treasurer of Yale University, 1900;  1.5).

1.5 Yale University departmental budget allotments, 1900

Nineteenth century U.S. colleges, unlike their Scottish counterparts, had no track record of scientific excellence, and no community of science researchers or educators (Ben-David, 1992). When reformers at Cornell and Harvard wanted to increase scientific research and teaching in the late 1800s, they struggled to find competent teachers and a place in the undergraduate curriculum for such courses (Ben-David, 1992). As a result, reformers shifted their attention to the creation of graduate schools at existing colleges.

Once graduate schools began producing rigorously trained academics, those scholars were able to modernize both research and undergraduate instruction in the sciences and social sciences (Ben-David, 1992). At that point, the U.S. system again diverged from much of Western Europe, where “first-rate scholars were not willing to teach courses in a program of general education” (Ben-David, 1992, p. 83). In the U.S., top-tier researchers conducted research, employed graduate students, and taught undergraduate students in university departments dedicated to the natural and social sciences. Their research built upon the work of European and U.S. scholars.

European pioneers of the social sciences included Adam Smith, Marie Jean Antoine Condorcet, and Johann Gottfried Herder (Ross, 1993). In the U.S., notable figures included Harvard’s William James and the University of Chicago’s John Dewey (Buxton, 1984; Thayer, 1982), not to be confused with Columbia University’s library science educator, Melvil Dewey. James and John Dewey led the development of pragmatism, a U.S.-born philosophical tradition that emphasized the value of grounded theories of people and the social world (Misak, 2013; Stanford Encyclopedia of Philosophy). Dewey’s view of pragmatism, like much early U.S. social science, hewed to natural science methods (Misak, 2013; Ross, 1993). According to historian Dorothy Ross, the U.S. social sciences were generally less heterogeneous and more derivative of the natural sciences than their European counterparts (1993). Throughout the twentieth century, Economics increasingly dominated the U.S. social sciences (Haskell, 1977; Siegfried, 2008). As evidence: the American Social Science Association, founded in late 1800s, now operates as a subset of the American Economic Association (see Haskell, 1977; Siegfried, 2008). The unique trajectory of the U.S. social sciences as empirical, experimental, and economics-dominated set the stage for the empirical legal research movements of the twentieth century.

 1.6 Notes from the Desk of Sarah E. Ryan: My empirical research on the ‘facebook’

In 2012, Yale Law School hired me as the nation’s first dedicated empirical legal research librarian (Miguel-Stearns & Ryan, 2014; Ryan & Miguel-Sterns, 2014). So, I had to build the new service area. My biggest issue was that I did not know where my patrons—especially the students—were starting from. Were they continental philosophers? Economists? Mechanical engineers? I mulled the idea as I began to take appointments. To prepare for one reference meeting, I looked up the patron in our print “facebook,” which contained a picture of each student, faculty, and staff member as well as students’ prior degrees. It dawned on me that the “facebook” was a data source and that mining it would be an interesting content analysis, or coding, project. For the “facebook” project, my chief aim was efficiency, so I adopted a binary (0/1) code for social science degree (i.e., either the student had a social science degree or didn’t). However, I quickly saw that colleges award a dizzying array of degrees, many of which are interdisciplinary. Further, I realized that I had never given much thought to what divides the humanities from the social sciences and the social sciences from the natural sciences. So, I reviewed the websites of prominent universities and kept a tally of what they counted as social sciences (Ryan, 2013). I then used a spreadsheet as my coding instrument and followed a simple rule: “students with degrees in six key social science fields would receive a one rather than a zero in the degree columns” (Ryan, 2013). The project yielded a useful statistic: more than 70% of Yale law students held a social science degree (Ryan, 2013). The empirical research was a fun way get to know my patrons, organize my work, and earn a first publication in law librarianship.

Legal Realism: The First Attempt to Merge Legal Research And Social Sciences

As the social sciences gained prominence in academia, legal scholarship began to move beyond formalism. While formalists championed the scientific application of context-neutral law, a new group of thinkers challenged the deductive, apolitical view of legal practice and justice. The legal realists of the early twentieth century adopted a contextual vision of the law and legal training (Hackney, 2016). In an effort to predict how judges and bureaucrats would enact law, for instance, realists studied their decision-making (Schlegal, 1980). Realists grounded this work in logic, philosophy, psychoanalysis, and scientific management theory (Cohen, 1923), and engaged with theorists outside of the law school. Underhill Moore, a realist concerned with how bankers assess risk (Moore & Hope, 1929), was encouraged to pursue his law-in-action research by leading social scientists of the day, including John Dewey (Schlegal, 1980). By the 1930s, dozens of scholars were conducting realist studies (Llewellyn, 1930; Hull, 1987). John Hopkins University launched an Institute of Law to foster realist research (John Hopkins University, 1929). But before long, the realists faced challenges from across their universities.

The realist movement coalesced around the New Deal and boasted prominent legal scholars (Suchman & Mertz, 2010). But by the mid-twentieth century, the movement had all but collapsed. Social scientists criticized legal realist work as crude and devoid of guiding theory or disciplinary tradition (Schlegal). Many traditional legal scholars dismissed the realists wholesale without appreciating their aims or methods (Schlegal). From all sides, the critiques understated differences within the realist movement—e.g., scholars focused more on reform rather than social science and vice versa—and between realism and formalism (Schlegal). Most importantly, the realists had neither an established place in the university nor in the legal profession (Schlegal). Without institutional support, the realists were unable to sustain their endeavor (Schlegal; Suchman & Mertz).

But even if the realists failed in their objectives, their approach to the law was a break with tradition that had lasting impacts on legal education, practice, and research. Realists like Underhill Moore, Charles Clark, William Douglas, Karl Llewellyn, and Louis Brandeis,

secured a place for social science in the law (Suchman & Mertz). As a result, realism paved the way for Empirical Legal Studies, the Law and Society movement, and other late 20th century legal subfields.

Contemporary Empirical Legal Studies

Today, the history of legal realism is itself a cottage industry. Some monographs focus on Roscoe Pound, others on Underhill Moore, and still others on Harold Lasswell and legal realists at Yale Law School. As professor Jack M. Balkin explains, “the choice of who is enshrined in the [realism] canon turns very much on the points one wants to make about the history of American law and legal education” (1998, p. 200). Similarly, the story of post-World War II empirical legal studies is contested and political.

Following World War II, law and social science projects gained steam again, most notably at the University of Chicago and Yale Law School. At Chicago, economist Ronald Coase led groundbreaking econometric studies of property law and transaction costs, and fostered a law and economics movement that survives to this day (Coase, 1993; Landes & Posner, 1993; Ho & Rubin, 2011). At Yale Law School, Underhill Moore was soon joined by Eugene Rostow, and then Boris Bittker and Guido Calebresi. Guido Calabresi produced empirical-theoretical treatises on strict liability that challenged formalism and neoclassical economics (Hackney, 2014). Calabresi’s progressive policy focus still informs descriptive and normative empirical scholarship. Elsewhere, social scientists such as Fred Kort used empirical methods to study judicial behavior and other legal topics (see Kort, 1957); their work is now seen as foundational to the field even if it was underappreciated by law professors at the time (Hall & Wright, 2008). The topical and methodological heterogeneity evidenced by Coase, Calebresi, and Kort’s work leads some scholars to view modern law and social science research as a continuation of legal realism (i.e., New Legal Realism; Suchman & Mertz, 2010). Fortunately, the broad field encompassing empirical legal studies, new legal realism, and law and social science has become more rigorous than legal realism ever was.

Until the late twentieth century, empirical legal research was an eclectic and undisciplined field. Content analysis was an early empirical research method, for instance, but most legal empiricists developed their analytic practices organically and without reference to methodological advances in fields such as Linguistics or Political Science (Hall & Wright, 2008; Suchman & Mertz, 2010). For instance, Richard A. Posner “read every published accident opinion of an American appellate court (state or federal, final or intermediate) issued in the first quarter of 1875, 1885, 1895, and 1905 . . . .” for his seminal article, A Theory of Negligence , but did not develop a codebook or utilize a content analysis instrument or employ a neutral coder to systematize and validate his findings (Posner, 1972; Hall & Wright).

Noting the need for methodological advancement, research training, and routine scholarly communication among empiricists, Theodore (Ted) Eisenberg of Cornell Law School and others founded the Society for Empirical Legal Studies (SELS), Journal of Empirical Legal Studies (JELS), and Conference on Empirical Legal Studies (CELS). Today, empirical legal researchers can present their early work at conferences featuring artificial intelligence and law, psychology and law, text analysis and law, and more. This organizing work was one of the factors in the rapid growth of empirical legal studies from the 1990s to the 2010s (Heise, 2011). So to was the development of specialized law journals. Today, law and social science scholars can publish their findings in a range of empirical research journals, including: Annual Law and Economics Review , Annual Review of Law and Social Science , Jurimetrics , Law and Human Behavior , Law & Society Review , and more. As a result, ELS offers many opportunities and resists simple classification. The field of several histories continues to branch out in many directions.

Reflection Questions

• Prior to reading this chapter, what was your impression of how U.S. legal instruction developed? Has that changed? If so, how? If not, which facts or ideas in this chapter reinforced your existing beliefs?
• Do you agree or disagree with the chapter’s premise that Economics has long dominated the social sciences in the U.S.? Explain. If you were educated in a non-U.S. country, which perspectives seem to dominate the social sciences in that country(ies)?
• Pragmatism played a key role in the development of the U.S. social sciences. How might pragmatism affect legal researchers’ choice of research subjects or methods today?
• How might you display historical information in this chapter visually (e.g., in presentation slides)?

Balkin, J. (1998). John Henry Schlegel, American Legal Realism and Empirical Social Science, Chapel Hill: The University of North Carolina Press, 1995 [book review].  Law and History Review ,  16 (1), 199-201.

Ben-David, J. (1992). Centers of learning: Britain, France, German, United States . Routledge.

Bondurant, B. C. (1909). The status of the classics in the South. The Classical Journal, 5 (2), 59-67.

Buxton, M. (1984). The influence of William James on John Dewey’s early work. Journal of the History of Ideas, 45 (3), 451-463.

Coase, R. H. (1993). Law and economics at Chicago. Journal of Law and Economics, 36 (1), 239-254.

Cohen, M. R. (1923). On the logic of fiction.  The Journal of Philosophy ,  20 (18), 477-488.

Eiseman, J., Bagnall, W., Kellett, C. & Lam, C. (2016). Litchfield unbound: Unlocking legal history with metadata, digitization, and digital tools.  Law and History Review ,  34 (4), 831-855.

Fernandez, A. (2012). Tapping Reeve, coverture and America’s first legal treatise. In A.Fernandez & M. D. Dubber (Eds.), Law books in action: Essays on the Anglo-American legal treatise (pp. 63-81). Hart Publishing.

Forgeus, E. (1939). Letters concerning some Litchfield Law School notebooks. Law Library Journal, 32 (5), 201-05.

Forgeus, E. (1942). An early Connecticut law school: Sylvester Gilbert’s school at Hebron. Law Library Journal, 35 (4), 200-03.

Forgeus, E. (1946). Sylvester Gilbert’s law school at Hebron, Connecticut: The students. Law Library Journal, 39 (2), 49-52.

Geiger, R. L. (2000). The era of the multipurpose colleges in American higher education. In R. Geiger (Ed.), The American college in the nineteenth century (pp. 127-152). Vanderbilt University Press.

Hackney, J.R., Jr. (2014). Guido Calabresi and the construction of contemporary American legal theory. Law and Contemporary Problems, 77 (1), 45-64.

Hall, M. A., & Wright, R. F. (2008). Systematic content analysis of judicial opinions.  California Law Review, 96 (1), 63-122.

Haskell, J. L. (1977). The emergence of professional social science: The American Social Science Association and the nineteenth-century crisis of authority . University of Illinois Press.

Heise, M. (2011). An empirical analysis of empirical legal scholarship production, 1990-2009. University of Illinois Law Review, 2011 (5), 1739-1752.

Ho, D. E., & Rubin, D. B. (2011). Credible causal inference for empirical legal studies.  Annual Review of Law and Social Science ,  7 , 17-40.

Hull, N. E. (1987). Some realism about the Llewellyn-Pound exchange over realism: The newly uncovered private correspondence, 1927-1931. Wisconsin Law Review, 1987 (6), 921-969.

John Hopkins University. (1929). The story of the Institute of Law at the John Hopkins University .  Norman T.A. Munder & Co.

Lochner v. New York , 198 U.S. 45 (1905).

Klafter, C. E. (1993). The influence of vocational law schools on the origins of American legal thought, 1779-1829. The American Journal of Legal History, 37 (3), 307-31.

Kort, F. (1957). Predicting Supreme Court decisions mathematically: A quantitative analysis of the “right to counsel” cases.  American Political Science Review ,  51 (1), 1-12.

Landes, W. M., & Posner, R. A. (1993). The influence of economics on law: A quantitative study.  The journal of law and economics ,  36 (1, Part 2), 385-424.

Langdell, C. C. (1871). A selection of cases on the law of contracts: With references and citations . Harvard Law School.

Llewellyn, K. N. (1930). A realistic jurisprudence –The next step.  Columbia Law Review, 30 (4), 431-465.

Miguel-Stearns, T., & Ryan, S. E. (2014). The empirical research law librarian. Part 1: Making the case and filling the role. Trends in Law Library Management and Technology, 24 , 1-6.

Misak, C. (2013). Rorty, pragmatism, and analytic philosophy. Humanities, 2 (3), 369-83.

Moore, U., & Hope, T. (1929). An institutional approach to the law of commercial banking. Yale Law Journal, 38 (6), 703-19.

Morrill Act of 1862. (2012). Codified at 7 U.S.C. § 301 et seq.

Posner, R. A. (1972). A theory of negligence.  The Journal of Legal Studies ,  1 (1), 29-96.

Ross, D. (1993). An historian’s view of American social science. Journal of the History of the Behavioral Sciences, 29 (2), 99-112.

Ryan, S. E. (2013, June). Digging in to patron data: What I learned from the law school “facebook.” AALL Spectrum , 20-21.

Ryan, S. E., & Miguel-Stearns, T. (2014). The empirical research law librarian Part 2: Developing the role. Trends in Law Library Management and Technology, 24 , 7-12.

Schlegel, J. H. (1980). American legal realism and empirical social science: The singular case of Underhill Moore. Buffalo Law Review, 29 (2),  195-323.

Siegfried, J. J. (2008). History of the meetings of the Allied Social Science Associations since World War II. The American Journal of Economics and Sociology, 67 (5), 973-84.

Stanford Encyclopedia of Philosophy. Pragmatism . https://plato.stanford.edu/entries/pragmatism/

Suchman, M.C. & Mertz, E. (2010). Toward a new legal empiricism: Empirical legal studies and new legal realism. Annual Review of Law and Social Science, 6 , 555-79.

Thayer, H. S. (Ed.). (1982). Pragmatism, the classic writings: Charles Sanders Peirce, William James, Clarence Irving Lewis, John Dewey, George Herbert Mead . Hackett Publishing.

Thomas, G. (2015). The founders and the idea of a national university: Constituting the American mind . Cambridge University Press.

Treasurer of Yale University. (1900). Report of the treasurer of Yale University . Yale University.

Veysey, L. R. (1965). The emergence of the American university . University of Chicago Press.

Empirical Legal Research Services by University of North Texas is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License , except where otherwise noted.

Share This Book

Volume 13, Issue 3, 2017

Special Issue Methodology of Legal Research: Challenges and Opportunities

Introduction

.jnl-nr0sjg{display:inline-block;color:inherit;}.jnl-nr0sjg:hover{text-decoration:none;} Editorial Methodology of Legal Research: Challenges and Opportunities

13 Dec 2017

Drafting (and Redrafting) Comparative Property Questionnaires

• Download links

22 Jan 2018

On Why Procedural Justice Matters in Court Hearings: Experimental Evidence that Behavioral Disinhibition Weakens the Association between Procedural Justice and Evaluations of Judges

12 Dec 2017

Methodology in Legal Research

The Study of Law as an Academic Discipline

The Victim’s Right to Intervene as an Injured Party in Criminal Proceedings: A Multidimensional and Interdisciplinary Assessment of Current Dutch Legal Practice

Odd Topics, Old Methods and the Cradle of the Ius Commune: Byzantine Law and the Italian City-States

Analysing Discursive Practices in Legal Research: How a Single Remark Implies a Paradigm

Building a Legal Citation Network: The Influence of the Court of Cassation on the Lower Judiciary

Evaluating the Quality of Dutch Academic Legal Publications: Results from a Survey

American Legal Realism: Research Programme and Policy Impact

Making the Case for Case Studies in Empirical Legal Research

Empirical Legal Studies

Learning about research methods, workshops for legal scholars, consultations.

• Selected Books & Articles on Research Methods
• Institutional Review
• Empirical Studies - Topics
• Organizations

In addition to reading books about methods, don't overlook the benefit of reading published studies. Not only do you get to read interesting findings, you will also learn about the process of research and you'll see examples of how to present data. Papers will generally have sections describing methods (e.g., if authors used a survey, how did they select a sample? how did they keep track of participants? how many responses did they get? how did they code answers?). They might also have an appendix with more details.

Graphic: Newton's Revolving Disc for Re-Composing Colours into White Light, from The Half Hour Library of Travel, Nature and Science for Young Readers (1896), available in British Library's photostream on Flickr

• Social Sciences Resources: Research Methods [UW Libraries] Links to resources on research design, quantitative methods, and qualitative methods.
• Sage Research Methods Online A research methods tool created to help researchers, faculty and students with their research projects. SAGE Research Methods links over 175,000 pages of SAGE’s renowned book, journal and reference content with truly advanced search and discovery tools. Researchers can explore methods concepts to help them design research projects, understand particular methods or identify a new method, conduct their research, and write up their findings. Since SAGE Research Methods focuses on methodology rather than disciplines, it can be used across the social sciences, health sciences, and more.

Experienced empirical researchers offer workshops aimed at legal scholars. For example, Northwestern University's Pritzker School of Law is offering a Workshop on Research Design for Causal Inference in summer 2024.

Sources for further workshop announcements include:

• Empirical Legal Studies blog Edited by several professors, covers new research, the " ELS blog serves as an online forum to discuss and provide links for emerging empirical legal scholarship, provide conference updates, discuss empirical claims that have emerged in public and political discourse, facilitate discussion for guest empirical scholars and assess current empirical findings and methodologies."

Graduate students (and sometimes undergrads) in the social sciences are required to take classes in the research methods of their fields. A partial list at the UW includes;

• ANTH 403 Qualitative Research Methods in Sociocultural Anthropology
• ANTH Advanced Qualitative Methods in Anthropology and Public Health

Communications

• COM 382 Social Scientific Approaches to Communication Research
• COM 383 Qualitative Communication Research Methods
• COM 501 Methods of Inquiry
• COM 513 Fieldwork Research Methods
• COM 517 Survey Research

Political Science

• POL S 500 Political Research Design and Analysis
• POL S 501 Advanced Political Research Design and Analysis
• POL S 502 Qualitative Research Methods
• POL S 503 Advanced Quantitative Political Methodology
• POL S 504 Multi-Method Field Research
• PSYCH 481 Seminar in Advanced Quantitative Methods
• PSYCH 524 Introduction to Statistics and Data Analysis
• PSYCH 531 Practical Issues in Data Analysis and Presentation
• SOC 220 Introduction to Sociological Methods
• SOC 221 Statistical Concepts and Methods for the Social Sciences
• SOC 506 Methodology: Quantitative Techniques in Sociology
• SOC 574 Seminar in Methods of Criminological Research

Foster School of Business

• BA RM 590 Behavioral Research Methods - Theory and Design
• BA RM 591 Behavioral Research Methods - Approaches and Applications

Evans School of Public Policy and Governance

• PUBPOL 525 Qualitative Field Methods and Analysis
• PUBPOL 527 Quantitative Analysis I
• PUBPOL 528 Quantitative Analysis II
• PUBPOL 529 Advanced Quantitative Methods for Policy Analysis

School of Public Health

• SPH 480 Research Methods in Public Health

School of Social Work

• SOC W 505 Foundations of Social Welfare Research
• SOC W 506 Social Welfare Research and Evaluation

Graphic: Poster by Sam Braun, Federal Art Project, 1937. "The World Wants New Knowledge, New Skills--Enroll: Federal Adult Schools." Library of Congress Prints and Photographs Division .

• Center for Statistics and the Social Sciences (CSSS) CSSS provides free statistical consulting to current UW faculty, staff, and students in affiliated departments.
• << Previous: Research Methods
• Next: Selected Books & Articles on Research Methods >>
• Last Updated: Apr 1, 2024 11:09 AM
• URL: https://lib.law.uw.edu/els

2024 Theses Doctoral

Charter-School Music Teacher Practitioners and Instructional Leaders’ Perception of Professional Development: A Multiple-Bounded Case Study

Moss, Jameon DeSean

This multiple-bounded case study explored charter-school music teacher practitioners’(MTPs’) and instructional leaders’ (ILs’) perceptions of professional development (PD) in four charter management organizations (CMOs). The purpose was to provide a rich description of these practitioners’ professional development, with the goal of spurring policy conversations and further research on music teachers and their experiences in the charter domain. Over two months in the fall of 2023, the researcher conducted one-on-one interviews with eight participants, which focused on ways of making change, methods of delivery, beneficial components of the methods of supporting music literacy, and forms of PD assessment from the perspectives of MTPs and ILs. In addition to holding two focus groups (one with each case), the researcher conducted four classroom and debrief observations. The interviews and observations were analyzed using the participants’ words as first-cycle analysis themes; these were then filtered through the study’s conceptual framework of Desimone’s (2009) core elements of effective professional development: content focus, active learning, coherence, sustained duration, and collective participation. The findings illustrate the participants’ experience with the professional development phenomenon through a series of main themes: instruction is classroom management, except PROFESSIONAL DEVELOPMENT IN CHARTER SCHOOLS when it is not, (b) the many moods of instructional coaching and workshops, (c) content expertise via cycles of inquiry, and (d) reflection is essential. Implications include framing future empirical research in this usually guarded sector as a partnership to identify best and emergent practices for practitioners that directly affect students and families. Framing research in this manner may resonate with charter management organizations that adhere to more formative professional development practices. Additionally, cycles of inquiry in which self-reflection can occur may be a way forward for myriad non-content-expert instructional leaders who support the professional development of music teacher practitioners in charter schools or traditional public schools. Further suggestions for future practice include hosting charter-specific sessions at music education conferences, which could be framed as dialogic sessions to foster collegial inquiry concerning practices at both charter and public schools. Because CMOs’ system structures are different, practitioners there experience some aspects of teaching and professional development differently than their traditional public counterparts. Offering sessions specifically tailored to charter practitioners’ needs could help ensure that their needs, as well as those of the ILs that support them, are met. Keywords: Professional Development, Charter Schools, In-Service Music Teacher Practitioners, Instructional Leaders, Instructional Coaching, Mentors, Workshops.

• Music--Instruction and study
• Music teachers--Training of
• Music teachers--Attitudes
• Charter schools
• Mentoring in education
• Professional development for teachers

More About This Work

• DOI Copy DOI to clipboard