qualitative approaches to empirical legal research

Even more »

Account Options

qualitative approaches to empirical legal research

Try the new Google Books
Advanced Book Search

Get this book in print

Oxford University Press
Barnes&Noble.com
Books-A-Million
Find in a library
All sellers »

Selected pages

Other editions - View all

Common terms and phrases, about the author (2012), bibliographic information.

Book chapter

Qualitative approaches to empirical legal research

Webley, L. 2010. Qualitative approaches to empirical legal research. in: Cane, P. and Kritzer, H. (ed.) Oxford handbook of empirical legal research Oxford Oxford University Press. pp. 926-950

Related outputs

Complete Public Law: Text, Cases, and Materials, 5th edition Webley, L. and Samuels, H. 2021. Complete Public Law: Text, Cases, and Materials, 5th edition. Oxford University Press (OUP).

Gender and the legal academy in the UK: a product of proxies and hiring and promotion practices Duff, L. and Webley, L. 2021. Gender and the legal academy in the UK: a product of proxies and hiring and promotion practices. in: Auchmuty, R.S., Ulrike Schultz, U., Shaw, G. and Thornton, M. (ed.) Gender and Careers in the Legal Academy UK Hart Publishing.

Women in Law Literature Review Duff, L., Mason, M. and Webley, L. 2019. Women in Law Literature Review. London Law Society.

Complete Public Law: Text, Cases, and Materials, 4th edition Webley, L. and Samuels, H. 2018. Complete Public Law: Text, Cases, and Materials, 4th edition. Oxford University Press (OUP).

When is mediation mediatory and when is it really adjudicatory? Religion, Norms and decision-making Webley, L. 2017. When is mediation mediatory and when is it really adjudicatory? Religion, Norms and decision-making. in: Bano, S. (ed.) Gender and Justice in Family Law Disputes: Women, Mediation, and Religious Arbitration University Press of New England.

Stumbling Blocks in Empirical Legal Research: Case Study Research Webley, L. 2016. Stumbling Blocks in Empirical Legal Research: Case Study Research. Law and Method. https://doi.org/10.5553/REM/.000020

Interception of communications and legal professional privilege and the rule of law Webley, L. 2016. Interception of communications and legal professional privilege and the rule of law. Legal Ethics. 19 (1), pp. 173-176. https://doi.org/10.1080/1460728x.2016.1193954

Diversity and Inclusion As the Key to Innovating Talent Management in Law Firms? Webley, L. and Duff, L. 2016. Diversity and Inclusion As the Key to Innovating Talent Management in Law Firms? in: Mottershead, T. (ed.) Innovating Talent Management in Law Firms National Association for Law Placement. pp. 235-254

Legal Writing (4th edition) Webley, L. 2016. Legal Writing (4th edition). Abingdon Routledge.

Access to a Career in the Legal Profession in England and Wales: Race, Class, and the Role of Educational Background Webley, L., Tomlinson, J., Muzio, D., Sommerlad, H. and Duff, L. 2016. Access to a Career in the Legal Profession in England and Wales: Race, Class, and the Role of Educational Background. in: Headworth, S., Nelson, R.L., Dinovitzer, R. and Wilkins, D.B. (ed.) Diversity In Practice: Race, Gender, and Class in Legal and Professional Careers Cambridge Cambridge University Press. pp. 198-225

European Citizenship, Freedom, Security, Justice and Equality Webley, L. 2015. European Citizenship, Freedom, Security, Justice and Equality. in: European Citizenship Challenges and Opportunities Porto Portugal Universidade Catolica Editora Porto. pp. 143-162

Complete Public Law: Text, Cases and Materials, 3rd edition Webley, L. and Samuels, H. 2015. Complete Public Law: Text, Cases and Materials, 3rd edition. Oxford Oxford University Press.

When is a Family Lawyer, a Lawyer? Webley, L. 2015. When is a Family Lawyer, a Lawyer? in: Mavis Maclean, John Eekalaar and Bastard Benoit (ed.) Delivering Family Justice in the 21st Century Oxford, UK Hart. pp. 305-321

Legal Professional De(Re)Regulation, Equality, and Inclusion, and the Contested Space of Professionalism Within the Legal Market in England and Wales Webley, L. 2015. Legal Professional De(Re)Regulation, Equality, and Inclusion, and the Contested Space of Professionalism Within the Legal Market in England and Wales. Fordham Law Review. 83 (5), pp. 2349-2367.

The Former Legal Director of the London Times, Legal Professional Privilege and the Duty Not to Mislead the Court in England and Wales Webley, L. 2014. The Former Legal Director of the London Times, Legal Professional Privilege and the Duty Not to Mislead the Court in England and Wales. Legal Ethics. 17 (2), pp. 310-312.

Legal Writing (3rd Edition) Webley, L. 2013. Legal Writing (3rd Edition). London, UK Routledge.

United Kingdom: What Robinson v Solicitors Regulation Authority Tells Us about the Contested Terrain of Race and Disciplinary Processes Webley, L. 2013. United Kingdom: What Robinson v Solicitors Regulation Authority Tells Us about the Contested Terrain of Race and Disciplinary Processes. Legal Ethics. 16 (1), pp. 236-241. https://doi.org/10.5235/1460728X.1.1.236

Structure, agency and the career strategies of women and BME individuals in the legal profession Tomlinson, J., Muzio, D., Sommerlad, H., Webley, L. and Duff, L. 2013. Structure, agency and the career strategies of women and BME individuals in the legal profession. Human Relations. 66 (2), pp. 245-269. https://doi.org/10.1177/0018726712460556

Complete Public Law: Text, Cases and Materials (2nd Edition) Webley, L. and Samuels, H. 2012. Complete Public Law: Text, Cases and Materials (2nd Edition). Oxford, UK Oxford University Press.

The Legal Education Training Review: Correspondent’s Report from the United Kingdom Webley, L. 2012. The Legal Education Training Review: Correspondent’s Report from the United Kingdom. Legal Ethics. 15 (1), pp. 140-141. https://doi.org/10.5235/146072812801292773

Legal Ethics and the Legal Education Training Review: Correspondent’s Report from the United Kingdom Webley, L. 2012. Legal Ethics and the Legal Education Training Review: Correspondent’s Report from the United Kingdom. Legal Ethics. 15 (2), pp. 402-404.

Gender, Hierarchy, Power and Inequality: What Sociological Theory Adds to our Understanding of Sex-Discrimination Webley, L. 2012. Gender, Hierarchy, Power and Inequality: What Sociological Theory Adds to our Understanding of Sex-Discrimination. Westminster Law Review . 1 (1), pp. 37-42.

Legal ethics in the academic curriculum: correspondent’s report from the United Kingdom Webley, L. 2011. Legal ethics in the academic curriculum: correspondent’s report from the United Kingdom. Legal Ethics. 14 (1), pp. 132-134.

Law Students as Peer Mentors: Developing the Reflective Practitioner and/or Embedding Legal Learning: Correspondent’s Report from the United Kingdom Webley, L. 2011. Law Students as Peer Mentors: Developing the Reflective Practitioner and/or Embedding Legal Learning: Correspondent’s Report from the United Kingdom. Legal Ethics. 14 (2), pp. 250-251.

Solicitors as imagined masculine, family mediators as fictive feminine and the hybridization of divorce solicitors Webley, L. 2011. Solicitors as imagined masculine, family mediators as fictive feminine and the hybridization of divorce solicitors. in: Bartlett, F., Mortenson, R. and Tranter, K. (ed.) Alternative perspectives on lawyers and legal ethics Abingdon Routledge. pp. 132-150

Gate-keeper, supervisor or mentor? The role of professional bodies in the regulation and professional development of solicitors and family mediators undertaking divorce matters in England and Wales Webley, L. 2010. Gate-keeper, supervisor or mentor? The role of professional bodies in the regulation and professional development of solicitors and family mediators undertaking divorce matters in England and Wales. Journal of Social Welfare and Family Law. 32 (2), pp. 119-133. https://doi.org/10.1080/09649069.2010.506308

Diversity in the legal profession: a business or an ethical rationale?: correspondent's report from the United Kingdom Webley, L. 2010. Diversity in the legal profession: a business or an ethical rationale?: correspondent's report from the United Kingdom. Legal Ethics. 13 (2), pp. 223-224.

Adversarialism and consensus? The professions’ construction of solicitor and family mediator identity and role Webley, L. 2010. Adversarialism and consensus? The professions’ construction of solicitor and family mediator identity and role. New Orleans Quid Pro.

Diversity in the legal profession in England and Wales: A qualitative study of barriers and individual choices Sommerlad, H., Webley, L., Duff, L., Muzio, D., Tomlinson, J. and Parnham, R. 2010. Diversity in the legal profession in England and Wales: A qualitative study of barriers and individual choices. London Legal Services Board.

Complete Public Law: Test Cases and Materials Webley, L. and Samuels, H. 2009. Complete Public Law: Test Cases and Materials. Oxford, UK Oxford University Press.

Legal writing. 2nd edition Webley, L. 2009. Legal writing. 2nd edition. London, UK Cavendish Publishing.

The next generation of legal aid solicitors: the LSC’s Training Grants Scheme Bacquet, S., Boon, A., Webley, L. and Whyte, A. 2009. The next generation of legal aid solicitors: the LSC’s Training Grants Scheme. in: Buck, A. (ed.) Reaching further: innovation, access and quality in legal services : papers from the Legal Services Research Centre's seventh international research conference, Reaching further, Royal Naval Academy in Greenwich, London, 18th to 20th June 2008 London Stationary Office. pp. 143-165

Evaluation of the Legal Service Commission's Training Contract Grant Scheme Bacquet, S., Boon, A., Webley, L. and Whyte, A. 2008. Evaluation of the Legal Service Commission's Training Contract Grant Scheme. Legal Services Commission.

The European criminal record in England and Wales Webley, L. 2008. The European criminal record in England and Wales. in: Stefanou, C. and Xanthaki, H. (ed.) Towards a European criminal record Cambridge Cambridge University Press. pp. 281-307

Paralegals Webley, L. 2008. Paralegals. in: Cane, P. and Conaghan, J. (ed.) The new Oxford companion to law Oxford Oxford University Press. pp. 862-863

Evaluation of the Legal Service Commission's Training Contract Grant Scheme Webley, L. 2008. Evaluation of the Legal Service Commission's Training Contract Grant Scheme. Society of Legal Scholars Annual Conference 2008: The Impact of Legal Scholarship. London School of Economics 15 - 18 Sep 2008

Encouraging the next generation of legal aid solicitors: the LSC’s Training Grants Scheme Webley, L. 2008. Encouraging the next generation of legal aid solicitors: the LSC’s Training Grants Scheme. Seventh Legal Services Research Centre (LSRC) International Research Conference, "Reaching Further: New Approaches to the Delivery of Legal Services". Naval College, Greenwich, London 18 - 20 Jun 2008

Costs of complaining and claiming Webley, L. 2008. Costs of complaining and claiming. in: Cane, P. and Conaghan, J. (ed.) The new Oxford companion to law Oxford Oxford University Press. pp. 243-245

Court-based mediation in England and Wales: are judges evading their responsibilities? Webley, L. 2007. Court-based mediation in England and Wales: are judges evading their responsibilities? Cleveland-Marshall College of Law, Cleveland State University Ohio, Seminar Series. Cleveland State University, Ohio 2007

Women solicitors as a barometer for problems within the legal profession: time to put values before profits? Webley, L. and Duff, L. 2007. Women solicitors as a barometer for problems within the legal profession: time to put values before profits? Journal of Law and Society. 34 (3), pp. 374-402. https://doi.org/10.1111/j.1467-6478.2007.00397.x

A review of how other countries provide information and advice to the vulnerable on consumer and social issues Webley, L. and Bacquet, S. 2006. A review of how other countries provide information and advice to the vulnerable on consumer and social issues. London, UK DTI. https://doi.org/URN06/1871

Mediation through the Birmingham court-based scheme Webley, L., Abrams, P. and Bacquet, S. 2006. Mediation through the Birmingham court-based scheme. Journal of Housing Law. 9 (1), pp. 9-12.

Evaluation of the Birmingham court-based Civil (non-family) mediation scheme Webley, L., Abrams, P. and Bacquet, S. 2006. Evaluation of the Birmingham court-based Civil (non-family) mediation scheme. London, UK Department for Constitutional Affairs.

Divorce solicitors and ethical approaches: the best interests of the client and/or the best interests of the family? Webley, L. 2005. Divorce solicitors and ethical approaches: the best interests of the client and/or the best interests of the family? Legal Ethics. 7 (2), pp. 231-250.

Criminal records and organised crime in England and Wales Webley, L. 2005. Criminal records and organised crime in England and Wales. in: Stefanou, C. and Xanthaki, H. (ed.) Financial crime in the EU: criminal records as effective tools or missed opportunities? The Hague, Netherlands Kluwer Law International. pp. 133-148

The loneliness of the pro bono coordinator Webley, L. 2004. The loneliness of the pro bono coordinator. Law Institute Journal. 78 (12), p. 92.

Legal aid and legal need Webley, L. 2004. Legal aid and legal need. Law Institute Journal. 78 (11), p. 91.

Access key in the UK Webley, L. 2004. Access key in the UK. Law Institute Journal. 78 (9), p. 83.

Equality and diversity: women solicitors research study 48 Duff, L. and Webley, L. 2004. Equality and diversity: women solicitors research study 48. London, UK Law Society.

Pro bono and young solicitors: views from the front line Webley, L. 2001. Pro bono and young solicitors: views from the front line. Legal Ethics. 3 (2), pp. 152-168.

Quality and cost: final report on the contracting of Civil, Non-Family Advice and Assistance Pilot Moorhead, R., Sherr, A., Webley, L., Rogers, S., Sherr, L., Patterson, A. and Domberger, S. 2001. Quality and cost: final report on the contracting of Civil, Non-Family Advice and Assistance Pilot. London, UK The Stationary Office.

Permalink - https://westminsterresearch.westminster.ac.uk/item/90872/qualitative-approaches-to-empirical-legal-research

Usage statistics

Help & FAQ

Qualitative approaches to empirical legal research

Birmingham Law School

Research output : Chapter in Book/Report/Conference proceeding › Chapter

Publication series

T1 - Qualitative approaches to empirical legal research

AU - Webley, Lisa

PY - 2010/11/11

Y1 - 2010/11/11

UR - https://global.oup.com/academic/product/the-oxford-handbook-of-empirical-legal-research-9780199542475?cc=gb&lang=en&

M3 - Chapter

SN - 9780199542475

SN - 9780199659944

T3 - Oxford Handbooks

BT - The Oxford Handbook of Empirical Legal Research

A2 - Cane, Peter

A2 - Kritzer, Herbert

PB - Oxford University Press

We will keep fighting for all libraries - stand with us!

Internet Archive Audio

This Just In
Grateful Dead
Old Time Radio
78 RPMs and Cylinder Recordings
Audio Books & Poetry
Computers, Technology and Science
Music, Arts & Culture
News & Public Affairs
Spirituality & Religion
Radio News Archive

Flickr Commons
Occupy Wall Street Flickr
NASA Images
Solar System Collection
Ames Research Center

All Software
Old School Emulation
MS-DOS Games
Historical Software
Classic PC Games
Software Library
Kodi Archive and Support File
Vintage Software
CD-ROM Software
CD-ROM Software Library
Software Sites
Tucows Software Library
Shareware CD-ROMs
Software Capsules Compilation
CD-ROM Images
ZX Spectrum
DOOM Level CD

Smithsonian Libraries
FEDLINK (US)
Lincoln Collection
American Libraries
Canadian Libraries
Universal Library
Project Gutenberg
Children's Library
Biodiversity Heritage Library
Books by Language
Additional Collections

Prelinger Archives
Democracy Now!
Occupy Wall Street
TV NSA Clip Library
Animation & Cartoons
Arts & Music
Computers & Technology
Cultural & Academic Films
Ephemeral Films
Sports Videos
Videogame Videos
Youth Media

Search the history of over 866 billion web pages on the Internet.

Mobile Apps

Wayback Machine (iOS)
Wayback Machine (Android)

Browser Extensions

Archive-it subscription.

Explore the Collections
Build Collections

Save Page Now

Capture a web page as it appears now for use as a trusted citation in the future.

Please enter a valid web address

Donate Donate icon An illustration of a heart shape

The Oxford handbook of empirical legal research

Bookreader item preview, share or embed this item, flag this item for.

Graphic Violence
Explicit Sexual Content
Hate Speech
Misinformation/Disinformation
Marketing/Phishing/Advertising
Misleading/Inaccurate/Missing Metadata

Obscured text on back cover due to sticker attached.

plus-circle Add Review comment Reviews

54 Previews

4 Favorites

DOWNLOAD OPTIONS

No suitable files to display here.

PDF access not available for this item.

IN COLLECTIONS

Uploaded by station23.cebu on December 27, 2022

Harvard Empirical Legal Studies Series

5005 Wasserstein Hall (WCC) 1585 Massachusetts Avenue Cambridge, MA02138

Contact the Graduate Program

The Harvard Empirical Legal Studies (HELS) Series explores a range of empirical methods, both qualitative and quantitative, and their application in legal scholarship in different areas of the law. It is a platform for engaging with current empirical research, hearing from leading scholars working in a variety of fields, and developing ideas and empirical projects.

HELS is open to all students and scholars with an interest in empirical research. No prior background in empirical legal research is necessary. If you would like to join HELS and receive information about our sessions, please subscribe to our mailing list by completing the HELS mailing list form .

If you have any questions, do not hesitate to contact the current HELS coordinator, Tiran Bajgiran.

All times are provided in U.S. Eastern Time (UTC/GMT-0400).

Spring 2024 Sessions

Empire and the shaping of american constitutional law.

Aziz Rana, BC Law

Monday, Mar. 25, 12:15 PM Lewis 202

This talk will explore how US imperial practice has influenced the methods and boundaries of American constitutional study.

Historical Approaches to Neoliberal Legality

Quinn Slobodian, Boston University

Thursday, Mar. 28, 12:15 PM Lewis 202

Fall 2023 Sessions

On critical quantitative methods.

Hendrik Theine , WU, Vienna/Univ. of Pennsylvania Monday, Nov. 6, 12:30 PM Lewis 202

Economic inequality is a profound challenge in the United States. Both income and wealth inequality increased remarkably since the 1980s. This growing concentration of economic inequality creates real-world political and societal problems which are increasingly reflected by social science scholarship. Among those detriments is for instance the increasing economic and political power of the super-rich. The research at hand takes a new radical look at media discourses of economic inequality over four decades in various elite US newspapers by way of quantitative critical discourse analysis. It shows that up until recently, there was minimal media coverage of economic inequality, but interest has steadily increased since then. Initially, the focus was primarily on income inequality, but over time, it has expanded to encompass broader issues of inequality. Notably, the discourse on economic inequality is significantly influenced by party politics and elections. The study also highlights certain limitations in the discourse. Critiques of inequality tend to remain at a general level, discussing concepts like capitalist and racial inequality. There is relatively less focus on policy-related discussions, such as tax reform, or discussions centered around specific actors, like the wealthy and their charitable contributions.

Spring 2023 Sessions

How to conduct qualitative empirical legal scholarship.

Jessica Silbey , Professor of Law at Boston University Yanakakis Faculty Research Scholar

Friday, March 31, 12:30 PM WCC 3034

This session explores the benefits and some limitations of qualitative research methods to study intellectual property law. It compares quantitative research methods and the economic analysis of law in the same field as other kinds of empirical inquiry that are helpful in collaboration but limited in isolation. Creativity and innovation, the practices intellectual property law purports to regulate, are not amenable to quantification without identifying qualitative variables. The lessons from this session apply across fields of legal research.

Fall 2022 Sessions

How to read quantitative empirical legal scholarship.

Holger Spamann , Lawrence R. Grove Professor of Law

Friday, September 13, 12:30 PM WCC 3007

As legal scholars, what tools do we need to read critically and engage productively with quantitative empirical scholarship? In the first session of the 2022-2023 Harvard Empirical Legal Studies Series, Harvard Law School Professor Holger Spamann will compare and discuss different quantitative studies. This session will be a first approximation to be able to understand and eventually produce empirical legal scholarship. All students and scholars interested in empirical research are welcome and encouraged to attend.

How do People Learn from Not Being Caught? An Experimental Investigation of a “Non-Occurrence Bias”

Tom Zur , John M. Olin Fellow and SJD candidate, HLS

Friday, November 4, 2:00 PM WCC 3007

The law and economics literature on specific deterrence has long theorized that offenders rationally learn from being caught and sanctioned. This paper presents evidence from a randomized controlled trial showing that offenders learn differently when not being caught as compared to being caught, which we call a “non-occurrence bias.” This implies that the socially optimal level of investment in law enforcement should be lower than stipulated by rational choice theory, even on grounds of deterrence alone.

Empirical Legal Research: Using Data and Methodology to Craft a Research Agenda

Florencia Marotta-Wurgler , NYU Boxer Family Professor of Law Faculty Director, NYU Law in Buenos Aires

Monday, November 14, 12:30 PM Lewis 202

Using a series of examples, this discussion will focus on strategies to conduct empirical legal research and develop a robust research agenda. Topics will include creating a data set and leveraging to answer unexplored questions, developing meaningful methodologies to address legal questions, building on existing work to develop a robust research agenda, and engaging the process of automation and scaling up to develop large scale data sets using machine learning approaches.

Resources for Empirical Research

HLS Library Empirical Research Service
Harvard Institute for Quantitative Social Research (IQSS)
Harvard Committee on the Use of Human Subjects
Qualtrics Harvard
Harvard Kennedy School Behavioral Insights Group

Past HELS Sessions

Holger Spamann (Lawrence R. Grove Professor of Law) – How to Read Quantitative Empirical Legal Scholarship?

Katerina Linos (Professor of Law at UC Berkeley School of Law) – Qualitative Methods for Law Review Writing

Aziza Ahmed (Professor of Law at UC Irvine School of Law) – Risk and Rage: How Feminists Transformed the Law and Science of AIDS

Amy Kapczynski and Yochai Benkler –(Professor of Law at Yale; Professor of Law at Harvard) Law & Political Economy and the Question of Method

Jessica Silbey – (Boston University School of Law) Ethnography in Legal Scholarship

Roberto Tallarita – (Lecturer on Law, and Associate Director of the Program on Corporate Governance at Harvard) The Limits of Portfolio Primacy

Susan S. Silbey – (Leon and Anne Goldberg Professor of Humanities, Sociology and Anthropology at MIT) HELS with Susan Silbey: Analyzing Ethnographic Data and Producting New Theory

Cass R. Sunstein (University Professor at Harvard) – Optimal Sludge? The Price of Program Integrity

Scott L. Cummings (Professor of Legal Ethics and Professor of Law at UCLA School of Law) – The Making of Public Interest Lawyers

Elliot Ash (Assistant Professor of Law, Economics, and Data Science at ETH Zürich) – Gender Attitudes in the Judiciary: Evidence from U.S. Circuit Courts

Kathleen Thelen (Ford Professor of Political Science at MIT) – Employer Organization in the United States: Historical Legacies and the Long Shadow of the American Courts

Omer Kimhi (Associate Professor at Haifa University Law School) – Caught In a Circle of Debt – Consumer Bankruptcy Discharge and Its Aftereffects

Suresh Naidu (Professor in Economics and International and Public Affairs, Columbia School of International and Public Affairs) – Ideas Have Consequences: The Impact of Law and Economics on American Justice

Vardit Ravitsky (Full Professor at the Bioethics Program, School of Public Health, University of Montreal) – Empirical Bioethics: The Example of Research on Prenatal Testing

Johnnie Lotesta (Postdoctoral Democracy Fellow at the Ash Center for Democratic Governance and Innovation at the Harvard Kennedy School) – Opinion Crafting and the Making of U.S. Labor Law in the States

David Hagmann (Harvard Kennedy School) – The Agent-Selection Dilemma in Distributive Bargaining

Cass R. Sunstein (Harvard Law School) – Rear Visibility and Some Problems for Economic Analysis (with Particular Reference to Experience Goods)

Talia Gillis (Ph.D. Candidate and S.J.D. Candidate, Harvard Business School and Graduate School of Arts and Sciences and Harvard Law School) – False Dreams of Algorithmic Fairness: The Case of Credit Pricing

Tzachi Raz (Ph.D. Candidate in Economics at Harvard University) – There’s No Such Thing as Free Land: The Homestead Act and Economic Development

Crystal Yang (Harvard Law School) – Fear and the Safety Net: Evidence from Secure Communities

Adaner Usmani (Harvard Sociology) – The Origins of Mass Incarceration

Jim Greiner (Harvard Law School) – Randomized Control Trials in the Legal Profession

Talia Shiff (Postdoctoral Fellow, Weatherhead Center for International Affairs and Department of Sociology, Harvard University) – Legal Standards and Moral Worth in Frontline Decision-Making: Evaluations of Victimization in US Asylum Determinations

Francesca Gino (Harvard Business School) – Rebel Talent

Joscha Legewie (Department of Sociology, Harvard University) – The Effects of Policing on Educational Outcomes and Health of Minority Youth

Ryan D. Enos (Department of Government, Harvard University) – The Space Between Us: Social Geography and Politics

Katerina Linos (Berkeley Law, University of California) – How Technology Transforms Refugee Law

Roie Hauser (Visiting Researcher at the Program on Corporate Governance, Harvard Law School) – Term Length and the Role of Independent Directors in Acquisitions

Anina Schwarzenbach (Fellow, National Security Program, the Belfer Center for Science and International Affairs, Harvard Kennedy School) – A Challenge to Legitimacy: Effects of Stop-and-Search Police Contacts on Young People’s Relations with the Police

Cass R. Sunstein (Harvard Law School) – Willingness to Pay to Use Facebook, Twitter, Youtube, Instagram, Snapchat, and More: A National Survey

Netta Barak-Corren (Hebrew University of Jerusalem) – The War Within

James Greiner & Holger Spamann (Harvard Law School) – Panel: Why Does the Legal Profession Resist Rigorous Empiricism?

Mila Versteeg (University of Virginia School of Law) (with Adam Chilton) – Do Constitutional Rights Make a Difference?

Susan S. Silbey (MIT Department of Anthropology) (with Patricia Ewick) – The Common Place of Law

Holger Spamann (Harvard Law School) – Empirical Legal Studies: What They Are and How NOT to Do Them

Arevik Avedian (Harvard Law School) – How to Read an Empirical Paper in Law

James Greiner (Harvard Law School) – Randomized Experiments in the Law

Robert MacCoun (Stanford Law School) – Coping with Rapidly Changing Standards and Practices in the Empirical Sciences (including ELS)

Mario Small (Harvard Department of Sociology) – Qualitative Research in the Big Data Era

Adam Chilton (University of Chicago Law School) – Trade Openness and Antitrust Law

Jennifer Lerner (Harvard Kennedy School and Department of Psychology) – Anger in Legal Decision Making

Sarah Dryden-Peterson (Harvard Graduate School of Education) – Respect, Reciprocity, and Relationships in Interview-Based Research

Charles Wang (Harvard Business School) – Natural Experiments and Court Rulings

Guhan Subramanian (Harvard Law School) – Determining Fair Value

James Greiner (Harvard Law School) – Randomized Control Trials and the Impact of Legal Aid

Maya Sen (Harvard Kennedy School) – The Political Ideologies of Law Clerks and their Judges

Daria Roithmayr (University of Southern California Law School) – The Dynamics of Police Violence

Crystal Yang (Harvard Law School) – Empiricism in the Service of Criminal Law and Theory

Oren Bar-Gill (Harvard Law School) – Is Empirical Legal Studies Changing Law and Economics?

Elizabeth Linos (Harvard Kennedy School; VP, Head of Research and Evaluation, North America, Behavioral Insights Team) – Behavioral Law and Economics in Action: BIT, BIG, and the policymaking of choice architecture

Meira Levinson (Harvard School of Education) – Justice in Schools: Qualitative Sociological Research and Normative Ethics in Schools

Howell Jackson (HLS) – Cost-Benefit Analysis

Michael Heise (Cornell Law School) – Quantitative Research in Law: An Introductory Workshop

Susan Silbey (MIT) – Interviews: An Introductory Workshop

Kevin Quinn (UC Berkeley) – Quantifying Judicial Decisions

Holger Spamman (Harvard Law School) – Comparative Empirical Research

James Greiner (Harvard Law School) – Randomized Controlled Trials in the Research of Legal Problems

Michael Heise (Cornell Law School) – Quantitative Research in Law

James Greiner (Harvard Law School) – A Typology of Empirical Methods in Law

David Wilkins (Harvard Law School) – Mixed Methods Work and the Legal Profession

Tom Tyler (Yale Law School) – Fairness and Policing

Modal Gallery

Gallery block modal gallery.

Empirical Legal Studies

Learning about research methods, workshops for legal scholars, consultations.

Selected Books & Articles on Research Methods
Institutional Review
Empirical Studies - Topics
Organizations

black and white etching of hand turning crank of scientific apparatus

In addition to reading books about methods, don't overlook the benefit of reading published studies. Not only do you get to read interesting findings, you will also learn about the process of research and you'll see examples of how to present data. Papers will generally have sections describing methods (e.g., if authors used a survey, how did they select a sample? how did they keep track of participants? how many responses did they get? how did they code answers?). They might also have an appendix with more details.

Graphic: Newton's Revolving Disc for Re-Composing Colours into White Light, from The Half Hour Library of Travel, Nature and Science for Young Readers (1896), available in British Library's photostream on Flickr

Social Sciences Resources: Research Methods [UW Libraries] Links to resources on research design, quantitative methods, and qualitative methods.
Sage Research Methods Online A research methods tool created to help researchers, faculty and students with their research projects. SAGE Research Methods links over 175,000 pages of SAGE’s renowned book, journal and reference content with truly advanced search and discovery tools. Researchers can explore methods concepts to help them design research projects, understand particular methods or identify a new method, conduct their research, and write up their findings. Since SAGE Research Methods focuses on methodology rather than disciplines, it can be used across the social sciences, health sciences, and more.

Experienced empirical researchers offer workshops aimed at legal scholars. For example, Northwestern University's Pritzker School of Law is offering a Workshop on Research Design for Causal Inference in summer 2024.

Sources for further workshop announcements include:

Empirical Legal Studies blog Edited by several professors, covers new research, the " ELS blog serves as an online forum to discuss and provide links for emerging empirical legal scholarship, provide conference updates, discuss empirical claims that have emerged in public and political discourse, facilitate discussion for guest empirical scholars and assess current empirical findings and methodologies."

Graduate students (and sometimes undergrads) in the social sciences are required to take classes in the research methods of their fields. A partial list at the UW includes;

Poster announcing free adult education classes and encouraging adults to return to school, showing earth.

ANTH 403 Qualitative Research Methods in Sociocultural Anthropology
ANTH Advanced Qualitative Methods in Anthropology and Public Health

Communications

COM 382 Social Scientific Approaches to Communication Research
COM 383 Qualitative Communication Research Methods
COM 501 Methods of Inquiry
COM 513 Fieldwork Research Methods
COM 517 Survey Research

Political Science

POL S 500 Political Research Design and Analysis
POL S 501 Advanced Political Research Design and Analysis
POL S 502 Qualitative Research Methods
POL S 503 Advanced Quantitative Political Methodology
POL S 504 Multi-Method Field Research
PSYCH 481 Seminar in Advanced Quantitative Methods
PSYCH 524 Introduction to Statistics and Data Analysis
PSYCH 531 Practical Issues in Data Analysis and Presentation
SOC 220 Introduction to Sociological Methods
SOC 221 Statistical Concepts and Methods for the Social Sciences
SOC 506 Methodology: Quantitative Techniques in Sociology
SOC 574 Seminar in Methods of Criminological Research

Foster School of Business

BA RM 590 Behavioral Research Methods - Theory and Design
BA RM 591 Behavioral Research Methods - Approaches and Applications

Evans School of Public Policy and Governance

PUBPOL 525 Qualitative Field Methods and Analysis
PUBPOL 527 Quantitative Analysis I
PUBPOL 528 Quantitative Analysis II
PUBPOL 529 Advanced Quantitative Methods for Policy Analysis

School of Public Health

SPH 480 Research Methods in Public Health

School of Social Work

SOC W 505 Foundations of Social Welfare Research
SOC W 506 Social Welfare Research and Evaluation

Graphic: Poster by Sam Braun, Federal Art Project, 1937. "The World Wants New Knowledge, New Skills--Enroll: Federal Adult Schools." Library of Congress Prints and Photographs Division .

Center for Statistics and the Social Sciences (CSSS) CSSS provides free statistical consulting to current UW faculty, staff, and students in affiliated departments.
<< Previous: Research Methods
Next: Selected Books & Articles on Research Methods >>
Last Updated: Apr 1, 2024 11:09 AM
URL: https://lib.law.uw.edu/els

Balancing Different Legal and Ethical Requirements in the Construction of Informed Consents in Qualitative International Collaborative Research Across Continents - Reflections from a Scandinavian Perspective

Published: 27 May 2024

Cite this article

Stinne Glasdam ORCID: orcid.org/0000-0002-0893-3054 1 ,
Katharina Ó. Cathaoir ORCID: orcid.org/0000-0002-1535-9134 2 &
Sigrid Stjernswärd ORCID: orcid.org/0000-0002-7152-9206 3

59 Accesses

1 Altmetric

Explore all metrics

International research collaborations engage multiple countries, researchers, and universities. This enhances the magnitude of contextual challenges, including legal and ethical dimensions across various jurisdictions, that must be bridged in qualitative research regardless of discipline, also in the construction of informed consents. From a Scandinavian perspective, this discussion paper explores challenges pertaining to the construction of informed consents related to EU data protection legislation, to which research institutions are subject when processing data related to EU residents. Next, it discusses challenges related to different traditions in terms of handling informed consent and research participants’ integrity, including the possibilities to waive anonymity in research. In international, multidisciplinary studies where researchers also operate in relatively ‘unknown territory’, it is especially important to be aware of and reflect on (inter)national possibilities and limitations related to laws, ethics, and culture/traditions in societies and within the academic fields. The variations in laws, ethical guidelines, and traditions in different countries demand that researchers are up to date with laws and ethical guidelines in the studied countries. Their practical implementation in the countries at stake in international, collaborative research endeavours are important, especially since such regulations and guidelines are far from static and change over time. The implementation of good ethical research practice requires democratic, reflexive, and responsive processes in all phases of research. Especially the preparation phase functions as a period to increase and ensure the knowledge and legal/ethical competences of the entire research team to meet the demands in the countries at stake.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA) Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Ethical Dilemmas in Cross-national Qualitative Research: A Reflection on Personal Experiences of Ethics from a Doctoral Research Project

Researchers’ views on, and experiences with, the requirement to obtain informed consent in research involving human participants: a qualitative study.

Compliance with research ethics in epidemiological studies targeted to conflict-affected areas in Western Ethiopia: validity of informed consent (VIC) by information comprehension and voluntariness (ICV)

Alkis-Tumturk, A. (2022). Uncertain future of transatlantic data flows: Will the United States ever achieve the ‘adequate level’ of data protection? Hungarian Journal of Legal Studies , 63 (3), 294–311. https://doi.org/10.1556/2052.2022.00376 .

Article Google Scholar

Appadurai, A. (1997). The research ethic and the spirit of internationalism. Items Social Science Research Council , 51 (4), 55–60. Part I.

Google Scholar

Ball, J., & Janyst, P. (2008). Enacting Research Ethics in partnerships with indigenous communities in Canada: Do it in a good way. Journal of Empirical Research on Human Research Ethics: An International Journal , 3 (2), 33–51. https://doi.org/10.1525/jer.2008.3.2.33 . https://doi-org.ludwig.lub.lu.se/ .

Becker, R., Chokoshvili, D., Thorogood, A., Dove, E. S., Molnár-Gábor, F., Ziaka, A., Tzortzatou-Nanopoulou, O., & Comandè, G. (2024). Purpose definition as a crucial step for determining the legal basis under the GDPR: Implications for scientific research. Journal of Law and the Biosciences , 11 (1), lsae001. https://doi.org/10.1093/jlb/lsae001 .

Canadian Institutes of Health Research, Natural Sciences and Engineering Research Council of Canada, & Social Sciences and Humanities Research Council of Canada (2022). TriCouncil Policy Statement. Ethical Conduct for Research Involving Humans. TCPS2 2022. Government of Canada https://ethics.gc.ca/eng/documents/tcps2-2022-en.pdf .

Collyer, F. M. (2018). Global patterns in the publishing of academic knowledge: Global North, global South. Current Sociology , 66 (1), 56–73. https://doi.org/10.1177/0011392116680020 .

Cool, A. (2019). Impossible, unknowable, accountable: Dramas and dilemmas of data law. Social Studies of Science , 49 (4), 503–530. https://doi.org/10.1177/0306312719846557 .

Danish National Center for Ethics (2017). Vejledning om brug af biologisk materiale i sundhedsvidenskabelige forskningsprojekter [Guidance on the use of biological material in health science research projects]. Danish National Center for Ethics https://nationaltcenterforetik.dk/videnskab/vejledninger/biobanker#9 .

Dovey, S., Hall, K., Makeham, M., Rosser, W., Kuzel, A., Van Weel, C., Esmail, A., & Phillips, R. (2011). Seeking ethical approval for an international study in primary care patient safety. The British Journal of General Practice , 61 (585), 197–204. https://doi.org/10.3399/bjgp11X567144 .

Dusdal, J., & Powell, J. J. W. (2021). Benefits, motivations, and challenges of international collaborative research: A sociology of science case study. Science and Public Policy , 48 (2), 235–245. https://doi.org/10.1093/scipol/scab010 .

Espinosa, P., Pichayayothin, R., Suavansri, N. B., French, P., Areekit, J. J., Nilchantuk, P., Jones, C., Mam, T. S., Moore, E., J. B., & Heaney, C. A. (2022). Found in translation: Reflections and lessons for qualitative research collaborations across language and culture. International Journal of Qualitative Methods , 21. https://doi.org/10.1177/16094069221101280 .

European Commission (2023). Data protection: European Commission adopts new adequacy decision for safe and trusted EU-US data flows. 10 July. https://ec.europa.eu/commission/presscorner/detail/en/ip_23_3721 .

European Data Protection Board (2021). Guidelines 07/2020 on the concepts of controller and processor in the GDPR. European Union. https://edpb.europa.eu/system/files/2021-07/eppb_guidelines_202007_controllerprocessor_final_en.pdf .

Gefenas, E., Lekstutiene, J., Lukaseviciene, V., Hartlev, M., Mourby, M., & Cathaoir, K. Ó. (2022). Controversies between regulations of research ethics and protection of personal data: Informed consent at a cross-road. Medicine, Health Care, and Philosophy , 25 (1), 23–30. https://doi.org/10.1007/s11019-021-10060-1 .

Gomez, C. J., Herman, A. C., & Parigi, P. (2022). Leading countries in global science increasingly receive more citations than other countries doing similar research. Nature Human Behaviour , 6 , 919–929. https://doi.org/10.1038/s41562-022-01351-5 .

Grimwood, B. S. R., Doubleday, N. C., Ljubicic, G. J., Donaldson, S. G., & Blangy, S. (2012). Engaged acclimatization: Towards responsible community-based participatory research in Nunavut. Canadian Geographer , 56 (2), 211–230. https://doi.org/10.1111/j.1541-0064.2012.00416.x .

Guillemin, M., & Gillam, L. (2004). Ethics, reflexivity, and ethically important moments in research. Qualitative Inquiry , 10 , 261–280. https://doi.org/10.1177/1077800403262360 .

Hamdan, O. A., Meschitti, V., & Burhan, M. (2022). How is leadership cultivated between principal investigators and research team members? Evidence from funded research projects in the UK. Higher Education Quarterly , 76 (4), 726–740. https://doi.org/10.1111/hequ.12342 .

Indenrigs- og Sundhedsministeriet. (2020). Bekendtgørelse Af Lov Om Videnskabsetisk behandling Af sundhedsvidenskabelige forskningsprojekter og sundhedsdatavidenskabelige forskningsprojekter. LBK Nr 1338 Af 01/09/2020 [Executive order of the Act on scientific ethical treatment of health science research projects and health data research projects. LBK no. 1338 of 01/09/2020] . Danish Ministry of the Interior and Health.

Kadam, R. A. (2017). Informed consent process: A step further towards making it meaningful! Perspectives in Clinical Research , 8 (3), 107–112. https://doi.org/10.4103/picr.PICR_147_16 .

Kirchhoffer, D. G. (2019). Dignity, being and becoming in research ethics. In D. G. Kirchhoffer, & B. J. Richards (Eds.), Beyond autonomy: Limits and alternatives to informed consent in research ethics and law (pp. 117–132). Cambridge University Press. http://doi.org.ludwig.lub.lu.se/ https://doi.org/10.1017/9781108649247.008 .

Lahey, T. (2013). The ethics of clinical research in low- and middle-income countries. Handbook of Clinical Neurology , 118 , 301–313. https://doi.org/10.1016/B978-0-444-53501-6.00025-1 .

Lawlor, R. T. (2023). The impact of GDPR on data sharing for European cancer research. The Lancet Oncology , 24 (1), 6–8. https://doi.org/10.1016/S1470-2045(22)00653-2 .

Puljak, L., Mladinić, A., Iphofen, R., & Koporc, Z. (2020). Before and after enforcement of GDPR: Personal data protection requests received by Croatian Personal Data Protection Agency from academic and research institutions. Biochemia Medica , 30(3), 030201. https://doi.org/10.11613/BM.2020.030201 .

MacDonald, K. (2017). My experiences with indigenist methodologies. Geographical Research , 55 (4), 369–378. https://doi.org/10.1111/1745-5871.12241 .

Marginson, S. (2021). All things are in flux’: China in global science. Higher Education: The International Journal of Higher Education Research , 1–30. https://doi.org/10.1007/s10734-021-00712-9 .

Ministry of Higher Education and Science. (2014). Danish code of conduct for research integrity . Ministry of Higher Education and Science.

Mladinić, A., Puljak, L., & Koporc, Z. (2021). Post-GDPR survey of data protection officers in research and non-research institutions in Croatia: A cross-sectional study. Biochemia Medica , 31 (3), 030703. https://doi.org/10.11613/BM.2021.030703 .

National Research Ethics Committees (2010). Forskningsbiobanker [Research biobanks]. https://www.forskningsetikk.no/ressurser/fbib/menneskelig-materiale/forskningsbiobanker/ .

Øye, C., Sørensen, N. Ø., & Glasdam, S. (2016). Qualitative research ethics on the spot: Not only on the desktop. Nursing Ethics, 23 (4), 455–464. https://doi.org/10.1177/0969733014567023 .

Prictor, M., Teare, H. J. A., & Kaye, J. (2018). Equitable participation in biobanks: The risks and benefits of a dynamic consent approach. Frontiers Public Health , 6 , 253. https://doi.org/10.3389/fpubh.2018.00253 .

Quaglio, G. (2019). International research collaboration – A key feature of the new global science landscape. European Parliamentary Research Service. https://epthinktank.eu/2019/09/30/international-research-collaboration-a-key-feature-of-the-new-global-science-landscape/ .

Raposo, H., Melo, S., & Egreja, C. (2022). Data protection in sociological health research: A critical narrative about the challenges of a new regulatory landscape. Sociological Research Online , 27 (4), 1060–1076. https://doi.org/10.1177/13607804221107676 .

Regulation (EU) 2016/679 of the European Parliament and of the council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing European Union, & Directive (2016). 95/46/EC (General Data Protection Regulation). https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:32016R0679_from=EN#d1e40-1-1

Reid, C., Calia, C., Guerra, C., Grant, L., Anderson, M., Chibwana, K., Kawale, P., & Action Amos. (2021). Ethics in global research: Creating a toolkit to support integrity and ethical action throughout the research journey. Research Ethics Review , 17. https://doi.org/10.1177/1747016121997522 .

Rothstein, M. A., Zawati, M. H., Thorogood, A., Beauvais, M. J. S., Joly, Y., Brothers, K. B., Lang, M., Andanda, P., Ho, C., Isasi, R., Kaye, J., Lee, W. B., Nnamuchi, O., Saltzman, A., & Knoppers, B. M. (2022). Streamlining ethics review for international health research. Science , 375 (6583), 825–826. https://doi.org/10.1126/science.abn0675 .

Schroeder, D., Chatfield, K., Singh, M., Chennells, R., & Herissone-Kelly, P. (2019). Ethics dumping and the need for a global code of conduct. Springer International Publishing . https://doi.org/10.1007/978-3-030-15745-6_1 .

Shih, T., & Forsberg, E. (2023). Origins, motives, and challenges in Western-Chinese research collaborations amid recent geopolitical tensions: Findings from swedish-chinese research collaborations. Higher Education: The International Journal of Higher Education Research , 85 (3), 651–667. https://doi.org/10.1007/s10734-022-00859-z .

Sidle, J. E., Were, E., Wools-Kaloustian, K., Chuani, C., Salmon, K., Tierney, W. M., & Meslin, E. M. (2006). A needs assessment to build international research ethics capacity. Journal of Empirical Research on Human Research Ethics: An International Journal , 1 (2), 23–38. https://doi.org/10.1525/jer.2006.1.2.23 .

Slokenberga, S. (2021). Setting the foundations: Individual rights, public interest, scientific research and biobanking. In S. Slokenberga, O. Tzortzatou, & J. Reichel (Eds.), GDPR and biobanking. Law, governance and technology series, 43 . Springer. https://doi.org/10.1007/978-3-030-49388-2_2 .

Staunton, C., Slokenberga, S., & Mascalzoni, D. (2919). The GDPR and the research exemption: Considerations on the necessary safeguards for research biobanks. European Journal of Human Genetics , 27 , 1159–1167. https://doi.org/10.1038/s41431-019-0386-5 .

Steinsbekk, K. S., Myskja, B., K., & Solberg, B. (2013). Broad consent versus dynamic consent in biobank research: Is passive participation an ethical problem? European Journal of Human Genetics , 21 (9), 897–902. https://doi.org/10.1038/ejhg.2012.282 .

Svalastog, A. L., & Eriksson, S. (2010). You can use my name; you don’t have to steal my story – A critique of anonymity in indigenous studies. Developing World Bioethics , 10 (2), 104–110. https://doi.org/10.1111/j.1471-8847.2010.00276.x .

Swedish Authority for Privacy Protection (2023). Transfer of personal data to a third country https://www.imy.se/en/organisations/data-protection/this-applies-accordning-to-gdpr/transfer-of-data-to-a-third-country/ .

Swedish Ethical Review Authority (n.d). Forskningsperson [Research person]. https://etikprovningsmyndigheten.se/for-forskningsperson/ .

Tang, L. (2019). Five ways China must cultivate research integrity. Nature , 575 (7784), 589–591.

Teare, H. J. A., Prictor, M., & Kaye, J. (2021). Reflections on dynamic consent in biomedical research: The story so far. European Journal of Human Genetics , 29 , 649–656. https://doi.org/10.1038/s41431-020-00771-z .

The Act concerning the Ethical Review of Research Involving Humans (2003:460). The Ministry of Education and Cultural Affairs, & Sweden (2003). https://www.onep.se/media/2348/the_ethical_review_act.pdf .

The Danish Data Protection Agency. (2018). Vejledning om de registreredes rettigheder [Guidance on the rights of data subjects] . The Danish Data Protection Agency.

The Danish Data Protection Agency (2022). Overførsel af personoplysninger til tredjelande. Vejledning [Transfer of personal data to third countries. Guidance]. The Danish Data Protection Agency. https://www.datatilsynet.dk/Media/637902777513932912/Vejledning%20om%20overf%C3%B8rsel%20til%20tredjelande.pdf .

The European Data Protection Board (2021). EDPB Document on response to the request from the European Commission for clarifications on the consistent application of the GDPR, focusing on health research. https://edpb.europa.eu/sites/default/files/files/file1/edpb_replyec_questionnaireresearch_final.pdf .

The National Committee for Medical and Health Research Ethics (2021). Medical and health research in low- and middle-income countries. Research ethics. The National Committee for Medical and Health Research Ethics (Norway). https://www.forskningsetikk.no/en/guidelines/medical-and-health-research/medical-and-health-research-in-low--and-middle-income-countries/ .

The Research Council of Norway (2023). Personal data protection – privacy statement. The Research Council of Norway. https://www.forskningsradet.no/en/privacy-policy/ .

Tusino, S., & Furfaro, M. (2022). Rethinking the role of research ethics committees in the light of regulation (EU) 536/2014 on clinical trials and the COVID-19 pandemic. British Journal of Clinical Pharmacology , 88 (1), 40–46. https://doi.org/10.1111/bcp.14871 .

van den Scott, L. J. K. (2020). Sociology and ethics: Doing the right thing. Springer International Publishing . https://doi.org/10.1007/978-3-030-16759-2_68 .

Wahlberg, A., Rehmann-Sutter, C., Sleeboom-Faulkner, M., Lu, G., Döring, O., Cong, Y., Laska-Formejster, A., He, J., Chen, H., Gottweis, H., & Rose, N. (2013). From global bioethics to ethical governance of biomedical research collaborations. Social Science & Medicine , 98 , 293–300. https://doi.org/10.1016/j.socscimed.2013.03.041 .

Wiles, R. (2013). What are qualitative research ethics? Bloomsbury. https://library.oapen.org/bitstream/handle/20.500.12657/58724/9781849666534.pdf?sequence=1 .

Wilson, E., Kenny, A., & Dickson-Swift, V. (2018). Ethical challenges of community based participatory research: Exploring researchers’ experience. International Journal of Social Research Methodology , 21 (1), 7–24. https://doi.org/10.1080/13645579.2017.1296714 .

Witze, A. (2016). Research gets increasingly international. Nature . https://doi.org/10.1038/nature.2016.19198 .

World Medical Association. (2013). WMA Declaration of Helsinki: Ethical principles for medical research involving human subjects . Fortaleza.

Download references

Acknowledgements

We thank Professor Christine Øye (Western Norway University of Applied Sciences) for valuable comments to the former drafts of the article and Professor Klaus Hoeyer (University of Copenhagen) for valuable discussions during the writing process.

No fundings was granted for the current article.

Author information

Authors and affiliations.

Integrative Health Research, Department of Health Sciences, Faculty of Medicine, Lund University, Sölvegatan 19, Lund, S- 222 42, Sweden

Stinne Glasdam

Welma, Faculty of Law, University of Copenhagen, Karen Blixens Plads 16, København S, DK- 2300, Denmark

Katharina Ó. Cathaoir

Mental Health, Activity and Participation, Department of Health Sciences, Faculty of Medicine, Lund University, Sölvegatan 19, Lund, S- 222 42, Sweden

Sigrid Stjernswärd

You can also search for this author in PubMed Google Scholar

Contributions

Stinne Glasdam: Project administration, Formal analysis, Writing- Original draft preparation, Writing - Review & Editing. Katharina Ó Cathaoir: Formal analysis, Writing- Original draft preparation, Writing - Review & Editing. Sigrid Stjernswärd: Formal analysis, Writing- Original draft preparation, Writing - Review & Editing.

Corresponding author

Correspondence to Stinne Glasdam .

Ethics declarations

Conflict of interest.

The authors have no competing interests to declare.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Glasdam, S., Cathaoir, K.Ó. & Stjernswärd, S. Balancing Different Legal and Ethical Requirements in the Construction of Informed Consents in Qualitative International Collaborative Research Across Continents - Reflections from a Scandinavian Perspective. J Acad Ethics (2024). https://doi.org/10.1007/s10805-024-09536-5

Download citation

Accepted : 06 May 2024

Published : 27 May 2024

DOI : https://doi.org/10.1007/s10805-024-09536-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Informed consent
International research collaboration
Legislation
Qualitative research
Research ethics
Find a journal
Publish with us
Track your research

Exploring Approaches to Legal Research, Berlin 2024

The 2024 ‘ Approaches to Legal Research ‘ course, held at Harnack House in Berlin in May, was an opportunity for Max Planck Law researchers to engage with a variety of research methodologies. This short residential course, the first of what is in effect a complete restructuring of the old Curriculum, brought together many early-stage PhD candidates as well as more advanced researchers from across the Max Planck Law network.

Participants explored a range of approaches, from comparative law and legal history to anthropology, sociology, economic analysis of law, and critical approaches. Experienced researchers from across the nine Max Planck Institutes led the sessions, as well as one guest professsor, each drawing upon practical experience in their respective fields.

Dr Matilde Cazzola guided attendees through the intricacies of historical analysis in legal research. She stressed the significance of understanding historical context to better grasp contemporary legal landscapes, emphasizing rigorous archival research as a cornerstone of this discipline.

Dr Johannes Rottmann presented on the economic analysis of law. His session highlighted the diverse applications of economic principles within the legal realm, which extends even to the economics of crime. Rottmann invited participants to consider how economic tools can illuminate human behaviour and incentives within legal contexts.

Dr Annette Mehlhorn and Pierre Druart delved into the relationship between law and anthropology. Their presentation emphasized, amongst other things, the value of ethnographic techniques and the blurred boundaries between anthropology and sociology. A major takeaway was an appreciation of how interdisciplinary work can deepen our understanding of legal phenomena.

Professor Dr Andreas Engert of FU Berlin discussed empirical legal research, beginning with a contrasts between quantitative and qualitative methods. Engert highlighted the objectivity aimed for in quantitative analysis while acknowledging the challenges of affordability and data availability, as well as the potential pitfalls of over-reliance on numerical data.

Professor Stefan Vogenauer, Director and Chair of Max Planck Law shared an example from his own work in comparative law. Through this, he was able to demonstrate the various aims of comparative law, such as legal harmonization and reform, and to address the ‘if’ and ‘how’ of its methodology. Progressing though a series of standard steps for conducting comparative law research, he simultaneously highlighted the common pitfalls to be aware of when taking this approach.

Dr Valerij Zisman introduced participants to a very specific critical approach in legal research. Drawing on his background in analytical philosophy, Zisman examined the logics of argumentation, offering tools to identify weaknesses and implicit assumptions in legal arguments.

A noteworthy feature of the event was the after-dinner roundtable discussion, where panellists reflected on the necessity and limits of interdisciplinarity in legal research.

After the course, participants provided excellent feedback, acknowledging its role in broadening their methodological toolkit. ‘Harnack House is a great location,’ one attendee noted, while another opined that ‘the workshop is a good introduction to different research methods.’ Though some participants admitted that they may not immediately apply the approaches to their current PhD projects, they nonetheless recognized their potential value for future research endeavours.

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

We're Hiring!
Help Center

Chapter 38 Qualitative Approaches to Empirical Legal Research

Publisher's description: "The empirical study of law, legal systems and legal institutions is widely viewed as one of the most exciting and important intellectual developments in the modern history of legal research. Motivated by a conviction that legal phenomena can and should be understood not only in normative terms but also as social practices of political, economic and ethical significance, empirical legal researchers have used quantitative and qualitative methods to illuminate many aspects of law's meaning, operation and impact. In the 43 chapters of The Oxford Handbook of Empirical Legal Research leading scholars provide accessible and original discussions of the history, aims and methods of empirical research about law, as well as its achievements and potential. The Handbook has three parts. The first deals with the development and institutional context of empirical legal research. The second - and largest - part consists of critical accounts of empirical research on many aspects of the legal world - on criminal law, civil law, public law, regulatory law and international law; on lawyers, judicial institutions, legal procedures and evidence; and on legal pluralism and the public understanding of law. The third part introduces readers to the methods of empirical research, and its place in the law school curriculum."

Related Papers

Hồng Hạnh Phạm

Ý thức pháp luật và xây dựng ý thức pháp luật trong điều kiện nhà nước pháp quyền Việt Nam hiện nay. DSpace/Manakin Repository. ... Login. Ý thức pháp luật và xây dựng ý thức pháp luật trong điều kiện nhà nước pháp quyền Việt Nam hiện nay. Show full item record. ...

VNU Journal of Science: Legal Studies

Hoàng Anh Nguyễn

Tạp chí Khoa học

Bài viết này trình bày tổng quan các quan niệm về phóng xạ tổng hợp được từ các nghiên cứu trên thế giới. Dựa vào kết quả tổng hợp được, chúng tôi xây dựng bảng hỏi và khảo sát các quan niệm về phóng xạ trên 505 sinh viên chuyên ngành vật lí ở các trường đại học. Các số liệu thống kê của khảo sát cho thấy rất nhiều sinh viên có quan niệm sai về phóng xạ dù đã được học về phóng xạ hạt nhân. Đã có một số đề xuất phương pháp khắc phục các quan niệm sai này từ các nghiên cứu nói trên, tuy nhiên kết quả vẫn chưa rõ ràng. Kết quả nghiên cứu và những đề xuất của chúng tôi có thể có ích cho những đề tài hay những nghiên cứu tiếp theo về việc dạy học nhằm khắc phục những quan niệm sai về phóng xạ.

Hue University Journal of Science: Social Sciences and Humanities

Phan Thị Hồng

Bài viết phân tích vấn đề đăng ký hợp đồng trong mối quan hệ giữa các chủ thể giao kết hợp đồng và mối quan hệ với người thứ ba nhằm làm rõ hậu quả pháp lý của hợp đồng được đăng ký và không đăng ký. Trên cơ sở phân tích quy định của pháp luật hiện hành và đối sánh với thực tiễn giải quyết tranh chấp hợp đồng chuyển nhượng quyền sử dụng đất tại Tòa án, tác giả đề xuất những kiến nghị nhằm góp phần hoàn thiện pháp luật dân sự và pháp luật đất đai về đăng ký hợp đồng.

Nabil Mabrouk

The Indonesian Journal of Social Studies

Devi Vitriana Purwanto

Assurance, Relevance, Interest, Assessment and Satisfaction (ARIAS) is one learning model to students center. This learning model emphasizes the delivery the activeness of students in the class. In the implementation of this model has the five components which consists of five integrated components needed in the learning activities (1) assurance which foster student’s self-confidence, (2) relevance which connect the material with student’s real life, (3) interest which grow the student’s interest, (4) assessment which is an student assessment, and (5) satisfaction which provide a sense of proud to student. The aim of this research was to describe the teacher's ability to manage learning, student activities, student learning, and students' response using ARIAS learning model. Research conducted on the use of the model of ARIAS in social studies so far shown significant result on increase students’ understanding of social studies consepts. The Application Assurance, Relevance,...

VNU Journal of Science: Policy and Management Studies

Đào Văn Minh

With the goal of objective and systematic evaluation of excellence research group development policy at VNU, in order to help policymakers of VNU can consult and develop scientific and technology development strategy. In this article, we focus on reviewing policies of excellence research group development in VNU, on the basis of analyzing and evaluating three overall groups of policies, including (1) Group of investment policies, human resource development; (2) Policy group to improve the environment and research conditions; (3) Policy group for cooperation and development. For each policy group, we will consider the content, the results achieved, assess the impact of the policy to point out the strengths and limitations to make recommendations for complete solutions

Siti Muyana

Pengembangan model layanan evaluasi berbasis context input process product (CIPP) merupakan salah satu bentuk solusi dari permasalahan guru BK dalam melaksanakan evaluasi terkait dengan layanan informasi yang telah dilakukan. Pengembangan model CIPP ini muncul karena kurang pahamnya guru BK dalam melakukan evaluasi sehingga menyebabkan kesulitan untuk mengetahui keberhasilan pelaksanaan layanan informasi di sekolah, lemahnya akuntabilitas dan kesulitan untuk melakukan perbaikan serta pengembangan. Melalui model context input process product (CIPP) ini, evaluasi layanan informasi dapat dilakukan secara komprehensif.

Revista do Curso de Direito do UNIFOR

Patrick Souto

O presente artigo trata das vantagens que a ação probatória autônoma de natureza não cautelar (art. 381, II e III do CPC) pode proporcionar àquele que é considerado juridicamente vulnerável pelo processo do trabalho — o empregado. Diante das recentes reformas que o direito processual trabalhista sofreu, principalmente com a imposição dos ônus de sucumbência também ao empregado, ainda que beneficiário da justiça gratuita, sustenta-se que a ação probatória autônoma com finalidade não cautelar pode facilitar o seu acesso à justiça, diminuindo ou, em algumas hipóteses eliminando, o risco de surpresa na relação processual e auxiliando na avaliação dos riscos da demanda.

38 Quantitative Approaches to Empirical Legal Research

Lee Epstein is the Ethan A. H. Shepley Distinguished University Professor at Washington University in St. Louis.

Andrew D. Martin is Professor of Law and Political Science at Washington University in St. Louis.

Published: 18 September 2012
Cite Icon Cite
Permissions Icon Permissions

This article deals with the objective nuances of empirical research, within the ambit of the quantitative kind. It begins with an overview of conducting empirical legal research, discussing its research design, implementation, and challenges faced. Theorizing in empirical legal scholarship comes in different forms: in some projects theories seek to provide insight into a wide range of phenomena, others are tailored to fit particular situations. In the clarification process the researcher translates abstract notions into concrete ones. To convert the data into an analyzable form, empirical legal researchers make use of a variety of data-generation mechanisms. Researchers can implement random sampling in various ways depending on the nature of the problem. Data analysis enables researchers to compare their overlap. The goal of empirical legal research is to find facts about the unknown. The last step of empirical legal research is to present its results, for which, documentation is a requisite.

Conducting Empirical Legal Research: An Overview 902

Designing Research 905

Collecting Data and Coding Variables 909

Analyzing Data 912

The Last Step: Presenting the Results of Empirical Legal Research 917

T he title of this Chapter seems too wordy. Why call it doing “empirical legal research,” and not simply doing “empirical research”? After all, regardless of whether empirical researchers are addressing a legal question or any other, they follow the same rules—the rules that enable them to draw inferences from the data they have collected (Epstein and King, 2002 ; King et al., 1994 ). What's more, because empirical research in law has methodological concerns that overlap with those in Biology, Chemistry, Economics, Medicine and Public Health, Political Science, Psychology, and Sociology, empirical legal researchers can adopt methods from these other disciplines to suit their own purposes.

On the other hand, in virtually every discipline that has developed a serious empirical research program—law not excepted—scholars discover methodological problems that are unique to the special concerns in that area. Each new data source often requires at least some adaptation of existing methods, and sometimes the development of new methods altogether. There is bioinformatics within Biology, biostatistics and epidemiology within Medicine and Public Health, econometrics within Economics, chemometrics within Chemistry, political methodology within Political Science, psychometrics within Psychology, sociological methodology within Sociology, and so on. As of this writing, there is no “legalmetrics” but that should happen soon enough (though probably not before this Chapter appears in print).

In short, with a few wording substitutions here and there, much of what follows pertains to all empirical research. But much is not all . Recognizing that empirical legal work is unique in various ways, as we describe the research process we also outline some of the field's distinct challenges—most notably, how to communicate complex statistical results to a community lacking in statistical training.

We begin by describing the research process. Then, in sections II-V, we flesh out the various components of the process: designing research, collecting and coding data, analyzing data, and presenting results.

I. Conducting Empirical Legal Research: An Overview

How do scholars implement quantitative empirical research? What challenges do they confront? To begin to formulate responses, consider a legal question at the center of hundreds, perhaps thousands, of lawsuits each year: do employers pay men more than women solely because of their gender? Next consider how researchers who faced absolutely no constraints—i.e., researchers with more powers than Batman, Superman, and Wonder Woman combined—would address this question. If we were the researchers, we would begin by creating a workplace, randomly drawing a worker from the workforce population, randomly assigning a sex (say, male) to the worker, instructing him to enter the workplace, and observing his wage. 2 Next, we would reverse time, and assign the same worker the other sex (female), send her into exactly the same workplace, and observe her wage. If we observed a difference in the wages of our two workers—such that the same employer paid the male version less than the female version—then we might conclude that, yes, gender causes pay inequities.

Unfortunately, researchers aren't superheroes; they usually don't have the power to create a workplace and assign a sex. And they certainly don't have the power to rerun history. This is known as fundamental problem of causal inference (Holland, 1986 : 947). It simply means that researchers can only observe the factual (e.g., a female worker's salary, if in fact the worker was a female) and not the counterfactual (e.g., a male worker's salary, if in fact the worker was female). 3

This is a problem without a solution but scholars have developed various fixes. The gold standard along these lines is a proper experiment—that is, an experiment in which the researcher randomly selects subjects from the population of interest and then randomly assigns the subjects to treatment and control conditions (see Ho et al., 2007 ). Very few experiments in empirical legal studies actually meet the first condition (random selection from the population) but some scholars have tried to meet the second. Jeffrey J. Rachlinski and his colleagues (2006), for example, recruited 113 bankruptcy court judges to participate in an experiment designed to detect whether the race of a party affected the judges' decisions. 4 They asked the judges to read the same case materials but unbeknownst to the judges, the researchers randomly assigned them to a control or treatment group. Those judges in the control group were led to believe that the debtor was white; those in the treatment group were led to believe that the debtor was black. (It turned out that race did not affect the judges' decisions.)

This is a reasonable approach to the fundamental problem of causal inference. But, sadly, it is infeasible for many empirical legal projects—including studies of pay equity (no experiment can assign a sex to workers). It is not even feasible for most analyses of judicial behavior (the Rachlinski et al. study is a notable exception). To provide but one example, suppose we wanted to investigate the extent to which female judges affect the decisions of their male colleagues. No U.S. Court of Appeals would allow us to manipulate the composition of panels so that we could identify a possible gender effect. We could say the same of the other institutions of government. Can you imagine the President of the United States agreeing to nominate two judicial candidates identical in all respects except that one is highly qualified and the other highly unqualified just to enable us to learn whether qualifications affect the confirmation votes of U.S. senators? We can't.

The upshot is that most empirical legal researchers simply do not have the luxury of analyzing data they developed in an experiment (i.e., experimental data). Instead, they must make use of data the world—not they—generated (i.e., observational data): salaries paid to workers by real companies; the decisions of judges in concrete cases; the votes cast by senators over the president's nominee to the federal courts. And this, of course, substantially complicates the task empirical legal researchers confront. While experimental data—generated by random assignment to treatment and control groups—effectively minimize the confounding effects of other variables, the same cannot be said of observational data. For those data, researchers must invoke statistical techniques (discussed below) to accomplish the same thing.

Because observational datasets are so much more common in quantitative empirical legal research, in what follows we focus on strategies for working with them. It is important to keep in mind, however, that other than issues of data generation and control (statistical versus experimental), experimental and observational studies are not altogether different for our purposes. Either way, scholars tend to execute them in four steps: they design their projects, collect and code data, conduct analyses, and present results. 5

Research design largely (though not exclusively) involves the process of moving from the conceptual to the concrete. To return to our example of pay equity, suppose the researcher hypothesizes that once she takes into account the experience of the workers, males earn no more than females. However plausible this hypothesis, the researcher confronts a non-trivial problem in assessing it: how to operationally define the concept of “experience.” Is it years from degree? Years in the workforce? Months in the same job? More generally, before researchers can answer empirical legal questions—actually before they can even collect the first piece of data—they must devise ways to clarify concepts such as experience so that they can observe them. All of this and more appear on that first (metaphorical) slide.

Data collection and coding entails translating information in a way that researchers can make use of it. For a study of pay equity, the researcher may have piles of pay stubs and employee records. Unless the researcher can transform the piles into data she can analyze the study cannot proceed.

Data analysis typically consists of two activities. First, researchers often summarize the data they have collected. If, for example, we collect information on a sample of 50 workers' salaries in a firm with 500 workers, it may be interesting to know the average salary for the men in our sample and the average salary for the women. Second, analysts use data to make inferences—to use facts they know (about the salaries, gender, experience, and so on of the 50 workers in their sample) to learn about facts they do not know (the salaries, gender, experience, and so on of the 500 workers). To perform inference in quantitative studies, researchers employ various statistical methods. Worth noting, though is that use of statistics presupposes that the study is well designed and the data are of a sufficiently high quality. If either the design is poor or the data inadequate, researchers will be unable to reach inferences of high quality. In other words, without a proper research design no statistical method can provide reliable answers; not even the best statistician cannot make lemonade without lemons.

Finally, once empirical legal analysts have drawn inferences from their data, they must be able to communicate their results to a community that may have little (or no) knowledge of even simple statistics. Doing so effectively blends both art and science, and requires careful consideration of both the project and the intended audience.

These are the contours of the research process. Let us now flesh them out to the extent possible given space constraints.

II. Designing Research

It should go without saying that before researchers can design their project, they must have one. To “have a project” usually means that the analyst has a question she wishes to answer and has theorized about possible responses.

Research questions in empirical legal studies come from everywhere and anywhere. Perhaps scholars see a gap in the existing literature or perhaps they think the literature is incomplete or even wrong. Sometimes questions come from current events—whether a new law is having the desired (or any) effect or whether a court decision is efficacious—and sometimes they come from history. A perusal of any socio-legal journal would provide evidence of these and other motivations.

The variation is not unexpected. Empirical legal scholars are a diverse lot, with equally diverse interests. What their questions have in common, though, may be just as important: virtually all are quite conceptual. Consider a variation on the question we asked at the onset:

Do males and females who have the same level of experience earn the same amount of money?

However important this question, it is not one that even the best empirical legal project can ever address. Rather, the question the study will actually answer comes closer to this:

Do males and females who have been in the workforce for the same number of years net the same salary per month?

Note that the first form of the question contains several concepts—“earn” and “experience”—which researchers cannot directly observe. Only by clarifying these concepts, as the second form does, can the researcher empirically answer the question. Because this is more or less true of every empirical project, a major research challenge is to tighten the fit between the question asked and the question actually answered. If it is too loose the researcher cannot, at the end of the day, claim to have answered the question she initially posed. 6

Once analysts have settled on a research question, they usually begin theorizing about possible answers they can use to develop observable implications (sometimes called hypotheses or expectations). 7 A theory is simply a reasonable and precise answer to the research question. An observable implication is a claim about what we would expect to observe in the real world if our theory is right—typically, a claim that specifies a relationship between (or among) a dependent variable (what we are trying to explain) and an independent variable(s) (what our theory suggests explains the dependent variable) (Epstein and King, 2002 : 61–2).

Theorizing is a big topic, one to which we can hardly do justice in this short Chapter. So two observations will have to suffice. First, theorizing in empirical legal scholarship comes in many different forms: in some projects theories are quite big and grand, seeking to provide insight into a wide range of phenomena (e.g., rational choice theory in law and economics); others are simple, small, or tailored to fit particular situations. For the purposes of conducting an empirical study, this distinction may not be very important.

What is important—and this takes us to the second key point—is that the researcher extract observable implications from the theory. The reason is simple. Just as analysts almost never actually answer the question they pose, they almost never directly test their theory. Rather, they only indirectly assess it by evaluating the observable implications that follow from it.

To see the point, return to our question about pay equity between males and females, and consider the following theories and their observable implications.

Difference Theory Owing to discriminatory judgments about worth, employers pay females less than comparable males. Observable Implication All else being equal (e.g., experience), if my theory is correct, we should observe females earning less than males. Efficiency Theory Because labor markets are efficient, any observed differences between male and female workers are a product of experience, quality, productivity, and so on. Observable Implication All else being equal (e.g., experience), if my theory is correct, we should observe females and males earning the same

Note that in neither instance—no matter how good their design, their data, and their methods—will the researchers be able to conclude that their theory is right or wrong (that discriminatory judgments lead to pay inequity or that efficient markets lead to pay equity). All they will be able to say is whether their data are consistent with the observable implications following from their theory.

And even saying that involves hard work. The problem, yet again, is that observable implications are conceptual claims about the relationship between (or among) variables. To evaluate these, researchers must delineate how they actually can observe them in the real world. They must, in short, move from the abstract to the concrete—a task that forms the core of research design and that Figure 1 depicts. Note that in the clarification process the researcher translates abstract notions, such as “experience” and “earnings,” into the far more concrete “years in the workforce” and “gross annual income.” Unlike the abstractions, researchers can observe and measure “years in the workforce” and so on.

The process of clarifying observable implications so that researchers can evaluate them .

Note that in the clarification process the researcher translates abstract notions, such as “experience” and “earnings,” into the far more concrete “years in the workforce” and “gross annual income.” Unlike the abstractions, researchers can observe and measure “years in the workforce” and so on.

But how do analysts evaluate their choices and procedures? Why “years in the workforce” and not “years from degree,” “months in the same position,” or any of the other many plausible measures of experience? Typically, researchers look to the reliability and validity of their measures. Reliability is the extent to which it is possible to replicate a measure, reproducing the same value (regardless of whether it is the right one) on the same standard for the same subject at the same time. Measures of high reliability are preferable to those with lower levels of reliability. Validity is the extent to which a reliable measure reflects the underlying concept being measured. Along these lines, we might consider whether the measure is facially valid, that is, whether it comports with prior evidence or existing knowledge, among other criteria.

There is another test to which many researchers put their measures: robustness checks. Suppose we settled on “years in the workforce” as our measure of experience but believed that “months in the same position” was plausible as well. In our statistical work, we might try both hoping to obtain consistent results regardless of the particular measure. This procedure does not tell us whether “years in the workforce” is a better measure than “years in the same position” but it does help to anticipate a question put to many empirical legal scholars: “what if you had used measure Y instead of measure X? Would your results have been the same?”

III Collecting Data and Coding Variables

Once researchers have designed their project—that is, they have filled out the first slide—they typically turn to collecting and coding their data—the makings of the second slide. By this point, it should go without saying, though we shall say it anyway, that we can hardly scratch the surface of either; both deserve Chapters of their own.

What we can do instead is offer some brief counsel, beginning with data collection—actually, with a crucial step before data collection: determining whether the data the researcher needs already exist in the form she needs it. For decades now, empirical legal scholars have been amassing datasets—some for particular projects and others, the so-called “multi-user” datasets, designed for application to a wide range of problems. Either way, it is entirely possible (even probable in some areas of empirical legal studies) that researchers can locate suitable data without having to invest in costly from-scratch data-collection efforts.

A few examples suffice to make the point. If analysts are interested in cases decided by the U.S. Supreme Court, they should proceed directly to the U.S. Supreme Court Database (〈 http;//supemecourtdatabase.org 〉) This remarkable resource houses scores of variables on Supreme Court cases decided since 1953, including the legal provisions under analysis, the identity of the majority opinion writer, and the votes of the justices. A similar dataset, the U.S. Courts of Appeals Database, exists for cases decided by the U.S. circuit courts (at: 〈 http;//www.cas.sc.edu/poli/juri/ 〉). For the researcher interested in public opinion, the General Social Survey and the American National Election Study (both available via an intuitive interface at: 〈 http;//sda.berkeley.edu/archive.htm 〉) are natural places to look for relevant data. For other types of projects, we recommend visiting the websites of the Inter-University Consortium for Political and Social Research (〈 http;//www.icpsr.umich.edu/ 〉) and the IQSS Dataverse Network 〈 http;//dvn.iq.harvard.edu/dvn/ 〉), both of which serve as repositories for (or have links to) existing datasets. Federal and state governments and agencies too retain enormous amounts of information of interest to empirical legal scholars, including data on population demographics, economic indicators, and court caseloads. Last but not least, experience has taught us that a well-formulated Internet search can unearth datasets that scholars maintain on their own websites.

If the data simply do not exist in an analyzable form, empirical legal researchers can and do make use of a wide variety of data-generation mechanisms. They amass numerical data from structured interviews or surveys, from field research, from public sources, from private papers, and on and on. Each has its strengths and weaknesses (as do archived datasets) and it is the researchers' job to learn, understand, and convey them.

Still, within all this variation, two principles governing the data-collection process apply to most empirical legal research projects. One is simple enough: as a general rule, researchers should collect as much data as resources and time allow because basing inferences on more data rather than less is almost always preferable. To see the point, think about a study designed to study gender pay equity in academia. The more professors included in the study, the more certain the conclusions the analyst can reach. As a practical matter, however, diminishing returns kick in and settling on a sample size (as opposed to including all professors) is good enough. For example, one can estimate a proportion with ±2% margin of error with a random sample of approximately 2400 observations; the number increases dramatically to 9,600 for ±1%. This is why most public opinion surveys query, at most, a couple thousand respondents. As discussed in more detail below, this “margin of error” is sometimes referred to as the “sampling error” or the “confidence interval” (e.g., “CI ±3% ” in examples below).

Second, if researchers cannot collect data on all members of the population of interest (e.g., all professors)—and they rarely can—they must invoke selection mechanisms that avoid selection bias (mechanisms that don't bias their sample for or against their theory). For large- n studies (where n = number of participants) only random probability sampling meets this criterion. 8 A random probability sample involves identifying the population of interest (all professors) and selecting a subset (the sample) according to known probabilistic rules. To perform these tasks, the researcher must assign each member of the population a selection probability and select each person into the observed sample according to these probabilities. (Collecting all the observations is a special case of random selection with a selection probability of 1.0 for every element in the population.) 9

Researchers can implement random sampling in various ways depending on the nature of the problem. For a study of pay equity in the academy, for example, we could draw an equal probability sample—a sample in which all professors have an equal chance of being selected. If, on the other hand, we wanted to include all racial and ethnic groups in our study and worried that our sample, by chance, might not include, say, any American Indians, stratified random sampling may be a better strategy. The idea is to draw separate equal-probability-of-selection random samples within each category of a variable (here, race/ethnicity).

Whatever the procedure (so long as it involves random selection for large- n samples!), the legal researcher will typically end up with piles or computer files of questionnaires, field notes, court cases, and so on. Coding variables is the process of translating the relevant properties or attributes of the world (i.e., variables) housed in the piles and files into a form that the researcher can then analyze systematically (presumably after they have chosen appropriate measures to tap the underlying variables of interest).

Coding is a near-universal task in empirical legal studies. No matter whether their data are quantitative or qualitative, from where their data come, or how they plan to analyze the information they have collected, researchers seeking to make claims or inferences based on observations of the real world must code their data. And yet, despite the common and fundamental role it plays in research, coding typically receives only the briefest mention in most volumes on empirical research; it has received almost no attention in empirical legal studies.

Why this is the case is a question on which we can only speculate, but an obvious response centers on the seemingly idiosyncratic nature of the undertaking. For some projects researchers may be best off coding inductively, that is, collecting their data, drawing a representative sample, examining the data in the sample, and then developing their coding scheme. For others, investigators proceed in a deductive manner, that is, they develop their schemes first and then collect/code their data. For still a third set, a combination of inductive and deductive coding may be most appropriate. 10

Nonetheless, we believe it is possible to offer three generalizations about the process of coding variables. First, regardless of the type of data they collect, the variables they intend to code, or even of whether they plan to code inductively or deductively, at some point empirical legal researchers require a coding schema, that is, a detailing of each variable of interest, along with the values of each variable. For example, in a study of the effect of female judges on the votes of their male colleagues, the variable Vote of the Judge would obviously figure prominently; for this variable we might code three values: the judge voted to “affirm,” to “reverse,” or “other.” With this sort of information in hand, investigators can prepare codebooks—or guides they employ to code their data and that others can use to replicate, reproduce, update, or build on the variables the resulting database contains and any analyses generated from it.

Second, depending on the type of data and variables, developing schema and creating codebooks are not always easy or straightforward tasks. To see this, reconsider the seemingly simple example of the variable Vote of the Judge. We just listed three values (affirm, reverse, and other) but what of a vote “affirming in part and reversing in part”? Should we code this as “other,” even if the judge gave the plaintiff some relief? For that matter, what should we make of the “other” category? Depending on the subjects under analysis, it may be appropriate (meaning that it would be an option exercised infrequently) or not. But our more general point should not be missed: accounting for the values of the variables of interest, even of seemingly simple ones, may be tricky. 11

To be sure, following best practices can help; for example, ensuring that the values of the variables are exhaustive, creating more (rather than fewer) values, establishing that the values of the variables are mutually exclusive, and more generally, pretesting the schema (for more details, see Epstein and Martin, 2005 ). But there is one assumption that all the rules and guidelines make—and this brings us to our third point: researchers must have a strong sense of their project, particularly about the piece of the legal world they are studying and how that piece generated the data they will be coding, as well as the observable implications of the theory that they will be assessing (see, e.g., Babbie, 2007 : 384; Frankfort-Nachmias and Nachmias, 2007 ). Even adhering to simple rules will be difficult, if not impossible, if the researcher lacks a deep understanding of the objects of her study and an underlying theory about whatever feature(s) of their behavior for she wishes to account.

IV. Analyzing Data

If research design is the first overhead slide and collecting and coding data, the second, then data analysis enables researchers to compare their overlap. When the overlap between the observable implications and data is substantial, analysts may conclude that the real world confirms their hunches; if the overlap is negligible, they may go back to the drawing board or even abandon the project altogether.

How do empirical legal scholars perform this task? The answer depends in no small part on their goals. If the goal is to summarize the data they have collected (say, the salaries of all male and female professors at their school), then some simple measures of central tendency (e.g., means, medians) and dispersion (e.g., standard deviations, ranges) might suffice. These will give researchers a feel for the distributions of their variables that, depending on the number of cases, they could not possibly develop from looking at a column of data.

For the vast majority of empirical legal projects, however, making inferences—using facts we know to learn about facts we do not know—is the goal. Rarely do we care much about the, say, 50 individuals or 100 cases in our sample. Rather, we care about what those 50 individuals or 100 cases can tell us about all the employees of the corporation or all the cases. In quantitative research, inferences come in two flavors: descriptive and causal . Descriptive claims themselves can take several forms but some seem quite a kin to data summaries. Suppose, for example, that we collected data on 100 court cases involving employment discrimination and learned that, on average, appellate court panels held for the plaintiff in 40% of the cases. In and of itself this figure of 40% (a summary of the data), probably isn't all that interesting to our readers or us. What we want to learn about is the fraction of all employment discrimination cases in which all courts held for the plaintiff. That is, we want to use what we know (the 100 cases we have collected) to learn about what we do not know (the cases we haven't collected). This is the task of drawing a descriptive inference. We do not perform it by summarizing facts; we make it by using facts we know—the small part of the world we have studied—to learn about facts we do not observe (the rest of the world). Researchers call the “small part” a sample and the “world” a population. (An important part of performing descriptive inference is quantifying the uncertainty we have about that inference. We discuss this in greater detail below.) It is important to keep in mind that when dealing with data coming from a non-probability sampling neither descriptive nor causal inferences can be drawn.

Causal inference too is about using facts we do know to learn about facts we do not know. In fact, a causal inference is the difference between two descriptive inferences —the average value the dependent variable (for example, the fraction of cases decided in favor of the plaintiff ) takes on when a “treatment” is applied (for example, a female judge serves on the panel) and the average value the dependent variable takes on when a “control” is applied (for example, if no female judge sits on the panel). The causal effect —the goal of the process of causal inference—is this difference, the amount the fraction of decisions in favor of the plaintiff increases or decreases when we move from all-male panels to panels with a female.

How do quantitative empirical researchers go about making descriptive or causal claims? Assuming they have appropriately designed their projects and appropriately amassed and coded their data, they make use of statistical inference , which entails examining a small piece of the world (the sample) to learn about the entire world (the population), along with evaluating the quality of the inference they reach. Conceptually, statistical inference is not all that hard to understand; actually we confront such inferences almost every day. When we open a newspaper, we might find the results of a survey showing that 70% (± 5% margin of error) of American voters have confidence in the US president. Or when we read about a scientific study indicating that a daily dose of aspirin helps 60% (95% CI ± 3% ) of Americans with heart disease. (95% CI and ± X% are explained below.) In neither of these instances, of course, did all Americans participate. The pollsters did not survey every voter, and the scientists did not study every person with heart problems. They rather made an inference (in these examples, a descriptive inference) about all voters and all those stricken with heart disease by drawing a sample of voters and of ill people.

But how do the researchers go about making the statistical inference (for example, 70% of all American voters have confidence in the president) and assess its quality (that is, indicate how un certain they are about the 70% figure, as indicated by the ± 5% )? It is one thing to say that 70% of the voters in the sample have confidence in the president (this is summarizing or describing the data); but it is quite another to say that 70% of all voters have conﬁdence (this is the descriptive inference).

To support the first claim, all analysts need do is tally (i.e., summarize ) the responses to their survey. To support (and evaluate) the second, they must (1) draw a random probability sample of the population of interest and (2) determine how certain (or uncertain) they are that the value they observe from their sample of voters (70% ), called the sample statistic , reflects the population of voters, the population parameter .

We already have discussed (1)—drawing a random sample—so we only need reiterate here that this step is crucial. If a sample is biased (for instance, if Democrats had a better chance of being in the pollsters' sample than Republicans), researchers cannot draw accurate conclusions.

Assuming researchers draw a random probability sample, they can move to (2) and make a (descriptive) inference about how well their sample reflects the population. Or, to put it another way, they can convey their degree of uncertainty about the sample statistic. Surveys reported in the press, for example, typically convey this degree of uncertainty as “the margin of error,” which is usually a 95% confidence interval (or 95% CI). When pollsters report the results of a survey—that 70% of the respondents have confidence in the president with a ±5 margin of error—they are supplying the level of uncertainty they have about the sample statistic of 70%. That is, the true fraction of voters who have confidence in the president will be captured in the stated confidence interval in 95 out of 100 applications of the same sampling procedure. The fact that the data come from a random sample is what makes it possible to use the rules of probability to compute these margins of error.

Note that this information does not say exactly where, or whether, the population (parameter) lies within this range. (In fact, the parameter either falls within the interval or not; only an all-knowing researcher would ever know.) What is critical, however, is that if the researcher continues to draw samples from a population of voters, the mean of the samples of voters will eventually equal the mean of the population, and if the researcher creates a specialized bar graph called a histogram showing the distribution of the individual sample means, the resulting shape would resemble a normal distribution. This is what enables researchers to make an inference—here, in the form of a sample statistic and a margin of error—about how all voters (the population) feel about the president by observing a single sample statistic. For the sake of illustration, consider Figure 2 . Here we show the confidence intervals computed from 50 random samples from a population where the known parameter of interest is ten. The 95% confidence intervals are constructed to contain the true parameter 95% of the time. Here in all but two samples the horizontal confidence intervals contain the known parameter value. Of course, in any application we do not know the parameter value (if we did we would not need to perform inference!), but we use confidence intervals that over repeated samples will return the right answer a high percentage of the time.

Confidence intervals for a known population mean ten for fifty random samples from a population .

This pertains to descriptive claims but it is important to draw a statistical inference when performing causal inference as well. Suppose that the average monthly income for the male professors in our sample of employees was $4,200, while for the females it was $3,900, yielding a difference of $300 in this sample. There are two possible explanations for the $300 difference (assuming all else is constant , a phrase we explain below). It might be the case that it is due solely to the particular sample we randomly drew; in other samples from the population the difference might only be $10, or women might make, on average, $250 more than men. It is also possible that in the population, men actually earn more than women.

The process researchers use to make this determination is called hypothesis testing . A hypothesis test tells us whether differences across groups are simply an artifact of sampling (the first possible explanation), or whether meaningful differences exist in the population (the second possibility). In the latter case we would say the difference is statistically significant . All statistical significance means is that sampling alone cannot explain the observed difference, and as such, it is likely that differences exist in the population. One would conclude a relationship is statistically insignificant when the difference in the sample can be explained by sampling alone.

In addition to statistical significance, it is important to consider the substantive significance of any finding. A $1,000 per month difference in salary is certainly large; an $8 per month difference is not. Both could be statistically significant, but only the first would be substantively significant. Accordingly, it is crucial for empirical researchers to compute and report the size of the differences—in addition to reporting the results of hypothesis tests—so that the reader can ascertain whether the findings are substantively important. In the following section we recommend using graphics to report these differences.

But before turning to data displays, one final topic deserves some attention: the assumption of “all else being constant” or “all things being equal.” This assumption takes us back to a point we made at the onset; namely, when working with data generated by the world, most of the time “all else is constant” or “all things being equal” is untenable. It is quite possible, for example, that male professors in our sample do not have the same experience as females. Thus, just naïvely comparing the average salaries across the two groups would not provide a reliable causal inference.

Today, there are two approaches commonly used for making causal inferences from observational data. One type of analysis is multiple regression analysis, and related regression models (such as logistic regression). Regression models work by allowing the researcher to hold all other measured variables constant while assessing the relationship of interest. In this example, we could see whether the difference in salaries persisted by controlling for experience. Regression models have been used for decades and are the most common tool in empirical legal research. For many types of research they work quite well, but they do require some strong assumptions about the relationship between the key causal variable and the outcome variable of interest (see Imai, 2005 ).

Another set of methods called matching methods is becoming more popular in applied statistics. These cutting-edge tools are making their way into empirical legal studies (Epstein et al. 2005 ; Greiner 2008 ), and for many reasons we predict that their use will increase in the coming decades. The idea, to return to the example of pay equity, is to match most-similar male and female professors, and then compute differences between the matched observations. Once researchers have made the matches, these methods allow them to treat observational data as if it were experimental.

Regardless of whether one uses regression analysis or matching to control for alternative explanations, a causal inference is just a statistical inference about a difference. At bottom what researchers want to know is whether observed differences in a sample represent the same differences in a population.

V. The Last Step: Presenting the Results of Empirical Legal Research 12

Just as scholars have been improving methods for causal inference, they have been working on approaches to convey the results of their studies. These developments should be of particular interest to quantitative empirical legal scholars who often must communicate their findings to judges, lawyers, and policy-makers—in other words, to audiences who have little or no training in statistics. Too often, though, analysts fail to take advantage of the new developments thus missing an opportunity to speak accessibly to their community.

To see the problem, consider an example adapted from a study that seeks to explain the votes cast by U.S. senators on Supreme Court nominees (Epstein et al., 2006 ). 13 Briefly, the authors operate under the assumption that electorally minded senators vote on the basis of their constituents' “principal concerns in the nomination process” (Cameron et al., 1990 : 528). These concerns primarily (though not exclusively) center on whether a candidate for the Court is (1) qualified for office and (2) ideologically proximate to the senator (i.e., to his or her constituents). Consequently, the two key causal variables in their statistical model are (1) the degree to which a senator perceives the candidate as qualified for office and (2) the ideological distance between the senator and the candidate, such that the more qualified the nominee and the closer the nominee is to the senator on the ideological spectrum, the more likely the senator is to cast a yea vote. Also following from the extant literature, the researchers control for two other possible determinants of senators' votes: whether the president was “strong” in the sense that his party controlled the Senate and he was not in his fourth year of office; and whether a senator is of the same political party as the president.

To assess the extent to which these variables help account for senators' votes, the researchers employed logistic regression, a common tool in legal scholarship when the dependent variable is binary. Table 1 displays the results, and they seem to lend support to the researchers' hypothesis. For example, the ⋆ on the coefficient for lack of qualifications variable tells us that a statistically significant relationship exists between qualifications and voting: the lower a nominee's qualifications, the higher the likelihood that a senator will vote against the nominee.

On the other hand, tables of this sort (which run rampant in empirical legal scholarship) are not just ugly and off-putting to most readers; they communicate virtually no information of value either to the audience or even to the researchers themselves. Most lawyers, judges, and even law professors do not understand terms such as “statistical significance,” much less “logit coefficient.”

How might empirical legal scholars improve their data presentations? Adhering to three general principles would be a good start. First, we recommend that analysts communicate substance, and not only statistics. Reconsider this statement:

In looking at Table 1 , we see that the coefficient on the variable lack of qualifications of - 4.11 is “statistically significant.”

This is not wrong but the emphasis on the coefficient is more than off-putting; it fails to convey useful information. In fact, all we learn from the -4.11 coefficient on lack of qualifications is that, controlling for all other factors, as we move from the most qualified to the most unqualified nominee we move down 4.11 on a logit scale. To make matters worse, because the logit scale is nonlinear, moving down 4.11 units will result in different probabilities of a yea vote depending on where we start on the scale.

Because few of their readers would understand what any of this means, it is no wonder many empirical legal scholars simply say “the coefficient on lack of qualifications is statistically significant at the.01 level.” But this too isn't an informative statement to many readers; it isn't even informative to readers with statistical training (a very small fraction of the legal community). It tells us is that qualified candidates are more likely to receive a yea vote than unqualified candidates but not how much more likely. 0.2 times more likely? 2 times? Or perhaps even 4 times? We probably wouldn't be very impressed, for example, if all else being equal, the predicted probability of a senator voting for a very qualified candidate was 0.11 and for a very unqualified candidate was 0.14. Certainly, a quantity such as a predicted probability is what matter most to readers of empirical legal scholarship. But it is not one that they can learn from a tabular display of logit coefficients.

This is why we recommend supplying readers with a quantity of interest; that is, replace “In looking at Table 1 , we see that the coefficient on the variable lack of qualifications of - 4.11 is statistically significant” with:

Other things being equal, 14 when a nominee is perceived as highly unqualified the likelihood of a senator casting a yea vote is only about 0.24 . That probability increases to 0.92 when the nominee is highly qualified.

Statements of this sort are easy to understand even by the most statistically challenged members of the legal community.

Second, we suggest that when they perform inference, researchers convey their uncertainty. To see the point, think about the statement above—that the likelihood of a senator casting a yea vote is only about 0.24 when the candidate is unqualified. This figure of 0.24 represents the researchers' “best guess” about the likelihood of a senator voting yea based on qualifications. But we know that error or uncertainty exists around that best guess. It is simply a fact of statistical analysis that we can never be certain about our guesses because they themselves are based on estimates.

Most quantitative empirical legal scholars appreciate this fact and supply the error surrounding their estimated coefficients . Statements such as this are not uncommon:

In looking at Table 1 , the coefficient on the variable lack of qualifications (- 4.11 with a standard error of 0.22) is statistically significant at the.01 level.

True, this conveys uncertainty in the form of a standard error around the estimate but of what value is it? None, it turns out, because all the error value supplies is an estimate of the standard deviation of the estimated coefficient—which, standing alone, is of interest to no one, readers and scientists alike. 15

One possible ﬁx is for empirical legal scholars to follow other disciplines and report far-more-meaningful 95% (or even 99% ) confidence intervals rather than (or in addition to) standard errors. In the case of lack of qualifications, the values of that interval are a lower bound of - 4.54 and an upper bound of - 3.69.

This interval comes closer than the standard error to conveying useful information: the researchers' best guess about the coefficient on lack of qualifications is - 4.11 but they are “95% certain” that it is in the range of - 4.54 to - 3.69. Because 0 is not in this range (the confidence interval), the researchers and their readers can safely reject the null hypothesis of no relationship between the nominees' qualifications and senators' votes.

But even denoting the confidence interval around a coefficient would not be making the most of the model's results. When researchers say they are “95% certain” that the true logit coefficient lies between - 4.54 to - 3.69, they lose half their audience. What we recommend instead is combining the lesson here of relating uncertainty with the first principle of conveying substantive information:

Other things being equal, when a nominee is perceived as highly unqualified the likelihood of a senator casting a yea vote is only about 0.24 (±0.05). That predicted probability increases to 0.93, (±0.02) when the nominee is highly qualified.

Now readers need no specialized knowledge about standard errors or even confidence intervals to understand the results of the study—including uncertainty about the results. They can easily see that the researchers' best guess about the predicted probability of yea vote for a highly unqualified candidate is 0.24, though it could be as low as 0.19 or as high 0.29. Such accessible communication creates a win-win for empirical legal researchers and their audience: both are now in a far better position to evaluate the study's conclusions.

Our final recommendation is that analysts graph their data and results. With this, we are trying to convey two ideas. One is just a general point: if the goal is to give readers a feel for patterns or trends in the data, graphs are superior to tables—even for small amounts of data. Figure 3 provides an example from the project on Supreme Court nominees.

To be sure, if we looked at the table long enough some of the patterns we observe in the figure would emerge but it takes a lot more cognitive work on the part of the reader. Plus, it is unlikely that readers of empirical legal studies need such specific, precise information as in the table. So in most instances graphic displays can convey the right information without losing much.

The second idea, more relevant to the communication of results (rather than data, as in Figure 3 ), is that figures enable analysts to combine the first two principles we set out above (substance and uncertainty) across many values. Think about it this way: while substantive claims of the form “When a nominee is perceived as highly unqualified the likelihood of a senator casting a yea vote is only about 0.24 (±0.05)” may be informative, they exclude a lot of information—the values in between “highly unqualified” and “highly qualified.” To provide these quantities, we could generate a long series of statements such as

Other things being equal, when a nominee is perceived as highly unqualified the likelihood of a senator casting a yea vote is 0.24 (±0.05).

Other things being equal, when a nominee is perceived as about average on the qualifications scale, the likelihood of a senator casting a yea vote is 0.83 (±0.03).

Other things being equal, when a nominee is perceived as highly qualified the likelihood of a senator casting a yea vote is 0.93 (±0.02).

Tables versus Figures .

Both the table and the figure provide information on the ideological distance between Senator Edward Kennedy (D-Mass.) and ten recent Supreme Court nominees. Juxtaposed against the table, the dot plot provides a more visually and cognitively appealing solution to the problem of providing the reader with information about variables of interest.

But graphing the results is a far more parsimonious, pleasing, and, for the readers of empirical legal work, cognitively less demanding approach. Underscoring these points is Figure 4 . Here the reader gets a real sense of the (1) results and (2) uncertainty across the values of qualifications without having to sift through a long series of claims.

Even better, and usually necessary in multivariate analysis, is to bring in other variables of interest, as Figure 5 does. Here, we've juxtaposed qualifications against another variable: ideology, when senators and nominees are ideologically very close and when they are very distant. Specifically, in the two panels we show the probability of a senator casting a yea vote across the range of lack of qualifications and when we set ideological distance at its minimum and maximum levels. In both panels we depict our uncertainty, in the form of 95% confidence intervals, with vertical lines.

This display, we believe, is a good example of what we mean by parsimony. It conveys a great deal of information—actually it encodes 66 pieces of information—quite efficiently or at least more efficiently than the 66 sentences it would have taken to describe each and every result depicted in the two panels and certainly more accessibly than a table of logit coefficients.

The effect of qualifications on Senate votes over Supreme Court nominees, from Black (1937) through Alito (2006) .

The figure shows the predicted probability of a senator casting a yea vote over the range of lack of qualifications (0 is the most qualified), when we set ideological distance at its mean and the other variables in the statistical model at 0. The small vertical bars are 95% confidence intervals. Created using S-Post.

Little more than a decade ago, implementing a graph of the sort depicted in Figure 4 would have been quite the chore: estimating the confidence intervals, in particular, was not possible for most empirical legal scholars. But now, because contemporary software packages use simulations (repeated sampling of the model parameters from their sampling distribution) to produce estimates of quantities of interest (e.g., predicted probabilities), generating assessments of error (e.g., confidence intervals) is quite easy. 16 Moreover, using the software requires no additional assumptions beyond those the researcher already has made to perform statistical inference.

Once researchers have prepared their results for presentation (and, ultimately, publication), their work would seem to be done. And, for the most part it is. But we in the empirical legal community should demand that they take one final step: archive their data and documentation. So doing ensures that empirical legal scholars adhere to the replication standard : Another researcher should be able to understand, evaluate, build on, and reproduce the research without any additional information from the author (King, 1995 ). This rule does not actually require anyone to replicate the results of an article or book; it only requires that researchers provide information—in the article or book or in some other publicly available or accessible form—sufficient to replicate the results in principle.

The effect of qualifications on Senate votes over Supreme Court nominees, from Black (1937) through Alito (2006), when the ideological distance between the Senator and nominee is very close (minimum) and very distant (maximum) and all other variables in the statistical model are set at 0 .

Why is such documentation a requisite step in conducting empirical research (regardless of whether the work is qualitative or quantitative in nature)? Epstein and King 2002 supply two answers. The first centers on the ability of outsiders to evaluate the research and its conclusions. In a broad sense, the point of the replication standard is to ensure that a published work stands alone so that readers can consume what it has to offer without any necessary connection with, further information from, or beliefs about the status or reputation of the author. The replication standard keeps empirical inquiry above the level of ad hominum attacks on or unquestioning acceptance of arguments by authority figures. The second reason is straightforward enough: as this Chapter has (hopefully!) made clear, the analyst's procedures may, and in most instances do, influence the outcomes they report. Readers deserve an opportunity to evaluate the researchers' choices, not to mention their data.

Designing research, collecting and coding data, analyzing data, and presenting results represent the four chief tasks of quantitative empirical legal scholarship, and we have tried to explain some of the basics. But readers should keep in mind that mastering the four requires far more than we can possibly convey here; it requires training. That is why PhD programs in the social sciences offer (at the least) a one-semester course on each.

Reading some of the books and articles we cite below would be a good start for legal scholars wishing to learn more—but only a start. To develop a full appreciation for the research process, we strongly recommend that readers contact their local social science departments.

Babbie, E. ( 2007 ). The Practice of Social Research , (11th edn.), Belmont, CA: Thomson.

Google Scholar

Google Preview

Boyd, C.L., Epstein, L., and Martin, A.D. ( 2010 ). “ Untangling the Causal Effects of Sex and Judging, ” American Journal of Political Science 54: 389–411.

Cameron, C.M., Cover, A.D., and Segal, J.A. ( 1990 ). “ Senate Voting on Supreme Court Nominees: A Neo-Institutional Model, ” American Political Science Review 85: 525–34.

Epstein, L. and King, G. ( 2002 ). “ The Rules of Inference, ” University of Chicago Law Review 69: 191–209.

Epstein, L. and Martin, A.D. ( 2005 ). “Coding Variables,” in K. Kempf-Leonard (ed.), The Encyclopedia of Social Measurement , San Diego: Academic Press.

Epstein, L., Ho, D.E., King, G., and Segal, J.A. ( 2005 ). “ The Supreme Court During Crisis, ” NYU Law Review 80: 1–116.

Epstein, L., Lindstadt, R., Segal, J.A., and Westerland, C. ( 2006 ). “ The Changing Dynamics of Senate Voting on Supreme Court Nominees, ” Journal of Politics 68: 296–307.

Epstein, L., Martin, A.D., and Schneider, M. ( 2006 ). “ On the Effective Communication of the Results of Empirical Studies, Part I, ” Vanderbilt Law Review 59: 1811–71.

Epstein, L., Martin, A.D., and Boyd, C. ( 2007 ). “ On the Effective Communication of the Results of Empirical Studies, Part II, ” Vanderbilt Law Review 60: 798–846.

Frankfort-Nachmias, C. and Nachmias, D. ( 2007 ). Research Methods in the Social Sciences , New York: Worth.

Gelman, A. et al. ( 2002 ). “ Let's Practice What We Preach: Turning Tables into Graphs, ” The American Statistician 56: 121.

Greiner, D.J. ( 2008 ). “ Causal Inference in Civil Rights Litigation, ” Harvard Law Review 122: 533.

Ho, D.E. et al. ( 2007 ). “ Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference, ” Political Analysis 15: 199.

Holland, P.W. ( 1986 ). “ Statistics and Causal Inference, ” Journal of American Statistical Association 81: 945–70.

Imai, K. ( 2005 ). “ Do Get-Out-The-Vote Calls Reduce Turnout: The Importance of Statistical Methods for Field Experiments, ” American Political Science Review 99: 283–300.

King, G. ( 1995 ). “ Replication, Replication, Replication, ” PS: Political Science and Politics 28: 443–99.

King, G., Keohane, R.O., and Verba, S. ( 1994 ). Designing Social Inquiry , Princeton University Press.

King, G., Tomz, M., and Wittenberg, J. ( 2000 ). “ Making the Most of Statistical Analyses, ” American Journal of Political Science 44: 50.

Neyman, J. ( 1935 ). “ Statistical Problems in Agricultural Experimentation, ” Journal of the Royal Statistical Society 2: 107–154.

Rachlinski, J.A., Guthrie, C., and Wistrich, A.J. ( 2006 ). “ Inside the Bankruptcy Judge's Mind, ” Boston University Law Review 86: 1227–65.

Rubin, D.B. ( 1973 ). “ Matching to Remove Bias in Observational Studies, ” Biometrics 29: 159–83.

Rubin, D.B. ( 1974 ). “ Estimating Causal Effects of Treatments in Randomized and Nonrandomized Studies, ” Journal of Educational Psychology 6: 688–701.

For research support, we thank the National Science Foundation, Northwestern University School of Law, and the Center for Empirical Research in the Law at Washington University. For their very helpful comments, we thank the editors of this volume. We adapt some of the material in this Chapter from Epstein and King 2002 ; Epstein and Martin 2005 ; Epstein, Martin, and Boyd (2007) ; Epstein, Martin, and Schneider (2006) ; and Epstein and Martin's annual workshop, Conducting Empirical Legal Research .

Though it should be obvious, for this hypothetical we are assuming that the employer is assigning wages intentionally, not randomly.

For a more formal accounting of this type of analysis, many scholars have adopted a potential outcomes framework—posited by Neyman 1935 and Rubin ( 1973 , 1974 ), thoroughly reviewed in Holland 1986 , and recently applied in the social sciences by Imai 2005 , Epstein et al. 2005 , and Boyd, et al. (2010) .

Their research tested for other biases as well, including anchoring and framing.

These are indeed the key components, and in the Sections to follow we describe them in order, from designing research to conducting analyses. Nonetheless, empirical legal scholars rarely regard their research as following a singular, mechanical process from which they can never deviate. Quite the opposite: scholars must have the flexibility of mind to overturn old ways of looking at the world, to ask new questions, to revise their blueprints as necessary, and to collect more (or different) data than they might have intended. On the other hand, being flexible does not mean that researchers do or should do ad or post hoc adjustment of theories to fit idiosyncrasies. Adjustments made to harmonize theory with data, of course, do not constitute any confirmation of the theory at all. While it is fine to use data to create theory, investigators know they must consult a brand new data set, or completely different and previously unanticipated testable consequences of the theory in the same data set, before concluding that data confirm their theory. For more on the idea of research as a “dynamic process conforming to fixed standards,” see Epstein and King 2001 .

How to ensure a good fit? We turn to this question when we tackle the subject of measurement.

Some might argue that these steps are unnecessary in research motivated purely by policy concerns. Not so. Because the statistical methods we describe momentarily are designed to test hypotheses, the researcher should, well, develop some hypotheses to test.

For advice on small- n studies, see Epstein and King 2002 : 112–13); King et al. 1994 : 124–8).

Dealing with data collected on a population raises some foundational statistical issues. One approach is to argue that an observed population is a “sample” from possible histories, and as such, traditional inferential statistics can be used. Another option is to simply summarize the data and not report measures of uncertainty. The ideal approach, from our perspective, is to adopt a Bayesian approach and treat the parameters as random variables, not the data.

Some writers associate inductive coding with research that primarily relies on qualitative data and deductive coding, with quantitative research. Given the [typically] dynamic nature of the processes of collecting data and coding, however, these associations do not always or perhaps even usually hold. Indeed, it is probably the case that most researchers, regardless of whether their data are qualitative or quantitative, invoke some combination of deductive and inductive coding.

More generally, the relative ease (or difficulty) of the coding task varies according to the types of data with which the researcher is working, the level of detail for which the coding scheme calls, and the amount of pretesting the analyst has conducted.

We draw material in this section from Epstein et al. (2007) ; Epstein et al. (2006) ; Gelman, et al. (2002) ; King et al. (2000) .

Since publication of their study, Epstein et al. have updated their dataset (available at: 〈 epstein.law.northwestern.edu/research/Bork.html 〉). We rely on the updated data.

We use the term “other things being equal” to signify that all variables in the model (other than the variable interest, here qualifications) are fixed at particular values. In this example, we set ideological distance at its mean and strong president and same party at 0.

Its value, rather, lies in computing confidence intervals.

King et al.'s (2000) Clarify is an example. It uses the Monte Carlo algorithm for the simulations, and can be implemented via the Clarify plug-in for Stata.

About Oxford Academic
Publish journals with us
University press partners
What we publish
New features
Open access
Institutional account management
Rights and permissions
Get help with access
Accessibility
Advertising
Media enquiries
Oxford University Press
Oxford Languages
University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

Cookie settings
Cookie policy
Privacy policy
Legal notice

This Feature Is Available To Subscribers Only

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

IMAGES

(PDF) Chapter 38 Qualitative Approaches to Empirical Legal Research
Which qualitative research approach should you select and apply in your
Methodological approaches in qualitative research.
6 Types of Qualitative Research Methods
Understanding Qualitative Research: An In-Depth Study Guide
What Is Empirical Research? Definition, Types & Samples

VIDEO

Quantitative v Qualitative Data for Legal Research 009
THE PROCESS OF CONDUCTING RESEARCH USING QUANTITATIVE AND QUALITATIVE APPROACHES
Group 1 (The process of Conducting Research Using Quantitative and Qualitative Approaches)
Model-Based or Design-Based? Methodological Approaches in Empirical Micro
Understanding the Case Study Approach in Qualitative Research
Empirical Legal Studies in Law and Psychology:| Class 6

COMMENTS

39 Qualitative Approaches to Empirical Legal Research
This article deals with the qualitative approach to empirical studies. This approach is presumed to be closer to the social sciences. Data collection in the qualitative approach follows a combination of these three methods—direct observations, in-depth interviews, and document analysis. It typically starts with the identification of methodology, data collection, analysis, ethical concerns ...
Qualitative approaches to empirical legal research
Data collection in the qualitative approach follows a combination of these three methods—direct observations, in-depth interviews, and document analysis. It typically starts with the identification of methodology, data collection, analysis, ethical concerns, and adapt to the dynamics if working in a team. Well-compiled qualitative research ...
The Oxford Handbook of Empirical Legal Research
The phrase "empirical legal research" in the title, The Oxford Handbook of Empirical Legal Research, is designed both to reflect and to celebrate the healthy pluralism of empirical approaches to the study of law and legal phenomena. Keywords: empirical legal studies, empirical investigation, legal systems, ELS movement, policing.
12 Qualitative Legal Research: A Methodological Discourse
The combination of two methods brings synergy. Worthy features of qualitative legal research consist of description of social setting, interpretation of social data, verification of assumptions, and evaluation of policies.
PDF An Introduction To Empirical Legal Research
Advanced Introduction to Empirical Legal Research Herbert M. Kritzer,2021-02-26 Herbert Kritzer presents a clear introduction to the history, methods and substance of empirical legal research (ELR). Quantitative methods dominate in empirical legal research, but an important segment of the field draws on qualitative methods, such as semi-structured interviews and observation. In this book both ...
Qualitative approaches to empirical legal research
Qualitative approaches to empirical legal research. Lisa Webley - 2010 - In Peter Cane & Herbert M. Kritzer (eds.), The Oxford Handbook of Empirical Legal Research. Oxford University Press.
The Oxford Handbook of Empirical Legal Research
In the 43 chapters of The Oxford Handbook of Empirical Legal Research leading scholars provide accessible and original discussions of the history, aims and methods of empirical research about law, as well as its achievements and potential. The Handbook has three parts.
Qualitative Approaches to Empirical Legal Research
This article focuses on the methodological lessons learned while conducting a legal research study of the lower criminal courts by gathering observational and interview data to understand why many…
Qualitative approaches to empirical legal research
Qualitative approaches to empirical legal research. Webley, L. 2010. Qualitative approaches to empirical legal research. in: Cane, P. and Kritzer, H. (ed.) Oxford handbook of empirical legal research Oxford Oxford University Press. pp. 926-950. Chapter title. Qualitative approaches to empirical legal research.
Qualitative approaches to empirical legal research
Webley, L. (2010). Qualitative approaches to empirical legal research. In P. Cane, & H. Kritzer (Eds.), The Oxford Handbook of Empirical Legal Research (pp. 926-948 ...
The Oxford handbook of empirical legal research
/ Tom Ginsburg and Gregory Shaffer -- Lawyers and other legal service providers / Richard Moorhead -- Legal pluralism / Margaret Davies -- Public images and understandings of courts / James L. Gibson -- Legal education and the legal academy / Fiona Cownie -- The (nearly) forgotten early empirical legal research / Herbert M. Kritzer ...
Chapter 38 Qualitative Approaches to Empirical Legal Research
Abstract. This article deals with the qualitative approach to empirical studies. This approach is presumed to be closer to the social sciences. Data collection in the qualitative approach follows ...
Empirical research on law and society.Advanced Introduction to
Investigating the relationship between law and society is often done by means of empirical legal research, which can be defined as the systematic collection of quantitative and/or qualitative data based on observations of what is going on in the legal world. 1 Thus, rather than focusing on law in the books, empirical legal research focuses on law in action, often by means of research methods ...
Harvard Empirical Legal Studies Series
The Harvard Empirical Legal Studies (HELS) Series explores a range of empirical methods, both qualitative and quantitative, and their application in legal scholarship in different areas of the law. It is a platform for engaging with current empirical research, hearing from leading scholars working in a variety of fields, and developing ideas and empirical projects.
Introduction
The phrase "empirical legal research" in the title, The Oxford Handbook of Empirical Legal Research, is designed both to reflect and to celebrate the healthy pluralism of empirical approaches to the study of law and legal phenomena. American legal realists were, perhaps, the first to appreciate the value and importance of, and to promote ...
Website: Empirical Legal Studies: Learning About Research Methods
Introduction to empirical legal studies--an approach to studying and law and legal institutions that draws on social science methods.
PDF Peter Cane, Herbert M. Kritzer. ISBN 978-0-19-954247-5 (alk. paper
The (Nearly) Forgotten Early Empirical Legal Research Erbert M. Kritzer Quantitative Approaches to Empirical Legal Research Lee Epstein and Andrew D. Martin Qualitative Approaches to Empirical Legal Research Lisa Webley The Need for MultiMethod Approaches in Empirical Legal Research Laura Beth Nielsen
Qualitative Approaches to Empirical Legal Research
Abstract: This research considers the EU's 21st century objective of mitigating climate change by promoting renewable electricity and the multiple legal conflicts between this objective and EU's core legal principles of free movement, the prohibition of distortion of competition and other forms of state aid.
Balancing Different Legal and Ethical Requirements in the ...
International research collaborations engage multiple countries, researchers, and universities. This enhances the magnitude of contextual challenges, including legal and ethical dimensions across various jurisdictions, that must be bridged in qualitative research regardless of discipline, also in the construction of informed consents. From a Scandinavian perspective, this discussion paper ...
Empirical Legal Research: Nature, Features, and Expanding Horizons
This chapter discusses the use and merits of empirical legal research as an evidence-based method of conducting research that systematically carries out the task of unearthing, analysing, and interpreting facts in relation to law and its functioning. This method, presupposing that scientific device, rather than tradition, shall determine the establishment and functioning of legal postulates ...
Exploring Approaches to Legal Research, Berlin 2024
Professor Dr Andreas Engert of FU Berlin discussed empirical legal research, beginning with a contrasts between quantitative and qualitative methods. Engert highlighted the objectivity aimed for in quantitative analysis while acknowledging the challenges of affordability and data availability, as well as the potential pitfalls of over-reliance ...
Chapter 38 Qualitative Approaches to Empirical Legal Research
The second - and largest - part consists of critical accounts of empirical research on many aspects of the legal world - on criminal law, civil law, public law, regulatory law and international law; on lawyers, judicial institutions, legal procedures and evidence; and on legal pluralism and the public understanding of law.
PDF Chapter 38: Qualitative Approaches to Empirical Legal Research
Yin describes a case study as 'an empirical study that investigates a contemporary phenomenon within its real-life context, especially when the boundaries between the phenomenon and context are ...
Nurse‐sensitive quality and benchmarking in hospitals striving for
Aim To examine if and how selected German hospitals use nurse-sensitive clinical indicators and perspectives on national/international benchmarking. Design Qualitative study. Methods In 2020, 18...
42 Empirical Legal Research and Policy-making
Empirical legal research (ELR) seeks to understand and explain how law works in the real world. Empirical research on law has become a recognized part of the social science research environment and the results of empirical research are central to an academic analysis of law. This article begins by offering some reflections on the relationship between research and government. It considers ...
Buildings
To address this knowledge gap, the current research aims to examine the current applications of LOS approaches in housing delivery in the UK through empirical and qualitative research methods.
38 Quantitative Approaches to Empirical Legal Research
This article deals with the objective nuances of empirical research, within the ambit of the quantitative kind. It begins with an overview of conducting empirical legal research, discussing its research design, implementation, and challenges faced. Theorizing in empirical legal scholarship comes in different forms: in some projects theories seek to provide insight into a wide range of ...

Account Options

Selected pages

Other editions - View all

Qualitative approaches to empirical legal research

Related outputs

Usage statistics

Qualitative approaches to empirical legal research

Publication series

Internet Archive Audio

Mobile Apps

Browser Extensions

Save Page Now

The Oxford handbook of empirical legal research

plus-circle Add Review comment Reviews

DOWNLOAD OPTIONS

IN COLLECTIONS

SIMILAR ITEMS (based on metadata)

Harvard Empirical Legal Studies Series

Spring 2024 Sessions

Historical Approaches to Neoliberal Legality

Fall 2023 Sessions

Spring 2023 Sessions

Fall 2022 Sessions

How do People Learn from Not Being Caught? An Experimental Investigation of a “Non-Occurrence Bias”

Empirical Legal Research: Using Data and Methodology to Craft a Research Agenda

Resources for Empirical Research

Past HELS Sessions

Modal Gallery

Empirical Legal Studies

Balancing Different Legal and Ethical Requirements in the Construction of Informed Consents in Qualitative International Collaborative Research Across Continents - Reflections from a Scandinavian Perspective

Cite this article

Access this article

Similar content being viewed by others

Ethical Dilemmas in Cross-national Qualitative Research: A Reflection on Personal Experiences of Ethics from a Doctoral Research Project

Compliance with research ethics in epidemiological studies targeted to conflict-affected areas in Western Ethiopia: validity of informed consent (VIC) by information comprehension and voluntariness (ICV)

Acknowledgements

Author information

Contributions

Corresponding author

Ethics declarations

Additional information

Rights and permissions

About this article

Share this article

Exploring Approaches to Legal Research, Berlin 2024

Chapter 38 Qualitative Approaches to Empirical Legal Research

Related Papers

RELATED PAPERS

38 Quantitative Approaches to Empirical Legal Research

I. Conducting Empirical Legal Research: An Overview

II. Designing Research

III Collecting Data and Coding Variables

IV. Analyzing Data

V. The Last Step: Presenting the Results of Empirical Legal Research 12

This Feature Is Available To Subscribers Only

IMAGES

VIDEO

COMMENTS