social work research design

Social Work Research Methods That Drive the Practice

Social workers advocate for the well-being of individuals, families and communities. But how do social workers know what interventions are needed to help an individual? How do they assess whether a treatment plan is working? What do social workers use to write evidence-based policy?

Social work involves research-informed practice and practice-informed research. At every level, social workers need to know objective facts about the populations they serve, the efficacy of their interventions and the likelihood that their policies will improve lives. A variety of social work research methods make that possible.

Data-Driven Work

Data is a collection of facts used for reference and analysis. In a field as broad as social work, data comes in many forms.

Quantitative vs. Qualitative

As with any research, social work research involves both quantitative and qualitative studies.

Quantitative Research

Answers to questions like these can help social workers know about the populations they serve — or hope to serve in the future.

• How many students currently receive reduced-price school lunches in the local school district?
• How many hours per week does a specific individual consume digital media?
• How frequently did community members access a specific medical service last year?

Quantitative data — facts that can be measured and expressed numerically — are crucial for social work.

Quantitative research has advantages for social scientists. Such research can be more generalizable to large populations, as it uses specific sampling methods and lends itself to large datasets. It can provide important descriptive statistics about a specific population. Furthermore, by operationalizing variables, it can help social workers easily compare similar datasets with one another.

Qualitative Research

Qualitative data — facts that cannot be measured or expressed in terms of mere numbers or counts — offer rich insights into individuals, groups and societies. It can be collected via interviews and observations.

• What attitudes do students have toward the reduced-price school lunch program?
• What strategies do individuals use to moderate their weekly digital media consumption?
• What factors made community members more or less likely to access a specific medical service last year?

Qualitative research can thereby provide a textured view of social contexts and systems that may not have been possible with quantitative methods. Plus, it may even suggest new lines of inquiry for social work research.

Mixed Methods Research

Combining quantitative and qualitative methods into a single study is known as mixed methods research. This form of research has gained popularity in the study of social sciences, according to a 2019 report in the academic journal Theory and Society. Since quantitative and qualitative methods answer different questions, merging them into a single study can balance the limitations of each and potentially produce more in-depth findings.

However, mixed methods research is not without its drawbacks. Combining research methods increases the complexity of a study and generally requires a higher level of expertise to collect, analyze and interpret the data. It also requires a greater level of effort, time and often money.

The Importance of Research Design

Data-driven practice plays an essential role in social work. Unlike philanthropists and altruistic volunteers, social workers are obligated to operate from a scientific knowledge base.

To know whether their programs are effective, social workers must conduct research to determine results, aggregate those results into comprehensible data, analyze and interpret their findings, and use evidence to justify next steps.

Employing the proper design ensures that any evidence obtained during research enables social workers to reliably answer their research questions.

Research Methods in Social Work

The various social work research methods have specific benefits and limitations determined by context. Common research methods include surveys, program evaluations, needs assessments, randomized controlled trials, descriptive studies and single-system designs.

Surveys involve a hypothesis and a series of questions in order to test that hypothesis. Social work researchers will send out a survey, receive responses, aggregate the results, analyze the data, and form conclusions based on trends.

Surveys are one of the most common research methods social workers use — and for good reason. They tend to be relatively simple and are usually affordable. However, surveys generally require large participant groups, and self-reports from survey respondents are not always reliable.

Program Evaluations

Social workers ally with all sorts of programs: after-school programs, government initiatives, nonprofit projects and private programs, for example.

Crucially, social workers must evaluate a program’s effectiveness in order to determine whether the program is meeting its goals and what improvements can be made to better serve the program’s target population.

Evidence-based programming helps everyone save money and time, and comparing programs with one another can help social workers make decisions about how to structure new initiatives. Evaluating programs becomes complicated, however, when programs have multiple goal metrics, some of which may be vague or difficult to assess (e.g., “we aim to promote the well-being of our community”).

Needs Assessments

Social workers use needs assessments to identify services and necessities that a population lacks access to.

Common social work populations that researchers may perform needs assessments on include:

• People in a specific income group
• Everyone in a specific geographic region
• A specific ethnic group
• People in a specific age group

In the field, a social worker may use a combination of methods (e.g., surveys and descriptive studies) to learn more about a specific population or program. Social workers look for gaps between the actual context and a population’s or individual’s “wants” or desires.

For example, a social worker could conduct a needs assessment with an individual with cancer trying to navigate the complex medical-industrial system. The social worker may ask the client questions about the number of hours they spend scheduling doctor’s appointments, commuting and managing their many medications. After learning more about the specific client needs, the social worker can identify opportunities for improvements in an updated care plan.

In policy and program development, social workers conduct needs assessments to determine where and how to effect change on a much larger scale. Integral to social work at all levels, needs assessments reveal crucial information about a population’s needs to researchers, policymakers and other stakeholders. Needs assessments may fall short, however, in revealing the root causes of those needs (e.g., structural racism).

Randomized Controlled Trials

Randomized controlled trials are studies in which a randomly selected group is subjected to a variable (e.g., a specific stimulus or treatment) and a control group is not. Social workers then measure and compare the results of the randomized group with the control group in order to glean insights about the effectiveness of a particular intervention or treatment.

Randomized controlled trials are easily reproducible and highly measurable. They’re useful when results are easily quantifiable. However, this method is less helpful when results are not easily quantifiable (i.e., when rich data such as narratives and on-the-ground observations are needed).

Descriptive Studies

Descriptive studies immerse the researcher in another context or culture to study specific participant practices or ways of living. Descriptive studies, including descriptive ethnographic studies, may overlap with and include other research methods:

• Informant interviews
• Census data
• Observation

By using descriptive studies, researchers may glean a richer, deeper understanding of a nuanced culture or group on-site. The main limitations of this research method are that it tends to be time-consuming and expensive.

Single-System Designs

Unlike most medical studies, which involve testing a drug or treatment on two groups — an experimental group that receives the drug/treatment and a control group that does not — single-system designs allow researchers to study just one group (e.g., an individual or family).

Single-system designs typically entail studying a single group over a long period of time and may involve assessing the group’s response to multiple variables.

For example, consider a study on how media consumption affects a person’s mood. One way to test a hypothesis that consuming media correlates with low mood would be to observe two groups: a control group (no media) and an experimental group (two hours of media per day). When employing a single-system design, however, researchers would observe a single participant as they watch two hours of media per day for one week and then four hours per day of media the next week.

These designs allow researchers to test multiple variables over a longer period of time. However, similar to descriptive studies, single-system designs can be fairly time-consuming and costly.

Learn More About Social Work Research Methods

Social workers have the opportunity to improve the social environment by advocating for the vulnerable — including children, older adults and people with disabilities — and facilitating and developing resources and programs.

Learn more about how you can earn your  Master of Social Work online at Virginia Commonwealth University . The highest-ranking school of social work in Virginia, VCU has a wide range of courses online. That means students can earn their degrees with the flexibility of learning at home. Learn more about how you can take your career in social work further with VCU.

From M.S.W. to LCSW: Understanding Your Career Path as a Social Worker

How Palliative Care Social Workers Support Patients With Terminal Illnesses

How to Become a Social Worker in Health Care

Gov.uk, Mixed Methods Study

MVS Open Press, Foundations of Social Work Research

Open Social Work Education, Scientific Inquiry in Social Work

Open Social Work, Graduate Research Methods in Social Work: A Project-Based Approach

Routledge, Research for Social Workers: An Introduction to Methods

SAGE Publications, Research Methods for Social Work: A Problem-Based Approach

Theory and Society, Mixed Methods Research: What It Is and What It Could Be

READY TO GET STARTED WITH OUR ONLINE M.S.W. PROGRAM FORMAT?

Want to learn more about the program and application process? Get in touch with the form below.

Bachelor’s degree is required to attend.

Chapter 5 Research Design

Research design is a comprehensive plan for data collection in an empirical research project. It is a “blueprint” for empirical research aimed at answering specific research questions or testing specific hypotheses, and must specify at least three processes: (1) the data collection process, (2) the instrument development process, and (3) the sampling process. The instrument development and sampling processes are described in next two chapters, and the data collection process (which is often loosely called “research design”) is introduced in this chapter and is described in further detail in Chapters 9-12.

Broadly speaking, data collection methods can be broadly grouped into two categories: positivist and interpretive. Positivist methods , such as laboratory experiments and survey research, are aimed at theory (or hypotheses) testing, while interpretive methods, such as action research and ethnography, are aimed at theory building. Positivist methods employ a deductive approach to research, starting with a theory and testing theoretical postulates using empirical data. In contrast, interpretive methods employ an inductive approach that starts with data and tries to derive a theory about the phenomenon of interest from the observed data. Often times, these methods are incorrectly equated with quantitative and qualitative research. Quantitative and qualitative methods refers to the type of data being collected (quantitative data involve numeric scores, metrics, and so on, while qualitative data includes interviews, observations, and so forth) and analyzed (i.e., using quantitative techniques such as regression or qualitative techniques such as coding). Positivist research uses predominantly quantitative data, but can also use qualitative data. Interpretive research relies heavily on qualitative data, but can sometimes benefit from including quantitative data as well. Sometimes, joint use of qualitative and quantitative data may help generate unique insight into a complex social phenomenon that are not available from either types of data alone, and hence, mixed-mode designs that combine qualitative and quantitative data are often highly desirable.

Key Attributes of a Research Design

The quality of research designs can be defined in terms of four key design attributes: internal validity, external validity, construct validity, and statistical conclusion validity.

Internal validity , also called causality, examines whether the observed change in a dependent variable is indeed caused by a corresponding change in hypothesized independent variable, and not by variables extraneous to the research context. Causality requires three conditions: (1) covariation of cause and effect (i.e., if cause happens, then effect also happens; and if cause does not happen, effect does not happen), (2) temporal precedence: cause must precede effect in time, (3) no plausible alternative explanation (or spurious correlation). Certain research designs, such as laboratory experiments, are strong in internal validity by virtue of their ability to manipulate the independent variable (cause) via a treatment and observe the effect (dependent variable) of that treatment after a certain point in time, while controlling for the effects of extraneous variables. Other designs, such as field surveys, are poor in internal validity because of their inability to manipulate the independent variable (cause), and because cause and effect are measured at the same point in time which defeats temporal precedence making it equally likely that the expected effect might have influenced the expected cause rather than the reverse. Although higher in internal validity compared to other methods, laboratory experiments are, by no means, immune to threats of internal validity, and are susceptible to history, testing, instrumentation, regression, and other threats that are discussed later in the chapter on experimental designs. Nonetheless, different research designs vary considerably in their respective level of internal validity.

External validity or generalizability refers to whether the observed associations can be generalized from the sample to the population (population validity), or to other people, organizations, contexts, or time (ecological validity). For instance, can results drawn from a sample of financial firms in the United States be generalized to the population of financial firms (population validity) or to other firms within the United States (ecological validity)? Survey research, where data is sourced from a wide variety of individuals, firms, or other units of analysis, tends to have broader generalizability than laboratory experiments where artificially contrived treatments and strong control over extraneous variables render the findings less generalizable to real-life settings where treatments and extraneous variables cannot be controlled. The variation in internal and external validity for a wide range of research designs are shown in Figure 5.1.

Research Design in Social Work Qualitative and Quantitative Methods

• Anne Campbell - Queen's University Belfast, UK
• Brian J. Taylor - University of Ulster, UK
• Anne McGlade - The Health and Social Care Board for Northern Ireland
• Description

More than just another research text, this book remains grounded in social work practice and has clear links to the Professional Capabilities Framework for Social Work.

See what’s new to this edition by selecting the Features tab on this page. Should you need additional information or have questions regarding the HEOA information provided for this title, including what is new to this edition, please email [email protected] . Please include your name, contact information, and the name of the title for which you would like more information. For information on the HEOA, please go to http://ed.gov/policy/highered/leg/hea08/index.html .

For assistance with your order: Please email us at [email protected] or connect with your SAGE representative.

SAGE 2455 Teller Road Thousand Oaks, CA 91320 www.sagepub.com

This is an easy to read comprehensive introduction to social science research methods. The textbook makes specific connections to social work and provides clear explanations of research concepts.

Coverage of qualitative, quantitative and mixed methods, which is very welcome in a social work text.

A very practical ready reference for students and practitioners.

Preview this book

For instructors, select a purchasing option, related products.

This title is also available on SAGE Knowledge , the ultimate social sciences online library. If your library doesn’t have access, ask your librarian to start a trial .

• USC Libraries
• Research Guides
• SOWK 546: The Science of Social Work
• Research Designs and Methods

SOWK 546: The Science of Social Work: Research Designs and Methods

• Finding Scholarly Articles
• Developing a Search Strategy
• Evaluating Research Effectively
• Appraising Research
• Evidence-based Practice Resources
• Understanding Journal Impact Factors
• Policy and Legislation Resources
• Demographics, Data & Statistics for Social Work
• Tests & Measurements
• Background on Theories/Reference Sources
• APA 7th Edition

Research Methods Map

Explore the methods map below from SAGE Research Methods online to learn more about various research methods and find definitions of research terms. Click on the image of the map to interact with the map online. 

• Research Design and Design Notation Guide

Resources for Research Methods

Use the resources below to get more background and information on various research designs and methods. 

Independent and Dependent Variables

The following information and examples are from the Encyclopedia of Research Design cited and linked above:

Independent Variables and Dependent Variables

In research design, independent variables are those that a researcher can manipulate, whereas dependent variables are the responses to the effects of independent variables (Salkind, 2010). 

Independent variables are predetermined by researchers before an experiment is started. They are carefully controlled in controlled experiments or selected in observational studies (i.e., they are manipulated by the researcher according to the purpose of a study).

The dependent variable is the effect to be observed and is the primary interest of the study (Salkind, 2010).

Consider a study on the relationship between physical inactivity and obesity in young children: The parameter(s) that measures physical inactivity, such as the hours spent on watching television and playing video games, and the means of transportation to and from daycares/schools is the independent variable. These are chosen by the researcher based on his or her preliminary research or on other reports in literature on the same subject prior to the study. The parameter(s) that measure obesity, such as the body mass index, is (are) the dependent variable (Salkind, 2010)

*Salkind, N. J. (2010).  Encyclopedia of research design.  Thousand Oaks, CA: SAGE Publications, Inc. doi: 10.4135/9781412961288

Internal and External Validity

Types of validity , internal validity .

• refers to the accuracy of statements made about the causal relationship between two variables, namely, the manipulated (treatment or independent) variable and the measured variable (dependent)
• internal validity claims are based on the procedures and operations used to conduct a research study, including the choice of design and measurement of variables.

*From Salkind, N. J. (2010).  Encyclopedia of research design.  Thousand Oaks, CA: SAGE Publications, Inc. doi: 10.4135/9781412961288

External Validity 

• refers to the degree to which the relations among variables observed in one sample of observations in one population will hold for other samples of observations within the same population or in other populations. i.e. how general are your results?

*From Frey, B. (2018).  The SAGE encyclopedia of educational research, measurement, and evaluation  (Vols. 1-4). Thousand Oaks,, CA: SAGE Publications, Inc. doi: 10.4135/9781506326139

Quick guide available from USC School of Social Work:  Threats to Internal Validity quick guide  

• << Previous: Developing a Search Strategy
• Next: Evaluating Research Effectively >>
• Last Updated: Apr 11, 2024 10:03 AM
• URL: https://libguides.usc.edu/sowk546

• Architecture and Design
• Asian and Pacific Studies
• Business and Economics
• Classical and Ancient Near Eastern Studies
• Computer Sciences
• Cultural Studies
• Engineering
• General Interest
• Geosciences
• Industrial Chemistry
• Islamic and Middle Eastern Studies
• Jewish Studies
• Library and Information Science, Book Studies
• Life Sciences
• Linguistics and Semiotics
• Literary Studies
• Materials Sciences
• Mathematics
• Social Sciences
• Sports and Recreation
• Theology and Religion
• Publish your article
• The role of authors
• Promoting your article
• Abstracting & indexing
• Publishing Ethics
• Why publish with De Gruyter
• How to publish with De Gruyter
• Our book series
• Our subject areas
• Your digital product at De Gruyter
• Contribute to our reference works
• Product information
• Tools & resources
• Product Information
• Promotional Materials
• Orders and Inquiries
• FAQ for Library Suppliers and Book Sellers
• Repository Policy
• Free access policy
• Open Access agreements
• Database portals
• For Authors
• Customer service
• People + Culture
• Journal Management
• How to join us
• Working at De Gruyter
• Mission & Vision
• De Gruyter Foundation
• De Gruyter Ebound
• Our Responsibility
• Partner publishers

Your purchase has been completed. Your documents are now available to view.

Research Design for Social Work and the Human Services

• Jeane Anastas
• X / Twitter

Please login or register with De Gruyter to order this product.

• Language: English
• Publisher: Columbia University Press
• Copyright year: 2000
• Edition: second edition
• Audience: Professional and scholarly;
• Main content: 608
• Published: January 28, 2000
• ISBN: 9780231529280

Organizing Your Social Sciences Research Paper: Types of Research Designs

• Purpose of Guide
• Writing a Research Proposal
• Design Flaws to Avoid
• Independent and Dependent Variables
• Narrowing a Topic Idea
• Broadening a Topic Idea
• The Research Problem/Question
• Academic Writing Style
• Choosing a Title
• Making an Outline
• Paragraph Development
• The C.A.R.S. Model
• Background Information
• Theoretical Framework
• Citation Tracking
• Evaluating Sources
• Reading Research Effectively
• Primary Sources
• Secondary Sources
• What Is Scholarly vs. Popular?
• Is it Peer-Reviewed?
• Qualitative Methods
• Quantitative Methods
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism [linked guide]
• Annotated Bibliography
• Grading Someone Else's Paper

Introduction

Before beginning your paper, you need to decide how you plan to design the study .

The research design refers to the overall strategy that you choose to integrate the different components of the study in a coherent and logical way, thereby, ensuring you will effectively address the research problem; it constitutes the blueprint for the collection, measurement, and analysis of data. Note that your research problem determines the type of design you should use, not the other way around!

De Vaus, D. A. Research Design in Social Research . London: SAGE, 2001; Trochim, William M.K. Research Methods Knowledge Base . 2006.

General Structure and Writing Style

The function of a research design is to ensure that the evidence obtained enables you to effectively address the research problem logically and as unambiguously as possible . In social sciences research, obtaining information relevant to the research problem generally entails specifying the type of evidence needed to test a theory, to evaluate a program, or to accurately describe and assess meaning related to an observable phenomenon.

With this in mind, a common mistake made by researchers is that they begin their investigations far too early, before they have thought critically about what information is required to address the research problem. Without attending to these design issues beforehand, the overall research problem will not be adequately addressed and any conclusions drawn will run the risk of being weak and unconvincing. As a consequence, the overall validity of the study will be undermined.

The length and complexity of describing research designs in your paper can vary considerably, but any well-developed design will achieve the following :

• Identify the research problem clearly and justify its selection, particularly in relation to any valid alternative designs that could have been used,
• Review and synthesize previously published literature associated with the research problem,
• Clearly and explicitly specify hypotheses [i.e., research questions] central to the problem,
• Effectively describe the data which will be necessary for an adequate testing of the hypotheses and explain how such data will be obtained, and
• Describe the methods of analysis to be applied to the data in determining whether or not the hypotheses are true or false.

The research design is usually incorporated into the introduction and varies in length depending on the type of design you are using. However, you can get a sense of what to do by reviewing the literature of studies that have utilized the same research design. This can provide an outline to follow for your own paper.

NOTE : Use the SAGE Research Methods Online and Cases and the SAGE Research Methods Videos databases to search for scholarly resources on how to apply specific research designs and methods . The Research Methods Online database contains links to more than 175,000 pages of SAGE publisher's book, journal, and reference content on quantitative, qualitative, and mixed research methodologies. Also included is a collection of case studies of social research projects that can be used to help you better understand abstract or complex methodological concepts. The Research Methods Videos database contains hours of tutorials, interviews, video case studies, and mini-documentaries covering the entire research process.

Creswell, John W. and J. David Creswell. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches . 5th edition. Thousand Oaks, CA: Sage, 2018; De Vaus, D. A. Research Design in Social Research . London: SAGE, 2001; Gorard, Stephen. Research Design: Creating Robust Approaches for the Social Sciences . Thousand Oaks, CA: Sage, 2013; Leedy, Paul D. and Jeanne Ellis Ormrod. Practical Research: Planning and Design . Tenth edition. Boston, MA: Pearson, 2013; Vogt, W. Paul, Dianna C. Gardner, and Lynne M. Haeffele. When to Use What Research Design . New York: Guilford, 2012.

Videos in Business and Management , Criminology and Criminal Justice , Education , and Media, Communication and Cultural Studies specifically created for use in higher education.

A literature review tool that highlights the most influential works in Business & Management, Education, Politics & International Relations, Psychology and Sociology. Does not contain full text of the cited works. Dates vary.

Encyclopedias, handbooks, ebooks, and videos published by Sage and CQ Press. 2000 to present

Causal Design

Definition and Purpose

Causality studies may be thought of as understanding a phenomenon in terms of conditional statements in the form, “If X, then Y.” This type of research is used to measure what impact a specific change will have on existing norms and assumptions. Most social scientists seek causal explanations that reflect tests of hypotheses. Causal effect (nomothetic perspective) occurs when variation in one phenomenon, an independent variable, leads to or results, on average, in variation in another phenomenon, the dependent variable.

Conditions necessary for determining causality:

• Empirical association -- a valid conclusion is based on finding an association between the independent variable and the dependent variable.
• Appropriate time order -- to conclude that causation was involved, one must see that cases were exposed to variation in the independent variable before variation in the dependent variable.
• Nonspuriousness -- a relationship between two variables that is not due to variation in a third variable.

What do these studies tell you ?

• Causality research designs assist researchers in understanding why the world works the way it does through the process of proving a causal link between variables and by the process of eliminating other possibilities.
• Replication is possible.
• There is greater confidence the study has internal validity due to the systematic subject selection and equity of groups being compared.

What these studies don't tell you ?

• Not all relationships are casual! The possibility always exists that, by sheer coincidence, two unrelated events appear to be related [e.g., Punxatawney Phil could accurately predict the duration of Winter for five consecutive years but, the fact remains, he's just a big, furry rodent].
• Conclusions about causal relationships are difficult to determine due to a variety of extraneous and confounding variables that exist in a social environment. This means causality can only be inferred, never proven.
• If two variables are correlated, the cause must come before the effect. However, even though two variables might be causally related, it can sometimes be difficult to determine which variable comes first and, therefore, to establish which variable is the actual cause and which is the  actual effect.

Beach, Derek and Rasmus Brun Pedersen. Causal Case Study Methods: Foundations and Guidelines for Comparing, Matching, and Tracing . Ann Arbor, MI: University of Michigan Press, 2016; Bachman, Ronet. The Practice of Research in Criminology and Criminal Justice . Chapter 5, Causation and Research Designs. 3rd ed. Thousand Oaks, CA: Pine Forge Press, 2007; Brewer, Ernest W. and Jennifer Kubn. “Causal-Comparative Design.” In Encyclopedia of Research Design . Neil J. Salkind, editor. (Thousand Oaks, CA: Sage, 2010), pp. 125-132; Causal Research Design: Experimentation. Anonymous SlideShare Presentation ; Gall, Meredith. Educational Research: An Introduction . Chapter 11, Nonexperimental Research: Correlational Designs. 8th ed. Boston, MA: Pearson/Allyn and Bacon, 2007; Trochim, William M.K. Research Methods Knowledge Base . 2006.

Cohort Design

Often used in the medical sciences, but also found in the applied social sciences, a cohort study generally refers to a study conducted over a period of time involving members of a population which the subject or representative member comes from, and who are united by some commonality or similarity. Using a quantitative framework, a cohort study makes note of statistical occurrence within a specialized subgroup, united by same or similar characteristics that are relevant to the research problem being investigated, r ather than studying statistical occurrence within the general population. Using a qualitative framework, cohort studies generally gather data using methods of observation. Cohorts can be either "open" or "closed."

• Open Cohort Studies [dynamic populations, such as the population of Los Angeles] involve a population that is defined just by the state of being a part of the study in question (and being monitored for the outcome). Date of entry and exit from the study is individually defined, therefore, the size of the study population is not constant. In open cohort studies, researchers can only calculate rate based data, such as, incidence rates and variants thereof.
• Closed Cohort Studies [static populations, such as patients entered into a clinical trial] involve participants who enter into the study at one defining point in time and where it is presumed that no new participants can enter the cohort. Given this, the number of study participants remains constant (or can only decrease).
• The use of cohorts is often mandatory because a randomized control study may be unethical. For example, you cannot deliberately expose people to asbestos, you can only study its effects on those who have already been exposed. Research that measures risk factors often relies upon cohort designs.
• Because cohort studies measure potential causes before the outcome has occurred, they can demonstrate that these “causes” preceded the outcome, thereby avoiding the debate as to which is the cause and which is the effect.
• Cohort analysis is highly flexible and can provide insight into effects over time and related to a variety of different types of changes [e.g., social, cultural, political, economic, etc.].
• Either original data or secondary data can be used in this design.
• In cases where a comparative analysis of two cohorts is made [e.g., studying the effects of one group exposed to asbestos and one that has not], a researcher cannot control for all other factors that might differ between the two groups. These factors are known as confounding variables.
• Cohort studies can end up taking a long time to complete if the researcher must wait for the conditions of interest to develop within the group. This also increases the chance that key variables change during the course of the study, potentially impacting the validity of the findings.
• Due to the lack of randominization in the cohort design, its external validity is lower than that of study designs where the researcher randomly assigns participants.

Healy P, Devane D. “Methodological Considerations in Cohort Study Designs.” Nurse Researcher 18 (2011): 32-36; Glenn, Norval D, editor. Cohort Analysis . 2nd edition. Thousand Oaks, CA: Sage, 2005; Levin, Kate Ann. Study Design IV: Cohort Studies. Evidence-Based Dentistry 7 (2003): 51–52; Payne, Geoff. “Cohort Study.” In The SAGE Dictionary of Social Research Methods . Victor Jupp, editor. (Thousand Oaks, CA: Sage, 2006), pp. 31-33; Study Design 101 . Himmelfarb Health Sciences Library. George Washington University, November 2011; Cohort Study . Wikipedia.

Cross-Sectional Design

Cross-sectional research designs have three distinctive features: no time dimension; a reliance on existing differences rather than change following intervention; and, groups are selected based on existing differences rather than random allocation. The cross-sectional design can only measure differences between or from among a variety of people, subjects, or phenomena rather than a process of change. As such, researchers using this design can only employ a relatively passive approach to making causal inferences based on findings.

• Cross-sectional studies provide a clear 'snapshot' of the outcome and the characteristics associated with it, at a specific point in time.
• Unlike an experimental design, where there is an active intervention by the researcher to produce and measure change or to create differences, cross-sectional designs focus on studying and drawing inferences from existing differences between people, subjects, or phenomena.
• Entails collecting data at and concerning one point in time. While longitudinal studies involve taking multiple measures over an extended period of time, cross-sectional research is focused on finding relationships between variables at one moment in time.
• Groups identified for study are purposely selected based upon existing differences in the sample rather than seeking random sampling.
• Cross-section studies are capable of using data from a large number of subjects and, unlike observational studies, is not geographically bound.
• Can estimate prevalence of an outcome of interest because the sample is usually taken from the whole population.
• Because cross-sectional designs generally use survey techniques to gather data, they are relatively inexpensive and take up little time to conduct.
• Finding people, subjects, or phenomena to study that are very similar except in one specific variable can be difficult.
• Results are static and time bound and, therefore, give no indication of a sequence of events or reveal historical or temporal contexts.
• Studies cannot be utilized to establish cause and effect relationships.
• This design only provides a snapshot of analysis so there is always the possibility that a study could have differing results if another time-frame had been chosen.
• There is no follow up to the findings.

Bethlehem, Jelke. "7: Cross-sectional Research." In Research Methodology in the Social, Behavioural and Life Sciences . Herman J Adèr and Gideon J Mellenbergh, editors. (London, England: Sage, 1999), pp. 110-43; Bourque, Linda B. “Cross-Sectional Design.” In  The SAGE Encyclopedia of Social Science Research Methods . Michael S. Lewis-Beck, Alan Bryman, and Tim Futing Liao. (Thousand Oaks, CA: 2004), pp. 230-231; Hall, John. “Cross-Sectional Survey Design.” In Encyclopedia of Survey Research Methods . Paul J. Lavrakas, ed. (Thousand Oaks, CA: Sage, 2008), pp. 173-174; Helen Barratt, Maria Kirwan. Cross-Sectional Studies: Design, Application, Strengths and Weaknesses of Cross-Sectional Studies . Healthknowledge, 2009. Cross-Sectional Study . Wikipedia.

Descriptive Design

Descriptive research designs help provide answers to the questions of who, what, when, where, and how associated with a particular research problem; a descriptive study cannot conclusively ascertain answers to why. Descriptive research is used to obtain information concerning the current status of the phenomena and to describe "what exists" with respect to variables or conditions in a situation.

• The subject is being observed in a completely natural and unchanged natural environment. True experiments, whilst giving analyzable data, often adversely influence the normal behavior of the subject [a.k.a., the Heisenberg effect whereby measurements of certain systems cannot be made without affecting the systems].
• Descriptive research is often used as a pre-cursor to more quantitative research designs with the general overview giving some valuable pointers as to what variables are worth testing quantitatively.
• If the limitations are understood, they can be a useful tool in developing a more focused study.
• Descriptive studies can yield rich data that lead to important recommendations in practice.
• Appoach collects a large amount of data for detailed analysis.
• The results from a descriptive research cannot be used to discover a definitive answer or to disprove a hypothesis.
• Because descriptive designs often utilize observational methods [as opposed to quantitative methods], the results cannot be replicated.
• The descriptive function of research is heavily dependent on instrumentation for measurement and observation.

Anastas, Jeane W. Research Design for Social Work and the Human Services . Chapter 5, Flexible Methods: Descriptive Research. 2nd ed. New York: Columbia University Press, 1999; Given, Lisa M. "Descriptive Research." In Encyclopedia of Measurement and Statistics . Neil J. Salkind and Kristin Rasmussen, editors. (Thousand Oaks, CA: Sage, 2007), pp. 251-254; McNabb, Connie. Descriptive Research Methodologies . Powerpoint Presentation; Shuttleworth, Martyn. Descriptive Research Design , September 26, 2008. Explorable.com website.

Experimental Design

A blueprint of the procedure that enables the researcher to maintain control over all factors that may affect the result of an experiment. In doing this, the researcher attempts to determine or predict what may occur. Experimental research is often used where there is time priority in a causal relationship (cause precedes effect), there is consistency in a causal relationship (a cause will always lead to the same effect), and the magnitude of the correlation is great. The classic experimental design specifies an experimental group and a control group. The independent variable is administered to the experimental group and not to the control group, and both groups are measured on the same dependent variable. Subsequent experimental designs have used more groups and more measurements over longer periods. True experiments must have control, randomization, and manipulation.

• Experimental research allows the researcher to control the situation. In so doing, it allows researchers to answer the question, “What causes something to occur?”
• Permits the researcher to identify cause and effect relationships between variables and to distinguish placebo effects from treatment effects.
• Experimental research designs support the ability to limit alternative explanations and to infer direct causal relationships in the study.
• Approach provides the highest level of evidence for single studies.
• The design is artificial, and results may not generalize well to the real world.
• The artificial settings of experiments may alter the behaviors or responses of participants.
• Experimental designs can be costly if special equipment or facilities are needed.
• Some research problems cannot be studied using an experiment because of ethical or technical reasons.
• Difficult to apply ethnographic and other qualitative methods to experimentally designed studies.

Anastas, Jeane W. Research Design for Social Work and the Human Services . Chapter 7, Flexible Methods: Experimental Research. 2nd ed. New York: Columbia University Press, 1999; Chapter 2: Research Design, Experimental Designs . School of Psychology, University of New England, 2000; Chow, Siu L. "Experimental Design." In Encyclopedia of Research Design . Neil J. Salkind, editor. (Thousand Oaks, CA: Sage, 2010), pp. 448-453; "Experimental Design." In Social Research Methods . Nicholas Walliman, editor. (London, England: Sage, 2006), pp, 101-110; Experimental Research . Research Methods by Dummies. Department of Psychology. California State University, Fresno, 2006; Kirk, Roger E. Experimental Design: Procedures for the Behavioral Sciences . 4th edition. Thousand Oaks, CA: Sage, 2013; Trochim, William M.K. Experimental Design . Research Methods Knowledge Base. 2006; Rasool, Shafqat. Experimental Research . Slideshare presentation.

Exploratory Design

An exploratory design is conducted about a research problem when there are few or no earlier studies to refer to or rely upon to predict an outcome . The focus is on gaining insights and familiarity for later investigation or undertaken when research problems are in a preliminary stage of investigation. Exploratory designs are often used to establish an understanding of how best to proceed in studying an issue or what methodology would effectively apply to gathering information about the issue.

The goals of exploratory research are intended to produce the following possible insights:

• Familiarity with basic details, settings, and concerns.
• Well grounded picture of the situation being developed.
• Generation of new ideas and assumptions.
• Development of tentative theories or hypotheses.
• Determination about whether a study is feasible in the future.
• Issues get refined for more systematic investigation and formulation of new research questions.
• Direction for future research and techniques get developed.
• Design is a useful approach for gaining background information on a particular topic.
• Exploratory research is flexible and can address research questions of all types (what, why, how).
• Provides an opportunity to define new terms and clarify existing concepts.
• Exploratory research is often used to generate formal hypotheses and develop more precise research problems.
• In the policy arena or applied to practice, exploratory studies help establish research priorities and where resources should be allocated.
• Exploratory research generally utilizes small sample sizes and, thus, findings are typically not generalizable to the population at large.
• The exploratory nature of the research inhibits an ability to make definitive conclusions about the findings. They provide insight but not definitive conclusions.
• The research process underpinning exploratory studies is flexible but often unstructured, leading to only tentative results that have limited value to decision-makers.
• Design lacks rigorous standards applied to methods of data gathering and analysis because one of the areas for exploration could be to determine what method or methodologies could best fit the research problem.

Cuthill, Michael. “Exploratory Research: Citizen Participation, Local Government, and Sustainable Development in Australia.” Sustainable Development 10 (2002): 79-89; Streb, Christoph K. "Exploratory Case Study." In Encyclopedia of Case Study Research . Albert J. Mills, Gabrielle Durepos and Eiden Wiebe, editors. (Thousand Oaks, CA: Sage, 2010), pp. 372-374; Taylor, P. J., G. Catalano, and D.R.F. Walker. “Exploratory Analysis of the World City Network.” Urban Studies 39 (December 2002): 2377-2394; Exploratory Research . Wikipedia.

Historical Design

The purpose of a historical research design is to collect, verify, and synthesize evidence from the past to establish facts that defend or refute a hypothesis. It uses secondary sources and a variety of primary documentary evidence, such as, diaries, official records, reports, archives, and non-textual information [maps, pictures, audio and visual recordings]. The limitation is that the sources must be both authentic and valid.

• The historical research design is unobtrusive; the act of research does not affect the results of the study.
• The historical approach is well suited for trend analysis.
• Historical records can add important contextual background required to more fully understand and interpret a research problem.
• There is often no possibility of researcher-subject interaction that could affect the findings.
• Historical sources can be used over and over to study different research problems or to replicate a previous study.
• The ability to fulfill the aims of your research are directly related to the amount and quality of documentation available to understand the research problem.
• Since historical research relies on data from the past, there is no way to manipulate it to control for contemporary contexts.
• Interpreting historical sources can be very time consuming.
• The sources of historical materials must be archived consistently to ensure access. This may especially challenging for digital or online-only sources.
• Original authors bring their own perspectives and biases to the interpretation of past events and these biases are more difficult to ascertain in historical resources.
• Due to the lack of control over external variables, historical research is very weak with regard to the demands of internal validity.
• It is rare that the entirety of historical documentation needed to fully address a research problem is available for interpretation, therefore, gaps need to be acknowledged.

Howell, Martha C. and Walter Prevenier. From Reliable Sources: An Introduction to Historical Methods . Ithaca, NY: Cornell University Press, 2001; Lundy, Karen Saucier. "Historical Research." In The Sage Encyclopedia of Qualitative Research Methods . Lisa M. Given, editor. (Thousand Oaks, CA: Sage, 2008), pp. 396-400; Marius, Richard. and Melvin E. Page. A Short Guide to Writing about History . 9th edition. Boston, MA: Pearson, 2015; Savitt, Ronald. “Historical Research in Marketing.” Journal of Marketing 44 (Autumn, 1980): 52-58;  Gall, Meredith. Educational Research: An Introduction . Chapter 16, Historical Research. 8th ed. Boston, MA: Pearson/Allyn and Bacon, 2007.

Longitudinal Design

A longitudinal study follows the same sample over time and makes repeated observations. For example, with longitudinal surveys, the same group of people is interviewed at regular intervals, enabling researchers to track changes over time and to relate them to variables that might explain why the changes occur. Longitudinal research designs describe patterns of change and help establish the direction and magnitude of causal relationships. Measurements are taken on each variable over two or more distinct time periods. This allows the researcher to measure change in variables over time. It is a type of observational study sometimes referred to as a panel study.

• Longitudinal data facilitate the analysis of the duration of a particular phenomenon.
• Enables survey researchers to get close to the kinds of causal explanations usually attainable only with experiments.
• The design permits the measurement of differences or change in a variable from one period to another [i.e., the description of patterns of change over time].
• Longitudinal studies facilitate the prediction of future outcomes based upon earlier factors.
• The data collection method may change over time.
• Maintaining the integrity of the original sample can be difficult over an extended period of time.
• It can be difficult to show more than one variable at a time.
• This design often needs qualitative research data to explain fluctuations in the results.
• A longitudinal research design assumes present trends will continue unchanged.
• It can take a long period of time to gather results.
• There is a need to have a large sample size and accurate sampling to reach representativness.

Anastas, Jeane W. Research Design for Social Work and the Human Services . Chapter 6, Flexible Methods: Relational and Longitudinal Research. 2nd ed. New York: Columbia University Press, 1999; Forgues, Bernard, and Isabelle Vandangeon-Derumez. "Longitudinal Analyses." In Doing Management Research . Raymond-Alain Thiétart and Samantha Wauchope, editors. (London, England: Sage, 2001), pp. 332-351; Kalaian, Sema A. and Rafa M. Kasim. "Longitudinal Studies." In Encyclopedia of Survey Research Methods . Paul J. Lavrakas, ed. (Thousand Oaks, CA: Sage, 2008), pp. 440-441; Menard, Scott, editor. Longitudinal Research . Thousand Oaks, CA: Sage, 2002; Ployhart, Robert E. and Robert J. Vandenberg. "Longitudinal Research: The Theory, Design, and Analysis of Change.” Journal of Management 36 (January 2010): 94-120; Longitudinal Study . Wikipedia.

Mixed-Method Design

• Narrative and non-textual information can add meaning to numeric data, while numeric data can add precision to narrative and non-textual information.
• Can utilize existing data while at the same time generating and testing a grounded theory approach to describe and explain the phenomenon under study.
• A broader, more complex research problem can be investigated because the researcher is not constrained by using only one method.
• The strengths of one method can be used to overcome the inherent weaknesses of another method.
• Can provide stronger, more robust evidence to support a conclusion or set of recommendations.
• May generate new knowledge new insights or uncover hidden insights, patterns, or relationships that a single methodological approach might not reveal.
• Produces more complete knowledge and understanding of the research problem that can be used to increase the generalizability of findings applied to theory or practice.
• A researcher must be proficient in understanding how to apply multiple methods to investigating a research problem as well as be proficient in optimizing how to design a study that coherently melds them together.
• Can increase the likelihood of conflicting results or ambiguous findings that inhibit drawing a valid conclusion or setting forth a recommended course of action [e.g., sample interview responses do not support existing statistical data].
• Because the research design can be very complex, reporting the findings requires a well-organized narrative, clear writing style, and precise word choice.
• Design invites collaboration among experts. However, merging different investigative approaches and writing styles requires more attention to the overall research process than studies conducted using only one methodological paradigm.
• Concurrent merging of quantitative and qualitative research requires greater attention to having adequate sample sizes, using comparable samples, and applying a consistent unit of analysis. For sequential designs where one phase of qualitative research builds on the quantitative phase or vice versa, decisions about what results from the first phase to use in the next phase, the choice of samples and estimating reasonable sample sizes for both phases, and the interpretation of results from both phases can be difficult.
• Due to multiple forms of data being collected and analyzed, this design requires extensive time and resources to carry out the multiple steps involved in data gathering and interpretation.

Burch, Patricia and Carolyn J. Heinrich. Mixed Methods for Policy Research and Program Evaluation . Thousand Oaks, CA: Sage, 2016; Creswell, John w. et al. Best Practices for Mixed Methods Research in the Health Sciences . Bethesda, MD: Office of Behavioral and Social Sciences Research, National Institutes of Health, 2010Creswell, John W. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches . 4th edition. Thousand Oaks, CA: Sage Publications, 2014; Domínguez, Silvia, editor. Mixed Methods Social Networks Research . Cambridge, UK: Cambridge University Press, 2014; Hesse-Biber, Sharlene Nagy. Mixed Methods Research: Merging Theory with Practice . New York: Guilford Press, 2010; Niglas, Katrin. “How the Novice Researcher Can Make Sense of Mixed Methods Designs.” International Journal of Multiple Research Approaches 3 (2009): 34-46; Onwuegbuzie, Anthony J. and Nancy L. Leech. “Linking Research Questions to Mixed Methods Data Analysis Procedures.” The Qualitative Report 11 (September 2006): 474-498; Tashakorri, Abbas and John W. Creswell. “The New Era of Mixed Methods.” Journal of Mixed Methods Research 1 (January 2007): 3-7; Zhanga, Wanqing. “Mixed Methods Application in Health Intervention Research: A Multiple Case Study.” International Journal of Multiple Research Approaches 8 (2014): 24-35 .

Observational Design

This type of research design draws a conclusion by comparing subjects against a control group, in cases where the researcher has no control over the experiment. There are two general types of observational designs. In direct observations, people know that you are watching them. Unobtrusive measures involve any method for studying behavior where individuals do not know they are being observed. An observational study allows a useful insight into a phenomenon and avoids the ethical and practical difficulties of setting up a large and cumbersome research project.

• Observational studies are usually flexible and do not necessarily need to be structured around a hypothesis about what you expect to observe [data is emergent rather than pre-existing].
• The researcher is able to collect in-depth information about a particular behavior.
• Can reveal interrelationships among multifaceted dimensions of group interactions.
• You can generalize your results to real life situations.
• Observational research is useful for discovering what variables may be important before applying other methods like experiments.
• Observation research designs account for the complexity of group behaviors.
• Reliability of data is low because seeing behaviors occur over and over again may be a time consuming task and are difficult to replicate.
• In observational research, findings may only reflect a unique sample population and, thus, cannot be generalized to other groups.
• There can be problems with bias as the researcher may only "see what they want to see."
• There is no possibility to determine "cause and effect" relationships since nothing is manipulated.
• Sources or subjects may not all be equally credible.
• Any group that is knowingly studied is altered to some degree by the presence of the researcher, therefore, potentially skewing any data collected.

Atkinson, Paul and Martyn Hammersley. “Ethnography and Participant Observation.” In Handbook of Qualitative Research . Norman K. Denzin and Yvonna S. Lincoln, eds. (Thousand Oaks, CA: Sage, 1994), pp. 248-261; Observational Research . Research Methods by Dummies. Department of Psychology. California State University, Fresno, 2006; Patton Michael Quinn. Qualitiative Research and Evaluation Methods . Chapter 6, Fieldwork Strategies and Observational Methods. 3rd ed. Thousand Oaks, CA: Sage, 2002; Payne, Geoff and Judy Payne. "Observation." In Key Concepts in Social Research . The SAGE Key Concepts series. (London, England: Sage, 2004), pp. 158-162; Rosenbaum, Paul R. Design of Observational Studies . New York: Springer, 2010;Williams, J. Patrick. "Nonparticipant Observation." In The Sage Encyclopedia of Qualitative Research Methods . Lisa M. Given, editor.(Thousand Oaks, CA: Sage, 2008), pp. 562-563.

• << Previous: Writing a Research Proposal
• Next: Design Flaws to Avoid >>
• Last Updated: Sep 8, 2023 12:19 PM
• URL: https://guides.library.txstate.edu/socialscienceresearch

21 13. Experimental design

Chapter outline.

• What is an experiment and when should you use one? (8 minute read)
• True experimental designs (7 minute read)
• Quasi-experimental designs (8 minute read)
• Non-experimental designs (5 minute read)
• Critical, ethical, and critical considerations  (5 minute read)

Content warning : examples in this chapter contain references to non-consensual research in Western history, including experiments conducted during the Holocaust and on African Americans (section 13.6).

13.1 What is an experiment and when should you use one?

Learning objectives.

Learners will be able to…

• Identify the characteristics of a basic experiment
• Describe causality in experimental design
• Discuss the relationship between dependent and independent variables in experiments
• Explain the links between experiments and generalizability of results
• Describe advantages and disadvantages of experimental designs

The basics of experiments

The first experiment I can remember using was for my fourth grade science fair. I wondered if latex- or oil-based paint would hold up to sunlight better. So, I went to the hardware store and got a few small cans of paint and two sets of wooden paint sticks. I painted one with oil-based paint and the other with latex-based paint of different colors and put them in a sunny spot in the back yard. My hypothesis was that the oil-based paint would fade the most and that more fading would happen the longer I left the paint sticks out. (I know, it’s obvious, but I was only 10.)

I checked in on the paint sticks every few days for a month and wrote down my observations. The first part of my hypothesis ended up being wrong—it was actually the latex-based paint that faded the most. But the second part was right, and the paint faded more and more over time. This is a simple example, of course—experiments get a heck of a lot more complex than this when we’re talking about real research.

Merriam-Webster defines an experiment   as “an operation or procedure carried out under controlled conditions in order to discover an unknown effect or law, to test or establish a hypothesis, or to illustrate a known law.” Each of these three components of the definition will come in handy as we go through the different types of experimental design in this chapter. Most of us probably think of the physical sciences when we think of experiments, and for good reason—these experiments can be pretty flashy! But social science and psychological research follow the same scientific methods, as we’ve discussed in this book.

As the video discusses, experiments can be used in social sciences just like they can in physical sciences. It makes sense to use an experiment when you want to determine the cause of a phenomenon with as much accuracy as possible. Some types of experimental designs do this more precisely than others, as we’ll see throughout the chapter. If you’ll remember back to Chapter 11  and the discussion of validity, experiments are the best way to ensure internal validity, or the extent to which a change in your independent variable causes a change in your dependent variable.

Experimental designs for research projects are most appropriate when trying to uncover or test a hypothesis about the cause of a phenomenon, so they are best for explanatory research questions. As we’ll learn throughout this chapter, different circumstances are appropriate for different types of experimental designs. Each type of experimental design has advantages and disadvantages, and some are better at controlling the effect of extraneous variables —those variables and characteristics that have an effect on your dependent variable, but aren’t the primary variable whose influence you’re interested in testing. For example, in a study that tries to determine whether aspirin lowers a person’s risk of a fatal heart attack, a person’s race would likely be an extraneous variable because you primarily want to know the effect of aspirin.

In practice, many types of experimental designs can be logistically challenging and resource-intensive. As practitioners, the likelihood that we will be involved in some of the types of experimental designs discussed in this chapter is fairly low. However, it’s important to learn about these methods, even if we might not ever use them, so that we can be thoughtful consumers of research that uses experimental designs.

While we might not use all of these types of experimental designs, many of us will engage in evidence-based practice during our time as social workers. A lot of research developing evidence-based practice, which has a strong emphasis on generalizability, will use experimental designs. You’ve undoubtedly seen one or two in your literature search so far.

The logic of experimental design

How do we know that one phenomenon causes another? The complexity of the social world in which we practice and conduct research means that causes of social problems are rarely cut and dry. Uncovering explanations for social problems is key to helping clients address them, and experimental research designs are one road to finding answers.

As you read about in Chapter 8 (and as we’ll discuss again in Chapter 15 ), just because two phenomena are related in some way doesn’t mean that one causes the other. Ice cream sales increase in the summer, and so does the rate of violent crime; does that mean that eating ice cream is going to make me murder someone? Obviously not, because ice cream is great. The reality of that relationship is far more complex—it could be that hot weather makes people more irritable and, at times, violent, while also making people want ice cream. More likely, though, there are other social factors not accounted for in the way we just described this relationship.

Experimental designs can help clear up at least some of this fog by allowing researchers to isolate the effect of interventions on dependent variables by controlling extraneous variables . In true experimental design (discussed in the next section) and some quasi-experimental designs, researchers accomplish this w ith the control group and the experimental group . (The experimental group is sometimes called the “treatment group,” but we will call it the experimental group in this chapter.) The control group does not receive the intervention you are testing (they may receive no intervention or what is known as “treatment as usual”), while the experimental group does. (You will hopefully remember our earlier discussion of control variables in Chapter 8 —conceptually, the use of the word “control” here is the same.)

In a well-designed experiment, your control group should look almost identical to your experimental group in terms of demographics and other relevant factors. What if we want to know the effect of CBT on social anxiety, but we have learned in prior research that men tend to have a more difficult time overcoming social anxiety? We would want our control and experimental groups to have a similar gender mix because it would limit the effect of gender on our results, since ostensibly, both groups’ results would be affected by gender in the same way. If your control group has 5 women, 6 men, and 4 non-binary people, then your experimental group should be made up of roughly the same gender balance to help control for the influence of gender on the outcome of your intervention. (In reality, the groups should be similar along other dimensions, as well, and your group will likely be much larger.) The researcher will use the same outcome measures for both groups and compare them, and assuming the experiment was designed correctly, get a pretty good answer about whether the intervention had an effect on social anxiety.

You will also hear people talk about comparison groups , which are similar to control groups. The primary difference between the two is that a control group is populated using random assignment, but a comparison group is not. Random assignment entails using a random process to decide which participants are put into the control or experimental group (which participants receive an intervention and which do not). By randomly assigning participants to a group, you can reduce the effect of extraneous variables on your research because there won’t be a systematic difference between the groups.

Do not confuse random assignment with random sampling. Random sampling is a method for selecting a sample from a population, and is rarely used in psychological research. Random assignment is a method for assigning participants in a sample to the different conditions, and it is an important element of all experimental research in psychology and other related fields. Random sampling also helps a great deal with generalizability , whereas random assignment increases internal validity .

We have already learned about internal validity in Chapter 11 . The use of an experimental design will bolster internal validity since it works to isolate causal relationships. As we will see in the coming sections, some types of experimental design do this more effectively than others. It’s also worth considering that true experiments, which most effectively show causality , are often difficult and expensive to implement. Although other experimental designs aren’t perfect, they still produce useful, valid evidence and may be more feasible to carry out.

Key Takeaways

• Experimental designs are useful for establishing causality, but some types of experimental design do this better than others.
• Experiments help researchers isolate the effect of the independent variable on the dependent variable by controlling for the effect of extraneous variables .
• Experiments use a control/comparison group and an experimental group to test the effects of interventions. These groups should be as similar to each other as possible in terms of demographics and other relevant factors.
• True experiments have control groups with randomly assigned participants, while other types of experiments have comparison groups to which participants are not randomly assigned.
• Think about the research project you’ve been designing so far. How might you use a basic experiment to answer your question? If your question isn’t explanatory, try to formulate a new explanatory question and consider the usefulness of an experiment.
• Why is establishing a simple relationship between two variables not indicative of one causing the other?

13.2 True experimental design

• Describe a true experimental design in social work research
• Understand the different types of true experimental designs
• Determine what kinds of research questions true experimental designs are suited for
• Discuss advantages and disadvantages of true experimental designs

True experimental design , often considered to be the “gold standard” in research designs, is thought of as one of the most rigorous of all research designs. In this design, one or more independent variables are manipulated by the researcher (as treatments), subjects are randomly assigned to different treatment levels (random assignment), and the results of the treatments on outcomes (dependent variables) are observed. The unique strength of experimental research is its internal validity and its ability to establish ( causality ) through treatment manipulation, while controlling for the effects of extraneous variable. Sometimes the treatment level is no treatment, while other times it is simply a different treatment than that which we are trying to evaluate. For example, we might have a control group that is made up of people who will not receive any treatment for a particular condition. Or, a control group could consist of people who consent to treatment with DBT when we are testing the effectiveness of CBT.

As we discussed in the previous section, a true experiment has a control group with participants randomly assigned , and an experimental group . This is the most basic element of a true experiment. The next decision a researcher must make is when they need to gather data during their experiment. Do they take a baseline measurement and then a measurement after treatment, or just a measurement after treatment, or do they handle measurement another way? Below, we’ll discuss the three main types of true experimental designs. There are sub-types of each of these designs, but here, we just want to get you started with some of the basics.

Using a true experiment in social work research is often pretty difficult, since as I mentioned earlier, true experiments can be quite resource intensive. True experiments work best with relatively large sample sizes, and random assignment, a key criterion for a true experimental design, is hard (and unethical) to execute in practice when you have people in dire need of an intervention. Nonetheless, some of the strongest evidence bases are built on true experiments.

For the purposes of this section, let’s bring back the example of CBT for the treatment of social anxiety. We have a group of 500 individuals who have agreed to participate in our study, and we have randomly assigned them to the control and experimental groups. The folks in the experimental group will receive CBT, while the folks in the control group will receive more unstructured, basic talk therapy. These designs, as we talked about above, are best suited for explanatory research questions.

Before we get started, take a look at the table below. When explaining experimental research designs, we often use diagrams with abbreviations to visually represent the experiment. Table 13.1 starts us off by laying out what each of the abbreviations mean.

Pretest and post-test control group design

In pretest and post-test control group design , participants are given a pretest of some kind to measure their baseline state before their participation in an intervention. In our social anxiety experiment, we would have participants in both the experimental and control groups complete some measure of social anxiety—most likely an established scale and/or a structured interview—before they start their treatment. As part of the experiment, we would have a defined time period during which the treatment would take place (let’s say 12 weeks, just for illustration). At the end of 12 weeks, we would give both groups the same measure as a post-test .

In the diagram, RA (random assignment group A) is the experimental group and RB is the control group. O 1 denotes the pre-test, X e denotes the experimental intervention, and O 2 denotes the post-test. Let’s look at this diagram another way, using the example of CBT for social anxiety that we’ve been talking about.

In a situation where the control group received treatment as usual instead of no intervention, the diagram would look this way, with X i denoting treatment as usual (Figure 13.3).

Hopefully, these diagrams provide you a visualization of how this type of experiment establishes time order , a key component of a causal relationship. Did the change occur after the intervention? Assuming there is a change in the scores between the pretest and post-test, we would be able to say that yes, the change did occur after the intervention. Causality can’t exist if the change happened before the intervention—this would mean that something else led to the change, not our intervention.

Post-test only control group design

Post-test only control group design involves only giving participants a post-test, just like it sounds (Figure 13.4).

But why would you use this design instead of using a pretest/post-test design? One reason could be the testing effect that can happen when research participants take a pretest. In research, the testing effect refers to “measurement error related to how a test is given; the conditions of the testing, including environmental conditions; and acclimation to the test itself” (Engel & Schutt, 2017, p. 444) [1] (When we say “measurement error,” all we mean is the accuracy of the way we measure the dependent variable.) Figure 13.4 is a visualization of this type of experiment. The testing effect isn’t always bad in practice—our initial assessments might help clients identify or put into words feelings or experiences they are having when they haven’t been able to do that before. In research, however, we might want to control its effects to isolate a cleaner causal relationship between intervention and outcome.

Going back to our CBT for social anxiety example, we might be concerned that participants would learn about social anxiety symptoms by virtue of taking a pretest. They might then identify that they have those symptoms on the post-test, even though they are not new symptoms for them. That could make our intervention look less effective than it actually is.

However, without a baseline measurement establishing causality can be more difficult. If we don’t know someone’s state of mind before our intervention, how do we know our intervention did anything at all? Establishing time order is thus a little more difficult. You must balance this consideration with the benefits of this type of design.

Solomon four group design

One way we can possibly measure how much the testing effect might change the results of the experiment is with the Solomon four group design. Basically, as part of this experiment, you have two control groups and two experimental groups. The first pair of groups receives both a pretest and a post-test. The other pair of groups receives only a post-test (Figure 13.5). This design helps address the problem of establishing time order in post-test only control group designs.

For our CBT project, we would randomly assign people to four different groups instead of just two. Groups A and B would take our pretest measures and our post-test measures, and groups C and D would take only our post-test measures. We could then compare the results among these groups and see if they’re significantly different between the folks in A and B, and C and D. If they are, we may have identified some kind of testing effect, which enables us to put our results into full context. We don’t want to draw a strong causal conclusion about our intervention when we have major concerns about testing effects without trying to determine the extent of those effects.

Solomon four group designs are less common in social work research, primarily because of the logistics and resource needs involved. Nonetheless, this is an important experimental design to consider when we want to address major concerns about testing effects.

• True experimental design is best suited for explanatory research questions.
• True experiments require random assignment of participants to control and experimental groups.
• Pretest/post-test research design involves two points of measurement—one pre-intervention and one post-intervention.
• Post-test only research design involves only one point of measurement—post-intervention. It is a useful design to minimize the effect of testing effects on our results.
• Solomon four group research design involves both of the above types of designs, using 2 pairs of control and experimental groups. One group receives both a pretest and a post-test, while the other receives only a post-test. This can help uncover the influence of testing effects.
• Think about a true experiment you might conduct for your research project. Which design would be best for your research, and why?
• What challenges or limitations might make it unrealistic (or at least very complicated!) for you to carry your true experimental design in the real-world as a student researcher?
• What hypothesis(es) would you test using this true experiment?

13.4 Quasi-experimental designs

• Describe a quasi-experimental design in social work research
• Understand the different types of quasi-experimental designs
• Determine what kinds of research questions quasi-experimental designs are suited for
• Discuss advantages and disadvantages of quasi-experimental designs

Quasi-experimental designs are a lot more common in social work research than true experimental designs. Although quasi-experiments don’t do as good a job of giving us robust proof of causality , they still allow us to establish time order , which is a key element of causality. The prefix quasi means “resembling,” so quasi-experimental research is research that resembles experimental research, but is not true experimental research. Nonetheless, given proper research design, quasi-experiments can still provide extremely rigorous and useful results.

There are a few key differences between true experimental and quasi-experimental research. The primary difference between quasi-experimental research and true experimental research is that quasi-experimental research does not involve random assignment to control and experimental groups. Instead, we talk about comparison groups in quasi-experimental research instead. As a result, these types of experiments don’t control the effect of extraneous variables as well as a true experiment.

Quasi-experiments are most likely to be conducted in field settings in which random assignment is difficult or impossible. They are often conducted to evaluate the effectiveness of a treatment—perhaps a type of psychotherapy or an educational intervention.  We’re able to eliminate some threats to internal validity, but we can’t do this as effectively as we can with a true experiment.  Realistically, our CBT-social anxiety project is likely to be a quasi experiment, based on the resources and participant pool we’re likely to have available. 

It’s important to note that not all quasi-experimental designs have a comparison group.  There are many different kinds of quasi-experiments, but we will discuss the three main types below: nonequivalent comparison group designs, time series designs, and ex post facto comparison group designs.

Nonequivalent comparison group design

You will notice that this type of design looks extremely similar to the pretest/post-test design that we discussed in section 13.3. But instead of random assignment to control and experimental groups, researchers use other methods to construct their comparison and experimental groups. A diagram of this design will also look very similar to pretest/post-test design, but you’ll notice we’ve removed the “R” from our groups, since they are not randomly assigned (Figure 13.6).

Researchers using this design select a comparison group that’s as close as possible based on relevant factors to their experimental group. Engel and Schutt (2017) [2] identify two different selection methods:

• Individual matching : Researchers take the time to match individual cases in the experimental group to similar cases in the comparison group. It can be difficult, however, to match participants on all the variables you want to control for.
• Aggregate matching : Instead of trying to match individual participants to each other, researchers try to match the population profile of the comparison and experimental groups. For example, researchers would try to match the groups on average age, gender balance, or median income. This is a less resource-intensive matching method, but researchers have to ensure that participants aren’t choosing which group (comparison or experimental) they are a part of.

As we’ve already talked about, this kind of design provides weaker evidence that the intervention itself leads to a change in outcome. Nonetheless, we are still able to establish time order using this method, and can thereby show an association between the intervention and the outcome. Like true experimental designs, this type of quasi-experimental design is useful for explanatory research questions.

What might this look like in a practice setting? Let’s say you’re working at an agency that provides CBT and other types of interventions, and you have identified a group of clients who are seeking help for social anxiety, as in our earlier example. Once you’ve obtained consent from your clients, you can create a comparison group using one of the matching methods we just discussed. If the group is small, you might match using individual matching, but if it’s larger, you’ll probably sort people by demographics to try to get similar population profiles. (You can do aggregate matching more easily when your agency has some kind of electronic records or database, but it’s still possible to do manually.)

Time series design

Another type of quasi-experimental design is a time series design. Unlike other types of experimental design, time series designs do not have a comparison group. A time series is a set of measurements taken at intervals over a period of time (Figure 13.7). Proper time series design should include at least three pre- and post-intervention measurement points. While there are a few types of time series designs, we’re going to focus on the most common: interrupted time series design.

But why use this method? Here’s an example. Let’s think about elementary student behavior throughout the school year. As anyone with children or who is a teacher knows, kids get very excited and animated around holidays, days off, or even just on a Friday afternoon. This fact might mean that around those times of year, there are more reports of disruptive behavior in classrooms. What if we took our one and only measurement in mid-December? It’s possible we’d see a higher-than-average rate of disruptive behavior reports, which could bias our results if our next measurement is around a time of year students are in a different, less excitable frame of mind. When we take multiple measurements throughout the first half of the school year, we can establish a more accurate baseline for the rate of these reports by looking at the trend over time.

We may want to test the effect of extended recess times in elementary school on reports of disruptive behavior in classrooms. When students come back after the winter break, the school extends recess by 10 minutes each day (the intervention), and the researchers start tracking the monthly reports of disruptive behavior again. These reports could be subject to the same fluctuations as the pre-intervention reports, and so we once again take multiple measurements over time to try to control for those fluctuations.

This method improves the extent to which we can establish causality because we are accounting for a major extraneous variable in the equation—the passage of time. On its own, it does not allow us to account for other extraneous variables, but it does establish time order and association between the intervention and the trend in reports of disruptive behavior. Finding a stable condition before the treatment that changes after the treatment is evidence for causality between treatment and outcome.

Ex post facto comparison group design

Ex post facto (Latin for “after the fact”) designs are extremely similar to nonequivalent comparison group designs. There are still comparison and experimental groups, pretest and post-test measurements, and an intervention. But in ex post facto designs, participants are assigned to the comparison and experimental groups once the intervention has already happened. This type of design often occurs when interventions are already up and running at an agency and the agency wants to assess effectiveness based on people who have already completed treatment.

In most clinical agency environments, social workers conduct both initial and exit assessments, so there are usually some kind of pretest and post-test measures available. We also typically collect demographic information about our clients, which could allow us to try to use some kind of matching to construct comparison and experimental groups.

In terms of internal validity and establishing causality, ex post facto designs are a bit of a mixed bag. The ability to establish causality depends partially on the ability to construct comparison and experimental groups that are demographically similar so we can control for these extraneous variables .

Quasi-experimental designs are common in social work intervention research because, when designed correctly, they balance the intense resource needs of true experiments with the realities of research in practice. They still offer researchers tools to gather robust evidence about whether interventions are having positive effects for clients.

• Quasi-experimental designs are similar to true experiments, but do not require random assignment to experimental and control groups.
• In quasi-experimental projects, the group not receiving the treatment is called the comparison group, not the control group.
• Nonequivalent comparison group design is nearly identical to pretest/post-test experimental design, but participants are not randomly assigned to the experimental and control groups. As a result, this design provides slightly less robust evidence for causality.
• Nonequivalent groups can be constructed by individual matching or aggregate matching .
• Time series design does not have a control or experimental group, and instead compares the condition of participants before and after the intervention by measuring relevant factors at multiple points in time. This allows researchers to mitigate the error introduced by the passage of time.
• Ex post facto comparison group designs are also similar to true experiments, but experimental and comparison groups are constructed after the intervention is over. This makes it more difficult to control for the effect of extraneous variables, but still provides useful evidence for causality because it maintains the time order[ /pb_glossary] of the experiment.
• Think back to the experiment you considered for your research project in Section 13.3. Now that you know more about quasi-experimental designs, do you still think it's a true experiment? Why or why not?
• What should you consider when deciding whether an experimental or quasi-experimental design would be more feasible or fit your research question better?

13.5 Non-experimental designs

Learners will be able to...

• Describe non-experimental designs in social work research
• Discuss how non-experimental research differs from true and quasi-experimental research
• Demonstrate an understanding the different types of non-experimental designs
• Determine what kinds of research questions non-experimental designs are suited for
• Discuss advantages and disadvantages of non-experimental designs

The previous sections have laid out the basics of some rigorous approaches to establish that an intervention is responsible for changes we observe in research participants. This type of evidence is extremely important to build an evidence base for social work interventions, but it's not the only type of evidence to consider. We will discuss qualitative methods, which provide us with rich, contextual information, in Part 4 of this text. The designs we'll talk about in this section are sometimes used in [pb_glossary id="851"] qualitative research, but in keeping with our discussion of experimental design so far, we're going to stay in the quantitative research realm for now. Non-experimental is also often a stepping stone for more rigorous experimental design in the future, as it can help test the feasibility of your research.

In general, non-experimental designs do not strongly support causality and don't address threats to internal validity. However, that's not really what they're intended for. Non-experimental designs are useful for a few different types of research, including explanatory questions in program evaluation. Certain types of non-experimental design are also helpful for researchers when they are trying to develop a new assessment or scale. Other times, researchers or agency staff did not get a chance to gather any assessment information before an intervention began, so a pretest/post-test design is not possible.

A significant benefit of these types of designs is that they're pretty easy to execute in a practice or agency setting. They don't require a comparison or control group, and as Engel and Schutt (2017) [3] point out, they "flow from a typical practice model of assessment, intervention, and evaluating the impact of the intervention" (p. 177). Thus, these designs are fairly intuitive for social workers, even when they aren't expert researchers. Below, we will go into some detail about the different types of non-experimental design.

One group pretest/post-test design

Also known as a before-after one-group design, this type of research design does not have a comparison group and everyone who participates in the research receives the intervention (Figure 13.8). This is a common type of design in program evaluation in the practice world. Controlling for extraneous variables is difficult or impossible in this design, but given that it is still possible to establish some measure of time order, it does provide weak support for causality.

Imagine, for example, a researcher who is interested in the effectiveness of an anti-drug education program on elementary school students’ attitudes toward illegal drugs. The researcher could assess students' attitudes about illegal drugs (O 1 ), implement the anti-drug program (X), and then immediately after the program ends, the researcher could once again measure students’ attitudes toward illegal drugs (O 2 ). You can see how this would be relatively simple to do in practice, and have probably been involved in this type of research design yourself, even if informally. But hopefully, you can also see that this design would not provide us with much evidence for causality because we have no way of controlling for the effect of extraneous variables. A lot of things could have affected any change in students' attitudes—maybe girls already had different attitudes about illegal drugs than children of other genders, and when we look at the class's results as a whole, we couldn't account for that influence using this design.

All of that doesn't mean these results aren't useful, however. If we find that children's attitudes didn't change at all after the drug education program, then we need to think seriously about how to make it more effective or whether we should be using it at all. (This immediate, practical application of our results highlights a key difference between program evaluation and research, which we will discuss in Chapter 23 .)

After-only design

As the name suggests, this type of non-experimental design involves measurement only after an intervention. There is no comparison or control group, and everyone receives the intervention. I have seen this design repeatedly in my time as a program evaluation consultant for nonprofit organizations, because often these organizations realize too late that they would like to or need to have some sort of measure of what effect their programs are having.

Because there is no pretest and no comparison group, this design is not useful for supporting causality since we can't establish the time order and we can't control for extraneous variables. However, that doesn't mean it's not useful at all! Sometimes, agencies need to gather information about how their programs are functioning. A classic example of this design is satisfaction surveys—realistically, these can only be administered after a program or intervention. Questions regarding satisfaction, ease of use or engagement, or other questions that don't involve comparisons are best suited for this type of design.

Static-group design

A final type of non-experimental research is the static-group design. In this type of research, there are both comparison and experimental groups, which are not randomly assigned. There is no pretest, only a post-test, and the comparison group has to be constructed by the researcher. Sometimes, researchers will use matching techniques to construct the groups, but often, the groups are constructed by convenience of who is being served at the agency.

Non-experimental research designs are easy to execute in practice, but we must be cautious about drawing causal conclusions from the results. A positive result may still suggest that we should continue using a particular intervention (and no result or a negative result should make us reconsider whether we should use that intervention at all). You have likely seen non-experimental research in your daily life or at your agency, and knowing the basics of how to structure such a project will help you ensure you are providing clients with the best care possible.

• Non-experimental designs are useful for describing phenomena, but cannot demonstrate causality.
• After-only designs are often used in agency and practice settings because practitioners are often not able to set up pre-test/post-test designs.
• Non-experimental designs are useful for explanatory questions in program evaluation and are helpful for researchers when they are trying to develop a new assessment or scale.
• Non-experimental designs are well-suited to qualitative methods.
• If you were to use a non-experimental design for your research project, which would you choose? Why?
• Have you conducted non-experimental research in your practice or professional life? Which type of non-experimental design was it?

13.6 Critical, ethical, and cultural considerations

• Describe critiques of experimental design
• Identify ethical issues in the design and execution of experiments
• Identify cultural considerations in experimental design

As I said at the outset, experiments, and especially true experiments, have long been seen as the gold standard to gather scientific evidence. When it comes to research in the biomedical field and other physical sciences, true experiments are subject to far less nuance than experiments in the social world. This doesn't mean they are easier—just subject to different forces. However, as a society, we have placed the most value on quantitative evidence obtained through empirical observation and especially experimentation.

Major critiques of experimental designs tend to focus on true experiments, especially randomized controlled trials (RCTs), but many of these critiques can be applied to quasi-experimental designs, too. Some researchers, even in the biomedical sciences, question the view that RCTs are inherently superior to other types of quantitative research designs. RCTs are far less flexible and have much more stringent requirements than other types of research. One seemingly small issue, like incorrect information about a research participant, can derail an entire RCT. RCTs also cost a great deal of money to implement and don't reflect “real world” conditions. The cost of true experimental research or RCTs also means that some communities are unlikely to ever have access to these research methods. It is then easy for people to dismiss their research findings because their methods are seen as "not rigorous."

Obviously, controlling outside influences is important for researchers to draw strong conclusions, but what if those outside influences are actually important for how an intervention works? Are we missing really important information by focusing solely on control in our research? Is a treatment going to work the same for white women as it does for indigenous women? With the myriad effects of our societal structures, you should be very careful ever assuming this will be the case. This doesn't mean that cultural differences will negate the effect of an intervention; instead, it means that you should remember to practice cultural humility implementing all interventions, even when we "know" they work.

How we build evidence through experimental research reveals a lot about our values and biases, and historically, much experimental research has been conducted on white people, and especially white men. [4] This makes sense when we consider the extent to which the sciences and academia have historically been dominated by white patriarchy. This is especially important for marginalized groups that have long been ignored in research literature, meaning they have also been ignored in the development of interventions and treatments that are accepted as "effective." There are examples of marginalized groups being experimented on without their consent, like the Tuskegee Experiment or Nazi experiments on Jewish people during World War II. We cannot ignore the collective consciousness situations like this can create about experimental research for marginalized groups.

None of this is to say that experimental research is inherently bad or that you shouldn't use it. Quite the opposite—use it when you can, because there are a lot of benefits, as we learned throughout this chapter. As a social work researcher, you are uniquely positioned to conduct experimental research while applying social work values and ethics to the process and be a leader for others to conduct research in the same framework. It can conflict with our professional ethics, especially respect for persons and beneficence, if we do not engage in experimental research with our eyes wide open. We also have the benefit of a great deal of practice knowledge that researchers in other fields have not had the opportunity to get. As with all your research, always be sure you are fully exploring the limitations of the research.

• While true experimental research gathers strong evidence, it can also be inflexible, expensive, and overly simplistic in terms of important social forces that affect the resources.
• Marginalized communities' past experiences with experimental research can affect how they respond to research participation.
• Social work researchers should use both their values and ethics, and their practice experiences, to inform research and push other researchers to do the same.
• Think back to the true experiment you sketched out in the exercises for Section 13.3. Are there cultural or historical considerations you hadn't thought of with your participant group? What are they? Does this change the type of experiment you would want to do?
• How can you as a social work researcher encourage researchers in other fields to consider social work ethics and values in their experimental research?
• Engel, R. & Schutt, R. (2016). The practice of research in social work. Thousand Oaks, CA: SAGE Publications, Inc. ↵
• Sullivan, G. M. (2011). Getting off the “gold standard”: Randomized controlled trials and education research. Journal of Graduate Medical Education ,  3 (3), 285-289. ↵

an operation or procedure carried out under controlled conditions in order to discover an unknown effect or law, to test or establish a hypothesis, or to illustrate a known law.

explains why particular phenomena work in the way that they do; answers “why” questions

variables and characteristics that have an effect on your outcome, but aren't the primary variable whose influence you're interested in testing.

the group of participants in our study who do not receive the intervention we are researching in experiments with random assignment

in experimental design, the group of participants in our study who do receive the intervention we are researching

the group of participants in our study who do not receive the intervention we are researching in experiments without random assignment

using a random process to decide which participants are tested in which conditions

The ability to apply research findings beyond the study sample to some broader population,

Ability to say that one variable "causes" something to happen to another variable. Very important to assess when thinking about studies that examine causation such as experimental or quasi-experimental designs.

the idea that one event, behavior, or belief will result in the occurrence of another, subsequent event, behavior, or belief

An experimental design in which one or more independent variables are manipulated by the researcher (as treatments), subjects are randomly assigned to different treatment levels (random assignment), and the results of the treatments on outcomes (dependent variables) are observed

a type of experimental design in which participants are randomly assigned to control and experimental groups, one group receives an intervention, and both groups receive pre- and post-test assessments

A measure of a participant's condition before they receive an intervention or treatment.

A measure of a participant's condition after an intervention or, if they are part of the control/comparison group, at the end of an experiment.

A demonstration that a change occurred after an intervention. An important criterion for establishing causality.

an experimental design in which participants are randomly assigned to control and treatment groups, one group receives an intervention, and both groups receive only a post-test assessment

The measurement error related to how a test is given; the conditions of the testing, including environmental conditions; and acclimation to the test itself

a subtype of experimental design that is similar to a true experiment, but does not have randomly assigned control and treatment groups

In nonequivalent comparison group designs, the process by which researchers match individual cases in the experimental group to similar cases in the comparison group.

In nonequivalent comparison group designs, the process in which researchers match the population profile of the comparison and experimental groups.

a set of measurements taken at intervals over a period of time

Graduate research methods in social work Copyright © 2021 by Matthew DeCarlo, Cory Cummings, Kate Agnelli is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

• Find My Rep

You are here

Research Design in Social Work Qualitative and Quantitative Methods

• Anne Campbell - Queen's University Belfast, UK
• Brian J. Taylor - University of Ulster, UK
• Anne McGlade - The Health and Social Care Board for Northern Ireland
• Description
• Author(s) / Editor(s)

More than just another research text, this book remains grounded in social work practice and has clear links to the Professional Capabilities Framework for Social Work.

This is an easy to read comprehensive introduction to social science research methods. The textbook makes specific connections to social work and provides clear explanations of research concepts.

Coverage of qualitative, quantitative and mixed methods, which is very welcome in a social work text.

A very practical ready reference for students and practitioners.

Preview this book

Anne campbell.

Anne Campbell is Lecturer in Social Work at Queens University, Belfast. She is responsible for co-ordinating a Master’s programme in Social Work studies and is also involved in teaching social work students at Undergraduate level. She has supervised and mentored a wide range of research students at Undergraduate, Masters and PHD levels over a number of years. She is also involved in developing knowledge-based and interactive e- learning tools for practice, research and social work education. She is currently active in research on drug and alcohol issues within regional and international contexts and online applications for social work... More About Author

Brian J. Taylor

Brian J. Taylor is Professor of Social Work at Ulster University in Northern Ireland, where he has the lead role for research in social work. He spent 10 years in practice and 15 years in professional training and organisation development in social work before moving to the university. He teaches research methods to Ph.D. students and to experienced social workers undertaking postgraduate, post-qualifying study. He was module coordinator for an innovative Introduction to Evidence Based Practice module on the B.Sc. qualifying social work programme. Brian leads the university social work research cluster on Decision, Assessment, Risk and... More About Author

Anne McGlade

Anne McGlade has been Social Care Research Lead for the Social Care and Children’s Directorate, Health and Social Care Board since October 2013. She is the lead on the development of the Research and Continuous Improvement Strategy (2015–2020) In Pursuit of Excellence in Evidence Informed Social Work Services in Northern Ireland. She has a long-standing career working in research and evaluation research in health and social care and other settings in England and Northern Ireland. She has a keen interest in the needs of older people, people with disabilities and people from black and minority ethnic groups. She has undertaken and published... More About Author

Purchasing options

Please select a format:

Order from:

• Buy from SAGE

Related Products

SAGE Knowledge is the ultimate social sciences digital library for students, researchers, and faculty. Hosting more than 4,400 titles, it includes an expansive range of SAGE eBook and eReference content, including scholarly monographs, reference works, handbooks, series, professional development titles, and more.

The platform allows researchers to cross-search and seamlessly access a wide breadth of must-have SAGE book and reference content from one source.

Scientific Inquiry in Social Work

(9 reviews)

Matthew DeCarlo, Radford University

Copyright Year: 2018

ISBN 13: 9781975033729

Publisher: Open Social Work Education

Language: English

Formats Available

Conditions of use.

Learn more about reviews.

Reviewed by Shannon Blajeski, Assistant Professor, Portland State University on 3/10/23

This book provides an introduction to research and inquiry in social work with an applied focus geared for the MSW student. The text covers 16 chapters, including several dedicated to understanding how to begin the research process, a chapter on... read more

Comprehensiveness rating: 5 see less

This book provides an introduction to research and inquiry in social work with an applied focus geared for the MSW student. The text covers 16 chapters, including several dedicated to understanding how to begin the research process, a chapter on ethics, and then eight chapters dedicated to research methods. The subchapters (1-5 per chapter) are concise and focused while also being tied to current knowledge and events so as to hold the reader's attention. It is comprehensive, but some of the later chapters covering research methods as well as the final chapter seem a bit scant and could be expanded. The glossary at the end of each chapter is helpful as is the index that is always accessible from the left-hand drop-down menu.

Content Accuracy rating: 4

The author pulls in relevant current and recent public events to illustrate important points about social research throughout the book. Each sub-chapter reads as accurate. I did not come across any inaccuracies in the text, however I would recommend a change in the title of Chapter 15 as "real world research" certainly encompasses more than program evaluation, single-subject designs, and action research.

Relevance/Longevity rating: 5

Another major strength of this book is that it adds currency to engage the reader while also maintaining its relevance to research methods. None of the current events/recent events that are described seem dated nor will they fade from relevance in a number of years. In addition, the concise nature of the modules should make them easy to update when needed to maintain relevancy in future editions.

Clarity rating: 5

Clarity is a major strength of this textbook. As described in the interface section, this book is written to be clear and concise, without unnecessary extra text that detracts from the concise content provided in each chapter. Any lengthy excerpts are also very engaging which lends itself to a clear presentation of content for the reader.

Consistency rating: 5

The text and content seems to be presented consistently throughout the book. Terminology and frameworks are balanced with real-world examples and current events.

Modularity rating: 5

The chapters of this textbook are appropriately spaced and easily digestible, particularly for readers with time constraints. Each chapter contains 3-5 sub-chapters that build upon each other in a scaffolding style. This makes it simple for the instructor to assign each chapter (sometimes two) per weekly session as well as add in additional assigned readings to complement the text.

Organization/Structure/Flow rating: 5

The overall organization of the chapters flow well. The book begins with a typical introduction to research aimed at social work practitioners or new students in social work. It then moves into a set of chapters on beginning a research project, reviewing literature, and asking research questions, followed by a chapter on ethics. Next, the text transitions to three chapters covering constructs, measurement, and sampling, followed by five chapters covering research methods, and a closing chapter on dissemination of research. This is one of the more logically-organized research methods texts that I have used as an instructor.

Interface rating: 5

The modular chapters are easy to navigate and the interface of each chapter follows a standard presentation style with the reading followed by a short vocabulary glossary and references. This presentation lends itself to a familiarity for students that helps them become more efficient with completing reading assignments, even in short bursts of time. This is particularly important for online and returning learners who may juggle their assignment time with family and work obligations.

Grammatical Errors rating: 5

No grammatical errors were noted.

Cultural Relevance rating: 4

At first glance at the table of contents, the book doesn't seem to be overtly committed to cultural representation, however, upon reading the chapters, it becomes clear that the author does try to represent and reference marginalized groups (e.g., women, individuals with disabilities, racial/ethnic/gender intersectionality) within the examples used. I also am very appreciative that the bottom of each introduction page for each chapter contains content trigger warnings for any possible topics that could be upsetting, e.g., substance abuse, violence.

As the author likely knows, social work students are eager to engage in learning that is current and relevant to their social causes. This book is written in a way that engages a non-researcher social worker into reading about research by weaving research information into topics that they might find compelling. It also does this in a concise way where short bits of pertinent information are presented, making the text accessible without needing to sustain long periods of attention. This is particularly important for online and returning learners who may need to sit with their readings in short bursts due to attending school while juggling work and family obligations.

Reviewed by Lynn Goerdt, Associate Professor, University of Wisconsin - Superior on 9/17/21

Text appears to be comprehensive in covering steps for typical SWK research class, taking students from the introduction of the purpose and importance of research to how to design and analyze research. Author covers the multitude of ways that... read more

Text appears to be comprehensive in covering steps for typical SWK research class, taking students from the introduction of the purpose and importance of research to how to design and analyze research. Author covers the multitude of ways that social workers engage in research as way of building knowledge and ways that social work practitioners conduct research to evaluate their practice, including outcome evaluation, single subject design, and action research. I particularly appreciated the last section on reporting research, which should be very practical.

Overall, content appears mostly accurate which few errors. Definitions and citations are mostly thorough and clear. Author does cite Wikipedia in at least one occasion which could be credible, depending on the source of the Wikipedia content. There were a few references within the text to comic or stories but the referenced material was not always apparent.

Relevance/Longevity rating: 4

The content of Scientific Inquiry for Social Work is relevant, as the field of social work research methods does not appear to change quickly, although there are innovations. The author referenced examples which appear to be recent and likely relatable to interests of current students. Primary area of innovation is in using technology for the collection and analysis of data, which could be expanded, particularly using social media for soliciting research participants.

Style is personable and content appears to be accessible, which is a unique attribute for a research textbook. Author uses first person in many instances, particularly in the beginning to present the content as relatable.

Format appears to be consistent in format and relative length. Each chapter includes learning objectives, content advisory (if applicable), key takeaways and glossary. Author uses color and text boxes to draw attention to these sections.

Modularity rating: 4

Text is divided into modules which could easily be assigned and reviewed in a class. The text modules could also be re-structured if desired to fit curricular uniqueness’s. Author uses images to illuminate the concepts of the module or chapter, but they often take about 1/3 of the page, which extends the size of the textbook quite a bit. Unclear if benefit of images outweighs additional cost if PDF version is printed.

Textbook is organized in a very logical and clear fashion. Each section appears to be approximately 6-10 pages in length which seems to be an optimal length for student attention and comprehension.

Interface rating: 4

There were some distortions of the text (size and visibility) but they were a fairly minor distraction and did not appear to reduce access to the content. Otherwise text was easy to navigate.

Grammatical Errors rating: 4

No grammatical errors were noted but hyperlinks to outside content were referenced but not always visible which occasionally resulted in an awkward read. Specific link may be in resources section of each chapter but occasionally they were also included in the text.

I did not recognize any text which was culturally insensitive or offensive. Images used which depicted people, appeared to represent diverse experiences, cultures, settings and persons. Did notice image depicting homelessness appeared to be stereotypical person sleeping on sidewalk, which can perpetuate a common perception of homelessness. Would encourage author to consider images representing a wider range of experiences of a social phenomena. Content advisories are used for each section, which is not necessarily cultural relevance but is respectful and recognizes the diversity of experiences and triggers that the readers may have.

Overall, I was very impressed and encouraged with the well organized content and thoughtful flow of this important textbook for social work students and instructors. The length and readability of each chapter would likely be appreciated by instructors as well as students, increasing the extent that the learning outcomes would be achieved. Teaching research is very challenging because the content and application can feel very intimidating. The author also has provided access to supplemental resources such as presentations and assignments.

Reviewed by elaine gatewood, Adjunct Faculty, Bridgewater State University on 6/15/21

The book provides concrete and clear information on using research as consumers, It provides a comprehensive review of each step to take to develop a research project from beginning to completion, with examples. read more

The book provides concrete and clear information on using research as consumers, It provides a comprehensive review of each step to take to develop a research project from beginning to completion, with examples.

Content Accuracy rating: 5

From my perspective, content is highly accurate in the field of learning research method and unbiased. It's all there!

The content is highly relevant and up-to-date in the field. The book is written and arranged in a way that its easy to follow along with adding updates.

The book is written in clear and concise. The book provides appropriate context for any jargon/technical terminology used along with examples which readers are able to follow along and understand.

The contents of the book flow quite well. The framework in the book is consistent.

The text appears easily adaptable for readers and the author also provides accompanying PowerPoint presentations; these are a good foundation tools for readers to use and implement.

Organization/Structure/Flow rating: 4

The contents of the book flow very well. Readers would be able to put into practice the key reading strategies shared in the book ) because its organization is laid out nicely

Interface rating: 3

The interface is generally good, but I was only able to download the .pdf. This may present issues for some student readers.

There are no grammatical errors.

The text was culturally relevant and provided diverse research and practice examples. The text could have benefited from sexamples of intersectional and anti-oppressive lenses for students to consider in their practice.

This text is a comprehensive introduction to research that can be easily adapted for a BSW/MSW research course.

Reviewed by Taylor Hall, Assistant Professor, Bridgewater State University on 6/30/20

This text is more comprehensive than the text I currently use in my Research Methods in Social Work course, which students have to pay for. This text not only covers both qualitative and quantitative research methods, but also all parts of the... read more

This text is more comprehensive than the text I currently use in my Research Methods in Social Work course, which students have to pay for. This text not only covers both qualitative and quantitative research methods, but also all parts of the research process from thinking about research ideas to questions all the way to evaluation after social work programs/policies have been employed.

Not much to say here- with research methods, things are black and white; it is or isn't. This content is accurate. I also like to way the content is explained in light of social work values and ethics. This is something our students can struggle with, and this is helpful in terms of showing why social work needs to pay attention to research.

There are upcoming changes to CSWE's competencies, therefore lots of text materials are going to need to be updated soon. Otherwise, case examples are pertinent and timely.

Clarity rating: 4

I think that research methods for social workers is a difficult field of study. Many go into the field to be clinicians, and few understand (off the bat) the importance of understanding methods of research. I think this textbook makes it clear to me, but hard to rate a 5 as I know from a student's perspective, lots of the terminology is so new.

Appears to be so- I was able to follow, seems consistent.

Yes- and I think this is a strong point of this text. This was easy to follow and read, and I could see myself easily divvying up different sections for students to work on in groups.

Yes- makes sense to me and the way I teach this course. I like the 30,000 ft view then honing in on specific types of research, all along the way explaining the different pieces of the research process and in writing a research paper.

I sometimes struggle with online platforms versus in person texts to read, and this OER is visually appealing- there is not too much text on the pages, it is spaced in a way that makes it easier to read. Colors are used well to highlight pertinent information.

Not something I found in this text.

Cultural Relevance rating: 3

This is a place where I feel the text could use some work. A nod to past wrongdoings in research methods on oppressed groups, and more of a discussion on social work's role in social justice with an eye towards righting the wrongs of the past. Updated language re: person first language, more diverse examples, etc.

This is a very useful text, and I am going to recommend my department check it out for future use, especially as many of our students are first gen and working class and would love to save money on textbooks where possible.

Reviewed by Olubunmi Oyewuwo-Gassikia, Assistant Professor, Northeastern Illinois University on 5/5/20

This text is an appropriate and comprehensive introduction to research methods for BSW students. It guides the reader through each stage of the research project, including identifying a research question, conducting and writing a literature... read more

This text is an appropriate and comprehensive introduction to research methods for BSW students. It guides the reader through each stage of the research project, including identifying a research question, conducting and writing a literature review, research ethics, theory, research design, methodology, sampling, and dissemination. The author explains complex concepts - such as paradigms, epistemology, and ontology - in clear, simple terms and through the use of practical, social work examples for the reader. I especially appreciated the balanced attention to quantitative and qualitative methods, including the explanation of data collection and basic analysis techniques for both. The text could benefit from the inclusion of an explanation of research design notations.

The text is accurate and unbiased. Additionally, the author effectively incorporates referenced sources, including sources one can use for further learning.

The content is relevant and timely. The author incorporates real, recent research examples that reflects the applicability of research at each level of social practice (micro, meso, and macro) throughout the text. The text will benefit from regular updates in research examples.

The text is written in a clear, approachable manner. The chapters are a reasonable length without sacrificing appropriate depth into the subject manner.

The text is consistent throughout. The author is effective in reintroducing previously explained terms from previous chapters.

The text appears easily adaptable. The instructions provided by the author on how to adapt the text for one's course are helpful to one who would like to use the text but not in its entirety. The author also provides accompanying PowerPoint presentations; these are a good foundation but will likely require tailoring based on the teaching style of the instructor.

Generally, the text flows well. However, chapter 5 (Ethics) should come earlier, preferably before chapter 3 (Reviewing & Evaluating the Literature). It is important that students understand research ethics as ethical concerns are an important aspect of evaluating the quality of research studies. Chapter 15 (Real-World Research) should also come earlier in the text, most suitably before or after chapter 7 (Design and Causality).

The interface is generally good, but I was only able to download the .pdf. The setup of the .pdf is difficult to navigate, especially if one wants to jump from chapter to chapter. This may present issues for the student reader.

The text was culturally relevant and provided diverse research and practice examples. The text could have benefited from more critical research examples, such as examples of research studies that incorporate intersectional and anti-oppressive lenses.

This text is a comprehensive introduction to research that can be easily adapted for a BSW level research course.

Reviewed by Smita Dewan, Assistant Professor, New York City College of Technology, Department of Human Services on 12/6/19

This is a very good introductory research methodology textbook for undergraduate students of social work or human services. For students who might be intimidated by social research, the text provides assurance that by learning basic concepts of... read more

Comprehensiveness rating: 4 see less

This is a very good introductory research methodology textbook for undergraduate students of social work or human services. For students who might be intimidated by social research, the text provides assurance that by learning basic concepts of research methodology, students will be better scholars and social work or human service practitioners. The content and flow of the text book supports a basic assignment of most research methodology courses which is to develop a research proposal or a research project. Each stage of research is explained well with many examples from social work practice that has the potential to keep the student engaged.

The glossary at the end of each chapter is very comprehensive but does not include the page number/s where the content is located. The glossary at the end of the book also lacks page numbers which might make it cumbersome for students seeking a quick reference.

The content is accurate and unbiased. Suggested exercises and prompts for students to engage in critical thinking and to identify biases in research that informs practice may help students understand the complexities of social research.

Content is up-to-date and concepts of research methodology presented is unlikely to be obsolete in the coming years. However, recent trends in research such as data mining, using algorithms for social policy and practice implications, privacy concerns, role of social media are topics that could be considered for inclusion in the forthcoming editions.

Content is presented very clearly for undergraduate students. Key takeaways and glossary for each section of the chapter is very useful for students.

Presentation of content, format and organization is consistent throughout the book.

Subsections within each chapter is very helpful for the students who might be assigned readings just in parts for the class.

Students would benefit from reading about research ethics right after the introductory chapter. I would also move Chapter 8 to right after the literature review which might inform creating and refining the research question. Content on evaluation research could also be moved up to follow the chapter on experimental designs. Regardless of the organization, the course instructors can assign chapters according to the course requirements.

PDF version of the book is very easy to use especially as students can save a copy on their computers and do not have to be online. Charts and tables are well presented but some of the images/photographs do not necessarily serve to enhance learning. Image attributions could be provided at the end of the chapter instead of being listed under the glossary. Students might also find it useful to be able to highlight the content and make annotations. This requires that students sign-in. Students should be able to highlight and annotate a downloaded version through Adobe Reader.

I did not find any grammatical errors.

Cultural Relevance rating: 5

Content is not insensitive or offensive in any way. Supporting examples in chapters are very diverse. Students would benefit from some examples of international research (both positive and negative examples) of protection of human subjects.

Reviewed by Jill Hoffman, Assistant Professor, Portland State University on 10/29/19

This text includes 16 chapters that cover content related to the process of conducting research. From identifying a topic and reviewing the literature, to formulating a question, designing a study, and disseminating findings, the text includes... read more

This text includes 16 chapters that cover content related to the process of conducting research. From identifying a topic and reviewing the literature, to formulating a question, designing a study, and disseminating findings, the text includes research basics that most other introductory social work research texts include. Content on ethics, theory, and to a lesser extent evaluation, single-subject design, and action research are also included. There is a glossary at the end of the text that includes information on the location of the terms. There is a practice behaviors index, but not an index in the traditional sense. If using the text electronically, search functions make it easy to find necessary information despite not having an index. If using a printed version, this would be more difficult. The text includes examples to illustrate concepts that are relevant to settings in which social workers might work. As most other introductory social work research texts, this book appears to come from a mainly positivist view. I would have appreciated more of a discussion related to power, privilege, and oppression and the role these play in the research topics that get studied and who benefits, along with anti-oppressive research. Related to evaluations, a quick mention of logic models would be helpful.

The information appears to be accurate and error free. The language in the text seems to emphasize "right/wrong" choices/decisions instead of highlighting the complexities of research and practice. Using gender-neutral pronouns would also make the language more inclusive.

Content appears to be up-to-date and relevant. Any updating would be straightforward to carry out. I found at least one link that did not work (e.g., NREPP) so if you use this text it will be important to check and make sure things are updated.

The content is clearly written, using examples to illustrate various concepts. I appreciated prompts for questions throughout each chapter in order to engage students in the content. Key terms are bolded, which helps to easily identify important points.

Information is presented in a consistent manner throughout the text.

Each chapter is divided into subsections that help with readability. It is easy to pick and choose various pieces of the text for your course if you're not using the entire thing.

There are many ways you can organize a social work research text. Personally, I prefer to talk about ethics and theory early on, so that students have this as a framework as they read about other's studies and design their own. In the case of this text, I'd put those two chapters right after chapter 1. As others have suggested, I'd also move up the content on research questions, perhaps after chapter 4.

In the online version, no significant interface issues arose. The only thing that would be helpful is to have chapter titles clearly presented when navigating through the text in the online version. For example, when you click through to a new chapter, the title simply says "6.0 Chapter introduction." In order to see the chapter title you have to click into the contents tab. Not a huge issue but could help with navigating the online version. In the pdf version, the links in the table of contents allowed me to navigate through to various sections. I did notice that some of the external links were not complete (e.g., on page 290, the URL is linked as "http://baby-").

Cultural representation in the text is similar to many other introductory social work research texts. There's more of an emphasis on white, western, cis-gendered individuals, particularly in the images. In examples, it appeared that only male/female pronouns were used.

Reviewed by Monica Roth Day, Associate Professor, Social Work, Metropolitan State University (Saint Paul, Minnesota) on 12/26/18

The book provides concrete and clear information on using research as consumers, then developing research as producers of knowledge. It provides a comprehensive review of each step to take to develop a research project from beginning to... read more

The book provides concrete and clear information on using research as consumers, then developing research as producers of knowledge. It provides a comprehensive review of each step to take to develop a research project from beginning to completion, with appropriate examples. More specific social work links would be helpful as students learn more about the field and the uses of research.

The book is accurate and communicates information and largely without bias. Numerous examples are provided from varied sources, which are then used to discuss potential for bias in research. The addition of critical race theory concepts would add to this discussion, to ground students in the importance of understanding implicit bias as researchers and ways to develop their own awareness.

The book is highly relevant. It provides historical and current examples of research which communicate concepts using accessible language that is current to social work. The text is written so that updates should be easy. Links need to be updated on a regular basis.

The book is accessible for students at it uses common language to communicate concepts while helping students build their research vocabulary. Terminology is communicate both within the text and in glossaries, and technical terms are minimally used.

The book is consistent in its use of terminology and framework. It follows a pattern of development, from consuming research to producing research. The steps are predictable and walk students through appropriate actions to take.

The book is easily readable. Each chapter is divided in sections that are easy to navigate and understand. Pictures and tables are used to support text.

Chapters are in logical order and follow a common pattern.

When reading the book online, the text was largely free of interface issues. As a PDF, there were issues with formatting. Be aware that students who may wish to download the book into a Kindle or other book reader may experience issues.

The text was grammatically correct with no misspellings.

While the book is culturally relevant, it lacks the application of critical race theory. While students will learn about bias in research, critical race theory would ground students in the importance of understanding implicit bias as researchers and ways to develop their own awareness. It would also help students understand why the background of researchers is important in relation to the ways of knowing.

Reviewed by Jennifer Wareham, Associate Professor, Wayne State University on 11/30/18

The book provides a comprehensive introduction to research methods from the perspective of the discipline of Social Work. The book borrows heavily from Amy Blackstone’s Principles of Sociological Inquiry – Qualitative and Quantitative Methods open... read more

The book provides a comprehensive introduction to research methods from the perspective of the discipline of Social Work. The book borrows heavily from Amy Blackstone’s Principles of Sociological Inquiry – Qualitative and Quantitative Methods open textbook. The book is divided into 16 chapters, covering: differences in reasoning and scientific thought, starting a research project, writing a literature review, ethics in social science research, how theory relates to research, research design, causality, measurement, sampling, survey research, experimental design, qualitative interviews and focus groups, evaluation research, and reporting research. Some of the more advanced concepts and topics are only covered at superficial level, which limits the intended population of readers to high school students, undergraduate students, or those with no background in research methods. Since the book is geared toward Social Work undergraduate students, the chapters and content address methodologies commonly used in this field, but ignore methodologies that may be more popular in other social science fields. For example, the material on qualitative methods is narrow and focuses on commonly used qualitative methods in Social Work. In addition, the chapter on evaluation is limited to a general overview of evaluation research, which could be improved with more in-depth discussion of different types of evaluation (e.g., needs assessment, evaluability assessment, process evaluation, impact/outcomes evaluation) and real-world examples of different types of evaluation implemented in Social Work. Overall, the author provides examples that are easy for practitioners in Social Work to understand, which are also easily relatable for students in similar disciplines such as criminal justice. The book provides a glossary of key terms. There is no index; however, users can search for terms using the find (Ctrl-F) function in the PDF version of the book.

Overall, the content inside this book is accurate, error-free, and unbiased. However, the content is limited to the Social Work perspective, which may be considered somewhat biased or inaccurate from the perspective of others in different disciplines.

The book describes classic examples used in most texts on social science research methods. It also includes contemporary and relevant examples. Some of the content (such as web addresses and contemporary news pieces) will need to be updated every few years. The text is written and arranged in such a way that any necessary updates should be relatively easy and straightforward to implement.

The book is written in clear and accessible prose. The book provides appropriate context for any jargon/technical terminology used. Readers from any social science discipline should be able to understand the content and context of the material presented in the book.

The framework and use of terminology in the book are consistent.

This book is highly modular. The author has even improved upon the modularity of the book from Blackstone’s open text (which serves as the basis of the present text). Each chapter is divided into short, related subsections. The design of the chapters and their subsections make it easy to divide the material into units of study across a semester or quarter of instruction.

Generally, the book is organized in a similar manner as other texts on social science research methods. However, the organization could be improved slightly. Chapters 2 through 4 describe the process of beginning a research project and conducting a literature review. Chapter 8 describes refining a research question. This chapter could be moved to follow the Chapter 4. Chapter 12 describes experimental design, while Chapter 15 provides a description and examples of evaluation research. Since evaluation research tends to rely on experimental and quasi-experimental design, this chapter should follow the experimental design chapter.

For the online version of the book, there were no interface issues. The images and charts were clear and readable. The hyperlinks to sources mentioned in the text worked. The Contents menu allowed for easy and quick access to any section of the book. For the PDF version of the book, there were interface issues. The images and charts were clear and readable. However, the URLs and hyperlinks were not active in the PDF version. Furthermore, the PDF version was not bookmarked, which made it more difficult to access specific sections of the book.

I did not find grammatical errors in the book.

Overall, the cultural relevance and sensitivity were consistent with other social science research methods texts. The author does a good job of using both female and male pronouns in the prose. While there are pictures of people of color, there could be more. Most of the pictures are of white people. Also, the context is generally U.S.-centric.

Table of Contents

• Chapter 1: Introduction to research
• Chapter 2: Beginning a research project
• Chapter 3: Reading and evaluating literature
• Chapter 4: Conducting a literature review
• Chapter 5: Ethics in social work research
• Chapter 6: Linking methods with theory
• Chapter 7: Design and causality
• Chapter 8: Creating and refining a research question
• Chapter 9: Defining and measuring concepts
• Chapter 10: Sampling
• Chapter 11: Survey research
• Chapter 12: Experimental design
• Chapter 13: Interviews and focus groups
• Chapter 14: Unobtrusive research: Qualitative and quantitative approaches
• Chapter 15: Real-world research: Evaluation, single-subjects, and action research
• Chapter 16: Reporting and reading research

Ancillary Material

• Open Social Work Education

About the Book

As an introductory textbook for social work students studying research methods, this book guides students through the process of creating a research project. Students will learn how to discover a researchable topic that is interesting to them, examine scholarly literature, formulate a proper research question, design a quantitative or qualitative study to answer their question, carry out the design, interpret quantitative or qualitative results, and disseminate their findings to a variety of audiences. Examples are drawn from the author's practice and research experience, as well as topical articles from the literature.

There are ancillary materials available for this book.  

About the Contributors

Matt DeCarlo earned his PhD in social work at Virginia Commonwealth University and is an Assistant Professor of Social Work at Radford University. He earned an MSW from George Mason University in 2010 and a BA in Psychology from the College of William and Mary in 2007. His research interests include open educational resources, self-directed Medicaid supports, and basic income. Matt is an Open Textbook Network Campus Leader for Radford University. He is the founder of Open Social Work Education, a non-profit collaborative advancing OER in social work education.

Contribute to this Page

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

21 12. Survey design

Chapter outline.

• What is survey research? (15 minute read time)
• Conducting a survey (18 minute read time)
• Creating a questionnaire (16 minute read time)
• Strengths and challenges of survey research (11 minute read time)

Content warning: examples in this chapter contain references to racial inequity, mental health treatment/symptoms/diagnosis, sex work, burnout and compassion fatigue, involuntary hospitalization, terrorism, religious beliefs and attitudes, drug use, physical (chronic) pain, workplace experience and discrimination.

12.1 What is survey research?

Learning Objectives

Learners will be able to…

• Demonstrate an understanding of survey research as a type of research design
• Think about the potential uses of survey research in their student research project

Surveys are a type of design

Congratulations! Your knowledge of social work research project has evolved. You have learned new terminology and the processes needed to develop good questions and to select the best measurement tools to answer your questions.  Now, we will the transition to a discussion on research design.

We are in Part 3: Using quantitative methods of this research text; therefore, the first designs we will discuss are those that focus on collecting data for quantitative analysis.  The first design we will discuss is survey design. Note: It is important to remember that even though survey design is featured in the quantitative methods section of this text, survey design research may also be used to collect qualitative data or a combination of both qualitative and quantitative data. In about six chapters from now, the following section of the text, Part 4: Qualitative Methods, will provide a more detailed focus on collecting qualitative data.

So, what do we mean when we use the term “research design?” When we think of research designs, we are thinking about an overall strategy or approach used to conduct research projects. [1] This chapter discusses survey design which involves strategies for conducting research that utilize a set of questions (contained in a questionnaire) to gain specific information from participants about their opinions, perceptions, reactions, knowledge, beliefs, values, or behaviors.

Caution: It is important to preface this chapter with a statement about the distinction between a questionnaire and survey design. Most people use these definitions interchangeably; however, they are quite different. The term  “survey” is used in research design and involves asking questions and collecting and using tools to analyze data. [2] Specifically, the term “survey” denotes the overall strategy or approach to answering questions. Conversely, the term questionnaire is the actual tool that collects data. So, in essence, researchers use a questionnaire to engage in survey research. This chapter will teach you how to employ a research approach that uses questionnaires to collect information.

The good news is that we have all been exposed to survey research. At the end of the semester when you complete your course evaluations, you are engaging in survey research. If you have ever completed any type of satisfaction questionnaire, you have completed survey research. In fact, every ten years, a random selection of individuals living in the United States are asked to participate in a large-scale survey research project that is conducted by the United States Census Bureau. So, survey research is widespread and familiar to many people, even those who do not have a formal understanding of research terminology.

This section further defines elements of survey research and provides an overview of the characteristics that distinguish survey research from other types of research. As you read this section, please think about your research project and how survey research might be used to help you answer your research question.

Survey research is frequently employed by social work researchers because we often seek to develop an understanding of how groups of people, communities, organizations, and population feel about a certain topic.  Social workers might seek to gather survey data from:

• Neighborhood residents
• People who possess certain characteristics or experiences
• Family members or people affected by a particular condition or experience
• Staff at an agency
• Service recipients
• The general public
• People with specialized knowledge in a given area
• Members of an organization or group

As you think about your research topic, you will likely select one (or maybe two) of these viewpoints to survey as you collect your data. However, it can be helpful to think about how these various perspectives might contribute to research in your given area. As a thought activity, try to fill out as many examples as you can of who you might consider collecting survey data from for your topic.

For example, suppose I am interested in researching the topic of perceptions of racial inequity.

• Neighborhood residents: I could survey two different neighborhoods, one that is more racial diverse and one that is more racially similar (homogenous) 
• People who possess certain characteristics or experiences: I could specifically survey people who are part of an interracial family
• Family members or people affected by a particular condition or experience: I could survey people who have a loved one that has been incarcerated 
• Staff at an agency: I could survey staff from agencies that serve predominately communities of color, but where the agency staff makeup is predominately white
• Service recipients: I could survey service recipients from agencies that serve predominately communities of color, but where the agency staff makeup is predominately white
• The general public: I could survey people at a large local shopping mall 
• People with specialized knowledge in a given area: I could survey state legislators   
• Members of an organization or group: I could survey members of racial justice advocacy organizations 

These are just a small sample of groups that could be surveyed. For each category, we could go in many different directions with many perspectives that can make valuable contributions to this topic.  That is what makes research so exciting…the possibilities are limitless!

Characteristics of survey research

Quite simply, survey research is a type of research design that has two important characteristics. First, the variables of interest are measured using self-reports. These self-reports are gathered by questionnaires, either completed independently by a participant or administered by a member of a research team. Researchers ask their participants , the people who have opted to participate in the research, to report directly on their own thoughts, feelings, and behaviors. Second, often survey research is conducted to understand something about a larger population; remember, this is known as generalizing results. Consequently, considerable attention is paid to the type of sampling and the number of cases used. In general, researchers using a survey design have a preference for large randomly selected samples because they provide the most accurate estimates of what is true in the population.

In previous chapters, we learned about the purposes of research ( exploratory , descriptive , and explanatory ). Survey research can be used for all of these types of research; however, it may be a little challenging to use with exploratory research. Why? The purpose of exploratory research is to uncover experiences in which little is known. Therefore, you may lack the knowledge base needed to develop your questionnaire.

Survey research is best suited for studies that have individual people as the unit of analysis . However, other units of analysis, such as families, groups, organizations, or communities may also be used in survey research. If researchers use a family, group, organization, or community as the unit of analysis,  they usually denote a specific person who is identified as a key informant or a “proxy” to complete the actual research tool. Researchers must be intentional with these choices, as they may introduce measurement error if the informant chosen does not have adequate knowledge or has a biased opinion about the phenomenon of interest.

For instance, many schools of social work are very interested in the school of social work rankings that are published annually by US News and World Report. For a full description of the methodology used in this process, please visit https://www.usnews.com/education/best-colleges/articles/how-us-news-calculated-the-rankings. Many students are not aware that these rankings are actually composite scores created by analyzing a variety of data sources. One type of data used in this process is known as peer review data, or data in which schools provide feedback on their perceptions of similar schools. A questionnaire is sent to several key informants at each school. Each key informant is asked to rank the other schools of social work on a variety of dimensions. These data are then collected and combined with other indicators to calculate the school rankings. However, what if an informant is unfamiliar with a school or has a personal bias against a school? This could significantly skew results. In summary, if you are not using individuals as the unit of analysis, it is important that you choose the right key informant who is knowledgeable about the topic of which you are asking, and who can provide an unbiased perspective.

Finally, most survey research is used to describe single variables (e.g., voter preferences, motivation, or social support) and to assess statistical relationships between variables (e.g., the relationship between income and health). For instance, Nesje (2016) used a survey design to understand the relationship between profession and personality traits. The author was interested in studying the relationship between two variables, personality (empathy and care) and selected profession (social work, nursing, or education). Specifically, Nesje sought to understand if a certain field of study had practitioners with higher levels of empathy and care than others. The author administered two tools, the Blau’s Career Commitment Scale and Orlinsky and Rønnestad’s Interpersonal Adjective Scale, to 1,765 students. Results failed to find a statistically significant difference between groups on the levels of empathy and care. [3]

The above example illustrates several characteristics of a survey research design. Please complete the following interactive exercise to see if you can identify the characteristics of survey research design that are found in this study.

History of survey research

Survey research has roots in English and American “social surveys” conducted around the turn of the 20th century by researchers and reformers who wanted to document the proliferation of social problems such as poverty (Converse, 1987) . [4] By the 1930s, the US government was conducting surveys to document economic and social conditions in the country. The need to draw conclusions about the entire population helped spur advances in sampling procedures. At about the same time, several researchers who had already made a name for themselves in market research studying consumer preferences for American businesses turned their attention to election polling. A watershed event was the presidential election of 1936 between Alf Landon and Franklin Roosevelt. A magazine called Literary Digest  conducted a survey by sending ballots (which were also subscription requests) to millions of Americans. Based on this “straw poll,” the editors predicted that Landon would win in a landslide. At the same time, the new pollsters were using scientific methods with much smaller samples to predict just the opposite—that Roosevelt would win in a landslide. In fact, one of them, George Gallup, publicly criticized the methods of  Literary Digest before the election and all but guaranteed that his prediction would be correct. And of course, it was. Interest in surveying around election times has led to several long-term projects, notably the Canadian Election Studies which has measured opinions of Canadian voters around federal elections since 1965.  Anyone can access the data and read about the results of the experiments in these studies (see  http://ces-eec.arts.ubc.ca/ )

From market research and election polling, survey research made its way into several academic fields, including political science, sociology, and public health—where it continues to be one of the primary approaches to collecting new data. Beginning in the 1930s, psychologists made important advances in questionnaire design, including techniques that are still used today, such as the Likert scale. We will discuss Likert scales later in this chapter.  Survey research has a strong historical association with the social psychological studies of attitudes, stereotypes, and prejudice. Survey research has also been used by social workers to understand a variety of conditions and experiences. 

In summary, survey research is a valuable research design, and one that may be used to study a variety of concepts. This flexibility of survey research allows it to be applied to many research projects, making it appealing for a variety of disciplines. Furthermore, its potential to gather information from a large number of people with a relatively low commitment of resources (compared to other methods) can also make it quite attractive to social science researchers.   

Survey research in social work

The above section mentioned concern with the sample size and type of sampling as being important considerations for survey research. In general, many studies using survey research have the goal of generalizable findings from a sample to a population. That said, if you conduct a literature search for studies using survey research, you will find that most large survey research studies utilizing random sampling are conducted by psychologists or sponsored by large non-profit or government research organizations such as the Pew Research ( https://www.pewresearch.org/ ) Center or the United States Census Bureau ( https://www.census.gov/ ). For example, each year, the Pew Research Center randomly selects and interviews thousands of people in order to study a variety of social attitudes and beliefs. Additionally, every ten years, the U.S Census bureau implements a large-scale data collection process to understand population characteristics and changes. Both of these organizations seek to generalize sample results to the larger US population. Finally, since 1984 the Center for Disease Control and Prevention (CDC) ( https://www.cdc.gov/) has maintained the Behavioral Risk Factor Surveillance System, “the nation’s premier system of telephone surveys that collect state-level data about health risk behaviors, chronic health conditions, and use of preventive services” [5] . While often gathered by professionals in other disciplines, all of these sources of survey data can be very useful for social workers seeking to look at quantitative data across a variety of topics.

So, why are social work researchers less likely to utilize large probability sampling techniques? Due to the nature of the client systems with which we work, sometimes collecting large random samples may not be feasible. Remember that in order to utilize a probability sample, you need to have access to a considerable  sampling frame .  Many of the populations with which we work are “hidden” or harder to access. Thus, securing a list of all possible cases would be challenging, if not impossible.  For example, think about a researcher wanting to study sex workers operating in a certain neighborhood. The researcher may have difficulty finding a list of all of the persons engaging in sex work in that neighborhood. The researcher could look at arrest records and seek to find all sex workers with an arrest record. However, having this list does not mean that the researcher would have access to sex workers. Next, sometimes social workers want to understand individual experiences so that they bring the perspectives of marginalized groups into the mainstream scholarly literature. These social workers may be less concerned with generalizing results and more concerned with “uncovering or discovering knowledge from oppressed groups”. For those social workers, a smaller-scale qualitative research project may be more feasible and allow the researcher to meet their goals.

As previously mentioned, social work practitioners are less likely to use large-scale probability samples. While they are less likely to implement these, there are situations where large-scale probability samples are used by social workers. For example,  university-affiliated social work academics who have received federal grants may conduct multi-site projects. Additionally, licensing organizations such as the NASW may utilize questionnaires to collect information about members’ practice experiences. Furthermore, social work researchers are often part of interdisciplinary teams that may extend resources and access to larger sampling frames.

Social work student projects and survey design research

Within social work schools, students are usually required to demonstrate their proficiency in basic research by implementing an empirical study. Many students end up implementing a project that utilizes survey design, often selected due to convenience. In addition, sometimes agencies have existing questionnaires they want to be used for the student research project. Agencies may feel more comfortable with students using survey design research instead of other designs. For example, interviewing clients may be seen as part of students’ existing responsibilities; whereas implementing an experimental or quasi-experimental design may seem more time-consuming and labor-intensive for the agency. Further, my s tudents have found survey research projects to be interesting, intellectually rewarding, and feasible. Below is a list of past social work research projects that were conducted by second-year MSW students. Can you see how each of these studies involves students asking participants to provide information (orally or in writing) that is then analyzed?

Past Student Research Projects

• What is the level of interpersonal relationship satisfaction among those diagnosed with an eating disorder?
• Does age, gender and/or DSM 5 diagnoses indicate the level of mental health support that clients receive?
•  For those seen in the XXX, Is there a difference in IPV injury patterns by gender?
• Does worker burn-out rate differ between departments within social service agencies?
• Is there a correlation between poor physical health and poor mental health functioning in college freshmen at XXX?
• Is there a relationship between burnout and compassion satisfaction among healthcare professionals who work in a mental health facility?
• Is there a difference in the levels of compassion fatigue and compassion satisfaction among the different types of direct service employees at the XXX agency?
• Is there a difference in the length of stay at XXX Hospital between individuals admitted voluntarily and those admitted involuntarily?
• What are the primary concerns that cause college students to present for services at their university’s counseling center?
• Does an individual’s level of stress influence treatment decisions?

Key Takeaways

• Survey research is common and used to gather a variety of information.
• Survey research is a design/approach, and a questionnaire is an actual tool used to collect data. While these words are often used interchangeably, they are different things.
• Two characteristics define survey research: participants being asked to provide information and a focus on sample size and sampling.
• Large random samples provide the opportunity to generalize results from your sample to the population from which it was drawn; however, this is often not possible for social work researchers.
• Successful questionnaire development takes time and requires feedback from multiple sources.

Think about your research project at this point.

• Why do you think this is the most appropriate way to gather data?
• Begin thinking about how you will access your population. What are some barriers you might experience to administering a survey?
• What made you decide not to use a survey? This is not to say you should use one!
• Are there related research questions to the one you chose that you could use a survey to answer?

12.2 Conducting a survey

• Define cross-sectional surveys, provide an example of a cross-sectional survey, and outline some of the drawbacks of cross-sectional research
• Describe the three types of longitudinal surveys
• Describe retrospective surveys and identify their strengths and weaknesses
• Discuss the benefits and drawbacks of the various methods of administering surveys

There is immense variety when it comes to surveys. This variety includes both how the survey is intended to reflect time and how the survey is administered or delivered to participants. In this section, we’ll look at variations across these two dimensions.

With respect to time, survey design is generally divided into two types: cross-sectional or longitudinal. Cross-sectional surveys are those that reflect responses that are given at just one point in time. These surveys offer researchers a snapshot in time and offer an idea about how things are for the respondents at the particular point in time that the survey is administered.

An example of a cross-sectional survey comes from Aniko Kezdy and colleagues’ study (Kezdy, Martos, Boland, & Horvath-Szabo, 2011) [1] of the association between religious attitudes, religious beliefs, and mental health among students in Hungary. These researchers administered a single, one-time-only, cross-sectional survey to a convenience sample of 403 high school and college students. The survey focused on how religious attitudes impact various aspects of one’s life and health. The researchers found from analysis of their cross-sectional data that anxiety and depression were highest among those who had both strong religious beliefs and some doubts about religion.

Yet another recent example of cross-sectional survey research can be seen in Bateman and colleagues’ study (Bateman, Pike, & Butler, 2011) [2] of how the perceived ‘publicness’ of social networking sites influences users’ self-disclosures. These researchers administered an online survey to undergraduate and graduate business students to understand perceptions and behaviors on this topic. They found that even though revealing information about oneself is viewed as key to realizing many of the benefits of social networking sites, respondents were less willing to disclose information about themselves as their perceptions of a social networking site’s publicness rose. That is, there was a negative relationship between perceived publicness of a social networking site and plans to self-disclose on the site.

One problem with cross-sectional surveys is that the events, opinions, behaviors, and other phenomena that such surveys are designed to assess don’t generally remain stagnant. They change over time and may be influenced by any number of things. Thus, generalizing from a cross-sectional survey about the way things are can be tricky; perhaps you can say something about the way things were in the moment that you administered your survey, but it is difficult to know whether things remained that way for long after you administered your survey. Think, for example, about how Americans might have responded if they received a survey asking for their opinions on terrorism on September 12, 2000. Now imagine how responses to the same set of questions might differ were they administered on September 12, 2001. The point is not that cross-sectional surveys are useless; they have many important uses. But researchers must remember what they have captured by administering a cross-sectional survey—that is, as previously noted, a snapshot of life as it was at the time that the survey was administered.

One way to overcome this sometimes-problematic aspect of cross-sectional surveys is to administer a longitudinal survey.  Longitudinal surveys are those that enable a researcher to make observations over some extended period of time. There are several types of longitudinal surveys, including trend, panel, and cohort surveys. We’ll discuss all three types here, along with retrospective surveys. Retrospective surveys fall somewhere in between cross-sectional and longitudinal surveys.

The first type of longitudinal survey is called a  trend survey . The main focus of a trend survey is, perhaps not surprisingly, trends. Researchers conducting trend surveys are interested in how people in a specific group change over time. Each time the researchers gather data, they ask different people from the group they are studying because their concern is capturing the sentiment of the group, not the individual people they survey. Let’s look at an example.

The Monitoring the Future Study ( http://www.monitoringthefuture.org/ ) is a trend study that described the substance use of high school children in the United States. It’s conducted annually by the National Institute on Drug Abuse (NIDA). Each year NIDA distributes surveys to children in high schools around the country to understand how substance use and abuse in that population changes over time. Perhaps surprisingly, fewer high school children reported using alcohol in the past month than at any point over the last 20 years. Recent data also reflected an increased use of e-cigarettes and the popularity of e-cigarettes with no nicotine over those with nicotine. The data points provide insight into targeting substance abuse prevention programs and resources. As you will note, this study is looking at general trends for this age group; it is not interested in tracking the changing attitudes or behaviors of specific students over time.

Unlike in a trend survey, in a  panel survey the same people participate in the survey each time it is administered. As you might imagine, panel studies can be difficult and costly. Imagine trying to administer a survey to the same 100 people every year for, say, 5 years in a row. Keeping track of where people live, when they move, how to contact them and when they die, etc. takes resources that researchers often don’t have. When they do, however, the results can be quite powerful. The Youth Development Study (YDS), administered from the University of Minnesota, offers an excellent example of a panel study.

Since 1988, YDS researchers have administered an annual survey to the same 1,000 people. Study participants were in ninth grade when the study began, and they are now in their thirties. Several hundred papers, articles, and books have been written using data from the YDS. One of the major lessons learned from this panel study is that work has a largely positive impact on young people (Mortimer, 2003).  [3] Contrary to popular beliefs about the impact of work on adolescents’ performance in school and transition to adulthood, work in fact increases confidence, enhances academic success, and prepares students for success in their future careers. Without this panel study, we may not be aware of the positive impact that working can have on young people. You can read more about the Youth Development Study at its website: https://cla.umn.edu/sociology/graduate/collaboration-opportunities/youth-development-study .

Another type of longitudinal survey is a cohort survey. In a  cohort survey , the participants have a defining characteristic that the researcher is interested in studying. The same people don’t necessarily participate from year to year, but all participants must meet whatever categorical criteria fulfill the researcher’s primary interest. Common cohorts that may be of interest to researchers include people of particular generations or those who were born around the same time period, graduating classes, people who began work in a given industry at the same time, or perhaps people who have some specific historical experience in common.

An example of this sort of research can be seen in Christine Percheski’s work (2008)  [4] on cohort differences in women’s employment. Percheski compared women’s employment rates across seven generational cohorts, from Progressives born between 1906 and 1915 to Generation Xers born between 1966 and 1975. She found, among other patterns, that professional women’s labor force participation had increased across all cohorts. She also found that professional women with young children from Generation X had higher labor force participation rates than similar women from previous generations, concluding that mothers do not appear to be opting out of the workforce as some journalists have speculated (Belkin, 2003).  [5]

All three types of longitudinal surveys share the strength in that they permit a researcher to make observations over time. This means that if whatever behavior or other phenomenon the researcher is interested in changes, either because of some world event or because people age, the researcher will be able to capture those changes. Table 12.1 summarizes these three types of longitudinal surveys.

Finally,  retrospective surveys are similar to other longitudinal studies in that they deal with changes over time, but like a cross-sectional study, they are administered only once. In a retrospective survey, participants are asked to report events from the past. By having respondents report past behaviors, beliefs, or experiences, researchers are able to gather longitudinal-like data without actually incurring the time or expense of a longitudinal survey. Of course, this benefit must be weighed against the highly likely possibility that people’s recollections of their pasts may be faulty, incomplete,or slightly modified by the passage of time. Imagine, for example, that you’re asked in a survey to respond to questions about where, how, and with whom you spent last Valentine’s Day. As last Valentine’s Day can’t have been more than 12 months ago, chances are good that you might be able to respond accurately to some survey questions about it. But now let’s say the researcher wants to know how last Valentine’s Day compares to previous Valentine’s Days, so she asks you to report on where, how, and with whom you spent the preceding six Valentine’s Days. How likely is it that you will remember? Will your responses be as accurate as they might have been had you been asked the question each year over the past 6 years, rather than asked to report on all years today?

In sum, when or with what frequency a survey is administered will determine whether your survey is cross-sectional or longitudinal. While longitudinal surveys are certainly preferable in terms of their ability to track changes over time, the time and cost required to administer a longitudinal survey can be prohibitive. Furthermore, by maintaining and accessing contact information for participants over long periods of time, we are increasing the opportunities for their privacy to be compromised. The issues of time described here are not necessarily unique to survey research. Other methods of data collection can be cross-sectional or longitudinal—these are larger matters of research design that really apply to all types of research. But we’ve placed our discussion of these terms here because they are most commonly used by survey researchers to describe the type of survey administered. Another aspect of survey design deals with how surveys are administered. We’ll examine that next.

Administration

Surveys vary not just in terms of the way they deal with time, but also in terms of how they are administered. One common way to administer surveys is through self-administered questionnaires . This means that a research participant is given a set of questions, in writing, to which they are asked to respond to autonomously.  These questionnaires can be hard copy or virtual. We’ll consider both modes of delivery here.

Hard copy self-administered questionnaires may be delivered to participants in person or via snail mail. Perhaps you’ve take a survey that was given to you in person; on many college campuses, it is not uncommon for researchers to administer surveys in large social science classes (as you might recall from the chapter on sampling). If you are ever asked to complete a survey in a similar setting, it might be interesting to note how your perspective on the survey and its questions could be shaped by the new knowledge you’re gaining about survey research in this chapter.

Researchers may also deliver surveys in person by going door-to-door or in public spaces by either asking people to fill them out right away or making arrangements for the researcher to return to pick up completed surveys or having them dropped off or mailed (with a self-addressed stamped envelope provided) to a designated location. The advent of online survey tools and greater widespread internet access has made door-to-door and snail mail delivery of surveys much less common, although I still see an occasional survey researcher at my door, especially around election time. This mode of gathering data is apparently still used by political campaign workers, at least in some areas of the country.

While choosing snail mail to disseminate your survey may not be ideal (imagine how much  less likely you’d probably be to return a survey that didn’t come with the researcher standing on your doorstep waiting to take it from you), sometimes it is the only available or the most practical option. As mentioned, though, this may not be the most ideal way of administering a survey because it can be difficult to convince people to take the time to complete and return your survey. Additionally, mail that is received and not recognized may be regarded with suspicion or ignored altogether.  If you are choosing to mail out your survey by post, make sure you are very thoughtful about the materials, including the envelope.  They should look professional, but also personalized whenever possible to help engage the participant quickly.  Chances are you worked hard on your study – the last thing you want is the potential participant to receive your survey in the mail and chuck it in the waste bin without even opening it!

Often survey researchers who deliver their surveys via snail mail may provide some advance notice to respondents about the survey to get people thinking about and preparing to complete it. They may also follow up with their sample a few weeks after their survey has been sent out. This can be done not only to remind those who have not yet completed the survey to please do so but also to thank those who have already returned the survey. Most survey researchers agree that this sort of follow-up is essential for improving mailed surveys’ return rates (Babbie, 2010).  [6]  Other helpful tools to increase response rate are to create an attractive and professional survey, offer monetary incentives, and provide a pre-addressed, stamped return envelope.

Earlier, I mentioned online delivery as another way to administer a survey. This delivery mechanism is becoming increasingly common, no doubt because it is easy to use, relatively cheap, and may be more efficient than knocking on doors or waiting for mailed surveys to be returned. To deliver a survey online, the most frequent method employed by researchers is to use an online survey management service or application.  These might be paid subscription services, like SurveyMonkey ( https://www.surveymonkey.com ) or Qualtrics ( https://www.qualtrics.com ), or free applications, like Google Forms. With any of these options you will design your survey online and then be provided a link to send out to your potential participants either via email or by posting the link in a virtually accessible space, like a forum, group, or webpage.  Wherever you choose to share the link, you will need to consider how you will gain permission to do so, which may mean getting permission to use a distribution list of emails or gaining permission from a group forum administer to post a link in the forum for members to access.

Many of the suggestions provided for improving the response rate on a hard copy questionnaire apply to online questionnaires as well. One difference of course is that the sort of incentives one can provide in an online format differ from those that can be given in person or sent through the mail. But this doesn’t mean that online survey researchers cannot offer completion incentives to their respondents. I’ve taken a number of online surveys; many of these did not come with an incentive other than the joy of knowing that I’d helped a fellow social scientist do their job. However, for participating in one survey, I was given a coupon code to use for $30 off any order at a major online retailer. I’ve taken other online surveys where on completion I could provide my name and contact information if I wished to be entered into a lottery together with other study participants to win a larger gift, such as a $50 gift card or an iPad.

Online surveys, however, may not be accessible to individuals with limited, unreliable, or no access to the internet or less skill at using a computer. If those issues are common in your target population, online surveys may not work as well for your research study. While online surveys may be faster and cheaper than mailed surveys, mailed surveys are more likely to reach your entire sample but also more likely to be lost and not returned. The choice of which delivery mechanism is best depends on a number of factors, including your resources, the resources of your study participants, and the time you have available to distribute surveys and wait for responses. Understanding the characteristics of your study’s population is key to identifying the appropriate mechanism for delivering your survey.

Sometimes surveys are administered by having a researcher pose questions verbally to respondents, rather than having respondents read the questions on their own. Researchers using phone or in-person surveys use an interview schedule which contains the list of questions and answer options that the researcher will read to respondents. Consistency in the way that questions and answer options are presented is very important with an interview schedule. The aim is to pose every question-and-answer option in the same way to every respondent. This is done to minimize interviewer effect, or possible changes in the way an interviewee responds based on how or when questions and answer options are presented by the interviewer. In-person surveys may be recorded, but because questions tend to be closed ended, taking notes during the interview is less disruptive than it can be during a qualitative interview.

Interview schedules are used in phone or in-person surveys and are also called quantitative interviews. Phone surveys are often conducted by political polling firms to understand how the electorate feels about certain candidates or policies. In both cases, researchers pose questions verbally to participants. As someone who has poor research karma, I often decline to participate in phone studies when I am called. It is easy, socially acceptable even, to hang up abruptly on an unwanted caller. Additionally, a distracted participant who is cooking dinner, tending to troublesome children, or driving may not provide accurate answers to your questions. Phone surveys make it difficult to control the environment in which a person answers your survey. Another challenge comes from the increasing number of people who only have cell phones and do not use landlines (Pew Research, n.d.).  [7]  Unlike landlines, cell phone numbers are portable across carriers, associated with individuals, not households, and do not change their first three numbers when people move to a new geographical area. Computer-assisted telephone interviewing (CATI) programs have also been developed to assist quantitative survey researchers. These programs allow an interviewer to enter responses directly into a computer as they are provided, thus saving hours of time that would otherwise have to be spent entering data into an analysis program by hand.

Quantitative interviews must also be administered in such a way that the researcher asks the same question the same way each time. While questions on hard copy questionnaires may create an impression based on the way they are presented, having a person administer questions introduces a slew of additional variables that might influence a respondent. Even a slight shift in emphasis on a word may bias the respondent to answer differently. As I’ve mentioned earlier, consistency is key with quantitative data collection—and human beings are not necessarily known for their consistency. On the positive side, quantitative interviews can help reduce a respondent’s confusion. If a respondent is unsure about the meaning of a question or answer option on a self-administered questionnaire, they probably won’t have the opportunity to get clarification from the researcher. An interview, on the other hand, gives the researcher an opportunity to clarify or explain any items that may be confusing. If a participant asks for clarification, the researcher often uses pre-determined responses to make sure each quantitative interview is exactly the same as the others.

In-person surveys are conducted in the same way as phone surveys but must also account for non-verbal expressions and behaviors. In-person surveys do carry one distinct benefit—they are more difficult to say “no” to. Because the participant is already in the room and sitting across from the researcher, they are less likely to decline than if they clicked “delete” for an emailed online survey or pressed “hang up” during a phone survey.  In-person surveys are also much more time consuming and expensive than mailing questionnaires. Thus, quantitative researchers may opt for self-administered questionnaires over in-person surveys on the grounds that they will be able to reach a large sample at a much lower cost than were they to interact personally with each and every respondent.

• Time is a factor in determining what type of survey a researcher administers; cross-sectional surveys are administered at one time, and longitudinal surveys are administered over time.
• Retrospective surveys offer some of the benefits of longitudinal research but also come with their own drawbacks.
• Self-administered questionnaires may be delivered in hard copy form to participants in person or via snail mail or online.
• Interview schedules are used with in-person or phone surveys.
• Each method of survey administration comes with benefits and drawbacks.

Think about the population you want to research.

• Which type of survey (i.e., in-person, telephone, web-based, by mail) do you think would most effectively reach your population? Why?
• Are there elements of your population you could miss by choosing one of these ways to administer your survey? How might this affect your results?

12.3 Writing a questionnaire

• Define different formats of questions
• Describe the principles of a good survey question
• Discuss the importance of pilot testing questions
• Understand principles of question development
• Evaluate questionnaire and interview questions

How are questionnaires developed? Developing an effective questionnaire takes a long time and is both a science and art. It is a science because the questionnaire should be developed based on accepted principles of questionnaire development that have evolved over time and practice. For instance, you must be attentive to issues of conceptual development, as well as reliability and validity. On the other hand, questionnaire development is also an art because it must take into account things such as color, font, use of white-space, etc. that will make a written questionnaire aesthetically pleasing. Researchers who develop questionnaires rely on colleagues and pilot testing to refine their measurement tools.

When implementing a survey, conduct an initial literature search to determine if there are existing questionnaires or interview questions you may use for your study. If not, you must create your own tool or tools, which may be a challenging process. You must have a strong understanding of what you want to ask, why you want to ask it, and how you want to ask. You need to be able to understand the potential barriers to your project and take these into account as you design your instrument(s). As discussed above, surveys are often self-administered. This means they must stand on their own so that they can be correctly understood and interpreted by your research participants.  While this may seem like an easy task, you would be surprised how quickly things get misinterpreted!

How to ask the right questions

How are items for questionnaires and interviews developed? Questions should be developed based on existing principles concerning item development. Remember that a questionnaire is developed to measure some variable or concept. We are often going to develop a series of questions that will help us to gather data about various aspects of that variable.  These questions should be grounded in the existing literature on your topic and should comprehensively assess the variable you are seeking to understand. For instance, if I develop a questionnaire about depression, but I don’t ask any questions about loss of interest in doing things, it would be a major gap in the information I am collecting about this variable. A good literature search will help me to identify the various areas that I will need to ask about in my questionnaire so that I can get the most complete picture of depression from participants. Questionnaire items must take into account idiosyncrasies regarding language, meaning that we need to anticipate the variety of ways that people might read and process the meaning of a question and its responses. Continuing on with the depression questionnaire example, we might ask a question about whether people feel blue much of the time. While it might be evident to you or I that the phrase “feeling blue” means experiencing low mood or sadness, that might not be interpreted the same by everyone, especially across cultural groups. Remember, being attentive to the way in which you ask questions is critical.

The next few sections will discuss the different characteristics of questionnaires and interviews and provide guidance on writing effective questions. Please note that this section discusses “guidelines.”  There may be times when these guidelines are not relevant. It is up to you as the researcher to read each guideline and determine if your study requires exceptions to them.

Guidelines for creating good questions

Crafting good questions is hard and requires thoughtful attention, feedback and revision. Below are some resources that will aid you in these tasks.

Participants in survey research are very sensitive to the types of questions asked. Poorly framed or ambiguous questions will likely result in meaningless responses with little value. Dillman (1978) provides several “rules” or guidelines for creating good questions: 

Every question should be carefully scrutinized for the following issues:

• Is the question clear and understandable? Questions should use very simple language, preferably in the active voice and without complicated words or jargon that may not be understood by a typical participant. All questions in the questionnaire should be worded in a similar manner to make it easy for respondents to read and understand them. The only exception is if your questionnaire is targeted at a specialized group of respondents, such as doctors, lawyers, and researchers, who use such jargon in their everyday work environment.
• Is the question worded in a negative manner? Negatively worded questions, such as “Should your local government not raise taxes?” tend to confuse participants  and lead to inaccurate responses. Such questions should be avoided, and in all cases, avoid double-negatives.
• Is the question ambiguous? Questions should not use words or expressions that may be interpreted differently by different participants (e.g., words like “any” or “just”). For instance, if you ask a respondent, what is your annual income, it is unclear whether you referring to salary/wages, or also dividend, rental, and other income, whether you referring to personal income, family income (including spouse’s wages), or personal and business income? Different interpretations will lead to incomparable responses that cannot be interpreted correctly.
• Does the question have biased or value-laden words? Bias refers to any property of a question that encourages participants to answer in a certain way. As social workers, we understand how we must be intentional with language. For instance, Kenneth Rasinky (1989) examined several studies on people’s attitudes toward government spending and observed that respondents tend to indicate stronger support for “assistance to the poor” and less for “welfare,” even though both terms had the same meaning. Remember the difference in public perception between “Obamacare” and the “Affordable Care Act?” Biased language or tone tends to skew observed responses. In summary, qu estions should be carefully evaluated to avoid biased language.
• Is the question double-barreled? Double-barreled questions are those that can have multiple answers. For example, are you satisfied with your professor’s grading style and lecturing? In this example, how should a respondent answer if they are satisfied with the grading style but not the lecturing and vice versa? It is always advisable to separate double-barreled questions into separate questions: (1) are you satisfied with your professor’s grading? and (2) are you satisfied with your professor’s lecturing? Another example: does your family favor public television? Some people may favor public television for themselves, but favor certain cable television programs such as Sesame Street for their children.
• Is the question too general? Sometimes, questions that are too general may not accurately convey respondents’ perceptions. If you asked someone how they liked a certain book and provide a response scale ranging from “not at all” to “extremely well”, and if that person selected “extremely well,” what do they mean? Instead, ask more specific behavioral questions, such as “Will you recommend this book to others?” or “Do you plan to read other books by the same author?” 
• Is the question too detailed? Avoid unnecessarily detailed questions that serve no specific research purpose. For instance, do you need the age of each child in a household or is just the number of children in the household acceptable? However, if unsure, it is better to err on the side of details than generality.
• Is the question presumptuous? Does your question make assumptions? For instance, if you ask, “what do you think the benefits of a tax cut would be?” you are presuming that the participant sees the tax cut as beneficial. But many people may not view tax cuts as beneficial. Some might see tax cuts as a precursor to less funding for public schools and fewer public services such as police, ambulance, and fire department. Avoid questions with built-in presumptions.
• Does the question ask the participant to imagine something? Is the question imaginary? A popular question on many television game shows is “if you won a million dollars on this show, how will you plan to spend it?” Most participants have never been faced with this large amount of money and have never thought about this scenario. In fact, most don’t even know that after taxes, the value of the million dollars will be greatly reduced. In addition, some game shows spread the amount over a 20-year period. Without understanding this “imaginary” situation, participants may not have the background information necessary to provide a meaningful response.

Another way to examine questions is to use the BRUSO model (Peterson, 2000) . [6] Note: Here this model is focused on questionnaires; however, it is also relevant for interview questions. An acronym, BRUSO  stands for “brief,” “relevant,” “unambiguous,” “specific,” and “objective.” Effective questionnaire items are  brief and to the point. They avoid long, overly technical, or unnecessary words. This brevity makes it easier for respondents to understand and faster for them to complete. Effective questionnaire items are also  relevant to the research question. If a respondent’s sexual orientation, marital status, or income is not relevant, then items requesting information on them should probably not be included. Again, this makes the questionnaire faster to complete, but it also avoids annoying respondents with what they will rightly perceive as irrelevant or even “nosy” questions. Effective questionnaire items are also unambiguous ; they can be interpreted in only one way. Part of the problem with the alcohol item presented earlier in this section is that different respondents might have different ideas about what constitutes “an alcoholic drink” or “a typical day.” Effective questionnaire items are also  specific   so that it is clear to respondents what their response  should  be about and clear to researchers what it  is about. A common problem here is closed-ended items that are “double-barreled.” They ask about two conceptually distinct issues but allow only one response. For example, “Please rate the extent to which you have been feeling anxious and depressed.” This item should probably be split into two separate items—one about anxiety and one about depression. Finally, effective questionnaire items are objective in the sense that they do not reveal the researcher’s own opinions or lead participants to answer in a particular way. 

Response formats

Questions may be found on questionnaires and in interview guides in a variety of formats. When developing questions, it is important to think about the type of data you will collect and how useful it will be to your project. Remember our discussion on levels of measurement ?  When you think about the format of your questions, it is also important to think about the level of measurement. Are you concerned with yes/no answers? Dichotomous response questions would work well for you. Do you have items where you really want participants to explain feelings or experiences? Perhaps open-ended items are best.  Is computing an overall score important? You might want to consider using interval-ratio response items or continuous response questions.

Below is a list of some of the different question formats. Remember, questions may be more than one type of format. For instance, you may have a filter question that is a dichotomous response item. As you look at this list, think about the questions that you have been asked in questionnaires or interviews. Which were the most common?

Question Formats

Based on Level of Measurement

• Nominal response question -Participants are presented with more than two un-ordered options, such as: What is your social work track ( Children and Families, Mental Health, Medical Social Work, International Social Work, Planning and Administration)?
• Ordinal response question- Participants have more than two ordered options, such as: what is your highest level of social work education (AS, BSW, MSW, PhD)?
• Interval response question -Participants are presented with an opportunity to indicate a numerical response in which the answer cannot be zero or none. For example, “how old are you?” This type of format can also include answers from a semantic differential scale or Guttman scale. Each of these scale types was discussed in the previous chapter.
• Continuous or ratio response question -Participants enter a continuous (ratio-scaled) value with a meaningful zero point, such as their age or tenure in a firm. These responses generally tend to be of the fill-in-the-blanks type.

Other Types of Questions

• Dichotomous response question -Participants are asked to select one of two possible choices, such as true/false, yes/no, or agree/disagree. An example of such a question is: Do you think those who receive public assistance should be drug tested (Yes or No)?
• Filter or Screening Questions– Questions that screen out/identify a certain type of respondent. For instance, let’s pretend that you want to survey your research class to determine how those with a letter of accommodation (for a disability) are navigating their field placement. One of the first questions is a filter question that asks students if they have a letter of accommodation. In other words, everyone receives the tool but you have a way to “screen in” those who can answer your research question. 
• Close-ended questions– Question type where participants are asked to choose their response from a list of existing responses. For instance, how many semesters of research should MSW students take: one, two, or three?
• Open-ended question– Question type in which participants are asked to provide a detailed answer to a question. For example, “How do you feel about the new medication-assisted recovery center?”
• Matrix question– Matrix questions are used to gather data across a number of variables that all have the same response categories. For examples, I might be interested in knowing “How likely you are to agree with the following statements: I prefer to study in the morning, I prefer to study with music playing, I prefer to study alone, I prefer to study in my room, I prefer to study in a coffee shop”. These are all separate questions, but the responses categories for all of these will be “Strongly Agree, Agree, Neither Agree nor Disagree, Disagree, Strongly Disagree”. When I set this question up I will develop a table or matrix, where the questions form the rows and the responses categories are the columns.

For visual examples, please see this book chapter on types of survey questions which includes some helpful diagrams.

A note about closed-ended questions

Closed-ended questions are used when researchers have a good idea of the different responses participants might make. They are more quantitative in nature, so they are also used when researchers are interested in a well-defined variable or construct such as participants’ level of agreement with some statement, perceptions of risk, or frequency of a particular behavior. Closed-ended items are more difficult to write because they must include an appropriate set of response options. However, they are relatively quick and easy for participants to complete. They are also much easier for researchers to analyze because the responses can be easily converted to numbers and entered into a spreadsheet. For these reasons, closed-ended items are much more common.

For closed-ended items, it is also important to create an appropriate response scale. For categorical variables, the categories presented should generally be mutually exclusive and exhaustive. Mutually exclusive categories do not overlap. For a religion item, for example, the categories of  Christian  and  Catholic  are not mutually exclusive but  Protestant  and  Catholic  are mutually exclusive. Exhaustive categories cover all possible responses. Although  Protestant  and  Catholic  are mutually exclusive, they are not exhaustive because there are many other religious categories that a respondent might select:  Jewish ,  Hindu ,  Buddhist , and so on. In many cases, it is not feasible to include every possible category, in which case an  Other category, with a space for the respondent to fill in a more specific response, is a good solution. If respondents could belong to more than one category (e.g., race), they should be instructed to choose all categories that apply. However, note that when you allow a participant to select more than one category, you need to realize that it may make analyzing your data more complicated. 

For rating scales, five or seven response options generally allow about as much precision as respondents are capable of. However, numerical scales with more options can sometimes be appropriate. For dimensions such as attractiveness, pain, and likelihood, a 0-to-10 scale will be familiar to many respondents and easy for them to use. Regardless of the number of response options, the most extreme ones should generally be “balanced” around a neutral or modal midpoint. 

Putting your questions together

An additional consideration is the “flow” of questions. Imagine being a participant in an interview. In the first scenario, the interviewer begins by asking you to answer questions that are very sensitive. Now imagine another scenario, one in which the interviewer begins with less intrusive questions. Which scenario sounds more appealing? In the first scenario, you might feel caught off guard and uncomfortable. In the second situation, you have time to develop rapport before moving into more sensitive questions.  The order in which you structure your questions matters. Generally,  questions should flow from the least sensitive to the most sensitive and from the general to the specific. A few other considerations are identified in the box below. 

General Rules for Question Sequencing And Other Important Considerations

• Start with easy non-threatening questions that can be easily recalled. Good options are demographics (age, gender, education level) for individual-level surveys and ‘firmographics’ (employee count, annual revenues, industry) for firm-level surveys.
• Never start with an open-ended question.
• If following a historical sequence of events, follow a chronological order from earliest to latest.
• Ask about one topic at a time. When switching topics, use a transition, such as “The next section examines your opinions about …”
• Use filter or contingency questions as needed, such as: “If you answered “yes” to question 5, please proceed to Section 2. If you answered “no” go to Section 3.”  

Also…

• People’s time is valuable. Be respectful of their time. Keep your questionnaire as short as possible and limit it to what is absolutely necessary. Participants do not like spending more than 10-15 minutes on any questionnaire, no matter how important or interesting the topic. Longer surveys tend to dramatically lower response rates.
• Always assure participants about the confidentiality of their responses, and how you will use their data (e.g., for academic research) and how the results will be reported (usually, in the aggregate). Your informed consent should be clear about these.
• For organizational questionnaires, assure participants that you will send a copy of the final results to the organization (and follow through!). 
• Thank respondents for their participation in your study. 
• Finally, and perhaps most importantly, pretest your questionnaire, by at least using a convenience sample, before administering it to your participants. Such pretesting may uncover ambiguity, lack of clarity, or biases in question-wording, which should be eliminated before administering to the intended sample. As a student, you might pretest with classmates, friends, other people at your field agency, etc.  
• Evaluating questions to be used in a questionnaire or interview is critical to the research project. There are many ways to examine your questions.
• There are different types of question formats. The researcher must select the type of question that is consistent with the type of data that they need to collect.
• Draft a few potential questions you might include on a questionnaire as part of a survey for your topic.

12.4 Strengths and challenges of survey research

• Understand the benefits of surveys as a raw data collection method
• Understand the drawbacks of surveys as a raw data collection method

Strengths of survey methods

Researchers employing survey methods to collect data enjoy a number of benefits. First, surveys are an excellent way to gather lots of information from many people. In a study of older people’s experiences in the workplace, researchers were able to mail a written questionnaire to around 500 people who lived throughout the state of Maine at a cost of just over $1,000. This cost included printing copies of a seven-page survey, printing a cover letter, addressing and stuffing envelopes, mailing the survey, and buying return postage for the survey. I realize that $1,000 is nothing to sneeze at, but just imagine what it might have cost to visit each of those people individually to interview them in person. You would have to dedicate a few weeks of your life at least, drive around the state, and pay for meals and lodging to interview each person individually. We could double, triple, or even quadruple our costs pretty quickly by opting for an in-person method of data collection over a mailed survey. Thus, surveys are relatively  cost-effective.

Related to the benefit of cost-effectiveness is a survey’s potential for generalizability. Because surveys allow researchers to collect data from very large samples for a relatively low cost, survey methods lend themselves to probability sampling techniques, which we discussed in Chapter 10. Of all the data collection methods described in this textbook, survey research is probably the best method to use when one hopes to gain a representative picture of the attitudes and characteristics of a large group.

Survey research also tends to be a  reliable method of inquiry. This is because surveys are standardized in that the same questions, phrased in exactly the same way, as they are posed to participants. Other methods, such as qualitative interviewing, which we’ll learn about in Chapter 18, do not offer the same consistency that a quantitative survey offers. This is not to say that all surveys are always reliable. A poorly phrased question can cause respondents to interpret its meaning differently, which can reduce that question’s reliability. Assuming well-constructed questions and survey design, one strength of this methodology is its potential to produce reliable results.

The versatility of survey research is also an asset. Surveys are used by all kinds of people in all kinds of professions. The versatility offered by survey research means that understanding how to construct and administer surveys is a useful skill to have for all kinds of jobs. Lawyers might use surveys in their efforts to select juries, social service and other organizations (e.g., churches, clubs, fundraising groups, activist groups) use them to evaluate the effectiveness of their efforts, businesses use them to learn how to market their products, governments use them to understand community opinions and needs, and politicians and media outlets use surveys to understand their constituencies.

In sum, the following are benefits of survey research:

• Cost-effectiveness
• Generalizability
• Reliability
• Versatility

Weaknesses of survey methods

As with all methods of data collection, survey research also comes with a few drawbacks. First, while one might argue that surveys are flexible in the sense that we can ask any number of questions on any number of topics in them, the fact is that the survey researcher is generally stuck with a single instrument for collecting data: the questionnaire. Surveys are in many ways rather inflexible. Let’s say you mail a survey out to 1,000 people and then discover, as responses start coming in, that your phrasing on a particular question seems to be confusing a number of respondents. At this stage, it’s too late for a do-over or to change the question for the respondents who haven’t yet returned their surveys. When conducting in-depth interviews, on the other hand, a researcher can provide respondents further explanation if they’re confused by a question and can tweak their questions as they learn more about how respondents seem to understand them.

Depth can also be a problem with surveys. Survey questions are standardized; thus, it can be difficult to ask anything other than very general questions that a broad range of people will understand. Because of this, survey results may not be as valid as results obtained using methods of data collection that allow a researcher to more comprehensively examine whatever topic is being studied. Let’s say, for example, that you want to learn something about voters’ willingness to elect an African American president, as in our opening example in this chapter. General Social Survey respondents were asked, “If your party nominated an African American for president, would you vote for him if he were qualified for the job?” Respondents were then asked to respond either yes or no to the question. But what if someone’s opinion was more complex than could be answered with a simple yes or no? What if, for example, a person was willing to vote for an African American woman but not an African American man?  [1]

In sum, potential drawbacks to survey research include the following:

• Inflexibility
• Lack of depth

Potential for bias

If you choose to use a survey design in your research project, you will have to weigh the pros and cons of that approach and make sure that it is appropriate to your research question. In addition, as you implement your survey, you should be aware of some potential issues that may arise in the data that result from conducting survey research.

Non-Response Bias

Survey research is generally notorious for its low response rates. A response rate of 15-20% is typical in a mail survey, even after two or three reminders. If the majority of the targeted respondents fail to respond to a survey, then a legitimate concern is whether non-respondents are not responding due to a systematic reason, which may raise questions about the validity of the study’s results, especially as this relates to the representativeness of the sample. This is known as non-response bias . For instance, dissatisfied customers tend to be more vocal about their experience than satisfied customers, and are therefore more likely to respond to satisfaction questionnaires. Hence, any respondent sample is likely to have a higher proportion of dissatisfied customers than the underlying population from which it is drawn. In this instance, not only will the results lack generalizability, but the observed outcomes may also be an artifact of the biased sample. Several strategies that can be employed to improve response rates are discussed in the box below.

Strategies to Improve Response Rate

• Advance notification : A short letter sent in advance to the targeted respondents soliciting their participation in an upcoming survey can prepare them and improve likelihood of response. The letter should state the purpose and importance of the study, mode of data collection (e.g., via a phone call, a survey form in the mail, etc.), and appreciation for their cooperation. A variation of this technique may request the respondent to return a postage-paid postcard indicating whether or not they are willing to participate in the study.
• Ensuring that content is relevant : If a survey examines issues of relevance or importance to respondents, then they are more likely to respond.
• Creating a respondent-friendly questionnaire : Shorter survey questionnaires tend to elicit higher response rates than longer questionnaires. Furthermore, questions that are clear, inoffensive, and easy to respond to tend to get higher response rates.
• Having the project endorsed : For organizational surveys, it helps to gain endorsement from a senior executive attesting to the importance of the study to the organization. Such endorsements can be in the form of a cover letter or a letter of introduction, which can improve the researcher’s credibility in the eyes of the respondents.
• Providing follow-up requests : Multiple follow-up requests may coax some non-respondents to respond, even if their responses are late.
• Ensuring that interviewers are properly trained : Response rates for interviews can be improved with skilled interviewers trained on how to request interviews, use computerized dialing techniques to identify potential respondents, and schedule callbacks for respondents who could not be reached.
• Providing incentives : Response rates, at least with certain populations, may increase with the use of incentives in the form of cash or gift cards, giveaways such as pens or stress balls, entry into a lottery, draw or contest, discount coupons, the promise of contribution to charity, and so forth.
• Providing non-monetary incentives : Businesses in particular are more prone to respond to non-monetary incentives than financial incentives. An example of such a non-monetary incentive is a benchmarking report comparing the business’s individual response against the aggregate of all responses to a survey.
• Making participants fully aware of confidentiality and privacy : Finally, assurances that respondents’ private data or responses will not fall into the hands of any third party may help improve response rates.

Sampling bias

Sampling bias is present when our sampling process results in a sample that does not represent our population in some way. Telephone surveys conducted by calling a random sample of publicly available telephone numbers will systematically exclude people with unlisted telephone numbers, mobile phone numbers, and will include a disproportionate number of respondents who have land-line telephone service with listed phone numbers and people who stay home during much of the day, such as the unemployed, the disabled, and the elderly. Likewise, online surveys tend to include a disproportionate number of students and younger people who are constantly on the Internet, and systematically exclude people with limited or no access to computers or the Internet, such as the poor and the elderly. Similarly, questionnaire surveys tend to exclude children and people who are unable to read, understand, or meaningfully respond to the questionnaire. A different kind of sampling bias relates to sampling the incorrect or incomplete population, such as asking teachers (or parents) about the academic learning of their students (or children) or asking CEOs about operational details in their company. Such biases make the respondent sample unrepresentative of the intended population and can hurt generalizability claims about inferences drawn from the biased sample.

Social desirability bias

Social desirability bias occurs when we create questions that lead respondents to answer in ways that don’t reflect their genuine thoughts or feelings to avoid being perceived negatively. With negative questions such as, “do you think that your project team is dysfunctional?”, “is there a lot of office politics in your workplace?”, or “have you ever illegally downloaded music files from the Internet?”, the researcher may not get truthful responses. This tendency among respondents to “spin the truth” in order to portray themselves in a socially desirable manner is called social desirability bias, which hurts the validity of responses obtained from survey research. There is practically no way of overcoming social desirability bias in a questionnaire survey outsides of designing questions that minimize the opportunity for social desirability bias to arise. However, in an interview setting, an astute interviewer may be able to spot inconsistent answers and ask probing questions or use personal observations to supplement respondents’ comments.

Recall bias

Responses to survey questions often depend on subjects’ motivation, memory, and ability to respond. Particularly when dealing with events that happened in the distant past, respondents may not adequately remember their own motivations or behaviors, or perhaps their memory of such events may have evolved with time and are no longer retrievable. This phenomenon is know as recall bias . For instance, if a respondent is asked to describe their utilization of computer technology one year ago, their response may not be accurate due to difficulties with recall. One possible way of overcoming the recall bias is by anchoring the respondent’s memory in specific events as they happened, rather than asking them to recall their perceptions and motivations from memory.

Common method bias

Common method bias refers to the amount of spurious covariance shared between independent and dependent variables that are measured at the same point in time, such as in a cross-sectional survey, and using the same instrument, such as a questionnaire. In such cases, the phenomenon under investigation may not be adequately separated from measurement artifacts. Standard statistical tests are available to test for common method bias, such as Harmon’s single-factor test (Podsakoff et al. 2003) [7] , Lindell and Whitney’s (2001) [8] market variable technique, and so forth. This bias can be potentially avoided if the independent and dependent variables are measured at different points in time, using a longitudinal survey design, or if these variables are measured using different methods, such as computerized recording of dependent variable versus questionnaire-based self-rating of independent variables.

Social Science Research: Principles, Methods, and Practices. Authored by: Anol Bhattacherjee. Provided by: University of South Florida. Located at: http://scholarcommons.usf.edu/oa_textbooks/3/. License: CC BY-NC-SA: Attribution-NonCommercial-ShareAlike

• Survey research has several strengths, including being versatile, cost-effective, and familiar to participants.
• Survey research may be used to examine a variety of variables as well as comparing the relationship(s) between variables.
• Limitations of survey research include several types of bias (non-response bias, sampling bias, social desirability bias, recall bias, and common method bias).
• There are strategies to help reduce bias.
• After what you learned in this section, what might be some potential sources of bias in survey results on your topic? How might you minimize those?
• Engel, R. & Schutt. (2013). The practice of research in social work (3rd. ed.) . Thousand Oaks, CA: SAGE. ↵
• Merriam-Webster. (n.d.). Survey. In Merriam-Webster.com dictionary . Retrieved from https://www.merriam-webster.com/dictionary/survey ↵
• Nesje, K. (2016). Personality and professional commitment of students in nursing, social work, and teaching: A comparative survey. International Journal of Nursing Studies, 53 , 173-181. ↵
• Converse, J. M. (1987). Survey research in the United States: Roots and emergence, 1890–1960. Berkeley, CA: University of California Press. ↵
• Center for Disease Control and Prevention, CDC. (n.d.). Behavioral risk factor surveillance system. cdc.gov, https://www.cdc.gov/chronicdisease/resources/publications/factsheets/brfss.htm ↵
• Peterson, R. A. (2000). Constructing effective questionnaires. Thousand Oaks, CA: Sage ↵
• Podsakoff, P. M., MacKenzie, S. B., Lee, J. Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research: a critical review of the literature and recommended remedies. Journal of Applied Psychology, 88 (5), 879. ↵
• Lindell, M. K., & Whitney, D. J. (2001). Accounting for common method variance in cross-sectional research designs. Journal of Applied Psychology, 86 (1), 114. ↵

The actual tool that collects data in survey research.

Those who are asked to contribute data in a research study; sometimes called respondents or subjects.

(as in generalization) to make claims about a large population based on a smaller sample of people or items

conducted during the early stages of a project, usually when a researcher wants to test the feasibility of conducting a more extensive study or if the topic has not been studied in the past

research that describes or defines a particular phenomenon

explains why particular phenomena work in the way that they do; answers “why” questions

entity that a researcher wants to say something about at the end of her study (individual, group, or organization)

Findings form a research study that apply to larger group of people (beyond the sample). Producing generalizable findings requires starting with a representative sample.

the list of people from which a researcher will draw her sample

Research that involves the use of data that represents human expression through words, pictures, movies, performance and other artifacts.

Research that collects data at one point in time.

Questionnaires that are distributed to participants (in person, by mail, virtually) and they are asked to complete them independently.

A detailed document that is used when a survey is read to a respondent that contains a list of questions and answer options that the researcher will read to respondents.

Biases are conscious or subconscious preferences that lead us to favor some things over others.

Testing out your research materials in advance on people who are not included as participants in your study.

An acronym, BRUSO for writing questions in survey research. The letters stand for: “brief,” “relevant,” “unambiguous,” “specific,” and “objective.”

Level of measurement that follows nominal level. Has mutually exclusive categories and a hierarchy (order).

A higher level of measurement. Denoted by having mutually exclusive categories, a hierarchy (order), and equal spacing between values. This last item means that values may be added, subtracted, divided, and multiplied.

The highest level of measurement. Denoted by mutually exclusive categories, a hierarchy (order), values can be added, subtracted, multiplied, and divided, and the presence of an absolute zero.

Mutually exclusive categories are options for closed ended questions that do not overlap.

The ability of a measurement tool to measure a phenomenon the same way, time after time. Note: Reliability does not imply validity.

Sampling bias is present when our sampling process results in a sample that does not represent our population in some way.

Social desirability bias occurs when we create questions that lead respondents to answer in ways that don't reflect their genuine thoughts or feelings to avoid being perceived negatively.

When respondents have difficult providing accurate answers to questions due to the passage of time.

Common method bias refers to the amount of spurious covariance shared between independent and dependent variables that are measured at the same point in time.

Graduate research methods in social work Copyright © 2020 by Matthew DeCarlo, Cory Cummings, Kate Agnelli is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is a Research Design | Types, Guide & Examples

What Is a Research Design | Types, Guide & Examples

Published on June 7, 2021 by Shona McCombes . Revised on November 20, 2023 by Pritha Bhandari.

A research design is a strategy for answering your   research question  using empirical data. Creating a research design means making decisions about:

• Your overall research objectives and approach
• Whether you’ll rely on primary research or secondary research
• Your sampling methods or criteria for selecting subjects
• Your data collection methods
• The procedures you’ll follow to collect data
• Your data analysis methods

A well-planned research design helps ensure that your methods match your research objectives and that you use the right kind of analysis for your data.

Table of contents

Step 1: consider your aims and approach, step 2: choose a type of research design, step 3: identify your population and sampling method, step 4: choose your data collection methods, step 5: plan your data collection procedures, step 6: decide on your data analysis strategies, other interesting articles, frequently asked questions about research design.

• Introduction

Before you can start designing your research, you should already have a clear idea of the research question you want to investigate.

There are many different ways you could go about answering this question. Your research design choices should be driven by your aims and priorities—start by thinking carefully about what you want to achieve.

The first choice you need to make is whether you’ll take a qualitative or quantitative approach.

Qualitative research designs tend to be more flexible and inductive , allowing you to adjust your approach based on what you find throughout the research process.

Quantitative research designs tend to be more fixed and deductive , with variables and hypotheses clearly defined in advance of data collection.

It’s also possible to use a mixed-methods design that integrates aspects of both approaches. By combining qualitative and quantitative insights, you can gain a more complete picture of the problem you’re studying and strengthen the credibility of your conclusions.

Practical and ethical considerations when designing research

As well as scientific considerations, you need to think practically when designing your research. If your research involves people or animals, you also need to consider research ethics .

• How much time do you have to collect data and write up the research?
• Will you be able to gain access to the data you need (e.g., by travelling to a specific location or contacting specific people)?
• Do you have the necessary research skills (e.g., statistical analysis or interview techniques)?
• Will you need ethical approval ?

At each stage of the research design process, make sure that your choices are practically feasible.

Prevent plagiarism. Run a free check.

Within both qualitative and quantitative approaches, there are several types of research design to choose from. Each type provides a framework for the overall shape of your research.

Types of quantitative research designs

Quantitative designs can be split into four main types.

• Experimental and   quasi-experimental designs allow you to test cause-and-effect relationships
• Descriptive and correlational designs allow you to measure variables and describe relationships between them.

With descriptive and correlational designs, you can get a clear picture of characteristics, trends and relationships as they exist in the real world. However, you can’t draw conclusions about cause and effect (because correlation doesn’t imply causation ).

Experiments are the strongest way to test cause-and-effect relationships without the risk of other variables influencing the results. However, their controlled conditions may not always reflect how things work in the real world. They’re often also more difficult and expensive to implement.

Types of qualitative research designs

Qualitative designs are less strictly defined. This approach is about gaining a rich, detailed understanding of a specific context or phenomenon, and you can often be more creative and flexible in designing your research.

The table below shows some common types of qualitative design. They often have similar approaches in terms of data collection, but focus on different aspects when analyzing the data.

Your research design should clearly define who or what your research will focus on, and how you’ll go about choosing your participants or subjects.

In research, a population is the entire group that you want to draw conclusions about, while a sample is the smaller group of individuals you’ll actually collect data from.

Defining the population

A population can be made up of anything you want to study—plants, animals, organizations, texts, countries, etc. In the social sciences, it most often refers to a group of people.

For example, will you focus on people from a specific demographic, region or background? Are you interested in people with a certain job or medical condition, or users of a particular product?

The more precisely you define your population, the easier it will be to gather a representative sample.

• Sampling methods

Even with a narrowly defined population, it’s rarely possible to collect data from every individual. Instead, you’ll collect data from a sample.

To select a sample, there are two main approaches: probability sampling and non-probability sampling . The sampling method you use affects how confidently you can generalize your results to the population as a whole.

Probability sampling is the most statistically valid option, but it’s often difficult to achieve unless you’re dealing with a very small and accessible population.

For practical reasons, many studies use non-probability sampling, but it’s important to be aware of the limitations and carefully consider potential biases. You should always make an effort to gather a sample that’s as representative as possible of the population.

Case selection in qualitative research

In some types of qualitative designs, sampling may not be relevant.

For example, in an ethnography or a case study , your aim is to deeply understand a specific context, not to generalize to a population. Instead of sampling, you may simply aim to collect as much data as possible about the context you are studying.

In these types of design, you still have to carefully consider your choice of case or community. You should have a clear rationale for why this particular case is suitable for answering your research question .

For example, you might choose a case study that reveals an unusual or neglected aspect of your research problem, or you might choose several very similar or very different cases in order to compare them.

Data collection methods are ways of directly measuring variables and gathering information. They allow you to gain first-hand knowledge and original insights into your research problem.

You can choose just one data collection method, or use several methods in the same study.

Survey methods

Surveys allow you to collect data about opinions, behaviors, experiences, and characteristics by asking people directly. There are two main survey methods to choose from: questionnaires and interviews .

Observation methods

Observational studies allow you to collect data unobtrusively, observing characteristics, behaviors or social interactions without relying on self-reporting.

Observations may be conducted in real time, taking notes as you observe, or you might make audiovisual recordings for later analysis. They can be qualitative or quantitative.

Other methods of data collection

There are many other ways you might collect data depending on your field and topic.

If you’re not sure which methods will work best for your research design, try reading some papers in your field to see what kinds of data collection methods they used.

Secondary data

If you don’t have the time or resources to collect data from the population you’re interested in, you can also choose to use secondary data that other researchers already collected—for example, datasets from government surveys or previous studies on your topic.

With this raw data, you can do your own analysis to answer new research questions that weren’t addressed by the original study.

Using secondary data can expand the scope of your research, as you may be able to access much larger and more varied samples than you could collect yourself.

However, it also means you don’t have any control over which variables to measure or how to measure them, so the conclusions you can draw may be limited.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

As well as deciding on your methods, you need to plan exactly how you’ll use these methods to collect data that’s consistent, accurate, and unbiased.

Planning systematic procedures is especially important in quantitative research, where you need to precisely define your variables and ensure your measurements are high in reliability and validity.

Operationalization

Some variables, like height or age, are easily measured. But often you’ll be dealing with more abstract concepts, like satisfaction, anxiety, or competence. Operationalization means turning these fuzzy ideas into measurable indicators.

If you’re using observations , which events or actions will you count?

If you’re using surveys , which questions will you ask and what range of responses will be offered?

You may also choose to use or adapt existing materials designed to measure the concept you’re interested in—for example, questionnaires or inventories whose reliability and validity has already been established.

Reliability and validity

Reliability means your results can be consistently reproduced, while validity means that you’re actually measuring the concept you’re interested in.

For valid and reliable results, your measurement materials should be thoroughly researched and carefully designed. Plan your procedures to make sure you carry out the same steps in the same way for each participant.

If you’re developing a new questionnaire or other instrument to measure a specific concept, running a pilot study allows you to check its validity and reliability in advance.

Sampling procedures

As well as choosing an appropriate sampling method , you need a concrete plan for how you’ll actually contact and recruit your selected sample.

That means making decisions about things like:

• How many participants do you need for an adequate sample size?
• What inclusion and exclusion criteria will you use to identify eligible participants?
• How will you contact your sample—by mail, online, by phone, or in person?

If you’re using a probability sampling method , it’s important that everyone who is randomly selected actually participates in the study. How will you ensure a high response rate?

If you’re using a non-probability method , how will you avoid research bias and ensure a representative sample?

Data management

It’s also important to create a data management plan for organizing and storing your data.

Will you need to transcribe interviews or perform data entry for observations? You should anonymize and safeguard any sensitive data, and make sure it’s backed up regularly.

Keeping your data well-organized will save time when it comes to analyzing it. It can also help other researchers validate and add to your findings (high replicability ).

On its own, raw data can’t answer your research question. The last step of designing your research is planning how you’ll analyze the data.

Quantitative data analysis

In quantitative research, you’ll most likely use some form of statistical analysis . With statistics, you can summarize your sample data, make estimates, and test hypotheses.

Using descriptive statistics , you can summarize your sample data in terms of:

• The distribution of the data (e.g., the frequency of each score on a test)
• The central tendency of the data (e.g., the mean to describe the average score)
• The variability of the data (e.g., the standard deviation to describe how spread out the scores are)

The specific calculations you can do depend on the level of measurement of your variables.

Using inferential statistics , you can:

• Make estimates about the population based on your sample data.
• Test hypotheses about a relationship between variables.

Regression and correlation tests look for associations between two or more variables, while comparison tests (such as t tests and ANOVAs ) look for differences in the outcomes of different groups.

Your choice of statistical test depends on various aspects of your research design, including the types of variables you’re dealing with and the distribution of your data.

Qualitative data analysis

In qualitative research, your data will usually be very dense with information and ideas. Instead of summing it up in numbers, you’ll need to comb through the data in detail, interpret its meanings, identify patterns, and extract the parts that are most relevant to your research question.

Two of the most common approaches to doing this are thematic analysis and discourse analysis .

There are many other ways of analyzing qualitative data depending on the aims of your research. To get a sense of potential approaches, try reading some qualitative research papers in your field.

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A research design is a strategy for answering your   research question . It defines your overall approach and determines how you will collect and analyze data.

A well-planned research design helps ensure that your methods match your research aims, that you collect high-quality data, and that you use the right kind of analysis to answer your questions, utilizing credible sources . This allows you to draw valid , trustworthy conclusions.

Quantitative research designs can be divided into two main categories:

• Correlational and descriptive designs are used to investigate characteristics, averages, trends, and associations between variables.
• Experimental and quasi-experimental designs are used to test causal relationships .

Qualitative research designs tend to be more flexible. Common types of qualitative design include case study , ethnography , and grounded theory designs.

The priorities of a research design can vary depending on the field, but you usually have to specify:

• Your research questions and/or hypotheses
• Your overall approach (e.g., qualitative or quantitative )
• The type of design you’re using (e.g., a survey , experiment , or case study )
• Your data collection methods (e.g., questionnaires , observations)
• Your data collection procedures (e.g., operationalization , timing and data management)
• Your data analysis methods (e.g., statistical tests  or thematic analysis )

A sample is a subset of individuals from a larger population . Sampling means selecting the group that you will actually collect data from in your research. For example, if you are researching the opinions of students in your university, you could survey a sample of 100 students.

In statistics, sampling allows you to test a hypothesis about the characteristics of a population.

Operationalization means turning abstract conceptual ideas into measurable observations.

For example, the concept of social anxiety isn’t directly observable, but it can be operationally defined in terms of self-rating scores, behavioral avoidance of crowded places, or physical anxiety symptoms in social situations.

Before collecting data , it’s important to consider how you will operationalize the variables that you want to measure.

A research project is an academic, scientific, or professional undertaking to answer a research question . Research projects can take many forms, such as qualitative or quantitative , descriptive , longitudinal , experimental , or correlational . What kind of research approach you choose will depend on your topic.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, November 20). What Is a Research Design | Types, Guide & Examples. Scribbr. Retrieved April 13, 2024, from https://www.scribbr.com/methodology/research-design/

Is this article helpful?

Shona McCombes

Other students also liked, guide to experimental design | overview, steps, & examples, how to write a research proposal | examples & templates, ethical considerations in research | types & examples, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

• Subject List
• Take a Tour
• For Authors
• Subscriber Services
• Publications
• African American Studies
• African Studies
• American Literature
• Anthropology
• Architecture Planning and Preservation
• Art History
• Atlantic History
• Biblical Studies
• British and Irish Literature
• Childhood Studies
• Chinese Studies
• Cinema and Media Studies
• Communication
• Criminology
• Environmental Science
• Evolutionary Biology
• International Law
• International Relations
• Islamic Studies
• Jewish Studies
• Latin American Studies
• Latino Studies
• Linguistics
• Literary and Critical Theory
• Medieval Studies
• Military History
• Political Science
• Public Health
• Renaissance and Reformation

Social Work

• Urban Studies
• Victorian Literature
• Browse All Subjects

How to Subscribe

• Free Trials

In This Article Expand or collapse the "in this article" section Experimental and Quasi-Experimental Designs

Introduction, introductory works.

• Manuals and Guides
• History of Experimental Design
• Appraising Experiments
• Statistical Principles and Analysis
• Cluster-Based Experiments
• Ethical Considerations
• Reporting Experiments
• Debate on Experimental Design

Related Articles Expand or collapse the "related articles" section about

About related articles close popup.

Lorem Ipsum Sit Dolor Amet

Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Aliquam ligula odio, euismod ut aliquam et, vestibulum nec risus. Nulla viverra, arcu et iaculis consequat, justo diam ornare tellus, semper ultrices tellus nunc eu tellus.

• Community-Based Participatory Research
• Economic Evaluation
• Evidence-based Social Work Practice
• Implementation Science and Practice
• Measurement, Scales, and Indices
• Meta-analysis
• Mixed Methods Research
• Practice Research
• Qualitative Research
• Research Ethics
• Social Intervention Research
• Social Work Research Methods
• Systematic Review Methods
• Triangulation

Other Subject Areas

Forthcoming articles expand or collapse the "forthcoming articles" section.

• Child Welfare Effectiveness
• Immigration and Child Welfare
• International Human Trafficking
• Find more forthcoming articles...
• Export Citations
• Share This Facebook LinkedIn Twitter

Experimental and Quasi-Experimental Designs by Matthew Morton , Paul Montgomery LAST REVIEWED: 01 May 2017 LAST MODIFIED: 29 June 2011 DOI: 10.1093/obo/9780195389678-0053

In strengthening social work’s ability to improve lives and communities, experimental design can play a critical role in helping stakeholders better understand what works in achieving positive impacts. Experimental design studies aim to test whether a specific “intervention” (or “treatment”) causes change in specific outcomes. Experiments test for this cause-and-effect relationship by exposing a group of research participants to the intervention and observing for any differences in changes of behavior between the intervention group and another group that does not receive the intervention. The group that does not receive the intervention is typically called a “control” or “comparison” group. Notably some literature reserves the term “experimental design” for studies in which participants are randomly assigned to intervention or control groups. Other literature, however, defines the term more broadly to include what some would classify as “quasi-experimental” or “nonrandomized” trials in which an intervention is applied to one group in order to detect changes but assignment to groups occurs through a method of selection other than randomization. This bibliography will consider experimental design in the broader context of both randomized and nonrandomized trials, but it will also supply references that clarify the special ability of randomized controlled trials to reduce bias and strengthen the credibility of experimental findings that guarantee causality. The field of experimental design includes considerable diversity with respect to specific methods, applications, and perspectives. This bibliography aims to organize some of the foremost texts and papers concerning experimental design to provide readers with (a) useful introductions to experimental design and basic principles, (b) practical references for specific audiences or topics of interest, and (c) a rounded tour of the views and debates surrounding experimental design.

This section presents texts and papers that aim to introduce the purpose and principles of experimental design to a wider audience. Chalmers 2003 provides a good first read that articulates cause for the evidence-based practice movement from which experimental design has gained increasing momentum. Rubin and Babbie 2008 , particularly chapter 10, offers an introduction to experimental design and critical concepts with the intention of reaching a social work student audience. Baker 2000 provides similar material applied for use by developmental impact researchers. Eccles, et al. 2003 and Kendall 2003 , though geared toward a health care readership, provide useful summaries of key concepts in experimental design for unfamiliar readers. Oakley, et al. 2003 , Rosen, et al. 2006 , and Sibbald and Roland 1998 articulate nontechnical cases for general audiences for the applicability and value of experimental design. For the advanced student, Kirk 2003 is a most useful text, as it provides a more sophisticated presentation of the topic area.

Baker, Judy L. 2000. Evaluating the impact of development projects on poverty: A handbook for practitioners . Washington, DC: World Bank.

Available free online, this handbook offers a user-friendly overview of the impact of evaluation issues and approaches in which experimental design is often situated. Different types of experimental and quasi-experimental designs are discussed.

Chalmers, Iain. 2003. Trying to do more good than harm in policy and practice: The role of rigorous, transparent, up-to-date evaluations. Annals of the American Academy of Political and Social Science 589.1: 22–40.

DOI: 10.1177/0002716203254762

Chalmers articulates a case for increasing the development and use of rigorous, transparent, and up-to-date experimental designs to improve the processes by which we make decisions about whether and how to intervene in the lives of others. He further argues for systematically reviewing the state of research on a given topic prior to initiating new trials.

Eccles, Martin, Jeremy Grimshaw, Marion Campbell, and Craig Ramsay. 2003. Research designs for studies evaluating the effectiveness of change and improvement strategies. Quality and Safety in Health Care 12.1: 47–52.

DOI: 10.1136/qhc.12.1.47

This article briefly surveys different kinds of experimental designs for evaluation of more complex, behavioral interventions and in doing so introduces readers to key concepts and terms.

Kendall, Jonathan M. 2003. Designing a research project: Randomised controlled trials and their principles. Emergency Medicine Journal 20.2: 164–168.

DOI: 10.1136/emj.20.2.164

This article provides a basic, nontechnical summary introduction to the features and applicability of randomized designs for an unfamiliar audience.

Kirk, Roger E. 2003. Experimental design. In Handbook of psychology , Vol. 2, Research methods in psychology . Edited by John A. Schinka, and Wayne F. Velicer, 3–32. Hoboken, NJ: Wiley.

Though brief, this overview introduces readers to more complex categories of experimental design (for example, hierarchical designs in which multiple treatments are nested within each other) relevant to readers interested in a more advanced introduction to approaches. Kirk characterizes experimental design by random assignment of participants.

Oakley, Ann, Vicki Strange, Tami Toroyan, Meg Wiggins, Ian Roberts, and Judith Stephenson. 2003. Using random allocation to evaluate social interventions: Three recent U.K. examples. Annals of the American Academy of Political and Social Science 589.1: 170–189.

DOI: 10.1177/0002716203254765

Oakley and colleagues argue for the applicability of robust experimental design to social interventions as it has been popularly used in health and medicine. The paper provides three examples of randomized controlled trials with social interventions in the United Kingdom to illustrate strategies for conducting successful experimental trials.

Rosen, Laura, Orly Manor, Dan Engelhard, and David Zucker. 2006. In defense of the randomized controlled trial for health promotion research. American Journal of Public Health 96.7: 1181–1186.

DOI: 10.2105/AJPH.2004.061713

This paper discusses the value of experimental design in evaluating health promotion interventions and responds to common criticisms of experimental design with suggestions for tailoring strategies and approaches to meet different conditions rather than abandoning experimental design altogether.

Rubin, Allen, and Earl R. Babbie. 2008. Research methods for social work . 6th ed. Belmont, CA: Thomson Brooks Cole.

This textbook, which can serve as a general textbook for graduate and upper-level undergraduate social work students on research methods, dedicates chapter 10 to experimental design, which could be used as an introductory read to the topic. Unique to this edition from previous versions, the authors make explicit links to the material throughout the book to the evidence-based practice movement.

Sibbald, Bonnie, and Martin Roland. 1998. Understanding controlled trials: Why are randomised controlled trials important? British Medical Journal 316.7126: 201.

This brief note discusses the features of experimental design that make it useful and authoritative for evaluating intervention impacts.

back to top

Users without a subscription are not able to see the full content on this page. Please subscribe or login .

Oxford Bibliographies Online is available by subscription and perpetual access to institutions. For more information or to contact an Oxford Sales Representative click here .

• About Social Work »
• Meet the Editorial Board »
• Adolescent Depression
• Adolescent Pregnancy
• Adolescents
• Adoption Home Study Assessments
• Adult Protective Services in the United States
• African Americans
• Aging out of foster care
• Aging, Physical Health and
• Alcohol and Drug Abuse Problems
• Alcohol and Drug Problems, Prevention of Adolescent and Yo...
• Alcohol Problems: Practice Interventions
• Alcohol Use Disorder
• Alzheimer's Disease and Other Dementias
• Anti-Oppressive Practice
• Asian Americans
• Asian-American Youth
• Autism Spectrum Disorders
• Baccalaureate Social Workers
• Behavioral Health
• Behavioral Social Work Practice
• Bereavement Practice
• Bisexuality
• Brief Therapies in Social Work: Task-Centered Model and So...
• Bullying and Social Work Intervention
• Canadian Social Welfare, History of
• Case Management in Mental Health in the United States
• Central American Migration to the United States
• Child Maltreatment Prevention
• Child Neglect and Emotional Maltreatment
• Child Poverty
• Child Sexual Abuse
• Child Welfare
• Child Welfare and Child Protection in Europe, History of
• Child Welfare Practice with LGBTQ Youth and Families
• Children of Incarcerated Parents
• Christianity and Social Work
• Chronic Illness
• Clinical Social Work Practice with Adult Lesbians
• Clinical Social Work Practice with Males
• Cognitive Behavior Therapies with Diverse and Stressed Pop...
• Cognitive Processing Therapy
• Cognitive-Behavioral Therapy
• Community Development
• Community Policing
• Community-Needs Assessment
• Comparative Social Work
• Computational Social Welfare: Applying Data Science in Soc...
• Conflict Resolution
• Council on Social Work Education
• Counseling Female Offenders
• Criminal Justice
• Crisis Interventions
• Cultural Competence and Ethnic Sensitive Practice
• Culture, Ethnicity, Substance Use, and Substance Use Disor...
• Dementia Care
• Dementia Care, Ethical Aspects of
• Depression and Cancer
• Development and Infancy (Birth to Age Three)
• Differential Response in Child Welfare
• Digital Storytelling for Social Work Interventions
• Direct Practice in Social Work
• Disabilities
• Disability and Disability Culture
• Domestic Violence Among Immigrants
• Early Pregnancy and Parenthood Among Child Welfare–Involve...
• Eating Disorders
• Ecological Framework
• Elder Mistreatment
• End-of-Life Decisions
• Epigenetics for Social Workers
• Ethical Issues in Social Work and Technology
• Ethics and Values in Social Work
• European Institutions and Social Work
• European Union, Justice and Home Affairs in the
• Evidence-based Social Work Practice: Finding Evidence
• Evidence-based Social Work Practice: Issues, Controversies...
• Experimental and Quasi-Experimental Designs
• Families with Gay, Lesbian, or Bisexual Parents
• Family Caregiving
• Family Group Conferencing
• Family Policy
• Family Services
• Family Therapy
• Family Violence
• Fathering Among Families Served By Child Welfare
• Fetal Alcohol Spectrum Disorders
• Field Education
• Financial Literacy and Social Work
• Financing Health-Care Delivery in the United States
• Forensic Social Work
• Foster Care
• Foster care and siblings
• Gender, Violence, and Trauma in Immigration Detention in t...
• Generalist Practice and Advanced Generalist Practice
• Grounded Theory
• Group Work across Populations, Challenges, and Settings
• Group Work, Research, Best Practices, and Evidence-based
• Harm Reduction
• Health Care Reform
• Health Disparities
• Health Social Work
• History of Social Work and Social Welfare, 1900–1950
• History of Social Work and Social Welfare, 1950-1980
• History of Social Work and Social Welfare, pre-1900
• History of Social Work from 1980-2014
• History of Social Work in China
• History of Social Work in Northern Ireland
• History of Social Work in the Republic of Ireland
• History of Social Work in the United Kingdom
• HIV/AIDS and Children
• HIV/AIDS Prevention with Adolescents
• Homelessness
• Homelessness: Ending Homelessness as a Grand Challenge
• Homelessness Outside the United States
• Human Needs
• Human Trafficking, Victims of
• Immigrant Integration in the United States
• Immigrant Policy in the United States
• Immigrants and Refugees
• Immigrants and Refugees: Evidence-based Social Work Practi...
• Immigration and Health Disparities
• Immigration and Intimate Partner Violence
• Immigration and Poverty
• Immigration and Spirituality
• Immigration and Substance Use
• Immigration and Trauma
• Impact of Emerging Technology in Social Work Practice
• Impaired Professionals
• Indigenous Peoples
• Individual Placement and Support (IPS) Supported Employmen...
• In-home Child Welfare Services
• Intergenerational Transmission of Maltreatment
• International Social Welfare
• International Social Work
• International Social Work and Education
• International Social Work and Social Welfare in Southern A...
• Internet and Video Game Addiction
• Interpersonal Psychotherapy
• Intervention with Traumatized Populations
• Interviewing
• Intimate-Partner Violence
• Juvenile Justice
• Kinship Care
• Korean Americans
• Latinos and Latinas
• Law, Social Work and the
• LGBTQ Populations and Social Work
• Mainland European Social Work, History of
• Major Depressive Disorder
• Management and Administration in Social Work
• Maternal Mental Health
• Medical Illness
• Men: Health and Mental Health Care
• Mental Health
• Mental Health Diagnosis and the Addictive Substance Disord...
• Mental Health Needs of Older People, Assessing the
• Mental Illness: Children
• Mental Illness: Elders
• Microskills
• Middle East and North Africa, International Social Work an...
• Military Social Work
• Moral distress and injury in social work
• Motivational Interviewing
• Multiculturalism
• Native Americans
• Native Hawaiians and Pacific Islanders
• Neighborhood Social Cohesion
• Neuroscience and Social Work
• Nicotine Dependence
• Occupational Social Work
• Organizational Development and Change
• Pain Management
• Palliative Care
• Palliative Care: Evolution and Scope of Practice
• Pandemics and Social Work
• Parent Training
• Personalization
• Person-in-Environment
• Philosophy of Science and Social Work
• Physical Disabilities
• Podcasts and Social Work
• Police Social Work
• Political Social Work in the United States
• Positive Youth Development
• Postmodernism and Social Work
• Postsecondary Education Experiences and Attainment Among Y...
• Post-Traumatic Stress Disorder (PTSD)
• Practice Interventions and Aging
• Practice Interventions with Adolescents
• Primary Prevention in the 21st Century
• Productive Engagement of Older Adults
• Profession, Social Work
• Program Development and Grant Writing
• Promoting Smart Decarceration as a Grand Challenge
• Psychiatric Rehabilitation
• Psychoanalysis and Psychodynamic Theory
• Psychoeducation
• Psychometrics
• Psychopathology and Social Work Practice
• Psychopharmacology and Social Work Practice
• Psychosocial Framework
• Psychosocial Intervention with Women
• Psychotherapy and Social Work
• Race and Racism
• Readmission Policies in Europe
• Redefining Police Interactions with People Experiencing Me...
• Rehabilitation
• Religiously Affiliated Agencies
• Reproductive Health
• Restorative Justice
• Risk Assessment in Child Protection Services
• Risk Management in Social Work
• Rural Social Work in China
• Rural Social Work Practice
• School Social Work
• School Violence
• School-Based Delinquency Prevention
• Services and Programs for Pregnant and Parenting Youth
• Severe and Persistent Mental Illness: Adults
• Sexual and Gender Minority Immigrants, Refugees, and Asylu...
• Sexual Assault
• Single-System Research Designs
• Social and Economic Impact of US Immigration Policies on U...
• Social Development
• Social Insurance and Social Justice
• Social Justice and Social Work
• Social Movements
• Social Planning
• Social Policy
• Social Policy in Denmark
• Social Security in the United States (OASDHI)
• Social Work and Islam
• Social Work and Social Welfare in East, West, and Central ...
• Social Work and Social Welfare in Europe
• Social Work Education and Research
• Social Work Leadership
• Social Work Luminaries: Luminaries Contributing to the Cla...
• Social Work Luminaries: Luminaries contributing to the fou...
• Social Work Luminaries: Luminaries Who Contributed to Soci...
• Social Work Regulation
• Social Work with Interpreters
• Solution-Focused Therapy
• Strategic Planning
• Strengths Perspective
• Strengths-Based Models in Social Work
• Supplemental Security Income
• Survey Research
• Sustainability: Creating Social Responses to a Changing En...
• Syrian Refugees in Turkey
• Task-Centered Practice
• Technology Adoption in Social Work Education
• Technology for Social Work Interventions
• Technology, Human Relationships, and Human Interaction
• Technology in Social Work
• Terminal Illness
• The Impact of Systemic Racism on Latinxs’ Experiences with...
• Transdisciplinary Science
• Translational Science and Social Work
• Transnational Perspectives in Social Work
• Transtheoretical Model of Change
• Trauma-Informed Care
• Tribal child welfare practice in the United States
• United States, History of Social Welfare in the
• Universal Basic Income
• Veteran Services
• Vicarious Trauma and Resilience in Social Work Practice wi...
• Vicarious Trauma Redefining PTSD
• Victim Services
• Virtual Reality and Social Work
• Welfare State Reform in France
• Welfare State Theory
• Women and Macro Social Work Practice
• Women's Health Care
• Work and Family in the German Welfare State
• Workforce Development of Social Workers Pre- and Post-Empl...
• Working with Non-Voluntary and Mandated Clients
• Young and Adolescent Lesbians
• Youth at Risk
• Youth Services
• Privacy Policy
• Cookie Policy
• Legal Notice
• Accessibility

Powered by:

• [66.249.64.20|195.158.225.244]
• 195.158.225.244

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

8.1 Experimental design: What is it and when should it be used?

Learning objectives.

• Define experiment
• Identify the core features of true experimental designs
• Describe the difference between an experimental group and a control group
• Identify and describe the various types of true experimental designs

Experiments are an excellent data collection strategy for social workers wishing to observe the effects of a clinical intervention or social welfare program. Understanding what experiments are and how they are conducted is useful for all social scientists, whether they actually plan to use this methodology or simply aim to understand findings from experimental studies. An experiment is a method of data collection designed to test hypotheses under controlled conditions. In social scientific research, the term experiment has a precise meaning and should not be used to describe all research methodologies.

Experiments have a long and important history in social science. Behaviorists such as John Watson, B. F. Skinner, Ivan Pavlov, and Albert Bandura used experimental design to demonstrate the various types of conditioning. Using strictly controlled environments, behaviorists were able to isolate a single stimulus as the cause of measurable differences in behavior or physiological responses. The foundations of social learning theory and behavior modification are found in experimental research projects. Moreover, behaviorist experiments brought psychology and social science away from the abstract world of Freudian analysis and towards empirical inquiry, grounded in real-world observations and objectively-defined variables. Experiments are used at all levels of social work inquiry, including agency-based experiments that test therapeutic interventions and policy experiments that test new programs.

Several kinds of experimental designs exist. In general, designs considered to be true experiments contain three basic key features:

• random assignment of participants into experimental and control groups
• a “treatment” (or intervention) provided to the experimental group
• measurement of the effects of the treatment in a post-test administered to both groups

Some true experiments are more complex.  Their designs can also include a pre-test and can have more than two groups, but these are the minimum requirements for a design to be a true experiment.

Experimental and control groups

In a true experiment, the effect of an intervention is tested by comparing two groups: one that is exposed to the intervention (the experimental group , also known as the treatment group) and another that does not receive the intervention (the control group ). Importantly, participants in a true experiment need to be randomly assigned to either the control or experimental groups. Random assignment uses a random number generator or some other random process to assign people into experimental and control groups. Random assignment is important in experimental research because it helps to ensure that the experimental group and control group are comparable and that any differences between the experimental and control groups are due to random chance. We will address more of the logic behind random assignment in the next section.

Treatment or intervention

In an experiment, the independent variable is receiving the intervention being tested—for example, a therapeutic technique, prevention program, or access to some service or support. It is less common in of social work research, but social science research may also have a stimulus, rather than an intervention as the independent variable. For example, an electric shock or a reading about death might be used as a stimulus to provoke a response.

In some cases, it may be immoral to withhold treatment completely from a control group within an experiment. If you recruited two groups of people with severe addiction and only provided treatment to one group, the other group would likely suffer. For these cases, researchers use a control group that receives “treatment as usual.” Experimenters must clearly define what treatment as usual means. For example, a standard treatment in substance abuse recovery is attending Alcoholics Anonymous or Narcotics Anonymous meetings. A substance abuse researcher conducting an experiment may use twelve-step programs in their control group and use their experimental intervention in the experimental group. The results would show whether the experimental intervention worked better than normal treatment, which is useful information.

The dependent variable is usually the intended effect the researcher wants the intervention to have. If the researcher is testing a new therapy for individuals with binge eating disorder, their dependent variable may be the number of binge eating episodes a participant reports. The researcher likely expects her intervention to decrease the number of binge eating episodes reported by participants. Thus, she must, at a minimum, measure the number of episodes that occur after the intervention, which is the post-test .  In a classic experimental design, participants are also given a pretest to measure the dependent variable before the experimental treatment begins.

Types of experimental design

Let’s put these concepts in chronological order so we can better understand how an experiment runs from start to finish. Once you’ve collected your sample, you’ll need to randomly assign your participants to the experimental group and control group. In a common type of experimental design, you will then give both groups your pretest, which measures your dependent variable, to see what your participants are like before you start your intervention. Next, you will provide your intervention, or independent variable, to your experimental group, but not to your control group. Many interventions last a few weeks or months to complete, particularly therapeutic treatments. Finally, you will administer your post-test to both groups to observe any changes in your dependent variable. What we’ve just described is known as the classical experimental design and is the simplest type of true experimental design. All of the designs we review in this section are variations on this approach. Figure 8.1 visually represents these steps.

An interesting example of experimental research can be found in Shannon K. McCoy and Brenda Major’s (2003) study of people’s perceptions of prejudice. In one portion of this multifaceted study, all participants were given a pretest to assess their levels of depression. No significant differences in depression were found between the experimental and control groups during the pretest. Participants in the experimental group were then asked to read an article suggesting that prejudice against their own racial group is severe and pervasive, while participants in the control group were asked to read an article suggesting that prejudice against a racial group other than their own is severe and pervasive. Clearly, these were not meant to be interventions or treatments to help depression, but were stimuli designed to elicit changes in people’s depression levels. Upon measuring depression scores during the post-test period, the researchers discovered that those who had received the experimental stimulus (the article citing prejudice against their same racial group) reported greater depression than those in the control group. This is just one of many examples of social scientific experimental research.

In addition to classic experimental design, there are two other ways of designing experiments that are considered to fall within the purview of “true” experiments (Babbie, 2010; Campbell & Stanley, 1963).  The posttest-only control group design is almost the same as classic experimental design, except it does not use a pretest. Researchers who use posttest-only designs want to eliminate testing effects , in which participants’ scores on a measure change because they have already been exposed to it. If you took multiple SAT or ACT practice exams before you took the real one you sent to colleges, you’ve taken advantage of testing effects to get a better score. Considering the previous example on racism and depression, participants who are given a pretest about depression before being exposed to the stimulus would likely assume that the intervention is designed to address depression. That knowledge could cause them to answer differently on the post-test than they otherwise would. In theory, as long as the control and experimental groups have been determined randomly and are therefore comparable, no pretest is needed. However, most researchers prefer to use pretests in case randomization did not result in equivalent groups and to help assess change over time within both the experimental and control groups.

Researchers wishing to account for testing effects but also gather pretest data can use a Solomon four-group design. In the Solomon four-group design , the researcher uses four groups. Two groups are treated as they would be in a classic experiment—pretest, experimental group intervention, and post-test. The other two groups do not receive the pretest, though one receives the intervention. All groups are given the post-test. Table 8.1 illustrates the features of each of the four groups in the Solomon four-group design. By having one set of experimental and control groups that complete the pretest (Groups 1 and 2) and another set that does not complete the pretest (Groups 3 and 4), researchers using the Solomon four-group design can account for testing effects in their analysis.

Solomon four-group designs are challenging to implement in the real world because they are time- and resource-intensive. Researchers must recruit enough participants to create four groups and implement interventions in two of them.

Overall, true experimental designs are sometimes difficult to implement in a real-world practice environment. It may be impossible to withhold treatment from a control group or randomly assign participants in a study. In these cases, pre-experimental and quasi-experimental designs–which we  will discuss in the next section–can be used.  However, the differences in rigor from true experimental designs leave their conclusions more open to critique.

Experimental design in macro-level research

You can imagine that social work researchers may be limited in their ability to use random assignment when examining the effects of governmental policy on individuals.  For example, it is unlikely that a researcher could randomly assign some states to implement decriminalization of recreational marijuana and some states not to in order to assess the effects of the policy change.  There are, however, important examples of policy experiments that use random assignment, including the Oregon Medicaid experiment. In the Oregon Medicaid experiment, the wait list for Oregon was so long, state officials conducted a lottery to see who from the wait list would receive Medicaid (Baicker et al., 2013).  Researchers used the lottery as a natural experiment that included random assignment. People selected to be a part of Medicaid were the experimental group and those on the wait list were in the control group. There are some practical complications macro-level experiments, just as with other experiments.  For example, the ethical concern with using people on a wait list as a control group exists in macro-level research just as it does in micro-level research.

Key Takeaways

• True experimental designs require random assignment.
• Control groups do not receive an intervention, and experimental groups receive an intervention.
• The basic components of a true experiment include a pretest, posttest, control group, and experimental group.
• Testing effects may cause researchers to use variations on the classic experimental design.
• Classic experimental design- uses random assignment, an experimental and control group, as well as pre- and posttesting
• Control group- the group in an experiment that does not receive the intervention
• Experiment- a method of data collection designed to test hypotheses under controlled conditions
• Experimental group- the group in an experiment that receives the intervention
• Posttest- a measurement taken after the intervention
• Posttest-only control group design- a type of experimental design that uses random assignment, and an experimental and control group, but does not use a pretest
• Pretest- a measurement taken prior to the intervention
• Random assignment-using a random process to assign people into experimental and control groups
• Solomon four-group design- uses random assignment, two experimental and two control groups, pretests for half of the groups, and posttests for all
• Testing effects- when a participant’s scores on a measure change because they have already been exposed to it
• True experiments- a group of experimental designs that contain independent and dependent variables, pretesting and post testing, and experimental and control groups

Image attributions

exam scientific experiment by mohamed_hassan CC-0

Foundations of Social Work Research Copyright © 2020 by Rebecca L. Mauldin is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

Data Specialist

• Madison, Wisconsin
• EXTENSION/AGRICULTURE INSTITUTE DIRECTOR
• Partially Remote
• Staff-Full Time
• Opening at: Apr 11 2024 at 16:30 CDT
• Closing at: May 2 2024 at 23:55 CDT

Job Summary:

On-farm and applied agricultural field trials are critical to engaging innovative farmers and better understanding the implications for agricultural management and adoption of innovative practices. The Data Specialist will work with UW faculty and Extension educators to strengthen field trial design and provide data analysis support for applied field research projects. This work is intended to assist farmers and industry partners in research-based decision making to improve the economic, environmental and social sustainability of Wisconsin farms. This position will also work closely with faculty to develop and analyze data to support agronomic and nutrient management recommendations. About Extension: UW-Madison's Division of Extension serves the people and communities of Wisconsin by addressing local, statewide and national issues, improving lives through research-based education, fostering partnerships and action, and facilitating positive impacts. The Agriculture Institute is one of six Institutes in the Division. About the Agriculture Institute's Crops and Soils Program: The Agriculture Institute's Crops and Soils Program works hand-in-hand with row crop, forage, fruit and vegetable producers to implement best practices for every aspect of the growing phase. The Crops and Soils program provides timely resources and information to help Wisconsin crop producers and their agricultural consultants manage crops efficiently and profitably. About this Position: The Data Specialist position is designed to work with crop producers and agribusiness professionals statewide to solve production challenges and incorporate new research findings into outreach and educational materials that improve the efficiency and profitability of crop production. Specific roles of the position will include: - Consult with colleagues on aspects of project design and data analysis in agricultural field research. - Develop systems for efficient data analysis and communication of results to farm and industry partners. - Compile and analyze existing datasets in collaboration with UW faculty to communicate findings and make management recommendations. - The ideal candidate will have experience with developing and evaluating educational programming and a track record of building positive relationships. The preferred location for this position is Madison, WI, but is flexible based on programmatic needs, successful candidate's preferences, and availability of suitable space. The Division of Extension has a deep and profound commitment to diversity, inclusion, and equity, believing that these values are foundational elements to eliminate disparities and expanding access for all. As Extension, we acknowledge the need for strategic and coordinated actions that help us form a more equitable, anti-racist, non-biased, and inclusive organization. ( https://blogs.extension.wisc.edu/oaic/call-to-action/ ) As such, all Extension employees are expected to foster and promote the values of diversity and inclusion.

Responsibilities:

• 30% Prepares data sets for analysis including cleaning/quality assurance, transformations, restructuring, and integration of multiple data sources
• 30% Independently identifies and implements appropriate data science techniques to find data patterns and answer research questions chosen by the lead researcher including data visualization, statistical analysis, machine learning, and data mining
• 10% Organizes and automates project steps for data preparation and analysis
• 10% Composes and assembles reproducible workflows and reports to clearly articulate patterns to researchers and/or administrators
• 10% Documents approaches to address research questions and contributes to the establishment of reproducible research methodologies and analysis workflows
• 10% Communicate plans, activities, and achievements to Program Managers, partners, and relevant stakeholders

Institutional Statement on Diversity:

Diversity is a source of strength, creativity, and innovation for UW-Madison. We value the contributions of each person and respect the profound ways their identity, culture, background, experience, status, abilities, and opinion enrich the university community. We commit ourselves to the pursuit of excellence in teaching, research, outreach, and diversity as inextricably linked goals. The University of Wisconsin-Madison fulfills its public mission by creating a welcoming and inclusive community for people from every background - people who as students, faculty, and staff serve Wisconsin and the world. For more information on diversity and inclusion on campus, please visit: Diversity and Inclusion

Required Master's Degree in a field related to this position's focus Preferred PhD in a field related to this position's focus

Qualifications:

Required: - Three or more years relevant professional experience; - Demonstrated skill with R or comparable statistical software - Demonstrated skill in data analysis and interpretation. - Demonstrated skills in interpreting, utilizing, and applying evidence-based information and research findings; - Demonstrated ability to communicate effectively, both written and verbal, through a variety of technologies; - Demonstrated ability to effectively work with people from different cultural backgrounds, including those associated with race, ethnicity, national origin, religion, socioeconomic status, age, gender, disability, sexual orientation, and other aspects of human diversity. Preferred: - Ability to conduct linear and non-linear regression analysis - Strong interpersonal relationship and problem-solving skills in a team setting - Demonstrated experience with building diverse, collaborative partnerships - Demonstrated ability to communicate scientific or technical materials in written and verbal forms for a variety of lay and other audiences

Full Time: 100% This position may require some work to be performed in-person, onsite, at a designated campus work location. Some work may be performed remotely, at an offsite, non-campus work location.

Appointment Type, Duration:

Ongoing/Renewable

Minimum $65,000 ANNUAL (12 months) Depending on Qualifications Employees in this position can expect to receive benefits such as generous vacation, holidays, and paid time off; competitive insurances and savings accounts; retirement benefits. Benefits information can be found at ( https://hr.wisc.edu/benefits/ ).

Additional Information:

Division of Extension headquarters are located within Madison, WI but the position location is flexible and will be determined based on programmatic needs and successful candidate's preferences and availability of suitable space. The University of Wisconsin-Madison has a remote work policy that offers the potential for remote or hybrid work. More about that policy within the Division of Extension can be found here: https://kb.wisc.edu/extension/113536 .  Please note that successful applicants are responsible for ensuring their eligibility to work in the United States (i.e. a citizen or national of the United States, a lawful permanent resident, a foreign national authorized to work in the United States without need of employer sponsorship) on or before the effective date of appointment.

How to Apply:

We are eager to learn more about how your experience and passion may align with this position. Please submit a cover letter referring to your related work experience and a resume detailing your educational and professional background.

Your cover letter should communicate your interest in the position and how your skillset aligns with the role. The application reviewers will be relying on written application materials to determine who may advance to preliminary interviews.

Anne Pfeiffer [email protected] 608-263-1095 Relay Access (WTRS): 7-1-1. See RELAY_SERVICE for further information.

Official Title:

Data Scientist II(RE021)

Department(s):

A47-EXTENSION/ANRCD/AGR/CROPS/CROPS&SOIL

Employment Class:

Academic Staff-Renewable

Job Number:

The university of wisconsin-madison is an equal opportunity and affirmative action employer..

You will be redirected to the application to launch your career momentarily. Thank you!

Frequently Asked Questions

Applicant Tutorial

Disability Accommodations

Pay Transparency Policy Statement

Refer a Friend

You've sent this job to a friend!

Website feedback, questions or accessibility issues: [email protected] .

Learn more about accessibility at UW–Madison .

© 2016–2024 Board of Regents of the University of Wisconsin System • Privacy Statement

Office of the Vice President for Research

Four clas faculty researchers secure prestigious early career awards.

Continuing  an upward trend of University of Iowa faculty securing prestigious early-career grants, four investigators from the Departments of Physics and Astronomy and Computer Science have been awarded notable grant awards to advance their careers.

DeRoo, Hoadley advance space instrumentation with Nancy Grace Roman Technology Fellowships in Astrophysics for Early Career Researchers

Casey DeRoo and Keri Hoadley , both assistant professors in the Department of Physics and Astronomy, each received a Nancy Grace Roman Technology Fellowship in Astrophysics for Early Career Researchers. The NASA fellowship provides each researcher with $500,000 over two years to support their research in space-based instrumentation. 

Hoadley’s research is two-pronged. She will design and ultimately prototype a mirror-based vacuum ultraviolet polarizer, which will allow researchers to access polarized light from space below 120-nanometer wavelength. Polarizing light at such a low wavelength is crucial to building optics for NASA’s future Habitable World Observatory (HWO), the agency’s next flagship astrophysics mission after the Nancy Grace Roman Space Telescope. 

“Our vacuum ultraviolet polarizer project is meant to help set up our lab to propose to NASA for one or more follow-up technology programs, including adapting this polarizer for use in vacuum systems, duplicating it and measuring its efficiency to measure additional flavors of polarized UV light, quantifying the polarization effects introduced by UV optical components that may be used on HWO, and building an astronomical instrument to measure the polarization of UV from around massive stars and throughout star-forming regions,” said Hoadley.

In addition, Hoadley and her team will build a facility to align, calibrate, and integrate small space telescopes before flight, using a vacuum chamber and wavelengths of light typically only accessible in space, which could help the university win future small satellite and suborbital missions from NASA. 

DeRoo will work to advance diffraction gratings made with electron beams that pattern structures on a nanometer scale.   Like a prism, diffraction gratings spread out and direct light coming from stars and galaxies, allowing researchers to deduce things like the temperature, density, or composition of an astronomical object.

The fellowship will allow DeRoo to upgrade the university’s Raith

 Voyager tool, a specialized fabrication tool hosted by OVPR’s Materials Analysis, Testing and Fabrication (MATFab) facility.

“These upgrades will let us perform algorithmic patterning, which uses computer code to quickly generate the patterns to be manufactured,” DeRoo said. “This is a major innovation that should enable us to make more complex grating shapes as well as make gratings more quickly.” DeRoo added that the enhancements mean his team may be able to make diffraction gratings that allow space instrument designs that are distinctly different from those launched to date.

“For faculty who develop space-based instruments, the Nancy Grace Roman Technology Fellowship is on par with the prestige of an NSF CAREER or Department of Energy Early Career award,” said Mary Hall Reno, professor and department chair. “Our track record with the program elevates our status as a destination university for astrophysics and space physics missions.”

Uppu pursues building blocks quantum computing with NSF CAREER Award

Ravitej Uppu, assistant professor in the Department of Physics and Astronomy, received a 5-year NSF CAREER award of $550,000 to conduct research aimed at amplifying the power of quantum computing and making its application more practical. 

Uppu and his team will explore the properties of light-matter interactions at the level of a single photon interacting with a single molecule, enabling them to generate efficient and high-quality multiphoton entangled states of light. Multiphoton entangled states, in which photons become inextricably linked, are necessary for photons to serve as practical quantum interconnects, transmitting information between quantum computing units, akin to classical cluster computers. 

“ In our pursuit of secure communication, exploiting quantum properties of light is the final frontier,” said Uppu. “However, unavoidable losses that occur in optical fiber links between users can easily nullify the secure link. Our research on multiphoton entangled states is a key building block for implementing ‘quantum repeaters’ that can overcome this challenge.”

Jiang tackles real-world data issues with NSF CAREER Award

Peng Jiang, assistant professor in the Department of Computer Science, received an NSF CAREER Award that will provide $548,944 over five years to develop tools to support the use of sampling-based algorithms. 

Sampling-based algorithms reduce computing costs by processing only a random selection of a dataset, which has made them increasingly popular, but the method still faces limited efficiency. Jiang will develop a suite of tools that simplify the implementation of sampling-based algorithms and improve their efficacy across wide range of computing and big data applications.

“ A simple example of a real-world application is subgraph matching,” Jiang said. “For example, one might be interested in finding a group of people with certain connections in a social network. The use of sampling-based algorithms can significantly accelerate this process.”

In addition to providing undergraduate students the opportunity to engage with this research, Jiang also plans for the project to enhance projects in computer science courses.

IOE - Faculty of Education and Society

• Departments and centres
• Innovation and enterprise
• Teacher Education College

Social class origin and job quality in the United Kingdom

24 April 2024, 1:00 pm–2:00 pm

Join this event to hear Mark Williams explore class origin gaps in five domains of (non-pay) job quality: non-pay rewards, prospective opportunities, work-life balance, job design, and workplace relations in the United Kingdom.

This event is free.

Event Information

Availability.

In recent years, research has documented that those from working class origins are paid substantially less than those from middle-class origins, even within the same destination class (‘the class pay gap’). At the same time, there has been growing academic and policy interest in the notion of ‘job quality’, an umbrella term denoting the multiple dimensions of what makes jobs ‘good’ and ‘bad’, which includes, but is not limited to, pay.

This seminar explores class origin gaps in five domains of (non-pay) job quality. Across 10 job quality indicators, Mark will discuss how class origin gaps, most of which cannot be completely accounted for by compositional factors such as qualifications and class destinations, imply the ‘class ceiling’ extends to job quality more broadly – concluding that social mobility research and debates must factor in job quality.

This event will be particularly useful for those interested in the labour market, social class, pay gap, and the class ceiling.

Please note this is a hybrid event and can be joined either in-person or online.

Related links

• QSS and CLS seminar series
• Quantitative Social Science
• Centre for Longitudinal Studies
• Social Research Institute

About the Speaker

Professor mark williams.

Professor of Human Resource Management at the School of Business and Management at Queen Mary University of London

Mark researches socio-economic disparities in the quality of jobs in the United Kingdom. Much of his work has focused on pay disparities across occupations and classes.

Over the years, his work has branched out into working conditions more broadly (e.g., job insecurity, job control) as well as in workers’ attitudes to their jobs (e.g., job satisfaction, job meaningfulness). More recently, his research has explored the relationship between labour market regulation and the quality of jobs.

Related News

Related events, related case studies, related research projects, press and media enquiries.

UCL Media Relations +44 (0)7747 565 056