random assignment psychology experiment

Random Assignment in Psychology (Definition + 40 Examples)

Have you ever wondered how researchers discover new ways to help people learn, make decisions, or overcome challenges? A hidden hero in this adventure of discovery is a method called random assignment, a cornerstone in psychological research that helps scientists uncover the truths about the human mind and behavior.

Random Assignment is a process used in research where each participant has an equal chance of being placed in any group within the study. This technique is essential in experiments as it helps to eliminate biases, ensuring that the different groups being compared are similar in all important aspects.

By doing so, researchers can be confident that any differences observed are likely due to the variable being tested, rather than other factors.

In this article, we’ll explore the intriguing world of random assignment, diving into its history, principles, real-world examples, and the impact it has had on the field of psychology.

History of Random Assignment

Stepping back in time, we delve into the origins of random assignment, which finds its roots in the early 20th century.

The pioneering mind behind this innovative technique was Sir Ronald A. Fisher , a British statistician and biologist. Fisher introduced the concept of random assignment in the 1920s, aiming to improve the quality and reliability of experimental research .

His contributions laid the groundwork for the method's evolution and its widespread adoption in various fields, particularly in psychology.

Fisher’s groundbreaking work on random assignment was motivated by his desire to control for confounding variables – those pesky factors that could muddy the waters of research findings.

By assigning participants to different groups purely by chance, he realized that the influence of these confounding variables could be minimized, paving the way for more accurate and trustworthy results.

Early Studies Utilizing Random Assignment

Following Fisher's initial development, random assignment started to gain traction in the research community. Early studies adopting this methodology focused on a variety of topics, from agriculture (which was Fisher’s primary field of interest) to medicine and psychology.

The approach allowed researchers to draw stronger conclusions from their experiments, bolstering the development of new theories and practices.

One notable early study utilizing random assignment was conducted in the field of educational psychology. Researchers were keen to understand the impact of different teaching methods on student outcomes.

By randomly assigning students to various instructional approaches, they were able to isolate the effects of the teaching methods, leading to valuable insights and recommendations for educators.

Evolution of the Methodology

As the decades rolled on, random assignment continued to evolve and adapt to the changing landscape of research.

Advances in technology introduced new tools and techniques for implementing randomization, such as computerized random number generators, which offered greater precision and ease of use.

The application of random assignment expanded beyond the confines of the laboratory, finding its way into field studies and large-scale surveys.

Researchers across diverse disciplines embraced the methodology, recognizing its potential to enhance the validity of their findings and contribute to the advancement of knowledge.

From its humble beginnings in the early 20th century to its widespread use today, random assignment has proven to be a cornerstone of scientific inquiry.

Its development and evolution have played a pivotal role in shaping the landscape of psychological research, driving discoveries that have improved lives and deepened our understanding of the human experience.

Principles of Random Assignment

Delving into the heart of random assignment, we uncover the theories and principles that form its foundation.

The method is steeped in the basics of probability theory and statistical inference, ensuring that each participant has an equal chance of being placed in any group, thus fostering fair and unbiased results.

Basic Principles of Random Assignment

Understanding the core principles of random assignment is key to grasping its significance in research. There are three principles: equal probability of selection, reduction of bias, and ensuring representativeness.

The first principle, equal probability of selection , ensures that every participant has an identical chance of being assigned to any group in the study. This randomness is crucial as it mitigates the risk of bias and establishes a level playing field.

The second principle focuses on the reduction of bias . Random assignment acts as a safeguard, ensuring that the groups being compared are alike in all essential aspects before the experiment begins.

This similarity between groups allows researchers to attribute any differences observed in the outcomes directly to the independent variable being studied.

Lastly, ensuring representativeness is a vital principle. When participants are assigned randomly, the resulting groups are more likely to be representative of the larger population.

This characteristic is crucial for the generalizability of the study’s findings, allowing researchers to apply their insights broadly.

Theoretical Foundation

The theoretical foundation of random assignment lies in probability theory and statistical inference .

Probability theory deals with the likelihood of different outcomes, providing a mathematical framework for analyzing random phenomena. In the context of random assignment, it helps in ensuring that each participant has an equal chance of being placed in any group.

Statistical inference, on the other hand, allows researchers to draw conclusions about a population based on a sample of data drawn from that population. It is the mechanism through which the results of a study can be generalized to a broader context.

Random assignment enhances the reliability of statistical inferences by reducing biases and ensuring that the sample is representative.

Differentiating Random Assignment from Random Selection

It’s essential to distinguish between random assignment and random selection, as the two terms, while related, have distinct meanings in the realm of research.

Random assignment refers to how participants are placed into different groups in an experiment, aiming to control for confounding variables and help determine causes.

In contrast, random selection pertains to how individuals are chosen to participate in a study. This method is used to ensure that the sample of participants is representative of the larger population, which is vital for the external validity of the research.

While both methods are rooted in randomness and probability, they serve different purposes in the research process.

Understanding the theories, principles, and distinctions of random assignment illuminates its pivotal role in psychological research.

This method, anchored in probability theory and statistical inference, serves as a beacon of reliability, guiding researchers in their quest for knowledge and ensuring that their findings stand the test of validity and applicability.

Methodology of Random Assignment

Implementing random assignment in a study is a meticulous process that involves several crucial steps.

The initial step is participant selection, where individuals are chosen to partake in the study. This stage is critical to ensure that the pool of participants is diverse and representative of the population the study aims to generalize to.

Once the pool of participants has been established, the actual assignment process begins. In this step, each participant is allocated randomly to one of the groups in the study.

Researchers use various tools, such as random number generators or computerized methods, to ensure that this assignment is genuinely random and free from biases.

Monitoring and adjusting form the final step in the implementation of random assignment. Researchers need to continuously observe the groups to ensure that they remain comparable in all essential aspects throughout the study.

If any significant discrepancies arise, adjustments might be necessary to maintain the study’s integrity and validity.

Tools and Techniques Used

The evolution of technology has introduced a variety of tools and techniques to facilitate random assignment.

Random number generators, both manual and computerized, are commonly used to assign participants to different groups. These generators ensure that each individual has an equal chance of being placed in any group, upholding the principle of equal probability of selection.

In addition to random number generators, researchers often use specialized computer software designed for statistical analysis and experimental design.

These software programs offer advanced features that allow for precise and efficient random assignment, minimizing the risk of human error and enhancing the study’s reliability.

Ethical Considerations

The implementation of random assignment is not devoid of ethical considerations. Informed consent is a fundamental ethical principle that researchers must uphold.

Informed consent means that every participant should be fully informed about the nature of the study, the procedures involved, and any potential risks or benefits, ensuring that they voluntarily agree to participate.

Beyond informed consent, researchers must conduct a thorough risk and benefit analysis. The potential benefits of the study should outweigh any risks or harms to the participants.

Safeguarding the well-being of participants is paramount, and any study employing random assignment must adhere to established ethical guidelines and standards.

Conclusion of Methodology

The methodology of random assignment, while seemingly straightforward, is a multifaceted process that demands precision, fairness, and ethical integrity. From participant selection to assignment and monitoring, each step is crucial to ensure the validity of the study’s findings.

The tools and techniques employed, coupled with a steadfast commitment to ethical principles, underscore the significance of random assignment as a cornerstone of robust psychological research.

Benefits of Random Assignment in Psychological Research

The impact and importance of random assignment in psychological research cannot be overstated. It is fundamental for ensuring the study is accurate, allowing the researchers to determine if their study actually caused the results they saw, and making sure the findings can be applied to the real world.

Facilitating Causal Inferences

When participants are randomly assigned to different groups, researchers can be more confident that the observed effects are due to the independent variable being changed, and not other factors.

This ability to determine the cause is called causal inference .

This confidence allows for the drawing of causal relationships, which are foundational for theory development and application in psychology.

Ensuring Internal Validity

One of the foremost impacts of random assignment is its ability to enhance the internal validity of an experiment.

Internal validity refers to the extent to which a researcher can assert that changes in the dependent variable are solely due to manipulations of the independent variable , and not due to confounding variables.

By ensuring that each participant has an equal chance of being in any condition of the experiment, random assignment helps control for participant characteristics that could otherwise complicate the results.

Enhancing Generalizability

Beyond internal validity, random assignment also plays a crucial role in enhancing the generalizability of research findings.

When done correctly, it ensures that the sample groups are representative of the larger population, so can allow researchers to apply their findings more broadly.

This representative nature is essential for the practical application of research, impacting policy, interventions, and psychological therapies.

Limitations of Random Assignment

Potential for implementation issues.

While the principles of random assignment are robust, the method can face implementation issues.

One of the most common problems is logistical constraints. Some studies, due to their nature or the specific population being studied, find it challenging to implement random assignment effectively.

For instance, in educational settings, logistical issues such as class schedules and school policies might stop the random allocation of students to different teaching methods .

Ethical Dilemmas

Random assignment, while methodologically sound, can also present ethical dilemmas.

In some cases, withholding a potentially beneficial treatment from one of the groups of participants can raise serious ethical questions, especially in medical or clinical research where participants' well-being might be directly affected.

Researchers must navigate these ethical waters carefully, balancing the pursuit of knowledge with the well-being of participants.

Generalizability Concerns

Even when implemented correctly, random assignment does not always guarantee generalizable results.

The types of people in the participant pool, the specific context of the study, and the nature of the variables being studied can all influence the extent to which the findings can be applied to the broader population.

Researchers must be cautious in making broad generalizations from studies, even those employing strict random assignment.

Practical and Real-World Limitations

In the real world, many variables cannot be manipulated for ethical or practical reasons, limiting the applicability of random assignment.

For instance, researchers cannot randomly assign individuals to different levels of intelligence, socioeconomic status, or cultural backgrounds.

This limitation necessitates the use of other research designs, such as correlational or observational studies , when exploring relationships involving such variables.

Response to Critiques

In response to these critiques, people in favor of random assignment argue that the method, despite its limitations, remains one of the most reliable ways to establish cause and effect in experimental research.

They acknowledge the challenges and ethical considerations but emphasize the rigorous frameworks in place to address them.

The ongoing discussion around the limitations and critiques of random assignment contributes to the evolution of the method, making sure it is continuously relevant and applicable in psychological research.

While random assignment is a powerful tool in experimental research, it is not without its critiques and limitations. Implementation issues, ethical dilemmas, generalizability concerns, and real-world limitations can pose significant challenges.

However, the continued discourse and refinement around these issues underline the method's enduring significance in the pursuit of knowledge in psychology.

By being careful with how we do things and doing what's right, random assignment stays a really important part of studying how people act and think.

Real-World Applications and Examples

Random assignment has been employed in many studies across various fields of psychology, leading to significant discoveries and advancements.

Here are some real-world applications and examples illustrating the diversity and impact of this method:

Medicine and Health Psychology: Randomized Controlled Trials (RCTs) are the gold standard in medical research. In these studies, participants are randomly assigned to either the treatment or control group to test the efficacy of new medications or interventions.
Educational Psychology: Studies in this field have used random assignment to explore the effects of different teaching methods, classroom environments, and educational technologies on student learning and outcomes.
Cognitive Psychology: Researchers have employed random assignment to investigate various aspects of human cognition, including memory, attention, and problem-solving, leading to a deeper understanding of how the mind works.
Social Psychology: Random assignment has been instrumental in studying social phenomena, such as conformity, aggression, and prosocial behavior, shedding light on the intricate dynamics of human interaction.

Let's get into some specific examples. You'll need to know one term though, and that is "control group." A control group is a set of participants in a study who do not receive the treatment or intervention being tested , serving as a baseline to compare with the group that does, in order to assess the effectiveness of the treatment.

Smoking Cessation Study: Researchers used random assignment to put participants into two groups. One group received a new anti-smoking program, while the other did not. This helped determine if the program was effective in helping people quit smoking.
Math Tutoring Program: A study on students used random assignment to place them into two groups. One group received additional math tutoring, while the other continued with regular classes, to see if the extra help improved their grades.
Exercise and Mental Health: Adults were randomly assigned to either an exercise group or a control group to study the impact of physical activity on mental health and mood.
Diet and Weight Loss: A study randomly assigned participants to different diet plans to compare their effectiveness in promoting weight loss and improving health markers.
Sleep and Learning: Researchers randomly assigned students to either a sleep extension group or a regular sleep group to study the impact of sleep on learning and memory.
Classroom Seating Arrangement: Teachers used random assignment to place students in different seating arrangements to examine the effect on focus and academic performance.
Music and Productivity: Employees were randomly assigned to listen to music or work in silence to investigate the effect of music on workplace productivity.
Medication for ADHD: Children with ADHD were randomly assigned to receive either medication, behavioral therapy, or a placebo to compare treatment effectiveness.
Mindfulness Meditation for Stress: Adults were randomly assigned to a mindfulness meditation group or a waitlist control group to study the impact on stress levels.
Video Games and Aggression: A study randomly assigned participants to play either violent or non-violent video games and then measured their aggression levels.
Online Learning Platforms: Students were randomly assigned to use different online learning platforms to evaluate their effectiveness in enhancing learning outcomes.
Hand Sanitizers in Schools: Schools were randomly assigned to use hand sanitizers or not to study the impact on student illness and absenteeism.
Caffeine and Alertness: Participants were randomly assigned to consume caffeinated or decaffeinated beverages to measure the effects on alertness and cognitive performance.
Green Spaces and Well-being: Neighborhoods were randomly assigned to receive green space interventions to study the impact on residents’ well-being and community connections.
Pet Therapy for Hospital Patients: Patients were randomly assigned to receive pet therapy or standard care to assess the impact on recovery and mood.
Yoga for Chronic Pain: Individuals with chronic pain were randomly assigned to a yoga intervention group or a control group to study the effect on pain levels and quality of life.
Flu Vaccines Effectiveness: Different groups of people were randomly assigned to receive either the flu vaccine or a placebo to determine the vaccine’s effectiveness.
Reading Strategies for Dyslexia: Children with dyslexia were randomly assigned to different reading intervention strategies to compare their effectiveness.
Physical Environment and Creativity: Participants were randomly assigned to different room setups to study the impact of physical environment on creative thinking.
Laughter Therapy for Depression: Individuals with depression were randomly assigned to laughter therapy sessions or control groups to assess the impact on mood.
Financial Incentives for Exercise: Participants were randomly assigned to receive financial incentives for exercising to study the impact on physical activity levels.
Art Therapy for Anxiety: Individuals with anxiety were randomly assigned to art therapy sessions or a waitlist control group to measure the effect on anxiety levels.
Natural Light in Offices: Employees were randomly assigned to workspaces with natural or artificial light to study the impact on productivity and job satisfaction.
School Start Times and Academic Performance: Schools were randomly assigned different start times to study the effect on student academic performance and well-being.
Horticulture Therapy for Seniors: Older adults were randomly assigned to participate in horticulture therapy or traditional activities to study the impact on cognitive function and life satisfaction.
Hydration and Cognitive Function: Participants were randomly assigned to different hydration levels to measure the impact on cognitive function and alertness.
Intergenerational Programs: Seniors and young people were randomly assigned to intergenerational programs to study the effects on well-being and cross-generational understanding.
Therapeutic Horseback Riding for Autism: Children with autism were randomly assigned to therapeutic horseback riding or traditional therapy to study the impact on social communication skills.
Active Commuting and Health: Employees were randomly assigned to active commuting (cycling, walking) or passive commuting to study the effect on physical health.
Mindful Eating for Weight Management: Individuals were randomly assigned to mindful eating workshops or control groups to study the impact on weight management and eating habits.
Noise Levels and Learning: Students were randomly assigned to classrooms with different noise levels to study the effect on learning and concentration.
Bilingual Education Methods: Schools were randomly assigned different bilingual education methods to compare their effectiveness in language acquisition.
Outdoor Play and Child Development: Children were randomly assigned to different amounts of outdoor playtime to study the impact on physical and cognitive development.
Social Media Detox: Participants were randomly assigned to a social media detox or regular usage to study the impact on mental health and well-being.
Therapeutic Writing for Trauma Survivors: Individuals who experienced trauma were randomly assigned to therapeutic writing sessions or control groups to study the impact on psychological well-being.
Mentoring Programs for At-risk Youth: At-risk youth were randomly assigned to mentoring programs or control groups to assess the impact on academic achievement and behavior.
Dance Therapy for Parkinson’s Disease: Individuals with Parkinson’s disease were randomly assigned to dance therapy or traditional exercise to study the effect on motor function and quality of life.
Aquaponics in Schools: Schools were randomly assigned to implement aquaponics programs to study the impact on student engagement and environmental awareness.
Virtual Reality for Phobia Treatment: Individuals with phobias were randomly assigned to virtual reality exposure therapy or traditional therapy to compare effectiveness.
Gardening and Mental Health: Participants were randomly assigned to engage in gardening or other leisure activities to study the impact on mental health and stress reduction.

Each of these studies exemplifies how random assignment is utilized in various fields and settings, shedding light on the multitude of ways it can be applied to glean valuable insights and knowledge.

Real-world Impact of Random Assignment

Random assignment is like a key tool in the world of learning about people's minds and behaviors. It’s super important and helps in many different areas of our everyday lives. It helps make better rules, creates new ways to help people, and is used in lots of different fields.

Health and Medicine

In health and medicine, random assignment has helped doctors and scientists make lots of discoveries. It’s a big part of tests that help create new medicines and treatments.

By putting people into different groups by chance, scientists can really see if a medicine works.

This has led to new ways to help people with all sorts of health problems, like diabetes, heart disease, and mental health issues like depression and anxiety.

Schools and education have also learned a lot from random assignment. Researchers have used it to look at different ways of teaching, what kind of classrooms are best, and how technology can help learning.

This knowledge has helped make better school rules, develop what we learn in school, and find the best ways to teach students of all ages and backgrounds.

Workplace and Organizational Behavior

Random assignment helps us understand how people act at work and what makes a workplace good or bad.

Studies have looked at different kinds of workplaces, how bosses should act, and how teams should be put together. This has helped companies make better rules and create places to work that are helpful and make people happy.

Environmental and Social Changes

Random assignment is also used to see how changes in the community and environment affect people. Studies have looked at community projects, changes to the environment, and social programs to see how they help or hurt people’s well-being.

This has led to better community projects, efforts to protect the environment, and programs to help people in society.

Technology and Human Interaction

In our world where technology is always changing, studies with random assignment help us see how tech like social media, virtual reality, and online stuff affect how we act and feel.

This has helped make better and safer technology and rules about using it so that everyone can benefit.

The effects of random assignment go far and wide, way beyond just a science lab. It helps us understand lots of different things, leads to new and improved ways to do things, and really makes a difference in the world around us.

From making healthcare and schools better to creating positive changes in communities and the environment, the real-world impact of random assignment shows just how important it is in helping us learn and make the world a better place.

So, what have we learned? Random assignment is like a super tool in learning about how people think and act. It's like a detective helping us find clues and solve mysteries in many parts of our lives.

From creating new medicines to helping kids learn better in school, and from making workplaces happier to protecting the environment, it’s got a big job!

This method isn’t just something scientists use in labs; it reaches out and touches our everyday lives. It helps make positive changes and teaches us valuable lessons.

Whether we are talking about technology, health, education, or the environment, random assignment is there, working behind the scenes, making things better and safer for all of us.

In the end, the simple act of putting people into groups by chance helps us make big discoveries and improvements. It’s like throwing a small stone into a pond and watching the ripples spread out far and wide.

Thanks to random assignment, we are always learning, growing, and finding new ways to make our world a happier and healthier place for everyone!

19+ Experimental Design Examples (Methods + Types)
Cluster Sampling vs Stratified Sampling
41+ White Collar Job Examples (Salary + Path)
47+ Blue Collar Job Examples (Salary + Path)
McDonaldization of Society (Definition + Examples)

Reference this article:

About The Author

Free Personality Test

Free Memory Test

Free IQ Test

PracticalPie.com is a participant in the Amazon Associates Program. As an Amazon Associate we earn from qualifying purchases.

Youtube Facebook Instagram X/Twitter

Psychology Resources

Developmental

Personality

Relationships

Psychologists

Serial Killers

Psychology Tests

Personality Quiz

Memory Test

Depression test

Type A/B Personality Test

What Is Random Assignment in Psychology?

Categories Research Methods

Random assignment means that every participant has the same chance of being chosen for the experimental or control group. It involves using procedures that rely on chance to assign participants to groups. Doing this means that every participant in a study has an equal opportunity to be assigned to any group.

For example, in a psychology experiment, participants might be assigned to either a control or experimental group. Some experiments might only have one experimental group, while others may have several treatment variations.

Using random assignment means that each participant has the same chance of being assigned to any of these groups.

Table of Contents

How to Use Random Assignment

So what type of procedures might psychologists utilize for random assignment? Strategies can include:

Flipping a coin
Assigning random numbers
Rolling dice
Drawing names out of a hat

How Does Random Assignment Work?

A psychology experiment aims to determine if changes in one variable lead to changes in another variable. Researchers will first begin by coming up with a hypothesis. Once researchers have an idea of what they think they might find in a population, they will come up with an experimental design and then recruit participants for their study.

Once they have a pool of participants representative of the population they are interested in looking at, they will randomly assign the participants to their groups.

Control group : Some participants will end up in the control group, which serves as a baseline and does not receive the independent variables.
Experimental group : Other participants will end up in the experimental groups that receive some form of the independent variables.

By using random assignment, the researchers make it more likely that the groups are equal at the start of the experiment. Since the groups are the same on other variables, it can be assumed that any changes that occur are the result of varying the independent variables.

After a treatment has been administered, the researchers will then collect data in order to determine if the independent variable had any impact on the dependent variable.

Random Assignment vs. Random Selection

It is important to remember that random assignment is not the same thing as random selection , also known as random sampling.

Random selection instead involves how people are chosen to be in a study. Using random selection, every member of a population stands an equal chance of being chosen for a study or experiment.

So random sampling affects how participants are chosen for a study, while random assignment affects how participants are then assigned to groups.

Examples of Random Assignment

Imagine that a psychology researcher is conducting an experiment to determine if getting adequate sleep the night before an exam results in better test scores.

Forming a Hypothesis

They hypothesize that participants who get 8 hours of sleep will do better on a math exam than participants who only get 4 hours of sleep.

Obtaining Participants

The researcher starts by obtaining a pool of participants. They find 100 participants from a local university. Half of the participants are female, and half are male.

Randomly Assign Participants to Groups

The researcher then assigns random numbers to each participant and uses a random number generator to randomly assign each number to either the 4-hour or 8-hour sleep groups.

Conduct the Experiment

Those in the 8-hour sleep group agree to sleep for 8 hours that night, while those in the 4-hour group agree to wake up after only 4 hours. The following day, all of the participants meet in a classroom.

Collect and Analyze Data

Everyone takes the same math test. The test scores are then compared to see if the amount of sleep the night before had any impact on test scores.

Why Is Random Assignment Important in Psychology Research?

Random assignment is important in psychology research because it helps improve a study’s internal validity. This means that the researchers are sure that the study demonstrates a cause-and-effect relationship between an independent and dependent variable.

Random assignment improves the internal validity by minimizing the risk that there are systematic differences in the participants who are in each group.

Key Points to Remember About Random Assignment

Random assignment in psychology involves each participant having an equal chance of being chosen for any of the groups, including the control and experimental groups.
It helps control for potential confounding variables, reducing the likelihood of pre-existing differences between groups.
This method enhances the internal validity of experiments, allowing researchers to draw more reliable conclusions about cause-and-effect relationships.
Random assignment is crucial for creating comparable groups and increasing the scientific rigor of psychological studies.

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

6.2 Experimental Design

Learning objectives.

Explain the difference between between-subjects and within-subjects experiments, list some of the pros and cons of each approach, and decide which approach to use to answer a particular research question.
Define random assignment, distinguish it from random sampling, explain its purpose in experimental research, and use some simple strategies to implement it.
Define what a control condition is, explain its purpose in research on treatment effectiveness, and describe some alternative types of control conditions.
Define several types of carryover effect, give examples of each, and explain how counterbalancing helps to deal with them.

In this section, we look at some different ways to design an experiment. The primary distinction we will make is between approaches in which each participant experiences one level of the independent variable and approaches in which each participant experiences all levels of the independent variable. The former are called between-subjects experiments and the latter are called within-subjects experiments.

Between-Subjects Experiments

In a between-subjects experiment , each participant is tested in only one condition. For example, a researcher with a sample of 100 college students might assign half of them to write about a traumatic event and the other half write about a neutral event. Or a researcher with a sample of 60 people with severe agoraphobia (fear of open spaces) might assign 20 of them to receive each of three different treatments for that disorder. It is essential in a between-subjects experiment that the researcher assign participants to conditions so that the different groups are, on average, highly similar to each other. Those in a trauma condition and a neutral condition, for example, should include a similar proportion of men and women, and they should have similar average intelligence quotients (IQs), similar average levels of motivation, similar average numbers of health problems, and so on. This is a matter of controlling these extraneous participant variables across conditions so that they do not become confounding variables.

Random Assignment

The primary way that researchers accomplish this kind of control of extraneous variables across conditions is called random assignment , which means using a random process to decide which participants are tested in which conditions. Do not confuse random assignment with random sampling. Random sampling is a method for selecting a sample from a population, and it is rarely used in psychological research. Random assignment is a method for assigning participants in a sample to the different conditions, and it is an important element of all experimental research in psychology and other fields too.

In its strictest sense, random assignment should meet two criteria. One is that each participant has an equal chance of being assigned to each condition (e.g., a 50% chance of being assigned to each of two conditions). The second is that each participant is assigned to a condition independently of other participants. Thus one way to assign participants to two conditions would be to flip a coin for each one. If the coin lands heads, the participant is assigned to Condition A, and if it lands tails, the participant is assigned to Condition B. For three conditions, one could use a computer to generate a random integer from 1 to 3 for each participant. If the integer is 1, the participant is assigned to Condition A; if it is 2, the participant is assigned to Condition B; and if it is 3, the participant is assigned to Condition C. In practice, a full sequence of conditions—one for each participant expected to be in the experiment—is usually created ahead of time, and each new participant is assigned to the next condition in the sequence as he or she is tested. When the procedure is computerized, the computer program often handles the random assignment.

Table 6.2 Block Randomization Sequence for Assigning Nine Participants to Three Conditions

Random assignment is not guaranteed to control all extraneous variables across conditions. It is always possible that just by chance, the participants in one condition might turn out to be substantially older, less tired, more motivated, or less depressed on average than the participants in another condition. However, there are some reasons that this is not a major concern. One is that random assignment works better than one might expect, especially for large samples. Another is that the inferential statistics that researchers use to decide whether a difference between groups reflects a difference in the population takes the “fallibility” of random assignment into account. Yet another reason is that even if random assignment does result in a confounding variable and therefore produces misleading results, this is likely to be detected when the experiment is replicated. The upshot is that random assignment to conditions—although not infallible in terms of controlling extraneous variables—is always considered a strength of a research design.

Treatment and Control Conditions

Between-subjects experiments are often used to determine whether a treatment works. In psychological research, a treatment is any intervention meant to change people’s behavior for the better. This includes psychotherapies and medical treatments for psychological disorders but also interventions designed to improve learning, promote conservation, reduce prejudice, and so on. To determine whether a treatment works, participants are randomly assigned to either a treatment condition , in which they receive the treatment, or a control condition , in which they do not receive the treatment. If participants in the treatment condition end up better off than participants in the control condition—for example, they are less depressed, learn faster, conserve more, express less prejudice—then the researcher can conclude that the treatment works. In research on the effectiveness of psychotherapies and medical treatments, this type of experiment is often called a randomized clinical trial .

There are different types of control conditions. In a no-treatment control condition , participants receive no treatment whatsoever. One problem with this approach, however, is the existence of placebo effects. A placebo is a simulated treatment that lacks any active ingredient or element that should make it effective, and a placebo effect is a positive effect of such a treatment. Many folk remedies that seem to work—such as eating chicken soup for a cold or placing soap under the bedsheets to stop nighttime leg cramps—are probably nothing more than placebos. Although placebo effects are not well understood, they are probably driven primarily by people’s expectations that they will improve. Having the expectation to improve can result in reduced stress, anxiety, and depression, which can alter perceptions and even improve immune system functioning (Price, Finniss, & Benedetti, 2008).

Placebo effects are interesting in their own right (see Note 6.28 “The Powerful Placebo” ), but they also pose a serious problem for researchers who want to determine whether a treatment works. Figure 6.2 “Hypothetical Results From a Study Including Treatment, No-Treatment, and Placebo Conditions” shows some hypothetical results in which participants in a treatment condition improved more on average than participants in a no-treatment control condition. If these conditions (the two leftmost bars in Figure 6.2 “Hypothetical Results From a Study Including Treatment, No-Treatment, and Placebo Conditions” ) were the only conditions in this experiment, however, one could not conclude that the treatment worked. It could be instead that participants in the treatment group improved more because they expected to improve, while those in the no-treatment control condition did not.

Figure 6.2 Hypothetical Results From a Study Including Treatment, No-Treatment, and Placebo Conditions

Fortunately, there are several solutions to this problem. One is to include a placebo control condition , in which participants receive a placebo that looks much like the treatment but lacks the active ingredient or element thought to be responsible for the treatment’s effectiveness. When participants in a treatment condition take a pill, for example, then those in a placebo control condition would take an identical-looking pill that lacks the active ingredient in the treatment (a “sugar pill”). In research on psychotherapy effectiveness, the placebo might involve going to a psychotherapist and talking in an unstructured way about one’s problems. The idea is that if participants in both the treatment and the placebo control groups expect to improve, then any improvement in the treatment group over and above that in the placebo control group must have been caused by the treatment and not by participants’ expectations. This is what is shown by a comparison of the two outer bars in Figure 6.2 “Hypothetical Results From a Study Including Treatment, No-Treatment, and Placebo Conditions” .

Of course, the principle of informed consent requires that participants be told that they will be assigned to either a treatment or a placebo control condition—even though they cannot be told which until the experiment ends. In many cases the participants who had been in the control condition are then offered an opportunity to have the real treatment. An alternative approach is to use a waitlist control condition , in which participants are told that they will receive the treatment but must wait until the participants in the treatment condition have already received it. This allows researchers to compare participants who have received the treatment with participants who are not currently receiving it but who still expect to improve (eventually). A final solution to the problem of placebo effects is to leave out the control condition completely and compare any new treatment with the best available alternative treatment. For example, a new treatment for simple phobia could be compared with standard exposure therapy. Because participants in both conditions receive a treatment, their expectations about improvement should be similar. This approach also makes sense because once there is an effective treatment, the interesting question about a new treatment is not simply “Does it work?” but “Does it work better than what is already available?”

The Powerful Placebo

Many people are not surprised that placebos can have a positive effect on disorders that seem fundamentally psychological, including depression, anxiety, and insomnia. However, placebos can also have a positive effect on disorders that most people think of as fundamentally physiological. These include asthma, ulcers, and warts (Shapiro & Shapiro, 1999). There is even evidence that placebo surgery—also called “sham surgery”—can be as effective as actual surgery.

Medical researcher J. Bruce Moseley and his colleagues conducted a study on the effectiveness of two arthroscopic surgery procedures for osteoarthritis of the knee (Moseley et al., 2002). The control participants in this study were prepped for surgery, received a tranquilizer, and even received three small incisions in their knees. But they did not receive the actual arthroscopic surgical procedure. The surprising result was that all participants improved in terms of both knee pain and function, and the sham surgery group improved just as much as the treatment groups. According to the researchers, “This study provides strong evidence that arthroscopic lavage with or without débridement [the surgical procedures used] is not better than and appears to be equivalent to a placebo procedure in improving knee pain and self-reported function” (p. 85).

Research has shown that patients with osteoarthritis of the knee who receive a “sham surgery” experience reductions in pain and improvement in knee function similar to those of patients who receive a real surgery.

Army Medicine – Surgery – CC BY 2.0.

Within-Subjects Experiments

In a within-subjects experiment , each participant is tested under all conditions. Consider an experiment on the effect of a defendant’s physical attractiveness on judgments of his guilt. Again, in a between-subjects experiment, one group of participants would be shown an attractive defendant and asked to judge his guilt, and another group of participants would be shown an unattractive defendant and asked to judge his guilt. In a within-subjects experiment, however, the same group of participants would judge the guilt of both an attractive and an unattractive defendant.

The primary advantage of this approach is that it provides maximum control of extraneous participant variables. Participants in all conditions have the same mean IQ, same socioeconomic status, same number of siblings, and so on—because they are the very same people. Within-subjects experiments also make it possible to use statistical procedures that remove the effect of these extraneous participant variables on the dependent variable and therefore make the data less “noisy” and the effect of the independent variable easier to detect. We will look more closely at this idea later in the book.

Carryover Effects and Counterbalancing

The primary disadvantage of within-subjects designs is that they can result in carryover effects. A carryover effect is an effect of being tested in one condition on participants’ behavior in later conditions. One type of carryover effect is a practice effect , where participants perform a task better in later conditions because they have had a chance to practice it. Another type is a fatigue effect , where participants perform a task worse in later conditions because they become tired or bored. Being tested in one condition can also change how participants perceive stimuli or interpret their task in later conditions. This is called a context effect . For example, an average-looking defendant might be judged more harshly when participants have just judged an attractive defendant than when they have just judged an unattractive defendant. Within-subjects experiments also make it easier for participants to guess the hypothesis. For example, a participant who is asked to judge the guilt of an attractive defendant and then is asked to judge the guilt of an unattractive defendant is likely to guess that the hypothesis is that defendant attractiveness affects judgments of guilt. This could lead the participant to judge the unattractive defendant more harshly because he thinks this is what he is expected to do. Or it could make participants judge the two defendants similarly in an effort to be “fair.”

Carryover effects can be interesting in their own right. (Does the attractiveness of one person depend on the attractiveness of other people that we have seen recently?) But when they are not the focus of the research, carryover effects can be problematic. Imagine, for example, that participants judge the guilt of an attractive defendant and then judge the guilt of an unattractive defendant. If they judge the unattractive defendant more harshly, this might be because of his unattractiveness. But it could be instead that they judge him more harshly because they are becoming bored or tired. In other words, the order of the conditions is a confounding variable. The attractive condition is always the first condition and the unattractive condition the second. Thus any difference between the conditions in terms of the dependent variable could be caused by the order of the conditions and not the independent variable itself.

There is a solution to the problem of order effects, however, that can be used in many situations. It is counterbalancing , which means testing different participants in different orders. For example, some participants would be tested in the attractive defendant condition followed by the unattractive defendant condition, and others would be tested in the unattractive condition followed by the attractive condition. With three conditions, there would be six different orders (ABC, ACB, BAC, BCA, CAB, and CBA), so some participants would be tested in each of the six orders. With counterbalancing, participants are assigned to orders randomly, using the techniques we have already discussed. Thus random assignment plays an important role in within-subjects designs just as in between-subjects designs. Here, instead of randomly assigning to conditions, they are randomly assigned to different orders of conditions. In fact, it can safely be said that if a study does not involve random assignment in one form or another, it is not an experiment.

There are two ways to think about what counterbalancing accomplishes. One is that it controls the order of conditions so that it is no longer a confounding variable. Instead of the attractive condition always being first and the unattractive condition always being second, the attractive condition comes first for some participants and second for others. Likewise, the unattractive condition comes first for some participants and second for others. Thus any overall difference in the dependent variable between the two conditions cannot have been caused by the order of conditions. A second way to think about what counterbalancing accomplishes is that if there are carryover effects, it makes it possible to detect them. One can analyze the data separately for each order to see whether it had an effect.

When 9 Is “Larger” Than 221

Researcher Michael Birnbaum has argued that the lack of context provided by between-subjects designs is often a bigger problem than the context effects created by within-subjects designs. To demonstrate this, he asked one group of participants to rate how large the number 9 was on a 1-to-10 rating scale and another group to rate how large the number 221 was on the same 1-to-10 rating scale (Birnbaum, 1999). Participants in this between-subjects design gave the number 9 a mean rating of 5.13 and the number 221 a mean rating of 3.10. In other words, they rated 9 as larger than 221! According to Birnbaum, this is because participants spontaneously compared 9 with other one-digit numbers (in which case it is relatively large) and compared 221 with other three-digit numbers (in which case it is relatively small).

Simultaneous Within-Subjects Designs

So far, we have discussed an approach to within-subjects designs in which participants are tested in one condition at a time. There is another approach, however, that is often used when participants make multiple responses in each condition. Imagine, for example, that participants judge the guilt of 10 attractive defendants and 10 unattractive defendants. Instead of having people make judgments about all 10 defendants of one type followed by all 10 defendants of the other type, the researcher could present all 20 defendants in a sequence that mixed the two types. The researcher could then compute each participant’s mean rating for each type of defendant. Or imagine an experiment designed to see whether people with social anxiety disorder remember negative adjectives (e.g., “stupid,” “incompetent”) better than positive ones (e.g., “happy,” “productive”). The researcher could have participants study a single list that includes both kinds of words and then have them try to recall as many words as possible. The researcher could then count the number of each type of word that was recalled. There are many ways to determine the order in which the stimuli are presented, but one common way is to generate a different random order for each participant.

Between-Subjects or Within-Subjects?

Almost every experiment can be conducted using either a between-subjects design or a within-subjects design. This means that researchers must choose between the two approaches based on their relative merits for the particular situation.

Between-subjects experiments have the advantage of being conceptually simpler and requiring less testing time per participant. They also avoid carryover effects without the need for counterbalancing. Within-subjects experiments have the advantage of controlling extraneous participant variables, which generally reduces noise in the data and makes it easier to detect a relationship between the independent and dependent variables.

A good rule of thumb, then, is that if it is possible to conduct a within-subjects experiment (with proper counterbalancing) in the time that is available per participant—and you have no serious concerns about carryover effects—this is probably the best option. If a within-subjects design would be difficult or impossible to carry out, then you should consider a between-subjects design instead. For example, if you were testing participants in a doctor’s waiting room or shoppers in line at a grocery store, you might not have enough time to test each participant in all conditions and therefore would opt for a between-subjects design. Or imagine you were trying to reduce people’s level of prejudice by having them interact with someone of another race. A within-subjects design with counterbalancing would require testing some participants in the treatment condition first and then in a control condition. But if the treatment works and reduces people’s level of prejudice, then they would no longer be suitable for testing in the control condition. This is true for many designs that involve a treatment meant to produce long-term change in participants’ behavior (e.g., studies testing the effectiveness of psychotherapy). Clearly, a between-subjects design would be necessary here.

Remember also that using one type of design does not preclude using the other type in a different study. There is no reason that a researcher could not use both a between-subjects design and a within-subjects design to answer the same research question. In fact, professional researchers often do exactly this.

Key Takeaways

Experiments can be conducted using either between-subjects or within-subjects designs. Deciding which to use in a particular situation requires careful consideration of the pros and cons of each approach.
Random assignment to conditions in between-subjects experiments or to orders of conditions in within-subjects experiments is a fundamental element of experimental research. Its purpose is to control extraneous variables so that they do not become confounding variables.
Experimental research on the effectiveness of a treatment requires both a treatment condition and a control condition, which can be a no-treatment control condition, a placebo control condition, or a waitlist control condition. Experimental treatments can also be compared with the best available alternative.

Discussion: For each of the following topics, list the pros and cons of a between-subjects and within-subjects design and decide which would be better.

You want to test the relative effectiveness of two training programs for running a marathon.
Using photographs of people as stimuli, you want to see if smiling people are perceived as more intelligent than people who are not smiling.
In a field experiment, you want to see if the way a panhandler is dressed (neatly vs. sloppily) affects whether or not passersby give him any money.
You want to see if concrete nouns (e.g., dog ) are recalled better than abstract nouns (e.g., truth ).
Discussion: Imagine that an experiment shows that participants who receive psychodynamic therapy for a dog phobia improve more than participants in a no-treatment control group. Explain a fundamental problem with this research design and at least two ways that it might be corrected.

Birnbaum, M. H. (1999). How to show that 9 > 221: Collect judgments in a between-subjects design. Psychological Methods, 4 , 243–249.

Moseley, J. B., O’Malley, K., Petersen, N. J., Menke, T. J., Brody, B. A., Kuykendall, D. H., … Wray, N. P. (2002). A controlled trial of arthroscopic surgery for osteoarthritis of the knee. The New England Journal of Medicine, 347 , 81–88.

Price, D. D., Finniss, D. G., & Benedetti, F. (2008). A comprehensive review of the placebo effect: Recent advances and current thought. Annual Review of Psychology, 59 , 565–590.

Shapiro, A. K., & Shapiro, E. (1999). The powerful placebo: From ancient priest to modern physician . Baltimore, MD: Johns Hopkins University Press.

Research Methods in Psychology Copyright © 2016 by University of Minnesota is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

school Campus Bookshelves
menu_book Bookshelves
perm_media Learning Objects
login Login
how_to_reg Request Instructor Account
hub Instructor Commons

Margin Size

Download Page (PDF)
Download Full Book (PDF)
Periodic Table
Physics Constants
Scientific Calculator
Reference & Cite
Tools expand_more
Readability

selected template will load here

This action is not available.

5.3: Experimental Design

Last updated
Save as PDF
Page ID 19647

Rajiv S. Jhangiani, I-Chant A. Chiang, Carrie Cuttler, & Dana C. Leighton

Rajiv S. Jhangiani, I-Chant A. Chiang, Carrie Cuttler, & Dana C. Leighton
Kwantlen Polytechnic U., Washington State U., & Texas A&M U.—Texarkana

$ \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } $

$ \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} $

$ \newcommand{\id}{\mathrm{id}}$ $ \newcommand{\Span}{\mathrm{span}}$

( \newcommand{\kernel}{\mathrm{null}\,}\) $ \newcommand{\range}{\mathrm{range}\,}$

$ \newcommand{\RealPart}{\mathrm{Re}}$ $ \newcommand{\ImaginaryPart}{\mathrm{Im}}$

$ \newcommand{\Argument}{\mathrm{Arg}}$ $ \newcommand{\norm}[1]{\| #1 \|}$

$ \newcommand{\inner}[2]{\langle #1, #2 \rangle}$

$ \newcommand{\Span}{\mathrm{span}}$

$ \newcommand{\id}{\mathrm{id}}$

$ \newcommand{\kernel}{\mathrm{null}\,}$

$ \newcommand{\range}{\mathrm{range}\,}$

$ \newcommand{\RealPart}{\mathrm{Re}}$

$ \newcommand{\ImaginaryPart}{\mathrm{Im}}$

$ \newcommand{\Argument}{\mathrm{Arg}}$

$ \newcommand{\norm}[1]{\| #1 \|}$

$ \newcommand{\Span}{\mathrm{span}}$ $ \newcommand{\AA}{\unicode[.8,0]{x212B}}$

$ \newcommand{\vectorA}[1]{\vec{#1}} % arrow$

$ \newcommand{\vectorAt}[1]{\vec{\text{#1}}} % arrow$

$ \newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } $

$ \newcommand{\vectorC}[1]{\textbf{#1}} $

$ \newcommand{\vectorD}[1]{\overrightarrow{#1}} $

$ \newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}} $

$ \newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}} $

Learning Objectives

Explain the difference between between-subjects and within-subjects experiments, list some of the pros and cons of each approach, and decide which approach to use to answer a particular research question.
Define random assignment, distinguish it from random sampling, explain its purpose in experimental research, and use some simple strategies to implement it
Define several types of carryover effect, give examples of each, and explain how counterbalancing helps to deal with them.

Between-Subjects Experiments

In a between-subjects experiment , each participant is tested in only one condition. For example, a researcher with a sample of 100 university students might assign half of them to write about a traumatic event and the other half write about a neutral event. Or a researcher with a sample of 60 people with severe agoraphobia (fear of open spaces) might assign 20 of them to receive each of three different treatments for that disorder. It is essential in a between-subjects experiment that the researcher assigns participants to conditions so that the different groups are, on average, highly similar to each other. Those in a trauma condition and a neutral condition, for example, should include a similar proportion of men and women, and they should have similar average IQs, similar average levels of motivation, similar average numbers of health problems, and so on. This matching is a matter of controlling these extraneous participant variables across conditions so that they do not become confounding variables.

Random Assignment

In its strictest sense, random assignment should meet two criteria. One is that each participant has an equal chance of being assigned to each condition (e.g., a 50% chance of being assigned to each of two conditions). The second is that each participant is assigned to a condition independently of other participants. Thus one way to assign participants to two conditions would be to flip a coin for each one. If the coin lands heads, the participant is assigned to Condition A, and if it lands tails, the participant is assigned to Condition B. For three conditions, one could use a computer to generate a random integer from 1 to 3 for each participant. If the integer is 1, the participant is assigned to Condition A; if it is 2, the participant is assigned to Condition B; and if it is 3, the participant is assigned to Condition C. In practice, a full sequence of conditions—one for each participant expected to be in the experiment—is usually created ahead of time, and each new participant is assigned to the next condition in the sequence as they are tested. When the procedure is computerized, the computer program often handles the random assignment.

Random assignment is not guaranteed to control all extraneous variables across conditions. The process is random, so it is always possible that just by chance, the participants in one condition might turn out to be substantially older, less tired, more motivated, or less depressed on average than the participants in another condition. However, there are some reasons that this possibility is not a major concern. One is that random assignment works better than one might expect, especially for large samples. Another is that the inferential statistics that researchers use to decide whether a difference between groups reflects a difference in the population takes the “fallibility” of random assignment into account. Yet another reason is that even if random assignment does result in a confounding variable and therefore produces misleading results, this confound is likely to be detected when the experiment is replicated. The upshot is that random assignment to conditions—although not infallible in terms of controlling extraneous variables—is always considered a strength of a research design.

Matched Groups

An alternative to simple random assignment of participants to conditions is the use of a matched-groups design . Using this design, participants in the various conditions are matched on the dependent variable or on some extraneous variable(s) prior the manipulation of the independent variable. This guarantees that these variables will not be confounded across the experimental conditions. For instance, if we want to determine whether expressive writing affects people’s health then we could start by measuring various health-related variables in our prospective research participants. We could then use that information to rank-order participants according to how healthy or unhealthy they are. Next, the two healthiest participants would be randomly assigned to complete different conditions (one would be randomly assigned to the traumatic experiences writing condition and the other to the neutral writing condition). The next two healthiest participants would then be randomly assigned to complete different conditions, and so on until the two least healthy participants. This method would ensure that participants in the traumatic experiences writing condition are matched to participants in the neutral writing condition with respect to health at the beginning of the study. If at the end of the experiment, a difference in health was detected across the two conditions, then we would know that it is due to the writing manipulation and not to pre-existing differences in health.

Within-Subjects Experiments

Carryover Effects and Counterbalancing

The primary disadvantage of within-subjects designs is that they can result in order effects. An order effect occurs when participants’ responses in the various conditions are affected by the order of conditions to which they were exposed. One type of order effect is a carryover effect. A carryover effect is an effect of being tested in one condition on participants’ behavior in later conditions. One type of carryover effect is a practice effect , where participants perform a task better in later conditions because they have had a chance to practice it. Another type is a fatigue effect , where participants perform a task worse in later conditions because they become tired or bored. Being tested in one condition can also change how participants perceive stimuli or interpret their task in later conditions. This type of effect is called a context effect (or contrast effect) . For example, an average-looking defendant might be judged more harshly when participants have just judged an attractive defendant than when they have just judged an unattractive defendant. Within-subjects experiments also make it easier for participants to guess the hypothesis. For example, a participant who is asked to judge the guilt of an attractive defendant and then is asked to judge the guilt of an unattractive defendant is likely to guess that the hypothesis is that defendant attractiveness affects judgments of guilt. This knowledge could lead the participant to judge the unattractive defendant more harshly because he thinks this is what he is expected to do. Or it could make participants judge the two defendants similarly in an effort to be “fair.”

There is a solution to the problem of order effects, however, that can be used in many situations. It is counterbalancing , which means testing different participants in different orders. The best method of counterbalancing is complete counterbalancing in which an equal number of participants complete each possible order of conditions. For example, half of the participants would be tested in the attractive defendant condition followed by the unattractive defendant condition, and others half would be tested in the unattractive condition followed by the attractive condition. With three conditions, there would be six different orders (ABC, ACB, BAC, BCA, CAB, and CBA), so some participants would be tested in each of the six orders. With four conditions, there would be 24 different orders; with five conditions there would be 120 possible orders. With counterbalancing, participants are assigned to orders randomly, using the techniques we have already discussed. Thus, random assignment plays an important role in within-subjects designs just as in between-subjects designs. Here, instead of randomly assigning to conditions, they are randomly assigned to different orders of conditions. In fact, it can safely be said that if a study does not involve random assignment in one form or another, it is not an experiment.

A more efficient way of counterbalancing is through a Latin square design which randomizes through having equal rows and columns. For example, if you have four treatments, you must have four versions. Like a Sudoku puzzle, no treatment can repeat in a row or column. For four versions of four treatments, the Latin square design would look like:

You can see in the diagram above that the square has been constructed to ensure that each condition appears at each ordinal position (A appears first once, second once, third once, and fourth once) and each condition precedes and follows each other condition one time. A Latin square for an experiment with 6 conditions would by 6 x 6 in dimension, one for an experiment with 8 conditions would be 8 x 8 in dimension, and so on. So while complete counterbalancing of 6 conditions would require 720 orders, a Latin square would only require 6 orders.

Finally, when the number of conditions is large experiments can use random counterbalancing in which the order of the conditions is randomly determined for each participant. Using this technique every possible order of conditions is determined and then one of these orders is randomly selected for each participant. This is not as powerful a technique as complete counterbalancing or partial counterbalancing using a Latin squares design. Use of random counterbalancing will result in more random error, but if order effects are likely to be small and the number of conditions is large, this is an option available to researchers.

When 9 Is “Larger” Than 221

Researcher Michael Birnbaum has argued that the lack of context provided by between-subjects designs is often a bigger problem than the context effects created by within-subjects designs. To demonstrate this problem, he asked participants to rate two numbers on how large they were on a scale of 1-to-10 where 1 was “very very small” and 10 was “very very large”. One group of participants were asked to rate the number 9 and another group was asked to rate the number 221 (Birnbaum, 1999) 221: Collect judgments in a between-subjects design. Psychological Methods, 4(3), 243-249." href="/Bookshelves/Psychology/Book:_Research_Methods_in_Psychology_(Jhangiani_Cuttler_and_Leighton)/05:_Experimental_Research/5.03:_Experimental_Design#footnote-63-1"> [1] . Participants in this between-subjects design gave the number 9 a mean rating of 5.13 and the number 221 a mean rating of 3.10. In other words, they rated 9 as larger than 221! According to Birnbaum, this difference is because participants spontaneously compared 9 with other one-digit numbers (in which case it is relatively large) and compared 221 with other three-digit numbers (in which case it is relatively small).

Simultaneous Within-Subjects Designs

Between-Subjects or Within-Subjects?

Almost every experiment can be conducted using either a between-subjects design or a within-subjects design. This possibility means that researchers must choose between the two approaches based on their relative merits for the particular situation.

Between-subjects experiments have the advantage of being conceptually simpler and requiring less testing time per participant. They also avoid carryover effects without the need for counterbalancing. Within-subjects experiments have the advantage of controlling extraneous participant variables, which generally reduces noise in the data and makes it easier to detect any effect of the independent variable upon the dependent variable. Within-subjects experiments also require fewer participants than between-subjects experiments to detect an effect of the same size.

A good rule of thumb, then, is that if it is possible to conduct a within-subjects experiment (with proper counterbalancing) in the time that is available per participant—and you have no serious concerns about carryover effects—this design is probably the best option. If a within-subjects design would be difficult or impossible to carry out, then you should consider a between-subjects design instead. For example, if you were testing participants in a doctor’s waiting room or shoppers in line at a grocery store, you might not have enough time to test each participant in all conditions and therefore would opt for a between-subjects design. Or imagine you were trying to reduce people’s level of prejudice by having them interact with someone of another race. A within-subjects design with counterbalancing would require testing some participants in the treatment condition first and then in a control condition. But if the treatment works and reduces people’s level of prejudice, then they would no longer be suitable for testing in the control condition. This difficulty is true for many designs that involve a treatment meant to produce long-term change in participants’ behavior (e.g., studies testing the effectiveness of psychotherapy). Clearly, a between-subjects design would be necessary here.

Birnbaum, M.H. (1999). How to show that 9>221: Collect judgments in a between-subjects design. Psychological Methods, 4 (3), 243-249. ↵

Random Assignment in Psychology (Intro for Students)

random assignment examples and definition, explained below

Random assignment is a research procedure used to randomly assign participants to different experimental conditions (or ‘groups’). This introduces the element of chance, ensuring that each participant has an equal likelihood of being placed in any condition group for the study.

It is absolutely essential that the treatment condition and the control condition are the same in all ways except for the variable being manipulated.

Using random assignment to place participants in different conditions helps to achieve this.

It ensures that those conditions are the same in regards to all potential confounding variables and extraneous factors .

Why Researchers Use Random Assignment

Researchers use random assignment to control for confounds in research.

Confounds refer to unwanted and often unaccounted-for variables that might affect the outcome of a study. These confounding variables can skew the results, rendering the experiment unreliable.

For example, below is a study with two groups. Note how there are more ‘red’ individuals in the first group than the second:

a representation of a treatment condition showing 12 red people in the cohort

There is likely a confounding variable in this experiment explaining why more red people ended up in the treatment condition and less in the control condition. The red people might have self-selected, for example, leading to a skew of them in one group over the other.

Ideally, we’d want a more even distribution, like below:

a representation of a treatment condition showing 4 red people in the cohort

To achieve better balance in our two conditions, we use randomized sampling.

Fact File: Experiments 101

Random assignment is used in the type of research called the experiment.

An experiment involves manipulating the level of one variable and examining how it affects another variable. These are the independent and dependent variables :

Independent Variable: The variable manipulated is called the independent variable (IV)
Dependent Variable: The variable that it is expected to affect is called the dependent variable (DV).

The most basic form of the experiment involves two conditions: the treatment and the control .

The Treatment Condition: The treatment condition involves the participants being exposed to the IV.
The Control Condition: The control condition involves the absence of the IV. Therefore, the IV has two levels: zero and some quantity.

Researchers utilize random assignment to determine which participants go into which conditions.

Methods of Random Assignment

There are several procedures that researchers can use to randomly assign participants to different conditions.

1. Random number generator

There are several websites that offer computer-generated random numbers. Simply indicate how many conditions are in the experiment and then click. If there are 4 conditions, the program will randomly generate a number between 1 and 4 each time it is clicked.

2. Flipping a coin

If there are two conditions in an experiment, then the simplest way to implement random assignment is to flip a coin for each participant. Heads means being assigned to the treatment and tails means being assigned to the control (or vice versa).

3. Rolling a die

Rolling a single die is another way to randomly assign participants. If the experiment has three conditions, then numbers 1 and 2 mean being assigned to the control; numbers 3 and 4 mean treatment condition one; and numbers 5 and 6 mean treatment condition two.

4. Condition names in a hat

In some studies, the researcher will write the name of the treatment condition(s) or control on slips of paper and place them in a hat. If there are 4 conditions and 1 control, then there are 5 slips of paper.

The researcher closes their eyes and selects one slip for each participant. That person is then assigned to one of the conditions in the study and that slip of paper is placed back in the hat. Repeat as necessary.

There are other ways of trying to ensure that the groups of participants are equal in all ways with the exception of the IV. However, random assignment is the most often used because it is so effective at reducing confounds.

Read About More Methods and Examples of Random Assignment Here

Potential Confounding Effects

Random assignment is all about minimizing confounding effects.

Here are six types of confounds that can be controlled for using random assignment:

Individual Differences: Participants in a study will naturally vary in terms of personality, intelligence, mood, prior knowledge, and many other characteristics. If one group happens to have more people with a particular characteristic, this could affect the results. Random assignment ensures that these individual differences are spread out equally among the experimental groups, making it less likely that they will unduly influence the outcome.
Temporal or Time-Related Confounds: Events or situations that occur at a particular time can influence the outcome of an experiment. For example, a participant might be tested after a stressful event, while another might be tested after a relaxing weekend. Random assignment ensures that such effects are equally distributed among groups, thus controlling for their potential influence.
Order Effects: If participants are exposed to multiple treatments or tests, the order in which they experience them can influence their responses. Randomly assigning the order of treatments for different participants helps control for this.
Location or Environmental Confounds: The environment in which the study is conducted can influence the results. One group might be tested in a noisy room, while another might be in a quiet room. Randomly assigning participants to different locations can control for these effects.
Instrumentation Confounds: These occur when there are variations in the calibration or functioning of measurement instruments across conditions. If one group’s responses are being measured using a slightly different tool or scale, it can introduce a confound. Random assignment can ensure that any such potential inconsistencies in instrumentation are equally distributed among groups.
Experimenter Effects: Sometimes, the behavior or expectations of the person administering the experiment can unintentionally influence the participants’ behavior or responses. For instance, if an experimenter believes one treatment is superior, they might unconsciously communicate this belief to participants. Randomly assigning experimenters or using a double-blind procedure (where neither the participant nor the experimenter knows the treatment being given) can help control for this.

Random assignment helps balance out these and other potential confounds across groups, ensuring that any observed differences are more likely due to the manipulated independent variable rather than some extraneous factor.

Limitations of the Random Assignment Procedure

Although random assignment is extremely effective at eliminating the presence of participant-related confounds, there are several scenarios in which it cannot be used.

Ethics: The most obvious scenario is when it would be unethical. For example, if wanting to investigate the effects of emotional abuse on children, it would be unethical to randomly assign children to either received abuse or not. Even if a researcher were to propose such a study, it would not receive approval from the Institutional Review Board (IRB) which oversees research by university faculty.
Practicality: Other scenarios involve matters of practicality. For example, randomly assigning people to specific types of diet over a 10-year period would be interesting, but it would be highly unlikely that participants would be diligent enough to make the study valid. This is why examining these types of subjects has to be carried out through observational studies . The data is correlational, which is informative, but falls short of the scientist’s ultimate goal of identifying causality.
Small Sample Size: The smaller the sample size being assigned to conditions, the more likely it is that the two groups will be unequal. For example, if you flip a coin many times in a row then you will notice that sometimes there will be a string of heads or tails that come up consecutively. This means that one condition may have a build-up of participants that share the same characteristics. However, if you continue flipping the coin, over the long-term, there will be a balance of heads and tails. Unfortunately, how large a sample size is necessary has been the subject of considerable debate (Bloom, 2006; Shadish et al., 2002).

“It is well known that larger sample sizes reduce the probability that random assignment will result in conditions that are unequal” (Goldberg, 2019, p. 2).

Applications of Random Assignment

The importance of random assignment has been recognized in a wide range of scientific and applied disciplines (Bloom, 2006).

Random assignment began as a tool in agricultural research by Fisher (1925, 1935). After WWII, it became extensively used in medical research to test the effectiveness of new treatments and pharmaceuticals (Marks, 1997).

Today it is widely used in industrial engineering (Box, Hunter, and Hunter, 2005), educational research (Lindquist, 1953; Ong-Dean et al., 2011)), psychology (Myers, 1972), and social policy studies (Boruch, 1998; Orr, 1999).

One of the biggest obstacles to the validity of an experiment is the confound. If the group of participants in the treatment condition are substantially different from the group in the control condition, then it is impossible to determine if the IV has an affect or if the confound has an effect.

Thankfully, random assignment is highly effective at eliminating confounds that are known and unknown. Because each participant has an equal chance of being placed in each condition, they are equally distributed.

There are several ways of implementing random assignment, including flipping a coin or using a random number generator.

Random assignment has become an essential procedure in research in a wide range of subjects such as psychology, education, and social policy.

Alferes, V. R. (2012). Methods of randomization in experimental design . Sage Publications.

Bloom, H. S. (2008). The core analytics of randomized experiments for social research. The SAGE Handbook of Social Research Methods , 115-133.

Boruch, R. F. (1998). Randomized controlled experiments for evaluation and planning. Handbook of applied social research methods , 161-191.

Box, G. E., Hunter, W. G., & Hunter, J. S. (2005). Design of experiments: Statistics for Experimenters: Design, Innovation and Discovery.

Dehue, T. (1997). Deception, efficiency, and random groups: Psychology and the gradual origination of the random group design. Isis , 88 (4), 653-673.

Fisher, R.A. (1925). Statistical methods for research workers (11th ed. rev.). Oliver and Boyd: Edinburgh.

Fisher, R. A. (1935). The Design of Experiments. Edinburgh: Oliver and Boyd.

Goldberg, M. H. (2019). How often does random assignment fail? Estimates and recommendations. Journal of Environmental Psychology , 66 , 101351.

Jamison, J. C. (2019). The entry of randomized assignment into the social sciences. Journal of Causal Inference , 7 (1), 20170025.

Lindquist, E. F. (1953). Design and analysis of experiments in psychology and education . Boston: Houghton Mifflin Company.

Marks, H. M. (1997). The progress of experiment: Science and therapeutic reform in the United States, 1900-1990 . Cambridge University Press.

Myers, J. L. (1972). Fundamentals of experimental design (2nd ed.). Allyn & Bacon.

Ong-Dean, C., Huie Hofstetter, C., & Strick, B. R. (2011). Challenges and dilemmas in implementing random assignment in educational research. American Journal of Evaluation , 32 (1), 29-49.

Orr, L. L. (1999). Social experiments: Evaluating public programs with experimental methods . Sage.

Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Quasi-experiments: interrupted time-series designs. Experimental and quasi-experimental designs for generalized causal inference , 171-205.

Stigler, S. M. (1992). A historical view of statistical concepts in psychology and educational research. American Journal of Education , 101 (1), 60-70.

Dave Cornell (PhD)

Dr. Cornell has worked in education for more than 20 years. His work has involved designing teacher certification for Trinity College in London and in-service training for state governments in the United States. He has trained kindergarten teachers in 8 countries and helped businessmen and women open baby centers and kindergartens in 3 countries.

Dave Cornell (PhD) https://helpfulprofessor.com/author/dave-cornell-phd/ 25 Positive Punishment Examples
Dave Cornell (PhD) https://helpfulprofessor.com/author/dave-cornell-phd/ 25 Dissociation Examples (Psychology)
Dave Cornell (PhD) https://helpfulprofessor.com/author/dave-cornell-phd/ 15 Zone of Proximal Development Examples
Dave Cornell (PhD) https://helpfulprofessor.com/author/dave-cornell-phd/ Perception Checking: 15 Examples and Definition

Chris Drew (PhD)

This article was peer-reviewed and edited by Chris Drew (PhD). The review process on Helpful Professor involves having a PhD level expert fact check, edit, and contribute to articles. Reviewers ensure all content reflects expert academic consensus and is backed up with reference to academic studies. Dr. Drew has published over 20 academic articles in scholarly journals. He is the former editor of the Journal of Learning Development in Higher Education and holds a PhD in Education from ACU.

Chris Drew (PhD) #molongui-disabled-link 25 Positive Punishment Examples
Chris Drew (PhD) #molongui-disabled-link 25 Dissociation Examples (Psychology)
Chris Drew (PhD) #molongui-disabled-link 15 Zone of Proximal Development Examples
Chris Drew (PhD) #molongui-disabled-link Perception Checking: 15 Examples and Definition

1.3 Defining Psychological Science: The Experimental Method

5 min read • january 6, 2023

Sadiyya Holsey

Attend a live cram event

Review all units live with expert teachers & students

Types of Research

Basic Research is performed to learn about something. It is curiosity-driven and used to expand upon knowledge. It doesn't have an immediate objective. An example of basic research would be a study assessing the impacts of caffeine ☕️ consumption on the brain. As you can see from the example, the goal is not to solve a problem it is only to increase knowledge about a particular topic.

Applied Research answers specific questions and is used to solve a problem or do something of practical use. An example of applied research would be trying to find a cure for obsessive-compulsive disorder.

The Scientific Method

Even though you have probably learned this a million times, here is a quick overview:

First, the researcher would make a theory to try and explain the behavior that we are observing.

This theory would then produce a hypothesis , or an educated guess/testable prediction.

However, theories could bias our observations. Think about it: if you want to prove your theory correct, you would try and make it so the results prove it.

To avoid this bias, psychologists use something called an operational definition.

How do you test something if every researcher describes it in a different way, possibly, even, in a biased way?🤔

Operational Definitions are statements of the exact procedures used in the study, which would eventually allow other researchers to replicate the research.

Here's an example: how would you describe human intelligence?

You may have said how smart someone is, measured by their grades, but this is a biased definition. The operational definition would be what an intelligence test (such as an IQ test) measures.

Types of Variables

There are several types of variables in an experiment. There is the independent variable , dependent variable , confounding variables , and control variables .

The independent variable is the variable that changes in an experiment. For example, a researcher wanted to see how sleep affects performance on a certain exam. The researcher would change the amounts of sleep given to the subjects in order to see any changes.

The dependent variable is the effect of the change in the experiment. This is what gets measured. For example, in the earlier example with sleep 😴 and performance on exams, sleep is the independent variable and the performance on the exam is the dependent variable , because performance on the exam “depends” on the independent variable , sleep.

The confounding variable is an outside influence that changes the effect of the dependent and independent variables. For example, say there is a correlation between crime and the sale of ice cream🍨. As the crime rate increases, ice cream sales also increase. So, one might suggest that criminals cause people to buy ice cream or that purchasing ice cream causes people to commit crimes. However, both are extremely unlikely.

The confounding variable includes a new outside variable not present in the original experiment. In our ice cream example, let's look at the weather, which could be the reason for the correlation. Ice cream is more often sold when it is hot outside, and people are more likely to commit crimes when it is hot outside because there is more social interaction. In the winter, people are less likely to buy ice cream, and there is also less social interaction. ☀️

To recall from key topic 1.2 , the Hawthorne Effect exists as well. If a researcher is observing people, those people would behave differently when they realize they are being watched, impacting the results of the naturalistic observation.

https://firebasestorage.googleapis.com/v0/b/fiveable-92889.appspot.com/o/images%2F-zNEVuhDfF5EX.png?alt=media&token=e8c81766-af65-4791-95e9-f0775f566518

Image Courtesy of Kiana Matthews .

The control variable is the variable that's kept the same throughout an experiment. For example, if a researcher wants to see how sleep affects performance on a test, the control variables could be the test, sleeping atmosphere, and the type of bed. These would all be kept the same throughout the research.

Why do we need a control variable?

Well, then you can prove why something happened without an alternate explanation. You can’t say the type of bed or the difficulty of the test impacted those results if the researcher kept those the same throughout the experiment.

Cause and Effect

Also, a researcher must use random assignment to demonstrate cause and effect . Random assignment is when participants are assigned to each experimental group with an equal chance ⚖️ of being chosen. Don't confuse this with a random sample : each individual in the population has an equal chance of participating in an experiment.

Random assignment is randomly selecting people to be in an experimental group while random sample is randomly selecting people from the population to be in the experiment as a whole. Both random sampling and random assignment ultimately lead to the most accurate results.

Types of Bias

Sampling bias is a result of a flawed sampling process that produces an unrepresentative sample.

Experimenter bias is when researchers influence the results of an experiment to portray a certain outcome. A double-blind procedure is when neither the researcher or the participants know what groups the participants have been assigned to. This helps prevent bias when the researcher is looking over the results.

Common Sense?

Researchers and scientists cannot rely on common sense because of three main concepts:

Hindsight Bias : the tendency to believe that you knew what was going to happen, as if you foresaw the event: "I knew it all along."

Overconfidence : we are often overconfident in what we find/believe, which misleads others about the truth.

We perceive order in events that are completely random. You can see this with coin flips. If you were to ask a group of students to flip a coin 50 times and record data, you'd be able to easily tell who actually did the assignment and who thought they could just make up the results. Those that actually did the assignment would have had long chains of heads or long chains of tails (HHHHHHH/TTTTTTT) while those that didn't would just alternate between the two (HTHTHTHHTTHTHT). We are generally unable to understand randomness since we always try to make sense of it.

Key Terms to Review ( 19 )

Applied Research

Basic Research

Confounding Variables

Control Variables

Dependent Variable

Double-Blind Procedure

Experimenter Bias

Hawthorne Effect

Hindsight Bias

Independent Variable

Operational Definitions

Overconfidence

Random Assignment

Random Sample

Sampling Bias

Scientific Method

Stay Connected

AP® and SAT® are trademarks registered by the College Board, which is not affiliated with, and does not endorse this website.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

Knowledge Base
Methodology
Random Assignment in Experiments | Introduction & Examples

Random Assignment in Experiments | Introduction & Examples

Published on 6 May 2022 by Pritha Bhandari . Revised on 13 February 2023.

In experimental research, random assignment is a way of placing participants from your sample into different treatment groups using randomisation.

With simple random assignment, every member of the sample has a known or equal chance of being placed in a control group or an experimental group. Studies that use simple random assignment are also called completely randomised designs .

Random assignment is a key part of experimental design . It helps you ensure that all groups are comparable at the start of a study: any differences between them are due to random factors.

Why does random assignment matter, random sampling vs random assignment, how do you use random assignment, when is random assignment not used, frequently asked questions about random assignment.

Random assignment is an important part of control in experimental research, because it helps strengthen the internal validity of an experiment.

In experiments, researchers manipulate an independent variable to assess its effect on a dependent variable, while controlling for other variables. To do so, they often use different levels of an independent variable for different groups of participants.

This is called a between-groups or independent measures design.

You use three groups of participants that are each given a different level of the independent variable:

A control group that’s given a placebo (no dosage)
An experimental group that’s given a low dosage
A second experimental group that’s given a high dosage

Random assignment to helps you make sure that the treatment groups don’t differ in systematic or biased ways at the start of the experiment.

If you don’t use random assignment, you may not be able to rule out alternative explanations for your results.

Participants recruited from pubs are placed in the control group
Participants recruited from local community centres are placed in the low-dosage experimental group
Participants recruited from gyms are placed in the high-dosage group

With this type of assignment, it’s hard to tell whether the participant characteristics are the same across all groups at the start of the study. Gym users may tend to engage in more healthy behaviours than people who frequent pubs or community centres, and this would introduce a healthy user bias in your study.

Although random assignment helps even out baseline differences between groups, it doesn’t always make them completely equivalent. There may still be extraneous variables that differ between groups, and there will always be some group differences that arise from chance.

Most of the time, the random variation between groups is low, and, therefore, it’s acceptable for further analysis. This is especially true when you have a large sample. In general, you should always use random assignment in experiments when it is ethically possible and makes sense for your study topic.

Prevent plagiarism, run a free check.

Random sampling and random assignment are both important concepts in research, but it’s important to understand the difference between them.

Random sampling (also called probability sampling or random selection) is a way of selecting members of a population to be included in your study. In contrast, random assignment is a way of sorting the sample participants into control and experimental groups.

While random sampling is used in many types of studies, random assignment is only used in between-subjects experimental designs.

Some studies use both random sampling and random assignment, while others use only one or the other.

Random sampling enhances the external validity or generalisability of your results, because it helps to ensure that your sample is unbiased and representative of the whole population. This allows you to make stronger statistical inferences .

You use a simple random sample to collect data. Because you have access to the whole population (all employees), you can assign all 8,000 employees a number and use a random number generator to select 300 employees. These 300 employees are your full sample.

Random assignment enhances the internal validity of the study, because it ensures that there are no systematic differences between the participants in each group. This helps you conclude that the outcomes can be attributed to the independent variable .

A control group that receives no intervention
An experimental group that has a remote team-building intervention every week for a month

You use random assignment to place participants into the control or experimental group. To do so, you take your list of participants and assign each participant a number. Again, you use a random number generator to place each participant in one of the two groups.

To use simple random assignment, you start by giving every member of the sample a unique number. Then, you can use computer programs or manual methods to randomly assign each participant to a group.

Random number generator: Use a computer program to generate random numbers from the list for each group.
Lottery method: Place all numbers individually into a hat or a bucket, and draw numbers at random for each group.
Flip a coin: When you only have two groups, for each number on the list, flip a coin to decide if they’ll be in the control or the experimental group.
Use a dice: When you have three groups, for each number on the list, roll a die to decide which of the groups they will be in. For example, assume that rolling 1 or 2 lands them in a control group; 3 or 4 in an experimental group; and 5 or 6 in a second control or experimental group.

This type of random assignment is the most powerful method of placing participants in conditions, because each individual has an equal chance of being placed in any one of your treatment groups.

Random assignment in block designs

In more complicated experimental designs, random assignment is only used after participants are grouped into blocks based on some characteristic (e.g., test score or demographic variable). These groupings mean that you need a larger sample to achieve high statistical power .

For example, a randomised block design involves placing participants into blocks based on a shared characteristic (e.g., college students vs graduates), and then using random assignment within each block to assign participants to every treatment condition. This helps you assess whether the characteristic affects the outcomes of your treatment.

In an experimental matched design , you use blocking and then match up individual participants from each block based on specific characteristics. Within each matched pair or group, you randomly assign each participant to one of the conditions in the experiment and compare their outcomes.

Sometimes, it’s not relevant or ethical to use simple random assignment, so groups are assigned in a different way.

When comparing different groups

Sometimes, differences between participants are the main focus of a study, for example, when comparing children and adults or people with and without health conditions. Participants are not randomly assigned to different groups, but instead assigned based on their characteristics.

In this type of study, the characteristic of interest (e.g., gender) is an independent variable, and the groups differ based on the different levels (e.g., men, women). All participants are tested the same way, and then their group-level outcomes are compared.

When it’s not ethically permissible

When studying unhealthy or dangerous behaviours, it’s not possible to use random assignment. For example, if you’re studying heavy drinkers and social drinkers, it’s unethical to randomly assign participants to one of the two groups and ask them to drink large amounts of alcohol for your experiment.

When you can’t assign participants to groups, you can also conduct a quasi-experimental study . In a quasi-experiment, you study the outcomes of pre-existing groups who receive treatments that you may not have any control over (e.g., heavy drinkers and social drinkers).

These groups aren’t randomly assigned, but may be considered comparable when some other variables (e.g., age or socioeconomic status) are controlled for.

In experimental research, random assignment is a way of placing participants from your sample into different groups using randomisation. With this method, every member of the sample has a known or equal chance of being placed in a control group or an experimental group.

Random selection, or random sampling , is a way of selecting members of a population for your study’s sample.

In contrast, random assignment is a way of sorting the sample into control and experimental groups.

Random sampling enhances the external validity or generalisability of your results, while random assignment improves the internal validity of your study.

Random assignment is used in experiments with a between-groups or independent measures design. In this research design, there’s usually a control group and one or more experimental groups. Random assignment helps ensure that the groups are comparable.

In general, you should always use random assignment in this type of experimental design when it is ethically possible and makes sense for your study topic.

To implement random assignment , assign a unique number to every member of your study’s sample .

Then, you can use a random number generator or a lottery method to randomly assign each number to a control or experimental group. You can also do so manually, by flipping a coin or rolling a die to randomly assign participants to groups.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

Bhandari, P. (2023, February 13). Random Assignment in Experiments | Introduction & Examples. Scribbr. Retrieved 14 May 2024, from https://www.scribbr.co.uk/research-methods/random-assignment-experiments/

Is this article helpful?

Pritha Bhandari

Other students also liked, a quick guide to experimental design | 5 steps & examples, controlled experiments | methods & examples of control, control groups and treatment groups | uses & examples.

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

9 Chapter 9: Simple Experiments

Simple experiments.

What Is an Experiment?

As we saw earlier, an experiment is a type of study designed specifically to answer the question of whether there is a causal relationship between two variables. Do changes in an independent variable cause changes in a dependent variable? Experiments have two fundamental features. The first is that the researchers manipulate, or systematically vary, the level of the independent variable. The different levels of the independent variable are called conditions. For example, in Darley and Latané’s experiment, the independent variable was the number of witnesses that participants believed to be present. The researchers manipulated this independent variable by telling participants that there were either one, two, or five other students involved in the discussion, thereby creating three conditions. The second fundamental feature of an experiment is that the researcher controls, or minimizes the variability in, variables other than the independent and dependent variable. These other variables are called extraneous variables. Darley and Latané tested all their participants in the same room, exposed them to the same emergency situation, and so on. They also randomly assigned their participants to conditions so that the three groups would be similar to each other to begin with. Notice that although the words manipulation and control have similar meanings in everyday language, researchers make a clear distinction between them. They manipulate the independent variable by systematically changing its levels and control other variables by holding them constant.

9.1 Experiment Basics

Internal Validity

Recall that the fact that two variables are statistically related does not necessarily mean that one causes the other. “Correlation does not imply causation.” For example, if it were the case that people who exercise regularly are happier than people who do not exercise regularly, this would not necessarily mean that exercising increases people’s happiness. It could mean instead that greater happiness causes people to exercise (the directionality problem) or that something like better physical health causes people to exercise and be happier (the third-variable problem).

The purpose of an experiment, however, is to show that two variables are statistically related and to do so in a way that supports the conclusion that the independent variable caused any observed differences in the dependent variable. The basic logic is this: If the researcher creates two or more highly similar conditions and then manipulates the independent variable to produce just one difference between them, then any later difference between the conditions must have been caused by the independent variable. For example, because the only difference between Darley and Latané’s conditions was the number of students that participants believed to be involved in the discussion, this must have been responsible for differences in helping between the conditions.

An empirical study is said to be high in internal validity if the way it was conducted supports the conclusion that the independent variable caused any observed differences in the dependent variable. Thus experiments are high in internal validity because the way they are conducted—with the manipulation of the independent variable and the control of extraneous variables—provides strong support for causal conclusions.

External Validity

At the same time, the way that experiments are conducted sometimes leads to a different kind of criticism. Specifically, the need to manipulate the independent variable and control extraneous variables means that experiments are often conducted under conditions that seem artificial or unlike “real life” (Stanovich, 2010). In many psychology experiments, the participants are all college undergraduates and come to a classroom or laboratory to fill out a series of paper-and-pencil questionnaires or to perform a carefully designed computerized task. Consider, for example, an experiment in which researcher Barbara Fredrickson and her colleagues had college students come to a laboratory on campus and complete a math test while wearing a swimsuit (Fredrickson, Roberts, Noll, Quinn, & Twenge, 1998). At first, this might seem silly. When will college students ever have to complete math tests in their swimsuits outside of this experiment?

The issue we are confronting is that of external validity. An empirical study is high in external validity if the way it was conducted supports generalizing the results to people and situations beyond those actually studied. As a general rule, studies are higher in external validity when the participants and the situation studied are similar to those that the researchers want to generalize to. Imagine, for example, that a group of researchers is interested in how shoppers in large grocery stores are affected by whether breakfast cereal is packaged in yellow or purple boxes. Their study would be high in external validity if they studied the decisions of ordinary people doing their weekly shopping in a real grocery store. If the shoppers bought much more cereal in purple boxes, the researchers would be fairly confident that this would be true for other shoppers in other stores. Their study would be relatively low in external validity, however, if they studied a sample of college students in a laboratory at a selective college who merely judged the appeal of various colors presented on a computer screen. If the students judged purple to be more appealing than yellow, the researchers would not be very confident that this is relevant to grocery shoppers’ cereal-buying decisions.

We should be careful, however, not to draw the blanket conclusion that experiments are low in external validity. One reason is that experiments need not seem artificial. Consider that Darley and Latané’s experiment provided a reasonably good simulation of a real emergency situation. Or consider field experiments that are conducted entirely outside the laboratory. In one such experiment, Robert Cialdini and his colleagues studied whether hotel guests choose to reuse their towels for a second day as opposed to having them washed as a way of conserving water and energy (Cialdini, 2005). These researchers manipulated the message on a card left in a large sample of hotel rooms. One version of the message emphasized showing respect for the environment, another emphasized that the hotel would donate a portion of their savings to an environmental cause, and a third emphasized that most hotel guests choose to reuse their towels. The result was that guests who received the message that most hotel guests choose to reuse their towels reused their own towels substantially more often than guests receiving either of the other two messages. Given the way they conducted their study, it seems very likely that their result would hold true for other guests in other hotels.

A second reason not to draw the blanket conclusion that experiments are low in external validity is that they are often conducted to learn about psychological processes that are likely to operate in a variety of people and situations. Let us return to the experiment by Fredrickson and colleagues. They found that the women in their study, but not the men, performed worse on the math test when they were wearing swimsuits. They argued that this was due to women’s greater tendency to objectify themselves—to think about themselves from the perspective of an outside observer—which diverts their attention away from other tasks. They argued, furthermore, that this process of self-objectification and its effect on attention is likely to operate in a variety of women and situations—even if none of them ever finds herself taking a math test in her swimsuit.

Manipulation of the Independent Variable

Again, to manipulate an independent variable means to change its level systematically so that different groups of participants are exposed to different levels of that variable, or the same group of participants is exposed to different levels at different times. For example, to see whether expressive writing affects people’s health, a researcher might instruct some participants to write about traumatic experiences and others to write about neutral experiences. The different levels of the independent variable are referred to as conditions, and researchers often give the conditions short descriptive names to make it easy to talk and write about them. In this case, the conditions might be called the “traumatic condition” and the “neutral condition.”

Notice that the manipulation of an independent variable must involve the active intervention of the researcher. Comparing groups of people who differ on the independent variable before the study begins is not the same as manipulating that variable. For example, a researcher who compares the health of people who already keep a journal with the health of people who do not keep a journal has not manipulated this variable and therefore not conducted an experiment. This is important because groups that already differ in one way at the beginning of a study are likely to differ in other ways too. For example, people who choose to keep journals might also be more conscientious, more introverted, or less stressed than people who do not. Therefore, any observed difference between the two groups in terms of their health might have been caused by whether or not they keep a journal, or it might have been caused by any of the other differences between people who do and do not keep journals. Thus the active manipulation of the independent variable is crucial for eliminating the third-variable problem.

Of course, there are many situations in which the independent variable cannot be manipulated for practical or ethical reasons and therefore an experiment is not possible. For example, whether or not people have a significant early illness experience cannot be manipulated, making it impossible to do an experiment on the effect of early illness experiences on the development of hypochondriasis. This does not mean it is impossible to study the relationship between early illness experiences and hypochondriasis—only that it must be done using non-experimental approaches. We will discuss this in detail later in the book.

In many experiments, the independent variable is a construct that can only be manipulated indirectly. For example, a researcher might try to manipulate participants’ stress levels indirectly by telling some of them that they have five minutes to prepare a short speech that they will then have to give to an audience of other participants. In such situations, researchers often include a manipulation check in their procedure. A manipulation check is a separate measure of the construct the researcher is trying to manipulate. For example, researchers trying to manipulate participants’ stress levels might give them a paper-and-pencil stress questionnaire or take their blood pressure—perhaps right after the manipulation or at the end of the procedure—to verify that they successfully manipulated this variable.

Control of Extraneous Variables

An extraneous variable is anything that varies in the context of a study other than the independent and dependent variables. In an experiment on the effect of expressive writing on health, for example, extraneous variables would include participant variables (individual differences) such as their writing ability, their diet, and their shoe size. They would also include situation or task variables such as the time of day when participants write, whether they write by hand or on a computer, and the weather. Extraneous variables pose a problem because many of them are likely to have some effect on the dependent variable. For example, participants’ health will be affected by many things other than whether or not they engage in expressive writing. This can make it difficult to separate the effect of the independent variable from the effects of the extraneous variables, which is why it is important to control extraneous variables by holding them constant.

One way to control extraneous variables is to hold them constant. This can mean holding situation or task variables constant by testing all participants in the same location, giving them identical instructions, treating them in the same way, and so on. It can also mean holding participant variables constant. For example, many studies of language limit participants to right-handed people, who generally have their language areas isolated in their left cerebral hemispheres. Left-handed people are more likely to have their language areas isolated in their right cerebral hemispheres or distributed across both hemispheres, which can change the way they process language and thereby add noise to the data.

In principle, researchers can control extraneous variables by limiting participants to one very specific category of person, such as 20-year-old, straight, female, right-handed, sophomore psychology majors. The obvious downside to this approach is that it would lower the external validity of the study—in particular, the extent to which the results can be generalized beyond the people actually studied. For example, it might be unclear whether results obtained with a sample of younger straight women would apply to older gay men. In many situations, the advantages of a diverse sample outweigh the reduction in noise achieved by a homogeneous one.

Extraneous Variables as Confounding Variables

The second way that extraneous variables can make it difficult to detect the effect of the independent variable is by becoming confounding variables. A confounding variable is an extraneous variable that differs on average across levels of the independent variable. For example, in almost all experiments, participants’ intelligence quotients (IQs) will be an extraneous variable. But as long as there are participants with lower and higher IQs at each level of the independent variable so that the average IQ is roughly equal, then this variation is probably acceptable (and may even be desirable). What would be bad, however, would be for participants at one level of the independent variable to have substantially lower IQs on average and participants at another level to have substantially higher IQs on average. In this case, IQ would be a confounding variable.

To confound means to confuse, and this is exactly what confounding variables do. Because they differ across conditions—just like the independent variable—they provide an alternative explanation for any observed difference in the dependent variable. Consider the results of a hypothetical study in which participants in a positive mood condition scored higher on a memory task than participants in a negative mood condition. If IQ is a confounding variable—with participants in the positive mood condition having higher IQs on average than participants in the negative mood condition—then it is unclear whether it was the positive moods or the higher IQs that caused participants in the first condition to score higher. One way to avoid confounding variables is by holding extraneous variables constant. For example, one could prevent IQ from becoming a confounding variable by limiting participants only to those with IQs of exactly 100. But this approach is not always desirable for reasons we have already discussed. A second and much more general approach—random assignment to conditions—will be discussed in detail shortly.

Key Takeaways

· An experiment is a type of empirical study that features the manipulation of an independent variable, the measurement of a dependent variable, and control of extraneous variables.

· Studies are high in internal validity to the extent that the way they are conducted supports the conclusion that the independent variable caused any observed differences in the dependent variable. Experiments are generally high in internal validity because of the manipulation of the independent variable and control of extraneous variables.

· Studies are high in external validity to the extent that the result can be generalized to people and situations beyond those actually studied. Although experiments can seem “artificial”—and low in external validity—it is important to consider whether the psychological processes under study are likely to operate in other people and situations.

9.2 Experimental Design

Between-Subjects Experiments

In a between-subjects experiment, each participant is tested in only one condition. For example, a researcher with a sample of 100 college students might assign half of them to write about a traumatic event and the other half write about a neutral event. Or a researcher with a sample of 60 people with severe agoraphobia (fear of open spaces) might assign 20 of them to receive each of three different treatments for that disorder. It is essential in a between-subjects experiment that the researcher assign participants to conditions so that the different groups are, on average, highly similar to each other. Those in a trauma condition and a neutral condition, for example, should include a similar proportion of men and women, and they should have similar average intelligence quotients (IQs), similar average levels of motivation, similar average numbers of health problems, and so on. This is a matter of controlling these extraneous participant variables across conditions so that they do not become confounding variables.

Random Assignment

The primary way that researchers accomplish this kind of control of extraneous variables across conditions is called random assignment, which means using a random process to decide which participants are tested in which conditions. Do not confuse random assignment with random sampling. Random sampling is a method for selecting a sample from a population, and it is rarely used in psychological research. Random assignment is a method for assigning participants in a sample to the different conditions, and it is an important element of all experimental research in psychology and other fields too.

One problem with coin flipping and other strict procedures for random assignment is that they are likely to result in unequal sample sizes in the different conditions. Unequal sample sizes are generally not a serious problem, and you should never throw away data you have already collected to achieve equal sample sizes. However, for a fixed number of participants, it is statistically most efficient to divide them into equal-sized groups. It is standard practice, therefore, to use a kind of modified random assignment that keeps the number of participants in each group as similar as possible. One approach is block randomization. In block randomization, all the conditions occur once in the sequence before any of them is repeated. Then they all occur again before any of them is repeated again. Within each of these “blocks,” the conditions occur in a random order. Again, the sequence of conditions is usually generated before any participants are tested, and each new participant is assigned to the next condition in the sequence. Random assignment is not guaranteed to control all extraneous variables across conditions. It is always possible that just by chance, the participants in one condition might turn out to be substantially older, less tired, more motivated, or less depressed on average than the participants in another condition. However, there are some reasons that this is not a major concern. One is that random assignment works better than one might expect, especially for large samples. Another is that the inferential statistics that researchers use to decide whether a difference between groups reflects a difference in the population takes the “fallibility” of random assignment into account. Yet another reason is that even if random assignment does result in a confounding variable and therefore produces misleading results, this is likely to be detected when the experiment is replicated. The upshot is that random assignment to conditions—although not infallible in terms of controlling extraneous variables—is always considered a strength of a research design.

Treatment and Control Conditions

Between-subjects experiments are often used to determine whether a treatment works. In psychological research, a treatment is any intervention meant to change people’s behavior for the better. This includes psychotherapies and medical treatments for psychological disorders but also interventions designed to improve learning, promote conservation, reduce prejudice, and so on. To determine whether a treatment works, participants are randomly assigned to either a treatment condition, in which they receive the treatment, or a control condition, in which they do not receive the treatment. If participants in the treatment condition end up better off than participants in the control condition—for example, they are less depressed, learn faster, conserve more, express less prejudice—then the researcher can conclude that the treatment works. In research on the effectiveness of psychotherapies and medical treatments, this type of experiment is often called a randomized clinical trial.

There are different types of control conditions. In a no-treatment control condition, participants receive no treatment whatsoever. One problem with this approach, however, is the existence of placebo effects. A placebo is a simulated treatment that lacks any active ingredient or element that should make it effective, and a placebo effect is a positive effect of such a treatment. Many folk remedies that seem to work—such as eating chicken soup for a cold or placing soap under the bedsheets to stop nighttime leg cramps—are probably nothing more than placebos. Although placebo effects are not well understood, they are probably driven primarily by people’s expectations that they will improve. Having the expectation to improve can result in reduced stress, anxiety, and depression, which can alter perceptions and even improve immune system functioning (Price, Finniss, & Benedetti, 2008).

Placebo effects are interesting in their own right, but they also pose a serious problem for researchers who want to determine whether a treatment works. Fortunately, there are several solutions to this problem. One is to include a placebo control condition, in which participants receive a placebo that looks much like the treatment but lacks the active ingredient or element thought to be responsible for the treatment’s effectiveness. When participants in a treatment condition take a pill, for example, then those in a placebo control condition would take an identical-looking pill that lacks the active ingredient in the treatment (a “sugar pill”). In research on psychotherapy effectiveness, the placebo might involve going to a psychotherapist and talking in an unstructured way about one’s problems. The idea is that if participants in both the treatment and the placebo control groups expect to improve, then any improvement in the treatment group over and above that in the placebo control group must have been caused by the treatment and not by participants’ expectations.

Of course, the principle of informed consent requires that participants be told that they will be assigned to either a treatment or a placebo control condition—even though they cannot be told which until the experiment ends. In many cases the participants who had been in the control condition are then offered an opportunity to have the real treatment. An alternative approach is to use a waitlist control condition, in which participants are told that they will receive the treatment but must wait until the participants in the treatment condition have already received it. This allows researchers to compare participants who have received the treatment with participants who are not currently receiving it but who still expect to improve (eventually). A final solution to the problem of placebo effects is to leave out the control condition completely and compare any new treatment with the best available alternative treatment. For example, a new treatment for simple phobia could be compared with standard exposure therapy. Because participants in both conditions receive a treatment, their expectations about improvement should be similar. This approach also makes sense because once there is an effective treatment, the interesting question about a new treatment is not simply “Does it work?” but “Does it work better than what is already available?”

Within-Subjects Experiments

In a within-subjects experiment, each participant is tested under all conditions. Consider an experiment on the effect of a defendant’s physical attractiveness on judgments of his guilt. Again, in a between-subjects experiment, one group of participants would be shown an attractive defendant and asked to judge his guilt, and another group of participants would be shown an unattractive defendant and asked to judge his guilt. In a within-subjects experiment, however, the same group of participants would judge the guilt of both an attractive and an unattractive defendant.

Carryover Effects and Counterbalancing

The primary disadvantage of within-subjects designs is that they can result in carryover effects. A carryover effect is an effect of being tested in one condition on participants’ behavior in later conditions. One type of carryover effect is a practice effect, where participants perform a task better in later conditions because they have had a chance to practice it. Another type is a fatigue effect, where participants perform a task worse in later conditions because they become tired or bored. Being tested in one condition can also change how participants perceive stimuli or interpret their task in later conditions. This is called a context effect. For example, an average-looking defendant might be judged more harshly when participants have just judged an attractive defendant than when they have just judged an unattractive defendant. Within-subjects experiments also make it easier for participants to guess the hypothesis. For example, a participant who is asked to judge the guilt of an attractive defendant and then is asked to judge the guilt of an unattractive defendant is likely to guess that the hypothesis is that defendant attractiveness affects judgments of guilt. This could lead the participant to judge the unattractive defendant more harshly because he thinks this is what he is expected to do. Or it could make participants judge the two defendants similarly in an effort to be “fair.”

There is a solution to the problem of order effects, however, that can be used in many situations. It is counterbalancing, which means testing different participants in different orders. For example, some participants would be tested in the attractive defendant condition followed by the unattractive defendant condition, and others would be tested in the unattractive condition followed by the attractive condition. With three conditions, there would be six different orders (ABC, ACB, BAC, BCA, CAB, and CBA), so some participants would be tested in each of the six orders. With counterbalancing, participants are assigned to orders randomly, using the techniques we have already discussed. Thus random assignment plays an important role in within-subjects designs just as in between-subjects designs. Here, instead of randomly assigning to conditions, they are randomly assigned to different orders of conditions. In fact, it can safely be said that if a study does not involve random assignment in one form or another, it is not an experiment.

Simultaneous Within-Subjects Designs

Between-Subjects or Within-Subjects?

· Experiments can be conducted using either between-subjects or within-subjects designs. Deciding which to use in a particular situation requires careful consideration of the pros and cons of each approach.

· Random assignment to conditions in between-subjects experiments or to orders of conditions in within-subjects experiments is a fundamental element of experimental research. Its purpose is to control extraneous variables so that they do not become confounding variables.

· Experimental research on the effectiveness of a treatment requires both a treatment condition and a control condition, which can be a no-treatment control condition, a placebo control condition, or a waitlist control condition. Experimental treatments can also be compared with the best available alternative.

9.3 Conducting Experiments

The information presented so far in this chapter is enough to design a basic experiment. When it comes time to conduct that experiment, however, several additional practical issues arise. In this section, we consider some of these issues and how to deal with them. Much of this information applies to non-experimental studies as well as experimental ones.

Recruiting Participants

Of course, you should be thinking about how you will obtain your participants from the beginning of any research project. Unless you have access to people with schizophrenia or incarcerated juvenile offenders, for example, then there is no point designing a study that focuses on these populations. But even if you plan to use a convenience sample, you will have to recruit participants for your study.

There are several approaches to recruiting participants. One is to use participants from a formal subject pool—an established group of people who have agreed to be contacted about participating in research studies. For example, at many colleges and universities, there is a subject pool consisting of students enrolled in introductory psychology courses who must participate in a certain number of studies to meet a course requirement. Researchers post descriptions of their studies and students sign up to participate, usually via an online system. Participants who are not in subject pools can also be recruited by posting or publishing advertisements or making personal appeals to groups that represent the population of interest. For example, a researcher interested in studying older adults could arrange to speak at a meeting of the residents at a retirement community to explain the study and ask for volunteers.

The Volunteer Subject

Even if the participants in a study receive compensation in the form of course credit, a small amount of money, or a chance at being treated for a psychological problem, they are still essentially volunteers. This is worth considering because people who volunteer to participate in psychological research have been shown to differ in predictable ways from those who do not volunteer. Specifically, there is good evidence that on average, volunteers have the following characteristics compared with non-volunteers (Rosenthal Rosnow, 1976):

· They are more interested in the topic of the research.

· They are more educated.

· They have a greater need for approval.

· They have higher intelligence quotients (IQs).

· They are more sociable.

· They are higher in social class.

This can be an issue of external validity if there is reason to believe that participants with these characteristics are likely to behave differently than the general population. For example, in testing different methods of persuading people, a rational argument might work better on volunteers than it does on the general population because of their generally higher educational level and IQ.

In many field experiments, the task is not recruiting participants but selecting them. For example, researchers Nicolas Guéguen and Marie-Agnès de Gail conducted a field experiment on the effect of being smiled at on helping, in which the participants were shoppers at a supermarket. A confederate walking down a stairway gazed directly at a shopper walking up the stairway and either smiled or did not smile. Shortly afterward, the shopper encountered another confederate, who dropped some computer diskettes on the ground. The dependent variable was whether or not the shopper stopped to help pick up the diskettes (Guéguen & de Gail, 2003). Notice that these participants were not “recruited,” but the researchers still had to select them from among all the shoppers taking the stairs that day. It is extremely important that this kind of selection be done according to a well-defined set of rules that is established before the data collection begins and can be explained clearly afterward. In this case, with each trip down the stairs, the confederate was instructed to gaze at the first person he encountered who appeared to be between the ages of 20 and 50. Only if the person gazed back did he or she become a participant in the study. The point of having a well-defined selection rule is to avoid bias in the selection of participants. For example, if the confederate was free to choose which shoppers he would gaze at, he might choose friendly-looking shoppers when he was set to smile and unfriendly-looking ones when he was not set to smile. As we will see shortly, such biases can be entirely unintentional.

Standardizing the Procedure

It is surprisingly easy to introduce extraneous variables during the procedure. For example, the same experimenter might give clear instructions to one participant but vague instructions to another. Or one experimenter might greet participants warmly while another barely makes eye contact with them. To the extent that such variables affect participants’ behaviour, they add noise to the data and make the effect of the independent variable more difficult to detect. If they vary across conditions, they become confounding variables and provide alternative explanations for the results. For example, if participants in a treatment group are tested by a warm and friendly experimenter and participants in a control group are tested by a cold and unfriendly one, then what appears to be an effect of the treatment might actually be an effect of experimenter demeanour.

Experimenter Expectancy Effects

It is well known that whether research participants are male or female can affect the results of a study. But what about whether the experimenter is male or female? There is plenty of evidence that this matters too. Male and female experimenters have slightly different ways of interacting with their participants, and of course participants also respond differently to male and female experimenters (Rosenthal, 1976). For example, in a recent study on pain perception, participants immersed their hands in icy water for as long as they could (Ibolya, Brake, & Voss, 2004). Male participants tolerated the pain longer when the experimenter was a woman, and female participants tolerated it longer when the experimenter was a man.

Researcher Robert Rosenthal has spent much of his career showing that this kind of unintended variation in the procedure does, in fact, affect participants’ behaviour. Furthermore, one important source of such variation is the experimenter’s expectations about how participants “should” behave in the experiment. This is referred to as an experimenter expectancy effect (Rosenthal, 1976). For example, if an experimenter expects participants in a treatment group to perform better on a task than participants in a control group, then he or she might unintentionally give the treatment group participants clearer instructions or more encouragement or allow them more time to complete the task. In a striking example, Rosenthal and Kermit Fode had several students in a laboratory course in psychology train rats to run through a maze. Although the rats were genetically similar, some of the students were told that they were working with “maze-bright” rats that had been bred to be good learners, and other students were told that they were working with “maze-dull” rats that had been bred to be poor learners. Sure enough, over five days of training, the “maze-bright” rats made more correct responses, made the correct response more quickly, and improved more steadily than the “maze-dull” rats (Rosenthal & Fode, 1963). Clearly it had to have been the students’ expectations about how the rats would perform that made the difference. But how? Some clues come from data gathered at the end of the study, which showed that students who expected their rats to learn quickly felt more positively about their animals and reported behaving toward them in a more friendly manner (e.g., handling them more).

The way to minimize unintended variation in the procedure is to standardize it as much as possible so that it is carried out in the same way for all participants regardless of the condition they are in. Here are several ways to do this:

· Create a written protocol that specifies everything that the experimenters are to do and say from the time they greet participants to the time they dismiss them.

· Create standard instructions that participants read themselves or that are read to them word for word by the experimenter.

· Automate the rest of the procedure as much as possible by using software packages for this purpose or even simple computer slide shows.

· Anticipate participants’ questions and either raise and answer them in the instructions or develop standard answers for them.

· Train multiple experimenters on the protocol together and have them practice on each other.

· Be sure that each experimenter tests participants in all conditions.

Another good practice is to arrange for the experimenters to be “blind” to the research question or to the condition that each participant is tested in. The idea is to minimize experimenter expectancy effects by minimizing the experimenters’ expectations. For example, in a drug study in which each participant receives the drug or a placebo, it is often the case that neither the participants nor the experimenter who interacts with the participants know which condition he or she has been assigned to. Because both the participants and the experimenters are blind to the condition, this is referred to as a double-blind study. (A single-blind study is one in which the participant, but not the experimenter, is blind to the condition.) Of course, there are many times this is not possible. For example, if you are both the investigator and the only experimenter, it is not possible for you to remain blind to the research question. Also, in many studies the experimenter must know the condition because he or she must carry out the procedure in a different way in the different conditions.

Record Keeping

It is essential to keep good records when you conduct an experiment. As discussed earlier, it is typical for experimenters to generate a written sequence of conditions before the study begins and then to test each new participant in the next condition in the sequence. As you test them, it is a good idea to add to this list basic demographic information; the date, time, and place of testing; and the name of the experimenter who did the testing. It is also a good idea to have a place for the experimenter to write down comments about unusual occurrences (e.g., a confused or uncooperative participant) or questions that come up. This kind of information can be useful later if you decide to analyze sex differences or effects of different experimenters, or if a question arises about a particular participant or testing session.

It can also be useful to assign an identification number to each participant as you test them. Simply numbering them consecutively beginning with 1 is usually sufficient. This number can then also be written on any response sheets or questionnaires that participants generate, making it easier to keep them together.

Pilot Testing

It is always a good idea to conduct a pilot test of your experiment. A pilot test is a small-scale study conducted to make sure that a new procedure works as planned. In a pilot test, you can recruit participants formally (e.g., from an established participant pool) or you can recruit them informally from among family, friends, classmates, and so on. The number of participants can be small, but it should be enough to give you confidence that your procedure works as planned. There are several important questions that you can answer by conducting a pilot test:

· Do participants understand the instructions?

· What kind of misunderstandings do participants have, what kind of mistakes do they make, and what kind of questions do they ask?

· Do participants become bored or frustrated?

· Is an indirect manipulation effective? (You will need to include a manipulation check.)

· Can participants guess the research question or hypothesis?

· How long does the procedure take?

· Are computer programs or other automated procedures working properly?

· Are data being recorded correctly?

Of course, to answer some of these questions you will need to observe participants carefully during the procedure and talk with them about it afterward. Participants are often hesitant to criticize a study in front of the researcher, so be sure they understand that this is a pilot test and you are genuinely interested in feedback that will help you improve the procedure. If the procedure works as planned, then you can proceed with the actual study. If there are problems to be solved, you can solve them, pilot test the new procedure, and continue with this process until you are ready to proceed.

· There are several effective methods you can use to recruit research participants for your experiment, including through formal subject pools, advertisements, and personal appeals. Field experiments require well-defined participant selection procedures.

· It is important to standardize experimental procedures to minimize extraneous variables, including experimenter expectancy effects.

· It is important to conduct one or more small-scale pilot tests of an experiment to be sure that the procedure works as planned.

References from Chapter 9

Birnbaum, M. H. (1999). How to show that 9 221: Collect judgments in a between-subjects design. Psychological Methods, 4, 243–249.

Cialdini, R. (2005, April). Don’t throw in the towel: Use social influence research. APS Observer. Retrieved from http://www.psychologicalscience.org/observer/getArticle.cfm?id=1762 .

Fredrickson, B. L., Roberts, T.-A., Noll, S. M., Quinn, D. M., & Twenge, J. M. (1998). The swimsuit becomes you: Sex differences in self-objectification, restrained eating, and math performance. Journal of Personality and Social Psychology, 75, 269–284.

Guéguen, N., & de Gail, Marie-Agnès. (2003). The effect of smiling on helping behavior: Smiling and good Samaritan behavior. Communication Reports, 16, 133–140.

Ibolya, K., Brake, A., & Voss, U. (2004). The effect of experimenter characteristics on pain reports in women and men. Pain, 112, 142–147.

Moseley, J. B., O’Malley, K., Petersen, N. J., Menke, T. J., Brody, B. A., Kuykendall, D. H., … & Wray, N. P. (2002). A controlled trial of arthroscopic surgery for osteoarthritis of the knee. The New England Journal of Medicine, 347, 81–88.

Price, D. D., Finniss, D. G., & Benedetti, F. (2008). A comprehensive review of the placebo effect: Recent advances and current thought. Annual Review of Psychology, 59, 565–590.

Rosenthal, R. (1976). Experimenter effects in behavioral research (enlarged ed.). New York, NY: Wiley.

Rosenthal, R., & Fode, K. (1963). The effect of experimenter bias on performance of the albino rat. Behavioral Science, 8, 183-189.

Rosenthal, R., & Rosnow, R. L. (1976). The volunteer subject. New York, NY: Wiley.

Shapiro, A. K., & Shapiro, E. (1999). The powerful placebo: From ancient priest to modern physician. Baltimore, MD: Johns Hopkins University Press.

Stanovich, K. E. (2010). How to think straight about psychology (9th ed.). Boston, MA: Allyn Bacon.

Share This Book

Bipolar Disorder
Therapy Center
When To See a Therapist
Types of Therapy
Best Online Therapy
Best Couples Therapy
Best Family Therapy
Managing Stress
Sleep and Dreaming
Understanding Emotions
Self-Improvement
Healthy Relationships
Student Resources
Personality Types
Guided Meditations
Verywell Mind Insights
2024 Verywell Mind 25
Mental Health in the Classroom
Editorial Process
Meet Our Review Board
Crisis Support

The Random Selection Experiment Method

Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

Emily is a board-certified science editor who has worked with top digital publishing brands like Voices for Biodiversity, Study.com, GoodTherapy, Vox, and Verywell.

When researchers need to select a representative sample from a larger population, they often utilize a method known as random selection. In this selection process, each member of a group stands an equal chance of being chosen as a participant in the study.

Random Selection vs. Random Assignment

How does random selection differ from random assignment ? Random selection refers to how the sample is drawn from the population as a whole, whereas random assignment refers to how the participants are then assigned to either the experimental or control groups.

It is possible to have both random selection and random assignment in an experiment.

Imagine that you use random selection to draw 500 people from a population to participate in your study. You then use random assignment to assign 250 of your participants to a control group (the group that does not receive the treatment or independent variable) and you assign 250 of the participants to the experimental group (the group that receives the treatment or independent variable).

Why do researchers utilize random selection? The purpose is to increase the generalizability of the results.

By drawing a random sample from a larger population, the goal is that the sample will be representative of the larger group and less likely to be subject to bias.

Factors Involved

Imagine a researcher is selecting people to participate in a study. To pick participants, they may choose people using a technique that is the statistical equivalent of a coin toss.

They may begin by using random selection to pick geographic regions from which to draw participants. They may then use the same selection process to pick cities, neighborhoods, households, age ranges, and individual participants.

Another important thing to remember is that larger sample sizes tend to be more representative. Even random selection can lead to a biased or limited sample if the sample size is small.

When the sample size is small, an unusual participant can have an undue influence over the sample as a whole. Using a larger sample size tends to dilute the effects of unusual participants and prevent them from skewing the results.

Lin L. Bias caused by sampling error in meta-analysis with small sample sizes . PLoS ONE . 2018;13(9):e0204056. doi:10.1371/journal.pone.0204056

Elmes DG, Kantowitz BH, Roediger HL. Research Methods in Psychology. Belmont, CA: Wadsworth; 2012.

By Kendra Cherry, MSEd Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

We're sorry, but some features of Research Randomizer require JavaScript. If you cannot enable JavaScript, we suggest you use an alternative random number generator such as the one available at Random.org .

RESEARCH RANDOMIZER

Random sampling and random assignment made easy.

Research Randomizer is a free resource for researchers and students in need of a quick way to generate random numbers or assign participants to experimental conditions. This site can be used for a variety of purposes, including psychology experiments, medical trials, and survey research.

GENERATE NUMBERS

In some cases, you may wish to generate more than one set of numbers at a time (e.g., when randomly assigning people to experimental conditions in a "blocked" research design). If you wish to generate multiple sets of random numbers, simply enter the number of sets you want, and Research Randomizer will display all sets in the results.

Specify how many numbers you want Research Randomizer to generate in each set. For example, a request for 5 numbers might yield the following set of random numbers: 2, 17, 23, 42, 50.

Specify the lowest and highest value of the numbers you want to generate. For example, a range of 1 up to 50 would only generate random numbers between 1 and 50 (e.g., 2, 17, 23, 42, 50). Enter the lowest number you want in the "From" field and the highest number you want in the "To" field.

Selecting "Yes" means that any particular number will appear only once in a given set (e.g., 2, 17, 23, 42, 50). Selecting "No" means that numbers may repeat within a given set (e.g., 2, 17, 17, 42, 50). Please note: Numbers will remain unique only within a single set, not across multiple sets. If you request multiple sets, any particular number in Set 1 may still show up again in Set 2.

Sorting your numbers can be helpful if you are performing random sampling, but it is not desirable if you are performing random assignment. To learn more about the difference between random sampling and random assignment, please see the Research Randomizer Quick Tutorial.

Place Markers let you know where in the sequence a particular random number falls (by marking it with a small number immediately to the left). Examples: With Place Markers Off, your results will look something like this: Set #1: 2, 17, 23, 42, 50 Set #2: 5, 3, 42, 18, 20 This is the default layout Research Randomizer uses. With Place Markers Within, your results will look something like this: Set #1: p1=2, p2=17, p3=23, p4=42, p5=50 Set #2: p1=5, p2=3, p3=42, p4=18, p5=20 This layout allows you to know instantly that the number 23 is the third number in Set #1, whereas the number 18 is the fourth number in Set #2. Notice that with this option, the Place Markers begin again at p1 in each set. With Place Markers Across, your results will look something like this: Set #1: p1=2, p2=17, p3=23, p4=42, p5=50 Set #2: p6=5, p7=3, p8=42, p9=18, p10=20 This layout allows you to know that 23 is the third number in the sequence, and 18 is the ninth number over both sets. As discussed in the Quick Tutorial, this option is especially helpful for doing random assignment by blocks.

Please note: By using this service, you agree to abide by the SPN User Policy and to hold Research Randomizer and its staff harmless in the event that you experience a problem with the program or its results. Although every effort has been made to develop a useful means of generating random numbers, Research Randomizer and its staff do not guarantee the quality or randomness of numbers generated. Any use to which these numbers are put remains the sole responsibility of the user who generated them.

Note: By using Research Randomizer, you agree to its Terms of Service .

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Chapter 6: Experimental Research

Experimental Design

Learning Objectives

Explain the difference between between-subjects and within-subjects experiments, list some of the pros and cons of each approach, and decide which approach to use to answer a particular research question.
Define random assignment, distinguish it from random sampling, explain its purpose in experimental research, and use some simple strategies to implement it.
Define what a control condition is, explain its purpose in research on treatment effectiveness, and describe some alternative types of control conditions.
Define several types of carryover effect, give examples of each, and explain how counterbalancing helps to deal with them.

Between-Subjects Experiments

In a between-subjects experiment , each participant is tested in only one condition. For example, a researcher with a sample of 100 university students might assign half of them to write about a traumatic event and the other half write about a neutral event. Or a researcher with a sample of 60 people with severe agoraphobia (fear of open spaces) might assign 20 of them to receive each of three different treatments for that disorder. It is essential in a between-subjects experiment that the researcher assign participants to conditions so that the different groups are, on average, highly similar to each other. Those in a trauma condition and a neutral condition, for example, should include a similar proportion of men and women, and they should have similar average intelligence quotients (IQs), similar average levels of motivation, similar average numbers of health problems, and so on. This matching is a matter of controlling these extraneous participant variables across conditions so that they do not become confounding variables.

Random Assignment

Random assignment is not guaranteed to control all extraneous variables across conditions. It is always possible that just by chance, the participants in one condition might turn out to be substantially older, less tired, more motivated, or less depressed on average than the participants in another condition. However, there are some reasons that this possibility is not a major concern. One is that random assignment works better than one might expect, especially for large samples. Another is that the inferential statistics that researchers use to decide whether a difference between groups reflects a difference in the population takes the “fallibility” of random assignment into account. Yet another reason is that even if random assignment does result in a confounding variable and therefore produces misleading results, this confound is likely to be detected when the experiment is replicated. The upshot is that random assignment to conditions—although not infallible in terms of controlling extraneous variables—is always considered a strength of a research design.

Treatment and Control Conditions

Between-subjects experiments are often used to determine whether a treatment works. In psychological research, a treatment is any intervention meant to change people’s behaviour for the better. This intervention includes psychotherapies and medical treatments for psychological disorders but also interventions designed to improve learning, promote conservation, reduce prejudice, and so on. To determine whether a treatment works, participants are randomly assigned to either a treatment condition , in which they receive the treatment, or a control condition , in which they do not receive the treatment. If participants in the treatment condition end up better off than participants in the control condition—for example, they are less depressed, learn faster, conserve more, express less prejudice—then the researcher can conclude that the treatment works. In research on the effectiveness of psychotherapies and medical treatments, this type of experiment is often called a randomized clinical trial .

Placebo effects are interesting in their own right (see Note “The Powerful Placebo” ), but they also pose a serious problem for researchers who want to determine whether a treatment works. Figure 6.2 shows some hypothetical results in which participants in a treatment condition improved more on average than participants in a no-treatment control condition. If these conditions (the two leftmost bars in Figure 6.2 ) were the only conditions in this experiment, however, one could not conclude that the treatment worked. It could be instead that participants in the treatment group improved more because they expected to improve, while those in the no-treatment control condition did not.

Fortunately, there are several solutions to this problem. One is to include a placebo control condition , in which participants receive a placebo that looks much like the treatment but lacks the active ingredient or element thought to be responsible for the treatment’s effectiveness. When participants in a treatment condition take a pill, for example, then those in a placebo control condition would take an identical-looking pill that lacks the active ingredient in the treatment (a “sugar pill”). In research on psychotherapy effectiveness, the placebo might involve going to a psychotherapist and talking in an unstructured way about one’s problems. The idea is that if participants in both the treatment and the placebo control groups expect to improve, then any improvement in the treatment group over and above that in the placebo control group must have been caused by the treatment and not by participants’ expectations. This difference is what is shown by a comparison of the two outer bars in Figure 6.2 .

Of course, the principle of informed consent requires that participants be told that they will be assigned to either a treatment or a placebo control condition—even though they cannot be told which until the experiment ends. In many cases the participants who had been in the control condition are then offered an opportunity to have the real treatment. An alternative approach is to use a waitlist control condition , in which participants are told that they will receive the treatment but must wait until the participants in the treatment condition have already received it. This disclosure allows researchers to compare participants who have received the treatment with participants who are not currently receiving it but who still expect to improve (eventually). A final solution to the problem of placebo effects is to leave out the control condition completely and compare any new treatment with the best available alternative treatment. For example, a new treatment for simple phobia could be compared with standard exposure therapy. Because participants in both conditions receive a treatment, their expectations about improvement should be similar. This approach also makes sense because once there is an effective treatment, the interesting question about a new treatment is not simply “Does it work?” but “Does it work better than what is already available?

The Powerful Placebo

Many people are not surprised that placebos can have a positive effect on disorders that seem fundamentally psychological, including depression, anxiety, and insomnia. However, placebos can also have a positive effect on disorders that most people think of as fundamentally physiological. These include asthma, ulcers, and warts (Shapiro & Shapiro, 1999) [2] . There is even evidence that placebo surgery—also called “sham surgery”—can be as effective as actual surgery.

Medical researcher J. Bruce Moseley and his colleagues conducted a study on the effectiveness of two arthroscopic surgery procedures for osteoarthritis of the knee (Moseley et al., 2002) [3] . The control participants in this study were prepped for surgery, received a tranquilizer, and even received three small incisions in their knees. But they did not receive the actual arthroscopic surgical procedure. The surprising result was that all participants improved in terms of both knee pain and function, and the sham surgery group improved just as much as the treatment groups. According to the researchers, “This study provides strong evidence that arthroscopic lavage with or without débridement [the surgical procedures used] is not better than and appears to be equivalent to a placebo procedure in improving knee pain and self-reported function” (p. 85).

Within-Subjects Experiments

Carryover Effects and Counterbalancing

The primary disad vantage of within-subjects designs is that they can result in carryover effects. A carryover effect is an effect of being tested in one condition on participants’ behaviour in later conditions. One type of carryover effect is a practice effect , where participants perform a task better in later conditions because they have had a chance to practice it. Another type is a fatigue effect , where participants perform a task worse in later conditions because they become tired or bored. Being tested in one condition can also change how participants perceive stimuli or interpret their task in later conditions. This type of effect is called a context effect . For example, an average-looking defendant might be judged more harshly when participants have just judged an attractive defendant than when they have just judged an unattractive defendant. Within-subjects experiments also make it easier for participants to guess the hypothesis. For example, a participant who is asked to judge the guilt of an attractive defendant and then is asked to judge the guilt of an unattractive defendant is likely to guess that the hypothesis is that defendant attractiveness affects judgments of guilt. This knowledge could lead the participant to judge the unattractive defendant more harshly because he thinks this is what he is expected to do. Or it could make participants judge the two defendants similarly in an effort to be “fair.”

An efficient way of counterbalancing is through a Latin square design which randomizes through having equal rows and columns. For example, if you have four treatments, you must have four versions. Like a Sudoku puzzle, no treatment can repeat in a row or column. For four versions of four treatments, the Latin square design would look like:

When 9 is “larger” than 221

Researcher Michael Birnbaum has argued that the lack of context provided by between-subjects designs is often a bigger problem than the context effects created by within-subjects designs. To demonstrate this problem, he asked participants to rate two numbers on how large they were on a scale of 1-to-10 where 1 was “very very small” and 10 was “very very large”. One group of participants were asked to rate the number 9 and another group was asked to rate the number 221 (Birnbaum, 1999) [4] . Participants in this between-subjects design gave the number 9 a mean rating of 5.13 and the number 221 a mean rating of 3.10. In other words, they rated 9 as larger than 221! According to Birnbaum, this difference is because participants spontaneously compared 9 with other one-digit numbers (in which case it is relatively large) and compared 221 with other three-digit numbers (in which case it is relatively small) .

Simultaneous Within-Subjects Designs

Between-Subjects or Within-Subjects?

A good rule of thumb, then, is that if it is possible to conduct a within-subjects experiment (with proper counterbalancing) in the time that is available per participant—and you have no serious concerns about carryover effects—this design is probably the best option. If a within-subjects design would be difficult or impossible to carry out, then you should consider a between-subjects design instead. For example, if you were testing participants in a doctor’s waiting room or shoppers in line at a grocery store, you might not have enough time to test each participant in all conditions and therefore would opt for a between-subjects design. Or imagine you were trying to reduce people’s level of prejudice by having them interact with someone of another race. A within-subjects design with counterbalancing would require testing some participants in the treatment condition first and then in a control condition. But if the treatment works and reduces people’s level of prejudice, then they would no longer be suitable for testing in the control condition. This difficulty is true for many designs that involve a treatment meant to produce long-term change in participants’ behaviour (e.g., studies testing the effectiveness of psychotherapy). Clearly, a between-subjects design would be necessary here.

Key Takeaways

Experiments can be conducted using either between-subjects or within-subjects designs. Deciding which to use in a particular situation requires careful consideration of the pros and cons of each approach.
Random assignment to conditions in between-subjects experiments or to orders of conditions in within-subjects experiments is a fundamental element of experimental research. Its purpose is to control extraneous variables so that they do not become confounding variables.
Experimental research on the effectiveness of a treatment requires both a treatment condition and a control condition, which can be a no-treatment control condition, a placebo control condition, or a waitlist control condition. Experimental treatments can also be compared with the best available alternative.
You want to test the relative effectiveness of two training programs for running a marathon.
Using photographs of people as stimuli, you want to see if smiling people are perceived as more intelligent than people who are not smiling.
In a field experiment, you want to see if the way a panhandler is dressed (neatly vs. sloppily) affects whether or not passersby give him any money.
You want to see if concrete nouns (e.g., dog ) are recalled better than abstract nouns (e.g., truth ).
Discussion: Imagine that an experiment shows that participants who receive psychodynamic therapy for a dog phobia improve more than participants in a no-treatment control group. Explain a fundamental problem with this research design and at least two ways that it might be corrected.
Price, D. D., Finniss, D. G., & Benedetti, F. (2008). A comprehensive review of the placebo effect: Recent advances and current thought. Annual Review of Psychology, 59 , 565–590. ↵
Shapiro, A. K., & Shapiro, E. (1999). The powerful placebo: From ancient priest to modern physician . Baltimore, MD: Johns Hopkins University Press. ↵
Moseley, J. B., O’Malley, K., Petersen, N. J., Menke, T. J., Brody, B. A., Kuykendall, D. H., … Wray, N. P. (2002). A controlled trial of arthroscopic surgery for osteoarthritis of the knee. The New England Journal of Medicine, 347 , 81–88. ↵
Birnbaum, M.H. (1999). How to show that 9>221: Collect judgments in a between-subjects design. Psychological Methods, 4(3), 243-249. ↵

An experiment in which each participant is only tested in one condition.

A method of controlling extraneous variables across conditions by using a random process to decide which participants will be tested in the different conditions.

All the conditions of an experiment occur once in the sequence before any of them is repeated.

Any intervention meant to change people’s behaviour for the better.

A condition in a study where participants receive treatment.

A condition in a study that the other condition is compared to. This group does not receive the treatment or intervention that the other conditions do.

A type of experiment to research the effectiveness of psychotherapies and medical treatments.

A type of control condition in which participants receive no treatment.

A simulated treatment that lacks any active ingredient or element that should make it effective.

A positive effect of a treatment that lacks any active ingredient or element to make it effective.

Participants receive a placebo that looks like the treatment but lacks the active ingredient or element thought to be responsible for the treatment’s effectiveness.

Participants are told that they will receive the treatment but must wait until the participants in the treatment condition have already received it.

Each participant is tested under all conditions.

An effect of being tested in one condition on participants’ behaviour in later conditions.

Participants perform a task better in later conditions because they have had a chance to practice it.

Participants perform a task worse in later conditions because they become tired or bored.

Being tested in one condition can also change how participants perceive stimuli or interpret their task in later conditions.

Testing different participants in different orders.

Research Methods in Psychology - 2nd Canadian Edition Copyright © 2015 by Paul C. Price, Rajiv Jhangiani, & I-Chant A. Chiang is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

Search Menu
Advance articles
Author Interviews
Research Curations
Author Guidelines
Open Access
Submission Site
Why Submit?
About Journal of Consumer Research
Editorial Board
Advertising and Corporate Services
Self-Archiving Policy
Dispatch Dates
Journals on Oxford Academic
Books on Oxford Academic

Article Contents

Four methods for collecting data on digital advertising platforms, non-causal evidence of psychological process, causal evidence of psychological process, external validity in the 21st century, reporting data from advertising platforms, an invitation to advertising platforms, author notes.

< Previous

Leveraging Digital Advertising Platforms for Consumer Research

Article contents
Figures & tables
Supplementary Data

Michael Braun, Bart de Langhe, Stefano Puntoni, Eric M Schwartz, Leveraging Digital Advertising Platforms for Consumer Research, Journal of Consumer Research , Volume 51, Issue 1, June 2024, Pages 119–128, https://doi.org/10.1093/jcr/ucad058

Permissions Icon Permissions

Digital advertising platforms have emerged as a widely utilized data source in consumer research; yet, the interpretation of such data remains a source of confusion for many researchers. This article aims to address this issue by offering a comprehensive and accessible review of four prominent data collection methods proposed in the marketing literature: “informal studies,” “multiple-ad studies without holdout,” “single-ad studies with holdout,” and “multiple-ad studies with holdout.” By outlining the strengths and limitations of each method, we aim to enhance understanding regarding the inferences that can and cannot be drawn from the collected data. Furthermore, we present seven recommendations to effectively leverage these tools for programmatic consumer research. These recommendations provide guidance on how to use these tools to obtain causal and non-causal evidence for the effects of marketing interventions, and the associated psychological processes, in a digital environment regulated by targeting algorithms. We also give recommendations for how to describe the testing tools and the data they generate and urge platforms to be more transparent on how these tools work.

Randomized controlled trials (RCTs) are the cornerstone of academic consumer research. Nevertheless, there is a lingering concern that the typical “lab experiment,” whether conducted in-person with student participants or through online platforms like Prolific, may lack the necessary realism to generalize findings to real-world contexts ( Inman et al. 2018 ; Schmitt et al. 2022 ). In response, researchers have embraced the digital revolution and expanded the range of data reported in consumer behavior papers, diversifying their sources and methods ( Blanchard et al. 2022 ).

In the pursuit of enhanced realism, consumer researchers have found an ally in digital advertising platforms like Meta and Google. These platforms facilitate swift and cost-effective data collection from real consumers, making them one of the fastest-growing data sources in academic journals. For example, Umashankar et al. (2023) feature five studies, all conducted using online advertising platforms. A non-exhaustive list of articles featuring data collected from these advertising platforms includes Atalay, El Kihal, and Ellsaesser (2023) , Banker and Park (2020) , Castelo, Bos, and Lehmann (2019) , Chan and Ilicic (2019) , Gupta and Hagtvedt (2021) , Hardisty and Weber (2020) , Hodges, Estes, and Warren (2023) , Humphreys, Isaac, and Wang (2021) , Kupor and Laurin (2020) , Mookerjee, Cornil, and Hoegg (2021) , Ostinelli and Luna (2022) , Paharia (2020) , Paharia and Swaminathan (2019) , Rifkin, Du, and Cutright (2023) , To and Patrick (2021) , Wang, Lisjak, and Mandel (2023) , Winterich, Nenkov, and Gonzales (2019) , Yin, Jia, and Zheng (2021) , and Zhou, Du, and Cutright (2022) .

The large majority of RCTs in consumer research include at least one variable that is randomly manipulated between subjects. Random assignment of participants to different levels of the between-subjects variable allows researchers to disentangle the effects of manipulated variables from those of both observed and unobserved background variables, thus enabling researchers to draw causal conclusions about consumer psychology. Unfortunately, it is often a mystery how between-subjects variables are manipulated when researchers conduct studies on commercial advertising platforms, due to the proprietary underlying ad-targeting algorithms. The explanations and tutorials available often prove to be ambiguous, misleading, or challenging to decipher, even for experts. While data obtained from advertising platforms offer heightened realism, which has many benefits ( Morales, Amir, and Lee 2017 ), without a thorough understanding of the data-generating process, it is difficult to accurately evaluate evidence.

This article aims to review four methods proposed or utilized in the literature for conducting field studies on digital advertising platforms. We evaluate their ability to provide (1) non-causal evidence of psychological processes, (2) causal evidence of psychological processes, and (3) external validity. Drawing from this analysis, we develop seven recommendations to effectively harness digital advertising platforms in consumer research.

There are four primary methods for collecting field data from studies on digital advertising platforms. Table 1 summarizes the key features of each method and, in the last two rows, the type of inferences they afford (as discussed in the subsequent sections). Regardless of the chosen method, researchers must make three critical decisions. First, they need to establish an advertising objective, which could be raising awareness (e.g., impressions), driving engagement (e.g., clicks), or generating sales (e.g., conversions). Second, they must set budgets and define the audience (e.g., “all female iOS users 18 to 65, who the platform has determined have an interest in gardening and environmental causes”). Third, researchers have the opportunity to customize the creative elements of ads. This ability to customize creative elements is what makes these platforms potentially valuable tools for consumer researchers.

MAIN FEATURES OF THE FOUR CURRENTLY AVAILABLE DATA COLLECTION METHODS FROM STUDIES ON DIGITAL ADVERTISING PLATFORMS

In an “informal study,” researchers are not using the testing tools provided by platforms to categorize consumers into distinct groups. Instead, all consumers are eligible to view all ads, and the targeting algorithm determines which specific ad(s) will be presented to each consumer. Consequently, some consumers may not see any ads, while others may see a single ad, and some may even be exposed to multiple ads.

In a “multiple-ad study without holdout,” the platform divides the customer base into (at least) two distinct groups: group A and group B. Consumers in group A can see ad A but are not eligible to see ad B. But consumers in group A will only see ad A if selected by the targeting algorithm. On the other hand, consumers in group B may see ad B if selected by the targeting algorithm but are not eligible to see ad A. In industry, this method is commonly known as “A/B testing.” The implementation of this method by Meta has become the most widely adopted approach among consumer researchers to collect data on advertising platforms.

In a “single-ad study with holdout,” the platform divides the consumer base into two groups: group A and a holdout group. There is only one focal ad, which only consumers in group A can see. Since consumers in the holdout group are not eligible to see the focal ad, the dependent variable must be an action that an unexposed consumer could plausibly undertake. As unexposed users cannot click on ads they have never seen, researchers need to track alternative metrics such as visits to offsite landing pages, sales, downloads, or find ways to engage unexposed consumers, such as administering brand awareness and attitude surveys. Different platforms construct the holdout group in slightly different ways. For instance, Google’s “ghost ads” approach creates a holdout group that is a representative subset of consumers who were targeted with and just about to be exposed to the focal ad, but then were randomly not shown the ad. Therefore, the mix of users in the holdout is the same as the mix in group A that was exposed to the focal ad, supporting an analysis of the average treatment effect on the treated ( Johnson, Lewis, and Nubbemeyer 2017 ). Meta’s “conversion lift” approach, on the other hand, forms a holdout with the same mix as all exposed and unexposed consumers in group A, supporting an intent-to-treat analysis, where the platform “intent” is targeting a user to be exposed regardless of whether they are available for exposure ( Gordon et al. 2019 ; Gordon, Moakler, and Zettelmeyer 2023 ).

A “multiple-ad study with holdout” combines elements from the two previous methods, to form a set of two or more single-ad studies with holdout. If there are two focal ads, then only consumers in group A are eligible to see ad A and only consumers in group B are eligible to see ad B. However, due to algorithmic targeting, not all consumers in these groups will see a focal ad. In addition, there are two holdout groups, one for each ad. Researchers compare the incremental effect of ad A (as measured against the holdout group) with the incremental effect of ad B.

A prominent example of an informal study comes from Matz et al. (2017) . To examine if persuasive appeals are more effective at influencing behavior when they are tailored to individuals’ psychological characteristics, Matz et al. conducted a study on Facebook described as a “2 (Ad Personality: Introverted vs. Extraverted) × 2 (Audience Personality: Extraverted vs. Introverted) between-subjects, full-factorial design.” Ad Personality was implemented by creating five ads with introverted design elements and five ads with extroverted design elements, while Audience Personality was implemented by targeting people who had previously liked a set of topics considered to be introverted or extroverted.

Informal studies have two notable shortcomings. First, they do not ensure mutual exclusivity, as consumers can be assigned to multiple ads. In Matz et al.’s study, a consumer may be targeted as an “extrovert” due to, for example, their interest in the reggae band Rebelution, while also being targeted as an “introvert” because of their affinity for computers. The same consumer may also be exposed to multiple ads, some featuring introverted design elements and others with extraverted elements. This complicates the interpretation of results. Second, the assignment of consumers to ads is not randomized, which undermines the ability to establish causal relationships. Meta’s algorithm aims to identify users who are most likely to exhibit the desired campaign objective, such as making an online purchase, if the users were to be targeted with that ad. Consequently, the algorithm serves the ad to these specific users. When participant assignment to conditions is properly randomized, there should be no significant differences in demographic variables, such as age and gender, between the conditions. However, Eckles, Gordon, and Johnson (2018) conducted statistical analyses on the demographic data reported by Matz et al. (2017) and found variations in demographics across conditions. This suggests that participants in different conditions may also differ in ways that are unobservable to the researcher.

R1 : Avoid informal studies.

Consumer researchers often follow a “programmatic” approach to presenting evidence within a paper, with different studies designed to satisfy different objectives. To achieve academic breakthroughs in marketing, the combination of internally valid lab experiments and realistic field studies proves to be a powerful template. For instance, Rifkin et al. (2023) conducted a series of lab experiments complemented by a multiple-ad study without holdout on Facebook to demonstrate that consumers exhibit a preference for spontaneity over planned behavior in various entertainment contexts. While the lab experiments provide causal evidence of psychological process, the multiple-ad study without holdout provides non-causal evidence and can significantly influence readers’ confidence in the accuracy and relevance of the overall conclusions. Given the nature of social media, Facebook and other similar platforms serve as an informative setting to test the central hypothesis. Moreover, field studies conducted on advertising platforms offer several rhetorical advantages for consumer researchers. Due to their heightened realism, these studies often captivate readers’ interest and amplify impact.

Nevertheless, it is important to recognize that multiple-ad studies without holdout do not provide causal evidence. The splitting of users into conditions in these studies only determines the eligibility of users to view specific ads. Once the audience is split, the targeting algorithm decides for each ad separately as to which users will be exposed to each ad, leading to divergent or skewed delivery ( Ali et al. 2019 ; Johnson 2023 ; Braun and Schwartz 2023 ). Algorithmic targeting with divergent delivery causes each ad to be shown to a subset of eligible users, and the characteristics of the mix of users within each subset may differ in ways that are unobservable to the experimenter. Consequently, a multiple-ad study without holdout cannot separate the effects of ad creatives (A vs. B) from the effects of individual background variables.

R2 : For non-causal evidence, do a multiple-ad study without holdout.

While non-causal evidence should not be presented as causal evidence, researchers may still be inclined to erroneously interpret non-causal evidence as if it were causal evidence for various reasons. First, advertising platforms often provide limited background variables, such as age and gender, aggregated across all study conditions. For instance, Adida et al. (2022) utilized Facebook’s A/B testing tool and noted that “since Facebook ad data only provides sample population level aggregate statistics of gender, we are unable to conduct traditional covariate-based balance tests” ( Adida et al. 2022 , online appendix B). This limited information may increase the perceived similarity between conditions, potentially leading researchers to erroneously conclude that consumers were properly randomized to test their hypotheses.

Second, even when background variables are reported by condition, researchers may fail to test for imbalances in these variables. In a recent multiple-ad study without holdout on coronavirus disease 2019 communications, conducted using Facebook’s A/B test or split test methodology ( Banker and Park 2020 ), the main analysis involved a logistic regression of click-throughs on different message framings. The message framings included self-focused (“protect yourself”), close prosocial (“protect your loved ones”), and distant prosocial (“protect your community”). The results revealed a significant decrease in click-through rates when the message had a distant prosocial framing compared to the self-focused or close prosocial framings. However, upon analyzing the demographic data presented by Banker and Park (2020) in their web appendix, we found evidence suggesting that participants exposed to ads in the test were not randomly assigned to the conditions. Although the demographics appeared similar at first glance, statistical analyses indicated significant differences. For example, the percentage (and counts) of female consumers in each condition were 60% ( N = 5,286, self-focused), 62% (5,366, close prosocial), and 59% (4,806, distant prosocial), differences that are statistically significant (χ 2 = 22.10, p = .000016).

Third, researchers may observe covariate balance based on the reported observables. However, it is important to note that the absence of evidence is not evidence of absence. Demographic variables typically reported are merely the tip of the iceberg. Targeting algorithms rely on numerous background variables derived from the past behaviors of all consumers. Moreover, the platforms may not even store every variable used in each targeting decision ( Gordon et al. 2019 ). Although demonstrating covariate balance along the observables reported to the researcher may appear promising, the countless unreported background variables used in targeting likely exhibit significant imbalances across treatment groups. Therefore, the absence of imbalance on observables across ads is not evidence of balance on all relevant unobservables.

Fourth, researchers may opt for interaction studies as a potential solution. In their reply to Eckles et al. (2018) , Matz et al. (2018) argue that non-random assignment due to targeting algorithms is a minimal threat when “studies tested for interaction effects between target group and advertising content, not main effects” (p. 5256). We hold a different perspective. To illustrate this, let us consider a hypothetical multiple-ad study without holdout aimed at testing the hypothesis that matching the color of an ad to someone’s political identity decreases the effectiveness of moderate claims (e.g., “Politics is Compromise”). The study considers two target audiences: one consisting of individuals self-identified as “Democrat” and the other as “Republican.” Within each audience, individuals were exposed to either blue or red ads, following “a 2 (Political Identity: Democrat vs. Republican) × 2 (Ad Color: Blue vs. Red) between-subjects, full-factorial design.” The results revealed a cross-over interaction: the click-through rate for Democrats was lower when the ad color was blue compared to red, whereas the click-through rate for Republicans was lower when the ad color was red compared to blue. At first glance, one might be tempted to conclude that these findings support our hypothesis, but ad color was not randomly assigned to individuals within each political group, which introduced a confound to the comparison. Suppose moderate Democrats are more inclined to click on red ads compared to extreme Democrats, while moderate Republicans are more likely to click on blue ads compared to extreme Republicans. The targeting algorithm would learn from these patterns and subsequently increase the exposure of red ads to moderate Democrats and blue ads to moderate Republicans. Consequently, the composition of the individuals exposed to blue versus red ads would differ, leading to a situation where moderate Democrats and moderate Republicans are more likely to encounter ads that do not align with their political affiliation. Their inclination to click on ads with moderate claims would be influenced not by the color of the ad, but by their moderate disposition. This non-random assignment of individuals to ad color creates a confound that cannot be resolved through study designs focusing on interaction effects.

R3 : Avoid the misconception that divergent delivery can be circumvented.

Gordon et al. (2019 , 2023 ) demonstrate that single-ad studies with holdout can be a valuable tool for marketing practitioners measuring the return on investing in an ad. However, a single-ad study with holdout may not be as useful for the development of theories in consumer psychology. The reason behind this limitation lies in the characteristics of the holdout group, which is defined not by exposure to an alternative ad that allows for a comparison of creative elements with the treatment ad, but simply by the absence of the specific treatment ad.

To illustrate this point, let us consider a hypothetical scenario where a researcher aims to investigate the impact of celebrity endorsements on sales. They could conduct a single-ad study with holdout where the treatment group is exposed to an ad featuring a celebrity endorsement, while the holdout group is exposed to whatever ad they would have seen otherwise. However, the difference in behavior between the treatment group and the holdout group confounds two distinct effects: the effect of seeing any ad for that same product (as opposed to seeing whatever the next best ad in the auction would be) and the effect specifically attributed to the presence of a celebrity endorser in the ad (as opposed to seeing a different ad without a celebrity for that same product). Consequently, the researcher is unable to determine whether an ad with the same creative elements as the treatment ad, but featuring a non-celebrity endorser, would have yielded the same difference.

To advance theories of consumer psychology, researchers typically require a comparative analysis of at least two distinct ad executions that manipulate the specific construct under investigation. While advertising platforms offer the option to conduct multiple-ad studies with holdout, there are still significant challenges that persist, similar to those encountered in multiple-ad studies without holdout. Due to divergent delivery, not only are consumers exposed to ad A inherently distinct from those exposed to ad B, but also the two unexposed holdout groups differ from each other. The presence of holdout groups in a multiple-ad study does not mitigate the impact of divergent delivery. Consequently, conducting a multiple-ad study with holdout can potentially lead consumer researchers to draw erroneous causal conclusions about psychological processes.

Consider an illustrative scenario in a simplified world with a test of two ad executions, Prevention-focused and Promotion-focused. Within this world, consumers possess an unobservable background variable utilized by the platform’s targeting algorithm, dividing them into two categories: Tightwads and Spendthrifts ( Rick et al. 2008 ; for simplicity, assume an equal proportion of 50% for each). These two consumer types differ in their likelihood of purchasing a product. Table 2 presents two examples within this world.

TWO EXAMPLES ILLUSTRATE HOW OBSERVED LIFTS AMONG TARGETED CONSUMERS IN A MULTIPLE-AD STUDY WITH HOLDOUT CAN BE OPPOSITE TO UNOBSERVED TRUE LIFTS IN THE POPULATION OF INTEREST

In example 1, without any advertising, Tightwads have a lower baseline conversion probability (10% chance of buying) compared to Spendthrifts (30%). However, the Prevention-focused and Promotion-focused ads influence the conversion probabilities of Tightwads and Spendthrifts differently. The Prevention-focused ad increases the conversion probability of Tightwads from 10% to 20%, while it only raises the conversion probability of Spendthrifts from 30% to 32%. On the other hand, the Promotion-focused ad has a smaller effect on Tightwads, increasing their conversion probability from 10% to 11%, but significantly impacts Spendthrifts by increasing their conversion probability from 30% to 38%. Considering the entire population of Tightwads and Spendthrifts, the Prevention-focused ad demonstrates greater effectiveness (with a true additive lift of 6.0%) compared to the Promotion-focused ad (with a true additive lift of 4.5%).

Given this population, a researcher aims to determine which ad execution, Prevention-focused or Promotion-focused, has a larger effect on conversion and by how much. In an attempt to ascertain the difference between Prevention-focused and Promotion-focused communication, the researcher conducts a multiple-ad study with holdout. The obtained results reveal that the Prevention-focused ad exhibits an additive lift of 3.6%, while the Promotion-focused ad displays an additive lift of 6.6%. Surprisingly, from the test results, the researcher infers a negative A–B difference of −3.0% (indicating the Promotion-focused ad being more effective), which stands in direct opposition to the true A–B difference of +1.5% (Prevention-focused more effective) within the overall population. How is such a reversal possible?

While the population consists of 50% Spendthrifts, the targeting algorithm, unbeknownst to the researcher, predominantly delivers ads to Spendthrifts (80%) compared to Tightwads (20%), who possess a higher overall conversion probability ( table 2 , example 1). Due to this non-representative targeting, the researcher’s sample data for the multiple-ad study with holdout become heavily skewed toward Spendthrifts, leading to the erroneous conclusion that the Promotion-focused ad is more effective for the overall population. Example 1 combines two crucial conditions that mislead the researcher: (1) each consumer type exhibits a differential response to the ads, with Spendthrifts favoring the Promotion-focused ad over the Prevention-focused ad, while Tightwads display the opposite preference, and (2) the targeting of consumer types is non-representative of the overall population but remains consistent across both ads, resulting in both the prevention-focused and promotion-focused conditions consisting of 20% Tightwads and 80% Spendthrifts. Consequently, the ad that the researcher infers to possess a higher lift, based on a multiple-ad study with holdout, actually exhibits a lower lift within the population.

Example 2 in table 2 highlights another scenario where multiple-ad studies with holdout can potentially mislead researchers and lead to incorrect conclusions. In this case, the conditions differ from the previous example in two ways: (1) the Prevention-focused ad proves to be more effective than the Promotion-focused ad for both Tightwads and Spendthrifts and (2) the targeting algorithm not only selects a non-representative subset of consumers from the population but also exhibits divergent delivery. Divergent delivery makes the proportion of each consumer type in the targeted mix to vary between the two ads. Specifically, the Prevention-focused ad targets a mix of 75% Tightwads and 25% Spendthrifts, while the Promotion-focused ad targets a mix of 30% Tightwads and 70% Spendthrifts. Example 2 demonstrates how the researcher would erroneously conclude that the A–B difference is −1.4% when, in reality, it is +1.5%. This is an example of a “Simpson’s Reversal” ( Pearl 2014 ). Despite the Prevention-focused ad consistently exhibiting a higher lift than the Promotion-focused ad for both Tightwads and Spendthrifts, as well as any other combination of these consumer types, the results mislead the researcher to believe that the Promotion-focused ad has a higher lift. Braun and Schwartz (2023) have conducted an analysis on how non-representative targeting and divergent delivery can introduce biased causal inferences in terms of both the magnitude and direction of effects. They demonstrate that a “Simpson’s Reversal,” as illustrated in example 2, can occur under plausible conditions.

R4: For causal evidence of consumer psychology, avoid online advertising platforms.

The preceding pages may lead readers to believe that the “field experiments” on advertising platforms are in fact “flawed experiments” and incapable of offering any causal evidence relevant to consumer researchers. However, we hold a different perspective. Online consumer behavior is influenced and regulated by targeting algorithms. A consumer searching for information online will “google” it and get results based on Google search algorithm’s predictions of what the consumer might find most relevant. A TikTok user enjoys an endless stream of videos tailored to their preferences thanks to ByteDance’s algorithms. A consumer searching for products on platforms like Amazon relies on recommender systems to find relevant items. The ads we encounter online are specifically selected to align with our personal tastes and interests. Consumer researchers often equate “consumer behavior” with “consumer psychology.” However, with the increasing prominence of the Internet in consumers’ lives and the role of algorithmic selection within that environment, it is crucial to acknowledge the limitations of this equivalence.

To fully appreciate our argument, it is helpful to revisit a key point from a seminal article published in this journal. “The external validity of experimental findings,” according to Lynch (1982 , 228), “depends upon whether background factors (e.g., subject or setting factors) that are held relatively constant over the cells of an experimental design interact in nature with the manipulated variables. If they do so, the relationships observed in experimental data would not be observed if an attempt were made to replicate the study while holding these background factors constant at different levels.” A significant goal of consumer research is to provide actionable insights, and as marketing activities increasingly shift online, it becomes crucial to consider a prominent “background factor”: the presence of targeting algorithms.

R5: For causal evidence of the combined impact of ads and algorithms online, do a single-ad or multiple-ad study with holdout.

R6: Report all procedural decisions (e.g., dates, campaign objectives, researcher-selected variables to define audience, budgets, ad creatives) and all variables recorded by the platform (e.g., impressions, unique users reached, clicks, demographics, and if available, website visits and conversions) broken down by conditions.

A hypothetical example of such reporting is provided here: “We conducted a multiple-ad study without holdout using Meta Advertising from May 20, 2023, to May 27, 2023. Selecting from options provided by the platform, we defined an audience comprising individuals aged 18–50 of any gender within the Philadelphia media market area, with an interest in yoga. The campaign was optimized to ‘Get more website visitors.’ The results table presents impression counts, unique users reached, clicks, segmented by age and gender for each ad treatment.” To facilitate interpretation, researchers could also include screenshots of the testing tool and report a table in a web appendix. This level of transparency not only strengthens the evaluation of current research but also facilitates future investigations using different platforms or during different time periods.

Transparent reporting offers numerous benefits, but interpretation can still be hindered by misleading language. For example, Meta writes that “A/B testing helps ensure your audiences will be evenly split and statistically comparable.” Due to the colloquial use of “A/B test” as a synonym for “randomized controlled trial,” we initially assumed that this method would enable us to randomly assign users to different ad executions and thus provide insights about the causal impact of being exposed to ad A versus ad B. This assumption was further reinforced when we read the commentary by Eckles et al. (2018) . In their critique, they fault Matz et al. (2017) for conducting informal studies because “this process does not create a randomized experiment: users are not randomly assigned to different ads.” In the final paragraph, they mention that “since the Matz et al. studies were conducted, some ad platforms, including Facebook, have introduced tools for advertisers to conduct randomized experiments, which may aid future work” and they refer to “split testing” (now known as “A/B testing” at Meta) as one such tool.

For the modal consumer researcher, labels like “A/B test,” “randomized experiment,” and “field experiment” give the impression that users are randomly assigned to various levels of the between-subject variable, without algorithmic selection. However, the truth is that users are initially divided randomly into two groups for ads A and B, and subsequently, an algorithm selects a subset of users within each group to target with each ad. Unfortunately, these labels continue to be widespread, causing ongoing confusion among researchers. In a recent article, Rifkin et al. (2023) write that “US-based Facebook users were randomly assigned to this two-cell between-subjects design (content: spontaneous vs. planning) study. We used Facebook Ads Managers’ A/B test feature (which allows marketers to compare two or more messages or ‘Creatives’ while holding other factors constant) to conduct a field experiment that lasted four days (October 26, 2021, through October 29, 2021). We budgeted $50 per ad per day and garnered nearly 40,000 total impressions (N = 39,211). To hold everything but the condition content constant, our settings were as follows: A/B test on creative; objective: traffic, age: 18–65+; location: United States; language: English; all devices; optimization: link clicks; bid strategy: highest volume.” In light of R5 above, the authors should be commended for the meticulous level of detail they provide regarding the selected settings. However, the description inadvertently perpetuates a misleading perception of causality, for instance, due to the phrases “randomly assigned” and “hold other factors constant.”

R7: Avoid using labels that suggest random assignment of users to ad executions, such as “field experiment” or “A/B test.” Instead use more neutral labels like “field study,” “multiple-ad study with holdout,” or “multiple-ad study without holdout.”

Consumer researchers are increasingly outsourcing essential steps in the data collection process to commercial actors like Meta, Google, or Amazon. While this collaboration offers numerous benefits, there is a significant risk that researchers lack a sufficient understanding of the data-generating process underlying the observed effects. Several factors contribute to the complexity of this understanding. First, the digital advertising landscape is constantly evolving, making it challenging for researchers to keep pace with the latest developments. The dynamic nature of the industry requires continuous learning and adaptation to accurately interpret the data collected. Moreover, advertising platforms often fail to effectively communicate the features and capabilities of their data collection tools. This lack of transparency further complicates researchers’ efforts to comprehend the data-generating process. In some cases, platforms may even provide ambiguous or deceitful communication, driven by their vested interest in portraying observational effects of advertising as causal ( De Langhe and Puntoni 2021a , 2021b ). Commercial interest does not necessarily align with the objectives of researchers or advertisers seeking to learn about consumer psychology through randomized experimentation. For instance, the platform has little incentive to permit advertisers to compare results with and without divergent delivery enabled, since this would allow advertisers to compare their ads’ effects isolated from targeting and “reverse engineer” the value of the platform’s targeting algorithm. In light of these challenges, we emphasize the importance of clear and unambiguous explanations from advertising platforms regarding their data collection tools for running tests. It is vital for researchers that these platforms provide comprehensive insights into how data are collected and generated, enabling them to make informed decisions and interpretations. Furthermore, advertising platforms should prioritize the development of experimentation tools that facilitate genuine randomized comparisons of multiple ads. By enabling robust experimentation and comparison, researchers can obtain more causal evidence about the psychological processes that underlie consumer behavior.

Michael Braun ( [email protected] ) is an associate professor of marketing and Corrigan Research Professor at the Cox School of Business, Southern Methodist University, Dallas, TX, USA.

Bart de Langhe ( [email protected] ) is a professor of marketing at KU Leuven and Vlerick Business School, Belgium.

Stefano Puntoni ( [email protected] ) is the Sebastian S. Kresge Professor of Marketing at The Wharton School, University of Pennsylvania, Philadelphia, PA, USA.

Eric M. Schwartz ( [email protected] ) is an associate professor of marketing at the Ross School of Business, University of Michigan, Ann Arbor, MI, USA.

The authors would like to thank Johannes Boegershausen and Johnny Yu for their help. The authors contributed equally.

https://www.facebook.com/business/help/169589021376481

Adida Claire L. , Lo Adeline , Prather Lauren , Williamson Scott ( 2022 ), “ Refugees to the Rescue? Motivating Pro-Refugee Public Engagement during the COVID-19 Pandemic ,” Journal of Experimental Political Science , 9 ( 3 ), 281 – 95 .

Google Scholar

Ali Muhammad , Sapiezynski Piotr , Bogen Miranda , Korolova Aleksandra , Mislove Alan , Rieke Aaron ( 2019 ), “ Discrimination through Optimization: How Facebook’s Ad Delivery Can Lead to Skewed Outcomes ,” Proceedings of the ACM on Human-Computer Interaction , 3 (CSCW) , 1 – 30 .

Atalay A. Selin , El Kihal Siham , Ellsaesser Florian ( 2023 ), “ Creating Effective Marketing Messages through Moderately Surprising Syntax ,” Journal of Marketing , 87 ( 5 ), 755 – 75 .

Banker Sachin , Park Joowon ( 2020 ), “ Evaluating Prosocial COVID-19 Messaging Frames: Evidence from a Field Study on Facebook ,” Judgment and Decision Making , 15 ( 6 ), 1037 – 43 .

Blanchard Simon J. , Goldenberg Jacob , Pauwels Koen , Schweidel David A. ( 2022 ), “ Promoting Data Richness in Consumer Research: How to Develop and Evaluate Articles with Multiple Data Sources ,” Journal of Consumer Research , 49 ( 2 ), 359 – 72 .

Braun Michael , Schwartz Eric M. ( 2023 ), “Where A-B Testing Goes Wrong: What Online Experiments Cannot (and Can) Tell You About How Customers Respond to Advertising,” Working paper. SMU Cox School of Business Research Paper No. 21-10, https://ssrn.com/abstract=3896024 .

Castelo Noah , Bos Maarten W. , Lehmann Donald R. ( 2019 ), “ Task-Dependent Algorithm Aversion ,” Journal of Marketing Research , 56 ( 5 ), 809 – 25 .

Chan Eugene Y. , Ilicic Jasmina ( 2019 ), “ Political Ideology and Brand Attachment ,” International Journal of Research in Marketing , 36 ( 4 ), 630 – 46 .

De Langhe Bart , Puntoni Stefano ( 2021a ), “ Facebook’s Misleading Campaign against Apple’s Privacy Policy ,” Harvard Business Review , https://hbr.org/2021/02/facebooks-misleading-campaign-against-apples-privacy-policy .

De Langhe Bart , Puntoni Stefano ( 2021b ), “ Does Personalized Advertising Work as Well as Tech Companies Claim? ,” Harvard Business Review , https://hbr.org/2021/12/does-personalized-advertising-work-as-well-as-tech-companies-claim .

Eckles Dean , Gordon Brett R. , Johnson Garrett A. ( 2018 ), “ Field Studies of Psychologically Targeted Ads Face Threats to Internal Validity ,” Proceedings of the National Academy of Sciences of the United States of America , 115 ( 23 ), E5254 – E5255 .

Gordon Brett R. , Moakler Robert , Zettelmeyer Florian ( 2023 ), “ Close Enough? A Large-Scale Exploration of Non-Experimental Approaches to Advertising Measurement ,” Marketing Science , 42 ( 4 ), 768 – 93 .

Gordon Brett R. , Zettelmeyer Florian , Bhargava Neha , Chapsky Dan ( 2019 ), “ A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook ,” Marketing Science , 38 ( 2 ), 193 – 225 .

Gupta Tanvi , Hagtvedt Henrik ( 2021 ), “ Safe Together, Vulnerable apart: How Interstitial Space in Text Logos Impacts Brand Attitudes in Tight versus Loose Cultures ,” Journal of Consumer Research , 48 ( 3 ), 474 – 91 .

Hardisty David J. , Weber Elke U. ( 2020 ), “ Impatience and Savoring vs. Dread: Asymmetries in Anticipation Explain Consumer Time Preferences for Positive vs. Negative Events ,” Journal of Consumer Psychology , 30 ( 4 ), 598 – 613 .

Hodges Brady T. , Estes Zachary , Warren Caleb ( 2023 ), “ Intel Inside: The Linguistic Properties of Effective Slogans ,” Journal of Consumer Research . https://doi.org/10.1093/jcr/ucad034 .

Humphreys Ashlee , Isaac Mathew S. , Wang Rebecca Jen-Hui ( 2021 ), “ Construal Matching in Online Search: Applying Text Analysis to Illuminate the Consumer Decision Journey ,” Journal of Marketing Research , 58 ( 6 ), 1101 – 19 .

Inman J. Jeffrey , Campbell Margaret C. , Kirmani Amna , Price Linda L. ( 2018 ), “ Our Vision for the Journal of Consumer Research: It’s All about the Consumer ,” Journal of Consumer Research , 44 ( 5 ), 955 – 9 .

Johnson Garrett A. ( 2023 ), “ Inferno: A Guide to Field Experiments in Online Display Advertising ,” Journal of Economics & Management Strategy , 32 ( 3 ), 469 – 90 .

Johnson Garrett A. , Lewis Randall A. , Nubbemeyer Elmar I. ( 2017 ), “ Ghost Ads: Improving the Economics of Measuring Online Ad Effectiveness ,” Journal of Marketing Research , 54 ( 6 ), 867 – 84 .

Kupor Daniella , Laurin Kristin ( 2020 ), “ Probable Cause: The Influence of Prior Probabilities on Forecasts and Perceptions of Magnitude ,” Journal of Consumer Research , 46 ( 5 ), 833 – 52 .

Lynch John G. ( 1982 ), “ On the External Validity of Experiments in Consumer Research ,” Journal of Consumer Research , 9 ( 3 ), 225 – 39 .

Matz Sandra C. , Kosinski Michal , Nave Gideon , Stillwell David J. ( 2017 ), “ Psychological Targeting as an Effective Approach to Digital Mass Persuasion ,” Proceedings of the National Academy of Sciences of the United States of America , 114 ( 48 ), 12714 – 9 .

Matz S. C. , Kosinski M. , Nave G. , Stillwell D. J. ( 2018 ), “ Reply to Eckles et al.: Facebook's Optimization Algorithms Are Highly Unlikely to Explain the Effects of Psychological Targeting ,” Proceedings of the National Academy of Sciences of the United States of America , 115 ( 23 ), E5256 – E5257 .

Melumad Shiri , Hadi Rhonda , Hildebrand Christian , Ward Adrian F. ( 2020 ), “ Technology-Augmented Choice: How Digital Innovations Are Transforming Consumer Decision Processes ,” Customer Needs and Solutions , 7 ( 3-4 ), 90 – 101 .

Mookerjee Siddhanth , Cornil Yann , Hoegg JoAndrea ( 2021 ), “ From Waste to Taste: How “Ugly” Labels Can Increase Purchase of Unattractive Produce ,” Journal of Marketing , 85 ( 3 ), 62 – 77 .

Morales Andrea C. , Amir On , Lee Leonard ( 2017 ), “ Keeping It Real in Experimental Research—Understanding When, Where, and How to Enhance Realism and Measure Consumer Behavior ,” Journal of Consumer Research , 44 ( 2 ), 465 – 76 .

Orazi Davide C. , Johnston Allen C. ( 2020 ), “ Running Field Experiments Using Facebook Split Test ,” Journal of Business Research , 118 , 189 – 98 .

Ostinelli Massimiliano , Luna David ( 2022 ), “ Syntax and the Illusion of Fit: How Grammatical Subject Influences Persuasion ,” Journal of Consumer Research , 48 ( 5 ), 885 – 903 .

Paharia Neeru ( 2020 ), “ Who Receives Credit or Blame? The Effects of Made-to-Order Production on Responses to Unethical and Ethical Company Production Practices ,” Journal of Marketing , 84 ( 1 ), 88 – 104 .

Paharia Neeru , Swaminathan Vanitha ( 2019 ), “ Who Is Wary of User Design? The Role of Power-Distance Beliefs in Preference for User-Designed Products ,” Journal of Marketing , 83 ( 3 ), 91 – 107 .

Pearl Judea ( 2014 ), “ Comment: Understanding Simpson’s Paradox ,” The American Statistician , 68 ( 1 ), 8 – 13 .

Rick Scott I , Cynthia E. Cryder , and , George Loewenstein ( 2008 ), “ Tightwads and Spendthrifts ,” Journal of Consumer Research , 34 ( 6 ), 767 – 82 .

Rifkin Jacqueline R. , Du Katherine M. , Cutright Keisha M. ( 2023 ), “ The Preference for Spontaneity in Entertainment ,” Journal of Consumer Research , 50 ( 3 ), 597 – 616 .

Schmitt Bernd H. , Cotte June , Giesler Markus , Stephen Andrew T. , Wood Stacy ( 2022 ), “ Relevance—Reloaded and Recoded ,” Journal of Consumer Research , 48 ( 5 ), 753 – 5 .

To Rita Ngoc , Patrick Vanessa M. ( 2021 ), “ How the Eyes Connect to the Heart: The Influence of Eye Gaze Direction on Advertising Effectiveness ,” Journal of Consumer Research , 48 ( 1 ), 123 – 46 .

Umashankar Nita , Grewal Dhruv , Guha Abhijit , Bohling Timothy (2023), “ Testing Work–Life Theory in Marketing: Evidence from Field Experiments on Social Media ,” Journal of Marketing Research . https://doi.org/10.1177/00222437231152894 .

Wang Qin , Lisjak Monika , Mandel Naomi ( 2023 ), “ On the Flexibility of Self-Repair: How Holistic versus Analytic Thinking Style Impacts Fluid Compensatory Consumption ,” Journal of Consumer Psychology , 33 ( 1 ), 3 – 20 .

Winterich Karen Page , Nenkov Gergana Y. , Gonzales Gabriel E. ( 2019 ), “ Knowing What It Makes: How Product Transformation Salience Increases Recycling ,” Journal of Marketing , 83 ( 4 ), 21 – 37 .

Yin Yunlu , Jia Jayson S. , Zheng Wanyi ( 2021 ), “ The Effect of Slow Motion Video on Consumer Inference ,” Journal of Marketing Research , 58 ( 5 ), 1007 – 24 .

Zhou Lingrui , Du Katherine M. , Cutright Keisha M. ( 2022 ), “ Befriending the Enemy: The Effects of Observing Brand-to-Brand Praise on Consumer Evaluations and Choices ,” Journal of Marketing , 86 ( 4 ), 57 – 72 .

Email alerts

Citing articles via.

Recommend to your Library

Affiliations

Online ISSN 1537-5277
Print ISSN 0093-5301
About Oxford Academic
Publish journals with us
University press partners
What we publish
New features
Open access
Institutional account management
Rights and permissions
Get help with access
Accessibility
Advertising
Media enquiries
Oxford University Press
Oxford Languages
University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

Cookie settings
Cookie policy
Privacy policy
Legal notice

This Feature Is Available To Subscribers Only

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

COMMENTS

Random Assignment in Psychology: Definition & Examples
Olivia Guy-Evans, MSc. In psychology, random assignment refers to the practice of allocating participants to different experimental groups in a study in a completely unbiased way, ensuring each participant has an equal chance of being assigned to any group. In experimental research, random assignment, or random placement, organizes participants ...
Random Assignment in Experiments
Revised on June 22, 2023. In experimental research, random assignment is a way of placing participants from your sample into different treatment groups using randomization. With simple random assignment, every member of the sample has a known or equal chance of being placed in a control group or an experimental group.
The Definition of Random Assignment In Psychology
The Definition of Random Assignment According to Psychology. Random assignment refers to the use of chance procedures in psychology experiments to ensure that each participant has the same opportunity to be assigned to any given group in a study to eliminate any potential bias in the experiment at the outset. Participants are randomly assigned ...
Random Assignment in Psychology (Definition + 40 Examples)
Stepping back in time, we delve into the origins of random assignment, which finds its roots in the early 20th century. The pioneering mind behind this innovative technique was Sir Ronald A. Fisher, a British statistician and biologist.Fisher introduced the concept of random assignment in the 1920s, aiming to improve the quality and reliability of experimental research.
What Is Random Assignment in Psychology?
Random assignment in psychology involves each participant having an equal chance of being chosen for any of the groups, including the control and experimental groups. It helps control for potential confounding variables, reducing the likelihood of pre-existing differences between groups. This method enhances the internal validity of experiments ...
6.2 Experimental Design
Random assignment is a method for assigning participants in a sample to the different conditions, and it is an important element of all experimental research in psychology and other fields too. In its strictest sense, random assignment should meet two criteria. One is that each participant has an equal chance of being assigned to each condition ...
Random assignment
Random assignment or random placement is an experimental technique for assigning human participants or animal subjects to different groups in an experiment (e.g., a treatment group versus a control group) using randomization, such as by a chance procedure (e.g., flipping a coin) or a random number generator. This ensures that each participant or subject has an equal chance of being placed in ...
Random Assignment in Psychology
Random assignment in psychology produces a true experiment and is the only way cause and effect relationships can be investigated. ... Psychologists rely on random assignment to assign subjects to ...
Random Assignment in Experiments
Random assignment helps you separation causation from correlation and rule out confounding variables. As a critical component of the scientific method, experiments typically set up contrasts between a control group and one or more treatment groups. The idea is to determine whether the effect, which is the difference between a treatment group ...
5.3: Experimental Design
Random assignment is a method for assigning participants in a sample to the different conditions, and it is an important element of all experimental research in psychology and other fields too. In its strictest sense, random assignment should meet two criteria.
Random Assignment in Psychology (Intro for Students)
If there are two conditions in an experiment, then the simplest way to implement random assignment is to flip a coin for each participant. Heads means being assigned to the treatment and tails means being assigned to the control (or vice versa). 3. Rolling a die. Rolling a single die is another way to randomly assign participants.
Randomized Control Trial (RCT)
A randomized control trial (RCT) is a type of study design that involves randomly assigning participants to either an experimental group or a control group to measure the effectiveness of an intervention or treatment. Randomized Controlled Trials (RCTs) are considered the "gold standard" in medical and health research due to their rigorous ...
The Experimental Method
Also, a researcher must use random assignment to demonstrate cause and effect. Random assignment is when participants are assigned to each experimental group with an equal chance ⚖️ of being chosen. Don't confuse this with a random sample: each individual in the population has an equal chance of participating in an experiment.. Random assignment is randomly selecting people to be in an ...
PDF Random assignment: It's all in the cards
assessment for the teacher, students apply the concept of random assignment to a hypothetical experiment. Alignment with APA's Guidelines for the Undergraduate Psychology Major Goal 1 Knowledge base in psychology Goal 2 Scientific Inquiry and Critical Thinking Outcomes 1.1 Describe key concepts, principles, and overarching themes in psychology
Random Assignment in Experiments
Random sampling (also called probability sampling or random selection) is a way of selecting members of a population to be included in your study. In contrast, random assignment is a way of sorting the sample participants into control and experimental groups. While random sampling is used in many types of studies, random assignment is only used ...
Experimental Design: Variables, Groups, and Random Assignment
In this video, Dr. Kushner outlines how to conduct a psychology experiment. The experimental method is a powerful tool for psychologists because it is the on...
Chapter 9: Simple Experiments
A second and much more general approach—random assignment to conditions—will be discussed in detail shortly. Key Takeaways. · An experiment is a type of empirical study that features the manipulation of an independent variable, the measurement of a dependent variable, and control of extraneous variables.
How Random Selection Is Used For Research
Random selection refers to how the sample is drawn from the population as a whole, whereas random assignment refers to how the participants are then assigned to either the experimental or control groups. It is possible to have both random selection and random assignment in an experiment. Imagine that you use random selection to draw 500 people ...
Natural Experiments: Missed Opportunities for Causal Inference in
A further reason for the lack of usage of and awareness about natural experiments in psychology might be a difference in research culture across fields. ... In an example for a standard natural experiment with random assignment, Lindqvist et al. (2020) compared Swedish lottery players who won a large sum of money with matched controls ...
Research Randomizer
RANDOM SAMPLING AND. RANDOM ASSIGNMENT MADE EASY! Research Randomizer is a free resource for researchers and students in need of a quick way to generate random numbers or assign participants to experimental conditions. This site can be used for a variety of purposes, including psychology experiments, medical trials, and survey research.
Experimental Design
Random assignment is a method for assigning participants in a sample to the different conditions, and it is an important element of all experimental research in psychology and other fields too. In its strictest sense, random assignment should meet two criteria. One is that each participant has an equal chance of being assigned to each condition ...
Leveraging Digital Advertising Platforms for Consumer Research
Random assignment of participants to different levels of the between-subjects variable allows researchers to disentangle the effects of manipulated variables from those of both observed and unobserved background variables, thus enabling researchers to draw causal conclusions about consumer psychology.

Random Assignment in Psychology (Definition + 40 Examples)

History of Random Assignment

Early Studies Utilizing Random Assignment

Evolution of the Methodology

Principles of Random Assignment

Basic Principles of Random Assignment

Theoretical Foundation

Differentiating Random Assignment from Random Selection

Methodology of Random Assignment

Tools and Techniques Used

Ethical Considerations

Conclusion of Methodology

Benefits of Random Assignment in Psychological Research

Facilitating Causal Inferences

Ensuring Internal Validity

Enhancing Generalizability

Limitations of Random Assignment

Ethical Dilemmas

Generalizability Concerns

Practical and Real-World Limitations

Response to Critiques

Real-World Applications and Examples

Real-world Impact of Random Assignment

Health and Medicine

Workplace and Organizational Behavior

Environmental and Social Changes

Technology and Human Interaction

Related posts:

About The Author

Free Personality Test

Free Memory Test

Free IQ Test

Psychology Resources

Psychology Tests

What Is Random Assignment in Psychology?

How to Use Random Assignment

How Does Random Assignment Work?

Random Assignment vs. Random Selection

Examples of Random Assignment

Forming a Hypothesis

Obtaining Participants

Randomly Assign Participants to Groups

Conduct the Experiment

Collect and Analyze Data

Why Is Random Assignment Important in Psychology Research?

Key Points to Remember About Random Assignment

6.2 Experimental Design

Between-Subjects Experiments

Random Assignment

Treatment and Control Conditions

The Powerful Placebo

Within-Subjects Experiments

Carryover Effects and Counterbalancing

When 9 Is “Larger” Than 221

Simultaneous Within-Subjects Designs

Between-Subjects or Within-Subjects?

Key Takeaways

Margin Size

5.3: Experimental Design

Learning Objectives

Between-Subjects Experiments

Random Assignment

Matched Groups

Within-Subjects Experiments

Carryover Effects and Counterbalancing

When 9 Is “Larger” Than 221

Simultaneous Within-Subjects Designs

Between-Subjects or Within-Subjects?

Random Assignment in Psychology (Intro for Students)

Why Researchers Use Random Assignment

Fact File: Experiments 101

Methods of Random Assignment

1. Random number generator

2. Flipping a coin

3. Rolling a die

4. Condition names in a hat

Potential Confounding Effects

Limitations of the Random Assignment Procedure

Applications of Random Assignment

Leave a Comment Cancel Reply