research studies gone wrong

• The Magazine
• Stay Curious
• The Sciences
• Environment
• Planet Earth

5 Unethical Medical Experiments Brought Out of the Shadows of History

Prisoners and other vulnerable populations often bore the brunt of unethical medical experimentation..

Most people are aware of some of the heinous medical experiments of the past that violated human rights. Participation in these studies was either forced or coerced under false pretenses. Some of the most notorious examples include the experiments by the Nazis, the Tuskegee syphilis study, the Stanford Prison Experiment, and the CIA’s LSD studies.

But there are many other lesser-known experiments on vulnerable populations that have flown under the radar. Study subjects often didn’t — or couldn’t — give consent. Sometimes they were lured into participating with a promise of improved health or a small amount of compensation. Other times, details about the experiment were disclosed but the extent of risks involved weren’t.

This perhaps isn’t surprising, as doctors who conducted these experiments were representative of prevailing attitudes at the time of their work. But unfortunately, even after informed consent was introduced in the 1950s , disregard for the rights of certain populations continued. Some of these researchers’ work did result in scientific advances — but they came at the expense of harmful and painful procedures on unknowing subjects.

Here are five medical experiments of the past that you probably haven’t heard about. They illustrate just how far the ethical and legal guidepost, which emphasizes respect for human dignity above all else, has moved.

The Prison Doctor Who Did Testicular Transplants

From 1913 to 1951, eugenicist Leo Stanley was the chief surgeon at San Quentin State Prison, California’s oldest correctional institution. After performing vasectomies on prisoners, whom he recruited through promises of improved health and vigor, Stanley turned his attention to the emerging field of endocrinology, which involves the study of certain glands and the hormones they regulate. He believed the effects of aging and decreased hormones contributed to criminality, weak morality, and poor physical attributes. Transplanting the testicles of younger men into those who were older would restore masculinity, he thought.  

Stanley began by using the testicles of executed prisoners — but he ran into a supply shortage. He solved this by using the testicles of animals, including goats and deer. At first, he physically implanted the testicles directly into the inmates. But that had complications, so he switched to a new plan: He ground up the animal testicles into a paste, which he injected into prisoners’ abdomens. By the end of his time at San Quentin, Stanley did an estimated 10,000 testicular procedures .

The Oncologist Who Injected Cancer Cells Into Patients and Prisoners

During the 1950s and 1960s, Sloan-Kettering Institute oncologist Chester Southam conducted research to learn how people’s immune systems would react when exposed to cancer cells. In order to find out, he injected live HeLa cancer cells into patients, generally without their permission. When patient consent was given, details around the true nature of the experiment were often kept secret. Southam first experimented on terminally ill cancer patients, to whom he had easy access. The result of the injection was the growth of cancerous nodules , which led to metastasis in one person.

Next, Southam experimented on healthy subjects , which he felt would yield more accurate results. He recruited prisoners, and, perhaps not surprisingly, their healthier immune systems responded better than those of cancer patients. Eventually, Southam returned to infecting the sick and arranged to have patients at the Jewish Chronic Disease Hospital in Brooklyn, NY, injected with HeLa cells. But this time, there was resistance. Three doctors who were asked to participate in the experiment refused, resigned, and went public.

The scandalous newspaper headlines shocked the public, and legal proceedings were initiated against Southern. Some in the scientific and medical community condemned his experiments, while others supported him. Initially, Southam’s medical license was suspended for one year, but it was then reduced to a probation. His career continued to be illustrious, and he was subsequently elected president of the American Association for Cancer Research.

The Aptly Named ‘Monster Study’

Pioneering speech pathologist Wendell Johnson suffered from severe stuttering that began early in his childhood. His own experience motivated his focus on finding the cause, and hopefully a cure, for stuttering. He theorized that stuttering in children could be impacted by external factors, such as negative reinforcement. In 1939, under Johnson’s supervision, graduate student Mary Tudor conducted a stuttering experiment, using 22 children at an Iowa orphanage. Half received positive reinforcement. But the other half were ridiculed and criticized for their speech, whether or not they actually stuttered. This resulted in a worsening of speech issues for the children who were given negative feedback.

The study was never published due to the multitude of ethical violations. According to The Washington Post , Tudor was remorseful about the damage caused by the experiment and returned to the orphanage to help the children with their speech. Despite his ethical mistakes, the Wendell Johnson Speech and Hearing Clinic at the University of Iowa bears Johnson's name and is a nod to his contributions to the field.

The Dermatologist Who Used Prisoners As Guinea Pigs

One of the biggest breakthroughs in dermatology was the invention of Retin-A, a cream that can treat sun damage, wrinkles, and other skin conditions. Its success led to fortune and fame for co-inventor Albert Kligman, a dermatologist at the University of Pennsylvania . But Kligman is also known for his nefarious dermatology experiments on prisoners that began in 1951 and continued for around 20 years. He conducted his research on behalf of companies including DuPont and Johnson & Johnson.

Kligman’s work often left prisoners with pain and scars as he used them as study subjects in wound healing and exposed them to deodorants, foot powders, and more for chemical and cosmetic companies. Dow once enlisted Kligman to study the effects of dioxin, a chemical in Agent Orange, on 75 inmates at Pennsylvania's Holmesburg Prison. The prisoners were paid a small amount for their participation but were not told about the potential side effects.

In the University of Pennsylvania’s journal, Almanac , Kligman’s obituary focused on his medical advancements, awards, and philanthropy. There was no acknowledgement of his prison experiments. However, it did mention that as a “giant in the field,” he “also experienced his fair share of controversy.”

The Endocrinologist Who Irradiated Prisoners

When the Atomic Energy Commission wanted to know how radiation affected male reproductive function, they looked to endocrinologist Carl Heller . In a study involving Oregon State Penitentiary prisoners between 1963 and 1973, Heller designed a contraption that would radiate their testicles at varying amounts to see what effect it had, particularly on sperm production. The prisoners also were subjected to repeated biopsies and were required to undergo vasectomies once the experiments concluded.

Although study participants were paid, it raised ethical issues about the potential coercive nature of financial compensation to prison populations. The prisoners were informed about the risks of skin burns, but likely were not told about the possibility of significant pain, inflammation, and the small risk of testicular cancer.

• personal health
• behavior & society

Already a subscriber?

Register or Log In

Keep reading for as low as $1.99!

Sign up for our weekly science updates.

Save up to 40% off the cover price when you subscribe to Discover magazine.

BREAKING: Angela Alsobrooks defeats self-funder David Trone in the Maryland Democratic Senate primary, NBC News projects

Ugly past of U.S. human experiments uncovered

Shocking as it may seem, U.S. government doctors once thought it was fine to experiment on disabled people and prison inmates. Such experiments included giving hepatitis to mental patients in Connecticut, squirting a pandemic flu virus up the noses of prisoners in Maryland, and injecting cancer cells into chronically ill people at a New York hospital.

Much of this horrific history is 40 to 80 years old, but it is the backdrop for a meeting in Washington this week by a presidential bioethics commission. The meeting was triggered by the government's apology last fall for federal doctors infecting prisoners and mental patients in Guatemala with syphilis 65 years ago.

U.S. officials also acknowledged there had been dozens of similar experiments in the United States — studies that often involved making healthy people sick.

An exhaustive review by The Associated Press of medical journal reports and decades-old press clippings found more than 40 such studies. At best, these were a search for lifesaving treatments; at worst, some amounted to curiosity-satisfying experiments that hurt people but provided no useful results.

Inevitably, they will be compared to the well-known Tuskegee syphilis study. In that episode, U.S. health officials tracked 600 black men in Alabama who already had syphilis but didn't give them adequate treatment even after penicillin became available.

These studies were worse in at least one respect — they violated the concept of "first do no harm," a fundamental medical principle that stretches back centuries.

"When you give somebody a disease — even by the standards of their time — you really cross the key ethical norm of the profession," said Arthur Caplan, director of the University of Pennsylvania's Center for Bioethics.

Attitude similar to Nazi experiments Some of these studies, mostly from the 1940s to the '60s, apparently were never covered by news media. Others were reported at the time, but the focus was on the promise of enduring new cures, while glossing over how test subjects were treated.

Attitudes about medical research were different then. Infectious diseases killed many more people years ago, and doctors worked urgently to invent and test cures. Many prominent researchers felt it was legitimate to experiment on people who did not have full rights in society — people like prisoners, mental patients, poor blacks. It was an attitude in some ways similar to that of Nazi doctors experimenting on Jews.

"There was definitely a sense — that we don't have today — that sacrifice for the nation was important," said Laura Stark, a Wesleyan University assistant professor of science in society, who is writing a book about past federal medical experiments.

The AP review of past research found:

• A federally funded study begun in 1942 injected experimental flu vaccine in male patients at a state insane asylum in Ypsilanti, Mich., then exposed them to flu several months later. It was co-authored by Dr. Jonas Salk, who a decade later would become famous as inventor of the polio vaccine.

Some of the men weren't able to describe their symptoms, raising serious questions about how well they understood what was being done to them. One newspaper account mentioned the test subjects were "senile and debilitated." Then it quickly moved on to the promising results.

• In federally funded studies in the 1940s, noted researcher Dr. W. Paul Havens Jr. exposed men to hepatitis in a series of experiments, including one using patients from mental institutions in Middletown and Norwich, Conn. Havens, a World Health Organization expert on viral diseases, was one of the first scientists to differentiate types of hepatitis and their causes.

A search of various news archives found no mention of the mental patients study, which made eight healthy men ill but broke no new ground in understanding the disease.

• Researchers in the mid-1940s studied the transmission of a deadly stomach bug by having young men swallow unfiltered stool suspension. The study was conducted at the New York State Vocational Institution, a reformatory prison in West Coxsackie. The point was to see how well the disease spread that way as compared to spraying the germs and having test subjects breathe it. Swallowing it was a more effective way to spread the disease, the researchers concluded. The study doesn't explain if the men were rewarded for this awful task.
• A University of Minnesota study in the late 1940s injected 11 public service employee volunteers with malaria, then starved them for five days. Some were also subjected to hard labor, and those men lost an average of 14 pounds. They were treated for malarial fevers with quinine sulfate. One of the authors was Ancel Keys, a noted dietary scientist who developed K-rations for the military and the Mediterranean diet for the public. But a search of various news archives found no mention of the study.
• For a study in 1957, when the Asian flu pandemic was spreading, federal researchers sprayed the virus in the noses of 23 inmates at Patuxent prison in Jessup, Md., to compare their reactions to those of 32 virus-exposed inmates who had been given a new vaccine.
• Government researchers in the 1950s tried to infect about two dozen volunteering prison inmates with gonorrhea using two different methods in an experiment at a federal penitentiary in Atlanta. The bacteria was pumped directly into the urinary tract through the penis, according to their paper.

The men quickly developed the disease, but the researchers noted this method wasn't comparable to how men normally got infected — by having sex with an infected partner. The men were later treated with antibiotics. The study was published in the Journal of the American Medical Association, but there was no mention of it in various news archives.

Though people in the studies were usually described as volunteers, historians and ethicists have questioned how well these people understood what was to be done to them and why, or whether they were coerced.

Victims for science Prisoners have long been victimized for the sake of science. In 1915, the U.S. government's Dr. Joseph Goldberger — today remembered as a public health hero — recruited Mississippi inmates to go on special rations to prove his theory that the painful illness pellagra was caused by a dietary deficiency. (The men were offered pardons for their participation.)

But studies using prisoners were uncommon in the first few decades of the 20th century, and usually performed by researchers considered eccentric even by the standards of the day. One was Dr. L.L. Stanley, resident physician at San Quentin prison in California, who around 1920 attempted to treat older, "devitalized men" by implanting in them testicles from livestock and from recently executed convicts.

Newspapers wrote about Stanley's experiments, but the lack of outrage is striking.

"Enter San Quentin penitentiary in the role of the Fountain of Youth — an institution where the years are made to roll back for men of failing mentality and vitality and where the spring is restored to the step, wit to the brain, vigor to the muscles and ambition to the spirit. All this has been done, is being done ... by a surgeon with a scalpel," began one rosy report published in November 1919 in The Washington Post.

Around the time of World War II, prisoners were enlisted to help the war effort by taking part in studies that could help the troops. For example, a series of malaria studies at Stateville Penitentiary in Illinois and two other prisons was designed to test antimalarial drugs that could help soldiers fighting in the Pacific.

It was at about this time that prosecution of Nazi doctors in 1947 led to the "Nuremberg Code," a set of international rules to protect human test subjects. Many U.S. doctors essentially ignored them, arguing that they applied to Nazi atrocities — not to American medicine.

The late 1940s and 1950s saw huge growth in the U.S. pharmaceutical and health care industries, accompanied by a boom in prisoner experiments funded by both the government and corporations. By the 1960s, at least half the states allowed prisoners to be used as medical guinea pigs.

But two studies in the 1960s proved to be turning points in the public's attitude toward the way test subjects were treated.

The first came to light in 1963. Researchers injected cancer cells into 19 old and debilitated patients at a Jewish Chronic Disease Hospital in the New York borough of Brooklyn to see if their bodies would reject them.

The hospital director said the patients were not told they were being injected with cancer cells because there was no need — the cells were deemed harmless. But the experiment upset a lawyer named William Hyman who sat on the hospital's board of directors. The state investigated, and the hospital ultimately said any such experiments would require the patient's written consent.

At nearby Staten Island, from 1963 to 1966, a controversial medical study was conducted at the Willowbrook State School for children with mental retardation. The children were intentionally given hepatitis orally and by injection to see if they could then be cured with gamma globulin.

Those two studies — along with the Tuskegee experiment revealed in 1972 — proved to be a "holy trinity" that sparked extensive and critical media coverage and public disgust, said Susan Reverby, the Wellesley College historian who first discovered records of the syphilis study in Guatemala.

'My back is on fire!' By the early 1970s, even experiments involving prisoners were considered scandalous. In widely covered congressional hearings in 1973, pharmaceutical industry officials acknowledged they were using prisoners for testing because they were cheaper than chimpanzees.

Holmesburg Prison in Philadelphia made extensive use of inmates for medical experiments. Some of the victims are still around to talk about it. Edward "Yusef" Anthony, featured in a book about the studies, says he agreed to have a layer of skin peeled off his back, which was coated with searing chemicals to test a drug. He did that for money to buy cigarettes in prison.

"I said 'Oh my God, my back is on fire! Take this ... off me!'" Anthony said in an interview with The Associated Press, as he recalled the beginning of weeks of intense itching and agonizing pain.

The government responded with reforms. Among them: The U.S. Bureau of Prisons in the mid-1970s effectively excluded all research by drug companies and other outside agencies within federal prisons.

As the supply of prisoners and mental patients dried up, researchers looked to other countries.

It made sense. Clinical trials could be done more cheaply and with fewer rules. And it was easy to find patients who were taking no medication, a factor that can complicate tests of other drugs.

Additional sets of ethical guidelines have been enacted, and few believe that another Guatemala study could happen today. "It's not that we're out infecting anybody with things," Caplan said.

Still, in the last 15 years, two international studies sparked outrage.

One was likened to Tuskegee. U.S.-funded doctors failed to give the AIDS drug AZT to all the HIV-infected pregnant women in a study in Uganda even though it would have protected their newborns. U.S. health officials argued the study would answer questions about AZT's use in the developing world.

The other study, by Pfizer Inc., gave an antibiotic named Trovan to children with meningitis in Nigeria, although there were doubts about its effectiveness for that disease. Critics blamed the experiment for the deaths of 11 children and the disabling of scores of others. Pfizer settled a lawsuit with Nigerian officials for $75 million but admitted no wrongdoing.

Last year, the U.S. Department of Health and Human Services' inspector general reported that between 40 and 65 percent of clinical studies of federally regulated medical products were done in other countries in 2008, and that proportion probably has grown. The report also noted that U.S. regulators inspected fewer than 1 percent of foreign clinical trial sites.

Monitoring research is complicated, and rules that are too rigid could slow new drug development. But it's often hard to get information on international trials, sometimes because of missing records and a paucity of audits, said Dr. Kevin Schulman, a Duke University professor of medicine who has written on the ethics of international studies.

Syphilis study These issues were still being debated when, last October, the Guatemala study came to light.

In the 1946-48 study, American scientists infected prisoners and patients in a mental hospital in Guatemala with syphilis, apparently to test whether penicillin could prevent some sexually transmitted disease. The study came up with no useful information and was hidden for decades.

Story: U.S. apologizes for Guatemala syphilis experiments

The Guatemala study nauseated ethicists on multiple levels. Beyond infecting patients with a terrible illness, it was clear that people in the study did not understand what was being done to them or were not able to give their consent. Indeed, though it happened at a time when scientists were quick to publish research that showed frank disinterest in the rights of study participants, this study was buried in file drawers.

"It was unusually unethical, even at the time," said Stark, the Wesleyan researcher.

"When the president was briefed on the details of the Guatemalan episode, one of his first questions was whether this sort of thing could still happen today," said Rick Weiss, a spokesman for the White House Office of Science and Technology Policy.

That it occurred overseas was an opening for the Obama administration to have the bioethics panel seek a new evaluation of international medical studies. The president also asked the Institute of Medicine to further probe the Guatemala study, but the IOM relinquished the assignment in November, after reporting its own conflict of interest: In the 1940s, five members of one of the IOM's sister organizations played prominent roles in federal syphilis research and had links to the Guatemala study.

So the bioethics commission gets both tasks. To focus on federally funded international studies, the commission has formed an international panel of about a dozen experts in ethics, science and clinical research. Regarding the look at the Guatemala study, the commission has hired 15 staff investigators and is working with additional historians and other consulting experts.

The panel is to send a report to Obama by September. Any further steps would be up to the administration.

Some experts say that given such a tight deadline, it would be a surprise if the commission produced substantive new information about past studies. "They face a really tough challenge," Caplan said.

• Bipolar Disorder
• Therapy Center
• When To See a Therapist
• Types of Therapy
• Best Online Therapy
• Best Couples Therapy
• Best Family Therapy
• Managing Stress
• Sleep and Dreaming
• Understanding Emotions
• Self-Improvement
• Healthy Relationships
• Student Resources
• Personality Types
• Guided Meditations
• Verywell Mind Insights
• 2024 Verywell Mind 25
• Mental Health in the Classroom
• Editorial Process
• Meet Our Review Board
• Crisis Support

Controversial and Unethical Psychology Experiments

Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

Shereen Lehman, MS, is a healthcare journalist and fact checker. She has co-authored two books for the popular Dummies Series (as Shereen Jegtvig).

There have been a number of famous psychology experiments that are considered controversial, inhumane, unethical, and even downright cruel—here are five examples. Thanks to ethical codes and institutional review boards, most of these experiments could never be performed today.

At a Glance

Some of the most controversial and unethical experiments in psychology include Harlow's monkey experiments, Milgram's obedience experiments, Zimbardo's prison experiment, Watson's Little Albert experiment, and Seligman's learned helplessness experiment.

These and other controversial experiments led to the formation of rules and guidelines for performing ethical and humane research studies.

Harlow's Pit of Despair

Psychologist Harry Harlow performed a series of experiments in the 1960s designed to explore the powerful effects that love and attachment have on normal development. In these experiments, Harlow isolated young rhesus monkeys, depriving them of their mothers and keeping them from interacting with other monkeys.

The experiments were often shockingly cruel, and the results were just as devastating.

The Experiment

The infant monkeys in some experiments were separated from their real mothers and then raised by "wire" mothers. One of the surrogate mothers was made purely of wire.

While it provided food, it offered no softness or comfort. The other surrogate mother was made of wire and cloth, offering some degree of comfort to the infant monkeys.

Harlow found that while the monkeys would go to the wire mother for nourishment, they preferred the soft, cloth mother for comfort.

Some of Harlow's experiments involved isolating the young monkey in what he termed a "pit of despair." This was essentially an isolation chamber. Young monkeys were placed in the isolation chambers for as long as 10 weeks.

Other monkeys were isolated for as long as a year. Within just a few days, the infant monkeys would begin huddling in the corner of the chamber, remaining motionless.

The Results

Harlow's distressing research resulted in monkeys with severe emotional and social disturbances. They lacked social skills and were unable to play with other monkeys.

They were also incapable of normal sexual behavior, so Harlow devised yet another horrifying device, which he referred to as a "rape rack." The isolated monkeys were tied down in a mating position to be bred.

Not surprisingly, the isolated monkeys also ended up being incapable of taking care of their offspring, neglecting and abusing their young.

Harlow's experiments were finally halted in 1985 when the American Psychological Association passed rules regarding treating people and animals in research.

Milgram's Shocking Obedience Experiments

Isabelle Adam/Flickr/CC BY-NC-ND 2.0

If someone told you to deliver a painful, possibly fatal shock to another human being, would you do it? The vast majority of us would say that we absolutely would never do such a thing, but one controversial psychology experiment challenged this basic assumption.

Social psychologist Stanley Milgram conducted a series of experiments to explore the nature of obedience . Milgram's premise was that people would often go to great, sometimes dangerous, or even immoral, lengths to obey an authority figure.

The Experiments

In Milgram's experiment, subjects were ordered to deliver increasingly strong electrical shocks to another person. While the person in question was simply an actor who was pretending, the subjects themselves fully believed that the other person was actually being shocked.

The voltage levels started out at 30 volts and increased in 15-volt increments up to a maximum of 450 volts. The switches were also labeled with phrases including "slight shock," "medium shock," and "danger: severe shock." The maximum shock level was simply labeled with an ominous "XXX."​

The results of the experiment were nothing short of astonishing. Many participants were willing to deliver the maximum level of shock, even when the person pretending to be shocked was begging to be released or complaining of a heart condition.

Milgram's experiment revealed stunning information about the lengths that people are willing to go in order to obey, but it also caused considerable distress for the participants involved.

Zimbardo's Simulated Prison Experiment

 Darrin Klimek / Getty Images

Psychologist Philip Zimbardo went to high school with Stanley Milgram and had an interest in how situational variables contribute to social behavior.

In his famous and controversial experiment, he set up a mock prison in the basement of the psychology department at Stanford University. Participants were then randomly assigned to be either prisoners or guards. Zimbardo himself served as the prison warden.

The researchers attempted to make a realistic situation, even "arresting" the prisoners and bringing them into the mock prison. Prisoners were placed in uniforms, while the guards were told that they needed to maintain control of the prison without resorting to force or violence.

When the prisoners began to ignore orders, the guards began to utilize tactics that included humiliation and solitary confinement to punish and control the prisoners.

While the experiment was originally scheduled to last two full weeks it had to be halted after just six days. Why? Because the prison guards had started abusing their authority and were treating the prisoners cruelly. The prisoners, on the other hand, started to display signs of anxiety and emotional distress.

It wasn't until a graduate student (and Zimbardo's future wife) Christina Maslach visited the mock prison that it became clear that the situation was out of control and had gone too far. Maslach was appalled at what was going on and voiced her distress. Zimbardo then decided to call off the experiment.

Zimbardo later suggested that "although we ended the study a week earlier than planned, we did not end it soon enough."

Watson and Rayner's Little Albert Experiment

If you have ever taken an Introduction to Psychology class, then you are probably at least a little familiar with Little Albert.

Behaviorist John Watson  and his assistant Rosalie Rayner conditioned a boy to fear a white rat, and this fear even generalized to other white objects including stuffed toys and Watson's own beard.

Obviously, this type of experiment is considered very controversial today. Frightening an infant and purposely conditioning the child to be afraid is clearly unethical.

As the story goes, the boy and his mother moved away before Watson and Rayner could decondition the child, so many people have wondered if there might be a man out there with a mysterious phobia of furry white objects.

Controversy

Some researchers have suggested that the boy at the center of the study was actually a cognitively impaired boy who ended up dying of hydrocephalus when he was just six years old. If this is true, it makes Watson's study even more disturbing and controversial.

However, more recent evidence suggests that the real Little Albert was actually a boy named William Albert Barger.

Seligman's Look Into Learned Helplessness

During the late 1960s, psychologists Martin Seligman and Steven F. Maier conducted experiments that involved conditioning dogs to expect an electrical shock after hearing a tone. Seligman and Maier observed some unexpected results.

When initially placed in a shuttle box in which one side was electrified, the dogs would quickly jump over a low barrier to escape the shocks. Next, the dogs were strapped into a harness where the shocks were unavoidable.

After being conditioned to expect a shock that they could not escape, the dogs were once again placed in the shuttlebox. Instead of jumping over the low barrier to escape, the dogs made no efforts to escape the box.

Instead, they simply lay down, whined and whimpered. Since they had previously learned that no escape was possible, they made no effort to change their circumstances. The researchers called this behavior learned helplessness .

Seligman's work is considered controversial because of the mistreating the animals involved in the study.

Impact of Unethical Experiments in Psychology

Many of the psychology experiments performed in the past simply would not be possible today, thanks to ethical guidelines that direct how studies are performed and how participants are treated. While these controversial experiments are often disturbing, we can still learn some important things about human and animal behavior from their results.

Perhaps most importantly, some of these controversial experiments led directly to the formation of rules and guidelines for performing psychology studies.

Blum, Deborah.  Love at Goon Park: Harry Harlow and the science of affection . New York: Basic Books; 2011.

Sperry L.  Mental Health and Mental Disorders: an Encyclopedia of Conditions, Treatments, and Well-Being . Santa Barbara, CA: Greenwood, an imprint of ABC-CLIO, LLC; 2016.

Marcus S. Obedience to Authority An Experimental View. By Stanley Milgram. illustrated . New York: Harper &. The New York Times. 

Le Texier T. Debunking the Stanford Prison Experiment .  Am Psychol . 2019;74(7):823‐839. doi:10.1037/amp0000401

Fridlund AJ, Beck HP, Goldie WD, Irons G.  Little Albert: A neurologically impaired child .  Hist Psychol.  2012;15(4):302-27. doi:10.1037/a0026720

Powell RA, Digdon N, Harris B, Smithson C. Correcting the record on Watson, Rayner, and Little Albert: Albert Barger as "psychology's lost boy" .  Am Psychol . 2014;69(6):600‐611. doi:10.1037/a0036854

Seligman ME. Learned helplessness .  Annu Rev Med . 1972;23:407‐412. doi:10.1146/annurev.me.23.020172.002203

By Kendra Cherry, MSEd Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

Featured Topics

Featured series.

A series of random questions answered by Harvard experts.

Explore the Gazette

Read the latest.

Epic science inside a cubic millimeter of brain

What is ‘original scholarship’ in the age of AI?

Complex questions, innovative approaches

Harvard’s Gary King (pictured) is one in a cohort of researchers rebutting a consortium of 270 scientists known as the Open Science Collaboration, which made worldwide headlines last year when it claimed that it could not replicate the results of 100 published psychology studies.

File photo by Stephanie Mitchell/Harvard Staff Photographer

Study that undercut psych research got it wrong

Peter Reuell

Harvard Staff Writer

Widely reported analysis that said much research couldn’t be reproduced is riddled with its own replication errors, researchers say

According to two Harvard professors and their collaborators, a widely reported study released last year that said more than half of all psychology studies cannot be replicated is itself wrong.

In an attempt to determine the “replicability” of psychological science, a consortium of 270 scientists known as the Open Science Collaboration (OSC) tried to reproduce the results of 100 published studies. More than half of them failed, creating sensational headlines worldwide about the “replication crisis” in psychology.

But an in-depth examination of the data by Daniel Gilbert , the Edgar Pierce Professor of Psychology at Harvard, Gary King , the Albert J. Weatherhead III University Professor at Harvard, Stephen Pettigrew, a Ph.D. student in the Department of Government at Harvard, and Timothy Wilson, the Sherrell J. Aston Professor of Psychology at the University of Virginia, has revealed that the OSC made some serious mistakes that make its pessimistic conclusion completely unwarranted.

The methods of many of the replication studies turn out to be remarkably different from the originals and, according to the four researchers, these “infidelities” had two important consequences.

First, the methods introduced statistical error into the data, which led the OSC to significantly underestimate how many of their replications should have failed by chance alone. When this error is taken into account, the number of failures in their data is no greater than one would expect if all 100 of the original findings had been true.

Second, Gilbert, King, Pettigrew, and Wilson discovered that the low-fidelity studies were four times more likely to fail than were the high-fidelity studies, suggesting that when replicators strayed from the original methods of conducting research, they caused their own studies to fail.

Finally, the OSC used a “low-powered” design. When the four researchers applied this design to a published data set that was known to have a high replication rate, it too showed a low replication rate, suggesting that the OSC’s design was destined from the start to underestimate the replicability of psychological science.

Individually, Gilbert and King said, each of these problems would be enough to cast doubt on the conclusion that most people have drawn from this study, but taken together, they completely repudiate it. The flaws are described in a commentary to be published Friday in Science.

Like most scientists who read the OSC’s article when it appeared, Gilbert, King, Pettigrew, and Wilson were shocked and chagrined. But when they began to scrutinize the methods and reanalyze the raw data, they immediately noticed problems, which started with how the replicators had selected the 100 original studies.

“If you want to estimate a parameter of a population,” said King, “then you either have to randomly sample from that population or make statistical corrections for the fact that you didn’t. The OSC did neither.”

‘Arbitrary list of sampling rules’

“What they did,” added Gilbert, “is create an idiosyncratic, arbitrary list of sampling rules that excluded the majority of psychology’s subfields from the sample, that excluded entire classes of studies whose methods are probably among the best in science from the sample, and so on. Then they proceeded to violate all of their own rules.

“Worse yet, they actually allowed some replicators to have a choice about which studies they would try to replicate. If they had used these same methods to sample people instead of studies, no reputable scientific journal would have published their findings. So the first thing we realized was that no matter what they found — good news or bad news — they never had any chance of estimating the reproducibility of psychological science, which is what the very title of their paper claims they did.”

“And that was just the beginning,” King said. “If you are going to replicate 100 studies, some will fail by chance alone. That’s basic sampling theory. So you have to use statistics to estimate how many of the studies are expected to fail by chance alone because otherwise the number that actually do fail is meaningless.”

According to King, the OSC did this, but made a critical error.

“When they did their calculations, they failed to consider the fact that their replication studies were not just new samples from the same population. They were often quite different from the originals in many ways, and those differences are a source of statistical error. So we did the calculation the right way and then applied it to their data. And guess what? The number of failures they observed was just about what you should expect to observe by chance alone — even if all 100 of the original findings were true. The failure of the replication studies to match the original studies was a failure of the replications, not of the originals.”

Gilbert noted that most people assume that a replication is a “replica”’ of the original study.

“Readers surely assumed that if a group of scientists did 100 replications, then they must have used the same methods to study the same populations. In this case, that assumption would be quite wrong. Replications always vary from originals in minor ways, of course. But if you read the reports carefully, as we did, you discover that many of the replication studies differed in truly astounding ways — ways that make it hard to understand how they could even be called replications.”

As an example, Gilbert described an original study that involved showing white students at Stanford University a video of four other Stanford students discussing admissions policies at their university. Three of those talking were white and one was black. During the discussion, a white student made offensive comments about affirmative action, and the researchers found that the observers looked significantly longer at the black student when they believed he could hear other comments than when they believed he could not.

“So how did they do the replication? With students at the University of Amsterdam!” Gilbert said. “They had Dutch students watch a video of Stanford students, speaking in English, about affirmative action policies at a university more than 5,000 miles away.”

In other words, unlike the participants in the original study, participants in the replication study watched students at a foreign university speaking in a foreign language about an issue of no relevance to them.

But according to Gilbert, that was not the most troubling part of the methodology.

Gilbert: ‘No one involved in this study was trying to deceive anyone. They just made mistakes, as scientists sometimes do.’

“If you dive deep into the data, you discover something else,” Gilbert said. “The replicators realized that doing this study in the Netherlands might have been a problem, so they wisely decided to run another version of it in the U.S. And when they did, they basically replicated the original result. And yet, when the OSC estimated the reproducibility of psychological science, they excluded the successful replication and included only the one from the University of Amsterdam that failed. So the public hears that ‘Yet another psychology study doesn’t replicate’ instead of ‘Yet another psychology study replicates just fine if you do it right, and not if you do it wrong,’ which isn’t a very exciting headline. Some of the replications were quite faithful to the originals, but anyone who carefully reads all the replication reports will find many more examples like this one.”

‘They introduce additional error’

“These infidelities were a problem for another reason,” King added, “namely, that they introduce additional error into the data set. That error can be calculated, and when we do, it turns out that the number of replication studies that actually failed is about what we should expect if every single one of the original findings had been true. Now, one could argue about how best to make this calculation, but the fact is that OSC didn’t make it at all. They simply ignored this potent source of error, and that caused them to draw the wrong conclusions from their data. That doesn’t mean that all 100 studies were true, of course, but it does mean that this article provides no evidence to the contrary.”

“So we now know that the infidelities created statistical noise,” said Gilbert, “but was that all they did? Or were the infidelities of a certain kind? In other words, did they just tend to change the original result, or did they tend to change it in a particular way?”

“To find out,” said King, “we needed a measure of how faithful each of the 100 replications was. Luckily, the OSC supplied it.”

Before each replication began, the OSC asked the original authors to examine the planned replication study and say whether they would endorse it as a faithful replication of their work, and about 70 percent did so.

“We used this as a rough index of fidelity, and when we did, we discovered something important: The low-fidelity replications were an astonishing four times more likely to fail,” King said. “What that suggests is that the infidelities did not just create random statistical noise — they actually biased the studies toward failure.”

In their “technical comment,” Gilbert, King, Pettigrew, and Wilson also note that the OSC used a “low-powered” design. They replicated each of the 100 studies once, using roughly the number of subjects used in the original studies. But according to King, this method artificially depresses the replication rate.

“To show how this happens, we took another published article that had examined the replicability of a group of classic psychology studies,” said King. “The authors of that paper had used a very high-powered design — they replicated each study with more than 30 times the original number of participants — and that high-powered design produced a very high replication rate. So we asked a simple question: What would have happened if these authors had used the low-powered design that was used by the OSC? The answer is that the replication rate would have been even lower than the replication rate found by the OSC.”

Despite uncovering serious problems with the landmark study, Gilbert and King emphasized that their critique does not suggest wrongdoing and is simply part of the normal process of scientific inquiry.

“Let’s be clear, Gilbert said. “No one involved in this study was trying to deceive anyone. They just made mistakes, as scientists sometimes do. Many of the OSC members are our friends, and the corresponding author, Brian Nosek, is actually a good friend who was both forthcoming and helpful to us as we wrote our critique. In fact, Brian is the one who suggested one of the methods we used for correcting the OSC’s error calculations. So this is not a personal attack, this is a scientific critique.

“We all care about the same things: doing science well and finding out what’s true. We were glad to see that in their response to our comment, the OSC quibbled about a number of minor issues but conceded the major one, which is that their paper does not provide evidence for the pessimistic conclusions that most people have drawn from it.”

“I think the big takeaway point here is that meta-science must obey the rules of science,” King said. “All the rules about sampling and calculating error and keeping experimenters blind to the hypothesis — all of those rules must apply whether you are studying people or studying the replicability of a science. Meta-science does not get a pass. It is not exempt. And those doing meta-science are not above the fray. They are part of the scientific process. If you violate the basic rules of science, you get the wrong answer, and that’s what happened here.”

“This [OSC] paper has had extraordinary impact,” Gilbert said. “It was Science magazine’s No. 3 ‘ Breakthrough of the Year’ across all fields of science. It led to changes in policy at many scientific journals, changes in priorities at funding agencies, and it seriously undermined public perceptions of psychology. So it is not enough now, in the sober light of retrospect, to say that mistakes were made. These mistakes had very serious repercussions. We hope the OSC will now work as hard to correct the public misperceptions of their findings as they did to produce the findings themselves.”

The OSC’s reply to “technical comments” by Gilbert and others, and Gilbert and others’ response to that reply, can be found here .

Share this article

You might like.

Researchers publish largest-ever dataset of neural connections

Symposium considers how technology is changing academia

Seven projects awarded Star-Friedman Challenge grants

Excited about new diet drug? This procedure seems better choice.

Study finds minimally invasive treatment more cost-effective over time, brings greater weight loss

How far has COVID set back students?

An economist, a policy expert, and a teacher explain why learning losses are worse than many parents realize

Subscribe or renew today

Every print subscription comes with full digital access

Science News

12 reasons research goes wrong.

FIXING THE NUMBERS   Massaging data, small sample sizes and other issues can affect the statistical analyses of studies and distort the results, and that's not all that can go wrong.

Justine Hirshfeld/ Science News

Share this:

By Tina Hesman Saey

January 13, 2015 at 2:23 pm

For more on reproducibility in science, see SN’s feature “ Is redoing scientific research the best way to find truth? “

Barriers to research replication are based largely in a scientific culture that pits researchers against each other in competition for scarce resources. Any or all of the factors below, plus others, may combine to skew results.

Pressure to publish

Research funds are tighter than ever and good positions are hard to come by. To get grants and jobs, scientists need to publish, preferably in big-name journals. That pressure may lead researchers to publish many low-quality studies instead of aiming for a smaller number of well-done studies. To convince administrators and grant reviewers of the worthiness of their work, scientists have to be cheerleaders for their research; they may not be as critical of their results as they should be.

Impact factor mania

For scientists, publishing in a top journal — such as Nature , Science or Cell  — with high citation rates or “impact factors” is like winning a medal. Universities and funding agencies award jobs and money disproportionately to researchers who publish in these journals. Many researchers say the science in those journals isn’t better than studies published elsewhere, it’s just splashier and tends not to reflect the messy reality of real-world data. Mania linked to publishing in high-impact journals may encourage researchers to do just about anything to publish there, sacrificing the quality of their science as a result.

Tainted cultures

Experiments can get contaminated and cells and animals may not be as advertised. In hundreds of instances since the 1960s, researchers misidentified cells they were working with. Contamination led to the erroneous report that the XMRV virus causes chronic fatigue syndrome, and a recent report suggests that bacterial DNA in lab reagents can interfere with microbiome studies.

Do the wrong kinds of statistical analyses and results may be skewed. Some researchers accuse colleagues of “p-hacking,” massaging data to achieve particular statistical criteria. Small sample sizes and improper randomization of subjects or “blinding” of the researchers can also lead to statistical errors. Data-heavy studies require multiple convoluted steps to analyze, with lots of opportunity for error. Researchers can often find patterns in their mounds of data that have no biological meaning.

Sins of omission

To thwart their competition, some scientists may leave out important details. One study found that 54 percent of research papers fail to properly identify resources, such as the strain of animals or types of reagents or antibodies used in the experiments. Intentional or not, the result is the same: Other researchers can’t replicate the results.

Biology is messy

Variability among and between people, animals and cells means that researchers never get exactly the same answer twice. Unknown variables abound and make replicating in the life and social sciences extremely difficult.

Peer review doesn’t work

Peer reviewers are experts in their field who evaluate research manuscripts and determine whether the science is strong enough to be published in a journal. A sting conducted by Science found some journals that don’t bother with peer review, or use a rubber stamp review process. Another study found that peer reviewers aren’t very good at spotting errors in papers. A high-profile case of misconduct concerning stem cells revealed that even when reviewers do spot fatal flaws, journals sometimes ignore the recommendations and publish anyway ( SN: 12/27/14, p. 25 ).

Some scientists don’t share

Collecting data is hard work and some scientists see a competitive advantage to not sharing their raw data. But selfishness also makes it impossible to replicate many analyses, especially those involving expensive clinical trials or massive amounts of data.

Research never reported

Journals want new findings, not repeats or second-place finishers. That gives researchers little incentive to check previously published work or to try to publish those findings if they do. False findings go unchallenged and negative results — ones that show no evidence to support the scientist’s hypothesis — are rarely published. Some people fear that scientists may leave out important, correct results that don’t fit a given hypothesis and publish only experiments that do.

Poor training produces sloppy scientists

Some researchers complain that young scientists aren’t getting proper training to conduct rigorous work and to critically evaluate their own and others’ studies.

Mistakes happen

Scientists are human, and therefore, fallible. Of 423 papers retracted due to honest error between 1979 and 2011, more than half were pulled because of mistakes, such as measuring a drug incorrectly.

Researchers who make up data or manipulate it produce results no one can replicate. However, fraud is responsible for only a tiny fraction of results that can’t be replicated. 

More Stories from Science News on Science & Society

A hidden danger lurks beneath Yellowstone

Online spaces may intensify teens’ uncertainty in social interactions

Want to see butterflies in your backyard? Try doing less yardwork

Ximena Velez-Liendo is saving Andean bears with honey

‘Flavorama’ guides readers through the complex landscape of flavor

Rain Bosworth studies how deaf children experience the world

Separating science fact from fiction in Netflix’s ‘3 Body Problem’ 

Language models may miss signs of depression in Black people’s Facebook posts

Subscribers, enter your e-mail address for full access to the Science News archives and digital editions.

Not a subscriber? Become one now .

10 Psychological Experiments That Could Never Happen Today

By meredith danko | sep 20, 2013.

Nowadays, the American Psychological Association has a Code of Conduct in place when it comes to ethics in psychological experiments. Experimenters must adhere to various rules pertaining to everything from confidentiality to consent to overall beneficence. Review boards are in place to enforce these ethics. But the standards were not always so strict, which is how some of the most famous studies in psychology came about. 

1. The Little Albert Experiment

At Johns Hopkins University in 1920, John B. Watson conducted a study of classical conditioning, a phenomenon that pairs a conditioned stimulus with an unconditioned stimulus until they produce the same result. This type of conditioning can create a response in a person or animal towards an object or sound that was previously neutral. Classical conditioning is commonly associated with Ivan Pavlov, who rang a bell every time he fed his dog until the mere sound of the bell caused his dog to salivate.

Watson tested classical conditioning on a 9-month-old baby he called Albert B. The young boy started the experiment loving animals, particularly a white rat. Watson started pairing the presence of the rat with the loud sound of a hammer hitting metal. Albert began to develop a fear of the white rat as well as most animals and furry objects. The experiment is considered particularly unethical today because Albert was never desensitized to the phobias that Watson produced in him. (The child died of an unrelated illness at age 6, so doctors were unable to determine if his phobias would have lasted into adulthood.)

2. Asch Conformity Experiments

Solomon Asch tested conformity at Swarthmore College in 1951 by putting a participant in a group of people whose task was to match line lengths. Each individual was expected to announce which of three lines was the closest in length to a reference line. But the participant was placed in a group of actors, who were all told to give the correct answer twice then switch to each saying the same incorrect answer. Asch wanted to see whether the participant would conform and start to give the wrong answer as well, knowing that he would otherwise be a single outlier.

Thirty-seven of the 50 participants agreed with the incorrect group despite physical evidence to the contrary. Asch used deception in his experiment without getting informed consent from his participants, so his study could not be replicated today.

3. The Bystander Effect

Some psychological experiments that were designed to test the bystander effect are considered unethical by today’s standards. In 1968, John Darley and Bibb Latané developed an interest in crime witnesses who did not take action. They were particularly intrigued by the murder of Kitty Genovese , a young woman whose murder was witnessed by many, but still not prevented.

The pair conducted a study at Columbia University in which they would give a participant a survey and leave him alone in a room to fill out the paper. Harmless smoke would start to seep into the room after a short amount of time. The study showed that the solo participant was much faster to report the smoke than participants who had the exact same experience, but were in a group.

The studies became progressively unethical by putting participants at risk of psychological harm. Darley and Latané played a recording of an actor pretending to have a seizure in the headphones of a person, who believed he or she was listening to an actual medical emergency that was taking place down the hall. Again, participants were much quicker to react when they thought they were the sole person who could hear the seizure.

4. The Milgram Experiment

Yale psychologist Stanley Milgram hoped to further understand how so many people came to participate in the cruel acts of the Holocaust. He theorized that people are generally inclined to obey authority figures, posing the question , “Could it be that Eichmann and his million accomplices in the Holocaust were just following orders? Could we call them all accomplices?” In 1961, he began to conduct experiments of obedience.

Participants were under the impression that they were part of a study of memory . Each trial had a pair divided into “teacher” and “learner,” but one person was an actor, so only one was a true participant. The drawing was rigged so that the participant always took the role of “teacher.” The two were moved into separate rooms and the “teacher” was given instructions. He or she pressed a button to shock the “learner” each time an incorrect answer was provided. These shocks would increase in voltage each time. Eventually, the actor would start to complain followed by more and more desperate screaming. Milgram learned that the majority of participants followed orders to continue delivering shocks despite the clear discomfort of the “learner.”

Had the shocks existed and been at the voltage they were labeled, the majority would have actually killed the “learner” in the next room. Having this fact revealed to the participant after the study concluded would be a clear example of psychological harm.

5. Harlow’s Monkey Experiments

In the 1950s, Harry Harlow of the University of Wisconsin tested infant dependency using rhesus monkeys in his experiments rather than human babies. The monkey was removed from its actual mother which was replaced with two “mothers,” one made of cloth and one made of wire. The cloth “mother” served no purpose other than its comforting feel whereas the wire “mother” fed the monkey through a bottle. The monkey spent the majority of his day next to the cloth “mother” and only around one hour a day next to the wire “mother,” despite the association between the wire model and food.

Harlow also used intimidation to prove that the monkey found the cloth “mother” to be superior. He would scare the infants and watch as the monkey ran towards the cloth model. Harlow also conducted experiments which isolated monkeys from other monkeys in order to show that those who did not learn to be part of the group at a young age were unable to assimilate and mate when they got older. Harlow’s experiments ceased in 1985 due to APA rules against the mistreatment of animals as well as humans . However, Department of Psychiatry Chair Ned H. Kalin, M.D. of the University of Wisconsin School of Medicine and Public Health has recently begun similar experiments that involve isolating infant monkeys and exposing them to frightening stimuli. He hopes to discover data on human anxiety, but is meeting with resistance from animal welfare organizations and the general public.

6. Learned Helplessness

The ethics of Martin Seligman’s experiments on learned helplessness would also be called into question today due to his mistreatment of animals. In 1965, Seligman and his team used dogs as subjects to test how one might perceive control. The group would place a dog on one side of a box that was divided in half by a low barrier. Then they would administer a shock, which was avoidable if the dog jumped over the barrier to the other half. Dogs quickly learned how to prevent themselves from being shocked.

Seligman’s group then harnessed a group of dogs and randomly administered shocks, which were completely unavoidable. The next day, these dogs were placed in the box with the barrier. Despite new circumstances that would have allowed them to escape the painful shocks, these dogs did not even try to jump over the barrier; they only cried and did not jump at all, demonstrating learned helplessness.

7. Robbers Cave Experiment

Muzafer Sherif conducted the Robbers Cave Experiment in the summer of 1954, testing group dynamics in the face of conflict. A group of preteen boys were brought to a summer camp, but they did not know that the counselors were actually psychological researchers. The boys were split into two groups, which were kept very separate. The groups only came into contact with each other when they were competing in sporting events or other activities.

The experimenters orchestrated increased tension between the two groups, particularly by keeping competitions close in points. Then, Sherif created problems, such as a water shortage, that would require both teams to unite and work together in order to achieve a goal. After a few of these, the groups became completely undivided and amicable.

Though the experiment seems simple and perhaps harmless, it would still be considered unethical today because Sherif used deception as the boys did not know they were participating in a psychological experiment. Sherif also did not have informed consent from participants.

8. The Monster Study

At the University of Iowa in 1939, Wendell Johnson and his team hoped to discover the cause of stuttering by attempting to turn orphans into stutterers. There were 22 young subjects, 12 of whom were non-stutterers. Half of the group experienced positive teaching whereas the other group dealt with negative reinforcement. The teachers continually told the latter group that they had stutters. No one in either group became stutterers at the end of the experiment, but those who received negative treatment did develop many of the self-esteem problems that stutterers often show. Perhaps Johnson’s interest in this phenomenon had to do with his own stutter as a child , but this study would never pass with a contemporary review board.

Johnson’s reputation as an unethical psychologist has not caused the University of Iowa to remove his name from its Speech and Hearing Clinic .

9. Blue Eyed versus Brown Eyed Students

Jane Elliott was not a psychologist, but she developed one of the most famously controversial exercises in 1968 by dividing students into a blue-eyed group and a brown-eyed group. Elliott was an elementary school teacher in Iowa, who was trying to give her students hands-on experience with discrimination the day after Martin Luther King Jr. was shot, but this exercise still has significance to psychology today. The famous exercise even transformed Elliott’s career into one centered around diversity training.

After dividing the class into groups, Elliott would cite phony scientific research claiming that one group was superior to the other. Throughout the day, the group would be treated as such. Elliott learned that it only took a day for the “superior” group to turn crueler and the “inferior” group to become more insecure. The blue eyed and brown eyed groups then switched so that all students endured the same prejudices.

Elliott’s exercise (which she repeated in 1969 and 1970) received plenty of public backlash, which is probably why it would not be replicated in a psychological experiment or classroom today. The main ethical concerns would be with deception and consent, though some of the original participants still regard the experiment as life-changing .

10. The Stanford Prison Experiment

In 1971, Philip Zimbardo of Stanford University conducted his famous prison experiment, which aimed to examine group behavior and the importance of roles. Zimbardo and his team picked a group of 24 male college students who were considered “healthy,” both physically and psychologically. The men had signed up to participate in a “ psychological study of prison life ,” which would pay them $15 per day. Half were randomly assigned to be prisoners and the other half were assigned to be prison guards. The experiment played out in the basement of the Stanford psychology department where Zimbardo’s team had created a makeshift prison. The experimenters went to great lengths to create a realistic experience for the prisoners, including fake arrests at the participants’ homes.

The prisoners were given a fairly standard introduction to prison life, which included being deloused and assigned an embarrassing uniform. The guards were given vague instructions that they should never be violent with the prisoners, but needed to stay in control. The first day passed without incident, but the prisoners rebelled on the second day by barricading themselves in their cells and ignoring the guards. This behavior shocked the guards and presumably led to the psychological abuse that followed. The guards started separating “good” and “bad” prisoners, and doled out punishments including push ups, solitary confinement, and public humiliation to rebellious prisoners.

Zimbardo explained , “In only a few days, our guards became sadistic and our prisoners became depressed and showed signs of extreme stress.” Two prisoners dropped out of the experiment; one eventually became a psychologist and a consultant for prisons . The experiment was originally supposed to last for two weeks, but it ended early when Zimbardo’s future wife, psychologist Christina Maslach, visited the experiment on the fifth day and told him , “I think it’s terrible what you’re doing to those boys.”

Despite the unethical experiment, Zimbardo is still a working psychologist today. He was even honored by the American Psychological Association with a Gold Medal Award for Life Achievement in the Science of Psychology in 2012 .

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Elsevier Sponsored Documents

The victims of unethical human experiments and coerced research under National Socialism

Paul weindling.

1 Oxford Brookes University, History, Philosophy and Religion, Headington Campus, Oxford OX3 0BP, United Kingdom

Anna von Villiez

2 Independent Data Analyst

Aleksandra Loewenau

3 Oxford Brookes University, Centre for Medical Humanities, Department of History, Philosophy and Religion, Gypsy Lane, Oxford OX3 0BP, United Kingdom

4 University of Calgary, Cumming School of Medicine, 2500 University Drive NW, Calgary T2N 1N4, Canada

Nichola Farron

5 Independent Holocaust Historian

• • Human experiments were more extensive than often assumed with a minimum of 15,750 documented victims.
• • Experiments rapidly increased from 1942, reaching a high point in 1943 and sustained until the end of the war.
• • There were more victims who survived than were killed as part of or as a result of the experiments. Many survived with severe injuries.
• • Victims came from diverse nationalities with Poles (Jews and Roman Catholics) as the largest national cohort.
• • Body parts, especially from euthanasia killings, were often retained for research and teaching after 1945.

There has been no full evaluation of the numbers of victims of Nazi research, who the victims were, and of the frequency and types of experiments and research. This paper gives the first results of a comprehensive evidence-based evaluation of the different categories of victims. Human experiments were more extensive than often assumed with a minimum of 15,754 documented victims. Experiments rapidly increased from 1942, reaching a high point in 1943. The experiments remained at a high level of intensity despite imminent German defeat in 1945. There were more victims who survived than were killed as part of or as a result of the experiments, and the survivors often had severe injuries.

The coerced human experiments and research under National Socialism constitute a reference point in modern bioethics. 7 Yet discussions of consent and the need for safeguards for research subjects to date lack a firm basis in historical evidence. There has been no full evaluation of the numbers of victims of Nazi research, who the victims were, and of the frequency and types of experiments and research. The one partial estimate is restricted to experiments cited at the Nuremberg Medical Trial. This paper gives the first results of a comprehensive evidence-based evaluation of the different categories of victims. In 1945 liberated prisoners from German concentration camps began to collect evidence of the experiments.

The scientific intelligence officer John Thompson then pointed out not only that 90% of all medical research under National Socialism was criminal, but also the need to evaluate all criminal experiments under National Socialism, and not just those whose perpetrators were available for arrest and prosecution. 8 The Nuremberg Medical Trial of 1946–47 was necessarily selective as to who was available for prosecution, and since then only clusters of victims have been identified. 9 In the early 1980s Günther Schwarberg named a set of child victims: his reconstruction the life histories of the ‘twenty children’ killed after transport from Auschwitz for a tuberculosis immunisation experiment at Neuengamme concentration camp was exemplary. 10 The question arises whether what Schwarberg achieved in microcosm can be achieved for the totality of victims. Our aim is to identify not just clusters of victims but all victims of unethical medical research under National Socialism. The methodology is that of record linkage to reconstruct life histories of the total population of all such research victims. This allows one to place individual survivors and groups of victims within a wider context.

This project on the victims of Nazi medical research represents the fulfilment of Thompson's original scheme of a complete record of all coerced experiments and their victims. 11 Our project identifies for the first time the victims of Nazi coercive research, and reconstructs their life histories as far as possible. Biographical data found in many different archives and collections is linked to compile a full life history, and subjective narratives and administrative data are compared. Results are aggregated here as cohorts because of undertakings as regards anonymisation, given in order to gain access to key sources. All data is verifiable through the project database.

The criterion for unethical research is whether coercion by researchers was involved, or whether the location was coercive. The project has covered involuntary research in clinical contexts as psychiatric hospitals, incarceration in concentration camps and prisoner of war camps, the ‘euthanasia’ killings of psychiatric patients with subsequent retention of body parts for research, and executions of political victims, when body parts were sent to university anatomical institutes, and persons subjected to anthropological research in coercive and life threatening situations as ghettoes and concentration camps.

Without a reliable, evidence-based historical analysis, compensation for surviving victims has involved many problems. Victim numbers have been consistently underestimated from the first compensation scheme in 1951 when the assumption was of only few hundred survivors. 12 The assumption was that most experiments were fatal. This project's use of several thousand compensation records in countries where victims lived (as Poland) or migrated to (as Israel), or were collected by the United Nations or the German government has corrected this impression. The availability of person-related evidence from the International Tracing Service at Bad Arolsen further helps to determine whether a victim survived. Major repositories of documents like the United States Holocaust Memorial Museum and the Yad Vashem archives, court records in war crimes proceedings, and oral history collections notably the Shoah Foundation have been consulted. Record linkage of named records is essential for the project, and shows how a single person could be the victim of research on multiple occasions. Father Leon Michałowski, born 22 March 1909 in Wąbrzeźno, was subjected to malaria in August 1942 and then to freezing experiments in October 1942 ( Figure 1 ).

Malaria card of Father Bruno Stachowski from Claus Schilling's research at Dachau. Approximately 1000 cards were kept back from destruction by the prisoner assistant Eugène Ost. International Tracing Service, source number 1079406301.

A further issue relates to the methods and organisation of the research. From the 1950s the experiments were viewed as ‘pseudo-science’, in effect marginalising them from mainstream science under National Socialism. For the purpose of this study, the experiments have been viewed as part of mainstream German medical research, as this renders rationales and supportive networks historically intelligible. It is clear that prestigious research institutions such as the Kaiser Wilhelm Society and funding agencies such as the German Research Fund were involved. 13 It has been argued more recently that some experiments were cutting edge science. 14 Another view is that the approach and methods were scientific albeit of varying quality. For the purpose of this study, the experiments have been viewed as part of mainstream German medical research, as this renders rationales and supportive networks intelligible.

Defining what constitutes research is problematic. For example, a listing of operations in a concentration camp may be nothing more than a clinical record, may have been undertaken by young surgeons seeking to improve their skills, or may indeed have involved research. As stated above, only confirmed data of research has been utilised in the project's category of a verified instance of unethical research. The only exception is the corpses sent to anatomical institutes for research purposes. 15 Separating these out often does not appear possible, but the project includes anatomical research on body parts and brains as separate categories.

The project has graded victim evidence into two categories, so that there should be a set of verifiable and proven victims established as incontestable evidence of having been a victim. The unexpectedly high numbers of identified experiment victims makes this necessary. The two categories are:

• 1. those who were identified as confirmed victims through a reliable source such as experimental records kept at the time.
• 2. those who have claimed to have been experimented on, but confirmation could not so far be obtained.

The project did not set out to adjudicate on the authenticity of victims’ claims. In Warsaw ca. 3600 compensation files of victims of human experiments were viewed, while there are a further 10,000 files representing claims deemed unsuccessful. It is sometimes unclear whether extensive injuries were retrospectively defined to have resulted from an experiment to meet the criteria of the compensation scheme offered by the Federal Republic of Germany in various forms since 1951, or whether experimentation had taken place in a hitherto unknown location. The project discounted claims of abuse when no experiment or research was involved, or when victims having misunderstood compensation schemes for experiments being about ‘experiences’. It is hoped that further research will provide confirmation of experiments in disputed locations like the concentration camps of Stutthof and Theresienstadt. 16 While Yugoslav victims were abused for experiments in German concentration camps, claims for experiments in the former Yugoslavia and Northern Norway have not so far been confirmed. The grading of victims’ claims into the verified and as yet unverified enable the project to establish verifiable minimum numbers, while indicating the possibility of higher numbers being confirmed by further research.

Project findings

The project is able to present results on: how many victims were killed in the course of the experiment, how many died from the consequences of the experiment or were killed as potential evidence of Nazi criminality, and how many survived? The project has covered experiments, as the most notorious experiments taken to the point of death and supported by the SS in concentration camps, as well as dispersed experiments in a variety of clinical contexts, particularly on psychiatric patients. Some sets of experiments and locations, not least those sponsored by German pharmaceutical companies remain shadowy, and require more detailed research possibly on the basis of further disclosure of documents held in company archives. The extent of involvement of German pharmaceutical companies like that of IG-Farben (using the branded product names of ‘Bayer’, ‘Hoechst’ and ‘Behringwerke’) remains contentious. The company supplied Helmuth Vetter with samples for experiments at Auschwitz and Mauthausen. Similarly problematic is the extent that Schering-Kahlbaum supported Clauberg's uses of X-ray contrast fluids and a substance to seal the fallopian tubes at Auschwitz. Initially, Clauberg asked for deliveries to his clinic at Königshütte (so making the experiments appear as taking place in a consensual clinical context), but later on to Auschwitz. The extent that the company's senior staff knew that their employee and Clauberg's pharmaceutical assistant Johannes Goebel worked at Auschwitz is contentious ( Figure 2 ). 17

Carl Clauberg and Horst Schumann at Block 10 in Auschwitz. United States Holocaust Memorial Museum W/S #67417.

The occurrence of unethical research provides insight into the structure of Nazi medical research. The project traced how Nazi coercive research began in the context of eugenic research in the mid-1930s. After numbers of experiments dipped in 1940 due to military call-up of medical researchers, the research rapidly intensified both in terms of numbers of experiments and victims, and in terms of severity for victims. This can be seen from 1942 with the notorious and often fatal experiments on low pressure, exposure to freezing temperatures, and infectious diseases when research could be taken to the point of death. Pharmacological experiments on therapies for tetanus, typhus and typhoid were spurred by the realisation that Allied military medical research on infectious diseases was outstripping German military medical expertise. From November 1942 racial priorities came increasingly to the fore, as exemplified by Schumann's X-ray sterilisation experiments on Jews in Auschwitz.

Victims were a highly international group. The above table ( Table 1 ) shows numbers of nationalities, using nationality as in 1938. The table indicates the distribution of nationalities. The largest national group, that of Polish victims, includes both Roman Catholics and Jews. There were high numbers of German and Austrian victims, in part as a result of the experiments and dissections that accompanied the killing of psychiatric patients. While there were other large groups, there are also smaller national groups, as Swiss, British and Irish, all highly remarkable in how their citizens became caught up in the experimentation. We find victims include a Swiss conscientious objector used for malaria experiments at Dachau, and British commandos captured in Norway used for amphetamine and high performance experiments on the shoe track at Sachsenhausen, and subsequently executed.

Nationality (as at March 1938).

Statistics on gender indicate a proportion of male to female of approximately 2:1 ( Table 2 ). One possible reason is the high number of military experiments as related to infectious diseases. Another is that more men than women were held in concentration camps, so that there was a higher male availability in the predominately male camps. In Ravensbrück the situation was reversed with the large female camp and a small male compound ( Figure 3 ).

Clandestine photograph of a mutilated leg of the Polish political prisoner Bogumiła Babińska-Dobrowska at Ravensbrück concentration camp. United States Holocaust Memorial Museum, courtesy of Anna Hassa Jarosky and Peter Hassa W/S #69340.

While for most nationalities male victims were the majority, in the case of certain national groups, female victims were in the majority. This is the case for victim groups from the Netherlands (in the case of sterilisation at Auschwitz), and Greece (for the Jewish skeleton collection). Children were often victims of experiments in psychiatric clinics. Later in the war, Roma and Jewish children were targeted for research by Mengele in Auschwitz.

The statistics show the age distribution was the same for men and women. While there was a very wide age spectrum, the peak is of victims born in 1921, so in their early twenties at the time of the experiment ( Figure 4 ). Several hundred Jewish children were held by Mengele for twin research, and batches of Jewish children were dispatched for hepatitis and tuberculosis research, and body parts of small children were retained by psychiatric researchers.

Age of victims at the start of experiment.

Ethnicity and religion have been recorded, as for the definitively confirmed experiment victims ( Table 3 ). Here, one is thrown back on the categories imposed by the Nazis. Thus a victim of the Jewish skeleton collection for the anatomy department at Strasbourg was baptised Protestant. 18 Generally, the Nazis used the generic and stigmatising term of ‘Zigeuner’ or gypsy rather than the self-identifying terms of ‘Sinti’ and ‘Roma’.

In addition to the experiment victims are Roma and Sinti victims of large scale anthropological investigations of Ritter, Justin, and Ehrhardt, amounting to at least a further 21,498 persons ( Figure 5 ).

A gypsy used for seawater experiments in Dachau to test methods of making seawater drinkable, ca. July–September 1944. United States Holocaust Memorial Museum, courtesy of National Archives and Records Administration, College Park W/S #78688.

If however one takes the year 1943 we find a higher proportion of Jewish victims, in part because of the intensification of experiments on Jews (particularly on women and children) at Auschwitz and Auschwitz-Birkenau. This would again indicate that there was an intensification of racial research ( Table 4 ).

Victim number indicates how from 1942 onwards there was an overall intensification of research ( Figure 6 ).

Start year of experiments.

The life history approach allows appraisal of both experiments and victim numbers over time. The period 1933–39 shows sporadic experimentation in the context of racial hygiene. Mixed race adolescents were sterilised and evaluated by anthropologists. The concerns of racial hygiene with mental illness explain why psychiatrists and neurologists conducted experiments in psychiatric institutions. The psychiatrist Georg Schaltenbrand pointed out that his neurological research subjects were transferred to other institutions, many as we now know to be killed. This interrupted his research on the transmissibility of multiple sclerosis. The numbers of brains and body parts increased. From 1942 onwards there was an overall intensification of research.

The chart ( Figure 7 ) shows when experiments started, but not the distribution of victims over time.

Victims by start year of each experiment.

The largest series of experiments were for infectious diseases. Malaria research at Dachau between 1942 and 1945 had 1091 confirmed victims, and after infection different combinations of drugs were tested. These experiments by Schilling began in 1942 and remarkably Schilling tried to continue the research after the liberation of the camp. 19

He pleaded at his trial to be allowed to continue the research, albeit on volunteers. The highest numbers were in 1943. The momentum continued even though the war was clearly lost. Other large groups included the twins researched on by Mengele, and to date 618 individuals are known ( Figure 8 ).

Twins Frank (lt) and Otto (rt) Klein attend a world gathering of survivors of Dr. Joseph Mengele's medical experiments at Auschwitz. United States Holocaust Memorial Museum W/S #05586.

The overall findings provide an accurate basis for analysis of experiments to date. First, nearly a quarter of confirmed victims were either killed to obtain their organs for research, or died as a result of experiments taking the research subject to the point of death (notoriously, the experiments on freezing and low pressure at Dachau). The euthanasia killings and executions were sources of bodies for research, and the extent that this happened and research conducted before and after the end of the war is still being documented. Of the fully documented victims died 781 died before the end of the war as a result of the experiments: research subjects were weakened by the strain of the experiment such as a deliberate infection or severe cold, or they were deliberately killed because it was feared that they would testify against the perpetrators ( Table 5 ).

Fatalities.

While, most subjects survived, amounting to 24,010 persons, many had severe physical disabilities with life-long consequences. 20

The analysis presented here shows that several types of unethical medical research occurred under National Socialism. Not only were large numbers of victims affected, but also overall, numbers of surviving victims were far higher than anticipated. The survivors were often seriously disabled and handicapped for the remainder of their lives. The experiments gained in numbers with the war and the implementation of the Holocaust, and were sustained at a high level of intensity despite imminent defeat.

One issue arising is that body parts of deceased victims were retained by medical research and teaching institutes, notably for anatomy and brain research. While there was meant to be full disclosure of specimens deriving from euthanasia victims and executed persons by 1990, specimens continue to be identified. 21 The complex data is to be further augmented and refined, the history of specimens retained for research during and after WW2 is being documented, and the narratives of survivors analysed in order to understand more fully the consequences of coerced research. This research provides a basis in historical evidence for discussions of the ethics of coerced medical research.

Acknowledgments

Wellcome Trust Grant No 096580/Z/11/A on research subject narratives.

AHRC GRANT AH/E509398/1 Human Experiments under National Socialism.

Conference for Jewish Material Claims Against Germany Application 8229/Fund SO 29.

7 Annas G, Grodin M. The Nazi Doctors and the Nuremberg Code . New York: Oxford University Press; 1992; Weindling P. Nazi Medicine and the Nuremberg Trials: From Medical War Crimes to Informed Consent . Basingstoke: Palgrave Macmillan; 2000.

8 Weindling P. John Thompson (1906–1965): Psychiatrist in the Shadow of the Holocaust . Rochester: Rochester University Press; 2010.

9 Klee E. Auschwitz, die NS-Medizin und ihre Opfer . Frankfurt am Main: S. Fischer Verlag; 1997; Mitscherlich A, Mielke F. Wissenschaft ohne Menschlichkeit . Heidelberg: Lambert Schneider; 1949. See also Schwarberg G. The Murders at Bullenhuser Damm: The SS Doctor and the Children . Bloomington: Indiana University Press; 1984.

10 Lang H-J. Die Namen der Nummern: Wie es gelang, die 86 Opfer eines NS-Verbrechens zu identifizieren. Hamburg: Hoffmann und Campe; 2004.

11 Brody H, Leonard S, Nie J-B, Weindling P. United States Responses to Japanese Wartime Inhuman Experimentation after World War II: National Security and Wartime Exigency. Cambridge Quarterly of Healthcare Ethics 2014; 23: 220–230.

12 Baumann S. Menschenversuche und Wiedergutmachung . München: Oldenbourg; 2009.

13 Schwerin A von. Experimentalisierung des Menschen: Der Genetiker Hans Nachtsheim und die vergleichende Erbpathologie 1920–1945 . Göttingen: Wallstein Verlag; 2004; Schmuhl H-W. The Kaiser Wilhelm Institute for Anthropology, Human Heredity and Eugenics, 1927–1945: Crossing Boundaries . Dordrecht: Springer; 2008.

14 Roelcke V. Fortschritt ohne Rücksicht, In: Eschebach I, Ley A. eds . Geschlecht und “Rasse” in der NS-Medizin . Berlin: Metropol Verlag; 2012: 101–114.

15 Hildebrandt S. Anatomy in the Third Reich: An outline, Part 1. National Socialist politics, anatomical institutions, and anatomists. Clinical Anatomy 2009; 22: 883–893.

16 Hunt N. The Soviet Experience of Nazi Medicine: Statistics, Stories and Stereotypes . MPhil thesis, Oxfrod Brookes University; 2011; and Loewenau A. The impact of Nazi medical experiments on Polish inmates at Dachau, Auschwitz and Ravensbück . PhD thesis, Oxford Brookes University; 2012.

17 Weindling, P. Victims and Survivors of Nazi Human Experiments: Science and suffering in the Holocaust. London: Bloomsbury; 2014.

18 www.die-namen-der-nummern.de/html/the_names.html (accessed 8 October 2014).

19 Eckart WU, Vondra H. Malaria and World War II: German malaria experiments 1939–45 . Parassitologia 2000; 42:53–58.

20 Loewenau A. Die “Kaninchen” von Ravensbrück: Eine Fotogeschichte. In: Eschebach I, Ley A eds. Geschlecht und “Rasse” in der NS-Medizin . Berlin: Metropol Verlag; 2012:115–140.

21 Weindling P. “Cleansing” Anatomical Collections: The Politics of Removing Specimens from German Anatomical and Medical Collections 1988–92. Annals of Anatomy 2012; 194:237–242.

Hundreds of Psychology Studies Are Wrong

One hidden factor means that hundreds of studies are completely useless..

Posted May 1, 2017 | Reviewed by Davia Sills

This might sound like nothing new if you’ve kept up with the replication crisis, but I’m referring to a completely different issue—one that is all too often overlooked.

Let me introduce the problem with an example. In a recent article from The Conversation , researchers from the University of Queensland describe a study in which they observed families during hospital visits and measured the anxiety and distress of both parents and children. The measures of children’s anxiety and pain were positively associated with parents’ distress levels.

The researchers interpret the role of the parents’ distress during the visit as causal, arguing that parents who are more anxious and distressed are less able to support their child during the procedure. This seems like a rather sensible and intuitive conclusion to draw; however, the researchers should have absolutely no confidence in doing so.

Anxiety sensitivity (i.e., how susceptible an individual is to feel anxious, ranging from completely normal to pathologically desensitized/hypersensitive) is substantially genetic. In fact, nearly half of the variation (i.e., the differences between people in the population) in anxiety sensitivity is likely to be genetic. The researchers, therefore, have no way to distinguish whether the children are responding to their parents’ distress or simply share their anxious disposition for genetic reasons, or more likely, some combination of the two.

In another instance, the New York Times published an article titled, "Yes, It’s Your Parents’ Fault" in which they quote research from the last 50 years to argue that if you have intimacy problems, then your parents are to blame because of their attachment behavior. They’re right, just for the wrong reasons.

Attachment styles are genetically conferred from parent to child , meaning it’s just as likely the genes you have inherited from your parents as their behavior that make you insecure as an adult. Consequently, genes confound any study correlating parents’ behavior with their child’s attachment, even in adulthood, making this data essentially useless in understanding how parenting affects our development.

Now, I know what you’re thinking: Sure, genes are important, but the environment is too, and you’re not wrong. The problem is that without measuring the impact of genes first, it’s impossible to say how important the environment is, or even which bits of it are responsible (e.g., parents, school, diet , learning). This is because the genes that influence our parents’ behavior, and therefore our environment, can be passed to us.

We can inherit the same genes that make our parents aggressive, rash, or impulsive, and the influence of these genes can be easily confused with the environment that we’re reared in. Without taking genes into account, it is impossible to determine if the environment is doing anything at all.

This is quite simply a problem of correlation being mistaken with causation.

For example, it was long thought that the absence of a father could cause daughters to become sexually promiscuous at an earlier age . However, using a genetically informative sample, researchers have now shown that the same genes that influence fathers’ tendency to leave home also predispose their daughters to early promiscuity , perhaps via a trait like novelty seeking. Conversely, the influence of the family environment is far less important than previously thought.

Researchers are able to disentangle the relative contributions of genes and environment because of the natural experiment provided by identical and non-identical twin pairs. By comparing the similarity between identical twins who share all the same genes and non-identical twins who only share on average 50 percent of their genes, scientists can model how important genes are for any given trait. If identical twins are more alike than non-identical twins, then genes may be important, and researchers can use complex statistics to generate an estimate of the relative importance of genes and the environment.

Unfortunately this body of literature has been ignored by a lot of psychologists, particularly those favoring environmental explanations for behavior, and consequently, the history of the field is replete with examples of scientists erroneously attributing a causal role to parents’ behavior.

One particularly damaging case comes from the 1940s when Leo Kanner undertook a case study of 11 children with autism and noted that “one other fact stands out prominently. In the whole group, there are very few really warmhearted fathers and mothers.”

The frosty nature of the parents was later forwarded as a cause for their children’s disorder, a position that was furthered in several later papers from Kanner, and became widely accepted within the medical community. As a result, the phrase "refrigerator mother" was coined as a label for parents of autistic children who suffered enormous stigma and were held responsible for their child’s disordered behavior.

Similarly, the households of schizophrenics have been found to have higher expressed emotions and for a long time were presumed to have contributed to the formation of the disease.

Moreover, t he behavior of the parents of obese children is frequently claimed to cause the child’s weight gain.

Yet, autism spectrum disorder is highly heritable . Schizophrenia is highly heritable . Obesity is highly heritable . In fact, most traits ever studied are heritable to some degree, from intelligence to how religious you are. In all of these instances, genes need to be considered when making causal claims about correlated variables, such as parenting and children’s behavior.

Now, none of this is to say that parenting doesn’t matter, and you can mistreat children, and they’ll turn out OK—far from it. The damaging effects of parental neglect and abuse are all too well established.

The problem is that when it comes to human behavior, scientists are studying the cumulative effects of genes and environment: Neglecting one source of influence means that it’s impossible to quantify the impact of either. This produces results that are uninterpretable. It is as simple as that.

This is bad science, and bad science is at best useless, and at worst dangerous, not to mention expensive. Moreover, any interventions that are developed off the back of such research are unfounded and likely to be unhelpful, if not harmful.

Perhaps the reluctance to acknowledge this position comes from a sense of resignation that the involvement of genes invokes. When people hear that genes have a causal role in establishing our character, they assume this is shorthand for unchangeable or immutable. With the exception of a few rare and particularly cruel diseases, this is not the case.

We are not automatons programmed from birth by a few small molecules. The journey from genes to tissue is complex enough to have defied our understanding despite rigorous inquiry for hundreds of years—let alone the marvelously sophisticated transition from physiology to thoughts, words, and actions. Genes are not the codex by which the rich tapestry of human achievement can be demystified—they are but one constituent.

Genes may be more important than other influences in establishing some traits, and less important for others, but at almost all levels, they provide some guiding principle of organization, small hands nudging here and there in confluence with the miasma of our lived experience. To assign them total responsibility and throw our hands in the air in defeat would be as futile an approach to understanding our history, and indeed our future, as denying them any role at all.

The truth, as it is often want to, lies somewhere between the two. To the extent that we obfuscate this with wishful thinking or outright denial , we hamstring our potential to uncover the truth of the world around us.

James Sherlock is a Ph.D. candidate in psychology at the University of Queensland.

• Find a Therapist
• Find a Treatment Center
• Find a Psychiatrist
• Find a Support Group
• Find Online Therapy
• United States
• Brooklyn, NY
• Chicago, IL
• Houston, TX
• Los Angeles, CA
• New York, NY
• Portland, OR
• San Diego, CA
• San Francisco, CA
• Seattle, WA
• Washington, DC
• Asperger's
• Bipolar Disorder
• Chronic Pain
• Eating Disorders
• Passive Aggression
• Personality
• Goal Setting
• Positive Psychology
• Stopping Smoking
• Low Sexual Desire
• Relationships
• Child Development
• Therapy Center NEW
• Diagnosis Dictionary
• Types of Therapy

Understanding what emotional intelligence looks like and the steps needed to improve it could light a path to a more emotionally adept world.

• Emotional Intelligence
• Gaslighting
• Affective Forecasting
• Neuroscience

• Share full article

Advertisement

Supported by

An N.Y.U. Study Gone Wrong, and a Top Researcher Dismissed

By Benedict Carey

• June 27, 2016

New York University’s medical school has quietly shut down eight studies at its prominent psychiatric research center and parted ways with a top researcher after discovering a series of violations in a study of an experimental, mind-altering drug.

A subsequent federal investigation found lax oversight of study participants, most of whom had serious mental issues. The Food and Drug Administration investigators also found that records had been falsified and researchers had failed to keep accurate case histories.

In one of the shuttered studies, people with a diagnosis of post-traumatic stress caused by childhood abuse took a relatively untested drug intended to mimic the effects of marijuana, to see if it relieved symptoms.

“I think their intent was good, and they were considerate to me,” said one of those subjects, Diane Ruffcorn, 40, of Seattle, who said she was sexually abused as a child. “But what concerned me, I was given this drug, and all these tests, and then it was goodbye, I was on my own. There was no follow-up.”

It’s a critical time for two important but still controversial areas of psychiatry: the search for a blood test or other biological sign of post-traumatic stress disorder , which has so far come up empty, and the use of recreational drugs like ecstasy and marijuana to treat it.

At least one trial of marijuana, and one using ecstasy, are in the works for traumatized veterans, and some psychiatrists and many patients see this work as having enormous promise to reshape and improve treatment for trauma. But obtaining approval to use the drugs in experiments is still politically sensitive. Doctors who have done studies with these drugs say that their uncertain effects on traumatic memory make close supervision during treatment essential.

The New York Times pieced together details of the research and the investigation through documents, letters and emails, as well as interviews with a participant and several researchers familiar with the project.

The violations “jeopardize subject safety and welfare, and raise concerns about the validity and integrity of the data collected at your site,” the F.D.A. said in a letter, obtained by The Times, to Dr. Alexander Neumeister, the studies’ lead investigator.

Dr. Charles Marmar, the chairman of the psychiatry department at N.Y.U., said that people working with Dr. Neumeister had reported concerns about the lab’s compliance with research standards. Once some of those issues were confirmed, Dr. Marmar said, the university placed Dr. Neumeister on leave, “and suspended all activity, suspended access to all accounts. I took control of those studies.” Dr. Neumeister later submitted his resignation.

Georges Lederman, a lawyer for Dr. Neumeister, said there may have been protocol violations, “but N.Y.U. has taken the position that those violations were more egregious than we believe they actually were.” The issues could have been easily remedied, he said, and noted that they did not cause the sponsor of the research, the pharmaceutical giant Pfizer, to shut it down.

Both the university and Mr. Lederman said there was no evidence that any study participant had been harmed.

Pfizer said that N.Y.U. was responsible for conducting the trial, and that the company had previously tested the same drug, known as an F.A.A.H. inhibitor, for osteoarthritic pain, without significant side effects. “The safety profile we observed does not preclude future development of our compound,” a Pfizer spokesman said by email.

Careful oversight is a centerpiece of ethical research practice, particularly when studies involve people taking an experimental drug. Earlier this year, six patients in a French trial of another experimental drug with similar, marijuana-like effects were hospitalized with severe neurological problems; one has reportedly died.

Study participants with mental disorders are especially susceptible to adverse reactions, experts said. “These are people who are more vulnerable to being exploited in the research process, and more vulnerable to things going wrong during the research, so you want extra vigilance,” said Elisa Hurley, the executive director of Public Responsibility in Medicine and Research , a national nonprofit devoted to promoting high research standards. “If someone in my family were in a situation like this, I would want to be sure that the institution was crossing its t’s and dotting its i’s.”

The study was an attempt to extend a small trial that Dr. Neumeister had done previously, suggesting that cannabis might relieve anxiety in some people with post-traumatic stress disorder. “We know very well that people with PTSD who use marijuana often experience more relief from their symptoms than they do from antidepressants and other psychiatric medications,” Dr. Neumeister said in a 2014 N.Y.U. news release after the first trial.

The research team decided to use a drug intended to produce some of marijuana’s effects, made by Pfizer, which financed the trial. Some participants took the drug over a seven-day period; others took a placebo pill. The N.Y.U. team performed scans on each person to see whether brain activation patterns correlated with symptom relief.

The study called for recruiting 50 people with a PTSD diagnosis, according to study documents. Only 14 enrolled at the N.Y.U. site, according to federal documents, and many had participated in previous studies by Dr. Neumeister. One had completed a study of another drug 16 days earlier, when the protocol called for a 30-day window, according to the F.D.A.

The federal inspection, from July 16 to Aug. 5 last year, found that the research team had failed to assess at least three subjects 24 hours after they had taken the experimental drug, contrary to study protocol, according to the F.D.A. letter. In several instances, the agency found, Dr. Neumeister had falsified documents by signing a fellow investigator’s name on reports. “However, in fact, you or another study employee actually conducted these study procedures,” not the colleague, the F.D.A. concluded.

Ms. Ruffcorn, who writes a popular Facebook blog on trauma called A Little Bent , said the most unsettling part of the experience was not the loose monitoring, but the preparations for the trial. To participate, the N.Y.U. team told her, she first had to stop taking all her medications. But the study had several false starts, requiring her to stop taking medication, then restart, then stop again — and restart.

“It was horrible,” she said. “I had flashbacks, returning nightmares, every symptom coming on full force, not to mention the withdrawal. After going off and back on four or five times, I told them, ‘I can’t do this anymore.’”

That’s when she was flown to New York to enter the trial.

Ms. Ruffcorn said she had several odd symptoms after the trial, including a hyper, wired sensation that occurred without the usual memories of abuse. For months, she tried to find out whether those reactions were tied to the experimental drug, but because the study was shut down and the data belonged to Pfizer, the N.Y.U. doctors could not tell her whether she’d taken the drug or a placebo.

Earlier this month, after much persistence, she found out that she’d taken the placebo. “It was a big relief,” she said.

Dr. Neumeister and N.Y.U. continue to disagree over the seriousness of the research violations, both sides said. But the university has tossed out all of the data as unreliable, and tracked down the study participants to check on their health, Dr. Marmar said.

“I honestly believed I had the best qualified and among the most ethical researchers on the faculty” running these studies, Dr. Marmar said.

Like the Science Times page on Facebook. | Sign up for the Science Times newsletter.

• International edition
• Australia edition
• Europe edition

Research findings that are probably wrong cited far more than robust ones, study finds

Academics suspect papers with grabby conclusions are waved through more easily by reviewers

Scientific research findings that are probably wrong gain far more attention than robust results, according to academics who suspect that the bar for publication may be lower for papers with grabbier conclusions.

Studies in top science, psychology and economics journals that fail to hold up when others repeat them are cited, on average, more than 100 times as often in follow-up papers than work that stands the test of time.

The finding – which is itself not exempt from the need for scrutiny – has led the authors to suspect that more interesting papers are waved through more easily by reviewers and journal editors and, once published, attract more attention.

“It could be wasting time and resources,” said Dr Marta Serra-Garcia, who studies behavioural and experimental economics at the University of California in San Diego. “But we can’t conclude that something is true or not based on one study and one replication.” What is needed, she said, is a simple way to check how often studies have been repeated, and whether or not the original findings are confirmed.

The study in Science Advances is the latest to highlight the “replication crisis” where results, mostly in social science and medicine, fail to hold up when other researchers try to repeat experiments. Following an influential paper in 2005 titled Why most published research findings are false , three major projects have found replication rates as low as 39% in psychology journals , 61% in economics journals , and 62% in social science studies published in the Nature and Science, two of the most prestigious journals in the world.

Working with Uri Gneezy, a professor of behavioural economics at UCSD, Serra-Garcia analysed how often studies in the three major replication projects were cited in later research papers. Studies that failed replication accrued, on average, 153 more citations in the period examined than those whose results held up. For the social science studies published in Science and Nature, those that failed replication typically gained 300 more citations than those that held up. Only 12% of the citations acknowledged that replication projects had failed to confirm the relevant findings.

The academic system incentivises journals and researchers to publish exciting findings, and citations are taken into account for promotion and tenure. But history suggests that the more dramatic the results , the more likely they are to be wrong . Dr Serra-Garcia said publishing the name of the overseeing editor on journal papers might help to improve the situation.

Prof Gary King, a political scientist at Harvard University, said the latest findings may be good news. He wants researchers to focus their efforts on claims that are subject to disagreement, so that they can gather more data and figure out the truth. “In some ways, then, we should regard the results of this interesting article as great news for the health of the scholarly community,” he said.

Prof Brian Nosek at the University of Virginia, who runs the Open Science Collaboration to assess reproducibility in psychology research, urged caution. “We presume that science is self-correcting. By that we mean that errors will happen regularly, but science roots out and removes those errors in the ongoing dialogue among scientists conducting, reporting, and citing each others research. If more replicable findings are less likely to be cited, it could suggest that science isn’t just failing to self-correct; it might be going in the wrong direction.’

“The evidence is not sufficient to draw such a conclusion, but it should get our attention and inspire us to look more closely at how the social systems of science foster self-correction and how they can be improved,” he added.

• Higher education

Most viewed

• Newsletters

Site search

• Israel-Hamas war
• Home Planet
• 2024 election
• Supreme Court
• All explainers
• Future Perfect

Filed under:

• Animal Welfare

What’s worse than a cruel animal experiment? A cruel and fake animal experiment.

Raising the consequences for animal testing experiments gone wrong.

Share this story

• Share this on Facebook
• Share this on Twitter
• Share this on Reddit
• Share All sharing options

Share All sharing options for: What’s worse than a cruel animal experiment? A cruel and fake animal experiment.

Last December, in the wake of animal cruelty allegations against Elon Musk’s brain chip startup Neuralink, Vox’s Kenny Torrella wrote about a concept he called “the moral math of animal testing”: the view held by many people that trading some amount of animal suffering is worth it if it can save enough human lives by advancing medicine.

Experimentation on live animals is a divisive, morally charged subject. Slightly more than half of Americans say they oppose using animals in scientific research, according to a 2018 Pew survey, but it depends a lot on how you phrase the question and who is asking. When asked by the biomedical industry whether they support “the humane use of animals” to develop “lifesaving medicines,” many more people say they do, or aren’t sure. These gaps reflect the public’s lack of understanding of how vivisection works in general: Most people don’t know whether animal testing is humane, effective, or necessary, nor do they always know how to define those terms.

Not everyone will agree with my view of vivisection, which is that it’s unjustifiable in nearly all circumstances. But I would think most people will agree that animal experiments should have to clear an especially high bar — that they have to be truly necessary for saving human lives and irreplaceable with non-animal methods.

That is, unfortunately, not how animal testing in the US works at all. Scientists harm and kill animals for all sorts of studies that have nothing to do with saving human lives. Researchers at Oregon Health & Science University, for example, have forced prairie voles to drink alcohol to test whether it makes them cheat on their partners. A Harvard neuroscientist recently came under fire for separating caged mother monkeys from their babies and giving them surrogate stuffed animals to bond with, thus demonstrating, she wrote in a top scientific journal, that “infant/mother bonds may be triggered by soft touch.”

The worst kind of fraud

Animal experimentation is also not immune to outright fraud, a problem that’s “disturbingly common” in science, as Vox’s Kelsey Piper wrote in June. Last week, federal investigators found that William Armstead, a former professor at the University of Pennsylvania’s medical school, had faked the results of multiple federally funded studies that involved cutting open piglets’ skulls and inducing brain injuries. The studies were meant to test drugs for treating brain injuries in humans. (Armstead left the university while he was under investigation for this misconduct.)

Some of Armstead’s fabrications, which included relabeling results from past studies as new ones, appear designed to make a drug his team was studying look more effective, Ivan Oransky, co-founder of Retraction Watch — a blog that tracks retractions of scientific papers — told the Philadelphia Inquirer last week. Armstead’s team doctored 51 scientific figures across five published studies, three federal grant applications, and other documents. The faked data renders the research useless; it’s now been retracted from journals and can’t be incorporated into future work. “A bunch of pigs were subjected to some pretty terrible conditions for no reason,” Oransky told the Inquirer.

Armstead now faces a seven-year ban on conducting federally funded research, a penalty that’s relatively rare in its severity. But his case isn’t an isolated one.

Last year, a pivotal 2006 mouse study, which had been thought to shed light on the pathology of Alzheimer’s disease and shaped years of federally funded research, was credibly accused of being fraudulent and remains under investigation.

Also last year, federal officials found Deepak Kaushal, then-head of the federally funded Southwest National Primate Research Center in San Antonio, to have falsified results in a published study of a tuberculosis treatment tested on monkeys, and used those results in two NIH grant applications. Kaushal was placed under a one-year supervision period — the lightest imaginable slap on the wrist — with no lasting consequences for his ability to experiment on animals. He also, according to initial reports, got to keep his job as director of the lab. After criticism from some in the research community and animal advocates, he was later demoted from that position; it’s unclear whether he’ll be reinstated after his supervision period.

The price of suffering

All these revelations should raise alarms about how misconduct is handled in research involving animal testing. When a top primate researcher is allowed to keep experimenting on monkeys after falsifying data, it sends a message to everyone in the research community that recklessly handling animal experiments, while temporarily embarrassing, may not be that big a deal.

“The NIH tends to give anybody on their pay line the benefit of the doubt,” neuroscientist Katherine Roe, who worked at NIH for more than eight years and is now chief of PETA’s science advancement and outreach division, told me. (PETA, despite its reputation, has a top-notch team of scientists challenging unethical animal research). “The penalties for research fraud are not what they should be.”

To shift the incentive structure, we need better federal regulation that raises the cost of torturing animals for botched experiments. Right now, the consequences for misconduct in federally funded research don’t take into account whether the work involved animal testing, Roe said. Federal research regulations could be amended so that scientists found responsible for misconduct in work involving vulnerable populations, including non-human animals, be permanently barred from testing on them in future federally sponsored research, a change that’s been proposed by PETA, explained Emily Trunnell, a senior scientist for the organization.

That would be a good start. But it would require the authorities who oversee science to view the animal experiments themselves, and not just lying about their results, as morally implicated, something the research community has been loath to do because it threatens to undermine the whole endeavor of animal testing.

On a higher level, we have to start seeing it as the public’s right and duty to make democratic decisions about whether and how animals are used in scientific research, especially when our money is paying for it. Scientists are an exalted class, often allowed to self-regulate, but their expertise in a narrow subject matter shouldn’t let them overrule democratic governance of research ethics. Ethics belongs to us all. And the public expects a much higher bar than too many animal researchers currently set for themselves.

Will you support Vox today?

We believe that everyone deserves to understand the world that they live in. That kind of knowledge helps create better citizens, neighbors, friends, parents, and stewards of this planet. Producing deeply researched, explanatory journalism takes resources. You can support this mission by making a financial gift to Vox today. Will you join us?

We accept credit card, Apple Pay, and Google Pay. You can also contribute via

Next Up In Future Perfect

Sign up for the newsletter today, explained.

Understand the world with a daily explainer plus the most compelling stories of the day.

Thanks for signing up!

Check your inbox for a welcome email.

Oops. Something went wrong. Please enter a valid email and try again.

How to prepare for another season of wildfire smoke

You might have another chance to see the northern lights tonight

Meme stocks like GameStop are soaring like it’s 2021

The silver lining in a very bad poll for Biden

Why is Biden blocking the cheapest, most popular EVs in the world?

Something weird is happening with tornadoes

share this!

July 5, 2018

Beware those scientific studies—most are wrong, researcher warns

by Ivan Couronne

A few years ago, two researchers took the 50 most-used ingredients in a cook book and studied how many had been linked with a cancer risk or benefit, based on a variety of studies published in scientific journals.

The result? Forty out of 50, including salt, flour, parsley and sugar. "Is everything we eat associated with cancer?" the researchers wondered in a 2013 article based on their findings.

Their investigation touched on a known but persistent problem in the research world: too few studies have large enough samples to support generalized conclusions.

But pressure on researchers, competition between journals and the media's insatiable appetite for new studies announcing revolutionary breakthroughs has meant such articles continue to be published.

"The majority of papers that get published, even in serious journals, are pretty sloppy," said John Ioannidis, professor of medicine at Stanford University, who specializes in the study of scientific studies.

This sworn enemy of bad research published a widely cited article in 2005 entitled: "Why Most Published Research Findings Are False."

Since then, he says, only limited progress has been made.

Some journals now insist that authors pre-register their research protocol and supply their raw data, which makes it harder for researchers to manipulate findings in order to reach a certain conclusion. It also allows other to verify or replicate their studies.

Because when studies are replicated, they rarely come up with the same results. Only a third of the 100 studies published in three top psychology journals could be successfully replicated in a large 2015 test.

Medicine, epidemiology, population science and nutritional studies fare no better, Ioannidis said, when attempts are made to replicate them.

"Across biomedical science and beyond, scientists do not get trained sufficiently on statistics and on methodology," Ioannidis said.

Too many studies are based solely on a few individuals, making it difficult to draw wider conclusions because the samplings have so little hope of being representative.

Coffee and Red Wine

"Diet is one of the most horrible areas of biomedical investigation," professor Ioannidis added—and not just due to conflicts of interest with various food industries.

"Measuring diet is extremely difficult," he stressed. How can we precisely quantify what people eat?

In this field, researchers often go in wild search of correlations within huge databases, without so much as a starting hypothesis.

Even when the methodology is good, with the gold standard being a study where participants are chosen at random, the execution can fall short.

A famous 2013 study on the benefits of the Mediterranean diet against heart disease had to be retracted in June by the most prestigious of medical journals, the New England Journal of Medicine , because not all participants were randomly recruited; the results have been revised downwards.

So what should we take away from the flood of studies published every day?

Ioannidis recommends asking the following questions: is this something that has been seen just once, or in multiple studies? Is it a small or a large study? Is this a randomized experiment? Who funded it? Are the researchers transparent?

These precautions are fundamental in medicine, where bad studies have contributed to the adoption of treatments that are at best ineffective, and at worst harmful.

In their book "Ending Medical Reversal," Vinayak Prasad and Adam Cifu offer terrifying examples of practices adopted on the basis of studies that went on to be invalidated, such as opening a brain artery with stents to reduce the risk of a new stroke.

It was only after 10 years that a robust, randomized study showed that the practice actually increased the risk of stroke.

The solution lies in the collective tightening of standards by all players in the research world, not just journals but also universities, public funding agencies. But these institutions all operate in competitive environments.

"The incentives for everyone in the system are pointed in the wrong direction," Ivan Oransky, co-founder of Retraction Watch, which covers the withdrawal of scientific articles, tells AFP. "We try to encourage a culture, an atmosphere where you are rewarded for being transparent."

The problem also comes from the media, which according to Oransky needs to better explain the uncertainties inherent in scientific research, and resist sensationalism.

"We're talking mostly about the endless terrible studies on coffee, chocolate and red wine," he said.

"Why are we still writing about those? We have to stop with that."

Journal information: New England Journal of Medicine

Explore further

Feedback to editors

Tiger beetles fight off bat attacks with ultrasonic mimicry

3 hours ago

Machine learning model uncovers new drug design opportunities

7 hours ago

Astronomers find the biggest known batch of planet ingredients swirling around young star

How 'glowing' plants could help scientists predict flash drought

New GPS-based method can measure daily ice loss in Greenland

New candidate genes for human male infertility found by analyzing gorillas' unusual reproductive system

9 hours ago

Study uncovers technologies that could unveil energy-efficient information processing and sophisticated data security

Scientists develop an affordable sensor for lead contamination

Chemists succeed in synthesizing a molecule first predicted 20 years ago

New optical tweezers can trap large and irregularly shaped particles

Relevant physicsforums posts, cover songs versus the original track, which ones are better, who is your favorite jazz musician and what is your favorite song.

4 hours ago

How does academic transcripts translation work?

Biographies, history, personal accounts.

17 hours ago

Today's Fusion Music: T Square, Cassiopeia, Rei & Kanade Sato

Music to lift your soul: 4 genres & honorable mention.

May 12, 2024

More from Art, Music, History, and Linguistics

Related Stories

Science says: what happens when researchers make mistakes.

Jun 13, 2018

One reason so many scientific studies may be wrong

Oct 6, 2016

New guidelines issued for reporting of genetic risk research

Mar 28, 2011

Two studies cast doubt on credibility of medical research

Jan 4, 2016

Study: Published research often is false

Aug 30, 2005

Researchers announce master plan for better science

Jan 10, 2017

Recommended for you

The power of ambiguity: Using computer models to understand the debate about climate change

May 13, 2024

Study finds avoiding social media before an election has little to no effect on people's political views

Researchers develop algorithms to understand how humans form body part vocabularies

Study shows AI conversational agents can help reduce interethnic prejudice during online interactions

May 10, 2024

Analysis of millions of posts shows that users seek out echo chambers on social media

May 9, 2024

The spread of misinformation varies by topic and by country in Europe, study finds

May 8, 2024

Let us know if there is a problem with our content

Use this form if you have come across a typo, inaccuracy or would like to send an edit request for the content on this page. For general inquiries, please use our contact form . For general feedback, use the public comments section below (please adhere to guidelines ).

Please select the most appropriate category to facilitate processing of your request

Thank you for taking time to provide your feedback to the editors.

Your feedback is important to us. However, we do not guarantee individual replies due to the high volume of messages.

E-mail the story

Your email address is used only to let the recipient know who sent the email. Neither your address nor the recipient's address will be used for any other purpose. The information you enter will appear in your e-mail message and is not retained by Phys.org in any form.

Newsletter sign up

Get weekly and/or daily updates delivered to your inbox. You can unsubscribe at any time and we'll never share your details to third parties.

More information Privacy policy

Donate and enjoy an ad-free experience

We keep our content available to everyone. Consider supporting Science X's mission by getting a premium account.

E-mail newsletter

When experiments go wrong: the U.S. perspective

Affiliation.

• 1 World Health Organization, Geneva.
• PMID: 15202353

The view that once prevailed in the U.S.--that research is no more dangerous than the activities of daily life--no longer holds in light of recent experience. Within the past few years, a number of subjects (including normal volunteers) have been seriously injured or killed in research conducted at prestigious institutions. Plainly, when we are talking about research going wrong, we're talking about something very important. We have seen that experiments can go wrong in several ways. Subjects can be injured--physically, mentally, or by having other interests violated. Investigators can commit fraud in data collection or can abuse subjects. And review mechanisms--such as IRBs--don't always work. The two major issues when research goes wrong in any of these ways are, first: What will be done for subjects who have suffered an injury or other wrong? and second: How will future problems be prevented? The present system in the U.S. is better at the second task than the first one. Part of the difficulty in addressing the first lies in knowing what "caused" an apparent injury. Moreover, since until recently the problem of research-related injuries was thought to be a small one, there was considerable resistance to setting up a non-fault compensation system, for fear that it would lead to payment in many cases where such compensation was not deserved. Now, with a further nudge from the NBAC there is renewed interest in developing a formal system to compensate for research injuries. Finally, I have tried to show that our system of local oversight is only partially effective in improving the design of experiments and the consent process in light of "unexpected (adverse) results." As many observers, including the federal General Accounting Office (GAO), have reported, the requirement for "continuing review" of approved research projects is the weak point in the IRB system. The probable solution would be to more strictly apply the requirement that investigators report back any adverse results, de-emphasizing the "screen" introduced by the present language about "unexpected" findings. Yet, despite its weaknesses, there are good aspects to the local basis of our oversight system, and when problems become severe enough, OHRP is likely to evaluate a system and insist on local improvements. Thus, while the U.S. system is far from perfect in responding when research goes wrong, our experience may be useful to others in crafting a system appropriate to their own circumstances. One of the major tasks will be to adequately define what triggers oversight--that is, who reports what to whom and when? The setting of this trigger needs to balance appropriate incentives and penalties. Any system, including our own, will, in my opinion, work much better once an accreditation process is in place, which will offer much more current and detailed information on how each IRB is functioning and what steps are needed to help avoid "experiments going wrong."

• Biomedical Research / ethics*
• Human Experimentation*
• Medical Errors* / legislation & jurisprudence
• Medical Errors* / prevention & control
• United States

Covering a story? Visit our page for journalists or call (773) 702-8360.

New Certificate Program

Top stories.

• Inside the Lab: A first-hand look at University of Chicago research
• Scav reflections: “for the love of the pursuit and the joy in the attempt”
• W. Ralph Johnson, pre-eminent UChicago critic of Latin poetry, 1933‒2024

• Big Brains: All Episodes

Where has Alzheimer’s research gone wrong? with Karl Herrup (Ep. 132)

Neurobiologist claims the leading definition of the disease may be flawed—and why we need to fix it to find a cure

For more than a century, scientists have been studying Alzheimer’s disease and developing theories about its underlying cause. The leading theory for decades has been that abnormal amyloid plaques in the brains of those who suffer from the disease are the central cause. But, according one renowned Alzheimer’s researcher, this myopic focus is not only flawed, but may be holding back our search for a cure. 

Neurobiologist Karl Herrup argues that we need to go back to the drawing board, redefine the disease and understand the many factors that could cause it before we can race for a cure. A professor of neurobiology and an investigator in the Alzheimer’s Disease Research Center at the University of Pittsburgh School of Medicine, Herrup is the author of  How Not to Study a Disease: The Story of Alzheimer’s .

Subscribe to Big Brains on  Apple Podcasts  and  Spotify .

(Episode published March 21, 2024)

Subscribe to the Big Brains newsletter.

Please rate and review the Big Brains podcast.

• The Dark Tangle of Alzheimer’s—The New York Review
• Does new research really mean the ‘beginning of the end of Alzheimer’s’?—I News
• Allegations of fabricated research undermine key Alzheimer’s theory—NBC News
• Defining Alzheimer’s, and the climate costs of AI: Books in brief—Nature

Transcript:

Paul Rand: The University of Chicago Leadership and Society Initiative guides accomplished executive leaders in transitioning from their longstanding careers into purposeful encore chapters of leadership for society. The initiative is currently accepting candidates for its second cohort of fellows. Your next chapter matters for you and for society. Learn more about this unique fellowship experience at leadforsociety.uchicago.edu. Over the last few decades, we’ve made notable advances in cancer research and heart disease, but when it comes to Alzheimer’s disease, why have we made so little progress?

Tape: 6.7 million Americans are living with Alzheimer’s, and that is expected to nearly double by 2050.

Paul Rand: Alarmingly, one third of people over age 85 have Alzheimer’s disease, according to the National Institute on Aging.

Tape: Anyone who’s had experience with Alzheimer’s disease knows the agony of watching someone fade away, as it steals memory and, at the end, a person’s own identity.

Paul Rand: Despite us researching this disease since 1906, we still haven’t found a cure.

Karl Herrup: We know more about the brain, we know more about the biochemistry and genetics of Alzheimer’s disease than we ever have, but I feel that, by closing our minds and closing our research avenues down to a few narrow passageways, we’ve missed so many opportunities to really advance the field.

Paul Rand: That’s Karl Herrup, a professor of neurobiology and investigator in the Alzheimer’s Disease Research Center at the University of Pittsburgh School of Medicine, and he recently wrote a book called How Not to Study a Disease: The Story of Alzheimer’s.

Karl Herrup: You just don’t know where your next good idea is coming from. Very often, the best way to study Alzheimer’s disease is not to study Alzheimer’s disease. The light bulb was not invented by somebody trying to make a better candle. I think that applies in spades to the scientific enterprise.

Paul Rand: Herrup argues that the reason we’ve made so little progress with Alzheimer’s starts with the fact that we don’t have an accurate definition of the disease.

Karl Herrup: One of the issues I have with the field as a whole is that our definition of the disease has become so distorted. It’s almost a worthless moniker.

Paul Rand: If you can’t even define the thing you’re trying to study, how can you hope define a cure?

Karl Herrup: I’m speaking now as a basic researcher, where, if I am going to study a disease, I’m interested in, what are its biological underpinnings? What is it about the brain and its biology that drives the symptoms that I’m seeing? And in what ways can I intervene? If I can’t define what the disease is, my hands are tied as a biologist, and I feel that’s the situation we’re in today.

Paul Rand: The story of how we got here is fascinating, surprising, and more than mildly infuriating, but it holds important lessons for understanding how science can get led astray and how we can get back on track. But Herrup believes that we can really find better ways to treat Alzheimer’s.

Karl Herrup: I do believe we will be able to intervene and make that process less aggressive, less rapid, and less debilitating.

Paul Rand: Welcome to Big Brains, where we translate the biggest ideas and complex discoveries into digestible brain food. Big Brains, little bites from the University of Chicago Podcast Network. I’m your host, Paul Rand. On today’s episode, our struggle to find a cure for Alzheimer’s disease. Herrup’s book is called How Not to Study a Disease. But when exactly did we start going down the wrong path with Alzheimer’s? The answer, sadly, goes all the way back to the very beginning. I wonder, Karl, if we could start off, actually, as the son of a woman who suffered from Alzheimer’s, of a woman who suffered from Alzheimer’s and eventually passed from complications, many, many of our listeners, of course, are going to be familiar with this disease. In 1906, if we can talk about the patient that really started all of this work and the two scientists who started really the actual discovery of this.

Karl Herrup: So the patient who started our path to the state of Alzheimer’s disease right now was a farmer’s wife known as Auguste Deter. She became aggressive and almost unmanageable. So her husband brought her to a clinic, and one of the young physicians in the clinic was a brain scientist known as Alois Alzheimer. This was in Frankfurt, Germany, and he was quite interested, studied her, interviewed her. She subsequently died, because the supervisor of the clinic where she had been seen was aware that Alzheimer himself was interested in the structure of the brain, as it might underlie the function and malfunction in the case of Auguste. And so, arranged to have an autopsy performed and shipped samples of the brain to Alzheimer in his new role in Munich. It was then that he sectioned her brain and stained it with modern techniques, using silver to reveal the structure of neurons and other elements of the brain. And he discovered deposits that he said were abnormal.

Paul Rand: And we talked to [inaudible 00:05:52] about as plaques. What does it look like when there’s a plaque in the brain?

Karl Herrup: Well, normally, at this point, I would say, “If I may have the first slide, please,” but since we’re doing this on audio, the only way to describe them is they’re blobs. They’re many times bigger than a nerve cell, maybe 10 to 20 times as big, and they just don’t belong there. So since he had never seen these deposits before, he said that it must be these deposits that were driving the behavioral problems.

Paul Rand: Great. Problem solved, right?

Karl Herrup: The problem is, looking back now or even at the time, thinking about it logically, when two things occur together, it doesn’t necessarily mean that one causes the other. And even if one did cause the other, based on the correlation alone, you have to be very careful about assuming which way who causes what. In the case of Alzheimer, if I had been a reviewer of his paper, I would’ve said, “Well, Dr. Alzheimer, are you sure that the disease itself wasn’t what caused plaques?” So first, the disease, then the plaques. It’s caused a huge problem, because in confusing correlation with causality, the field has been trapped into this model of the disease.

Paul Rand: And your story of how it got to be accepted is also pretty ironic, isn’t it? Because this is not typically how peer review papers get accepted, put into science, and go from there. But this actually took a very different turn that gives the explanation.

Karl Herrup: That is correct. Alzheimer ultimately published his observations, and he published in what’s known as a case study, which is a very valuable form of scientific literature. It’s a description a physician gives, where he’s seen a patient or maybe a couple patients, and they have a very particular condition and he speculates as to how it might relate to the broader field of medicine. The irony was his boss, Emil Kraepelin, decided that he wanted to take it a step further. So Kraepelin was the author of a very influential textbook of neurological diseases, and he too was a big fan of the idea that the structure of the brain determined its function. And so, he too was a big fan of this idea that plaques therefore must be causing the dementia. But he took the rather audacious step of actually putting Auguste Deter’s case into his textbook, giving the condition a name, which he called Alzheimer’s disease.

Paul Rand: It’s nice he gave him credit for it.

Karl Herrup: Well, he gave him credit for it. Good guy. But we’re still trying to dig out from the hole that was created by that decision.

Paul Rand: That single decision to tie the definition of Alzheimer’s to the presence of these plaques would define how we study this disease and the search for a cure all the way to the present day. And what was originally just a hypothesis eventually solidified after a discovery in the 1980s and nineties.

Karl Herrup: A couple of clever biochemists, Glenner and Wong, were able to isolate the plaques and determine their amino acid sequence. In other words, what they were made of. And they discovered this small peptide that’s the A beta peptide, they discovered that the gene that encoded that peptide was on chromosome 21. And that got everybody excited, because chromosome 21, when it’s in three copies instead of the normal two, is the genetic condition known as Down syndrome. Sad though it may be, individuals with Down syndrome, as they age, undergo a premature onset of dementia and also build up plaques in their brain. And that led to geneticists, John Hardy and Higgins, to put forward what became known as the Amyloid Cascade Hypothesis.

Their idea was that, “Okay, we’ve got these plaques, sort of the pathological evidence of Alzheimer’s disease, and now, we have the biochemistry and the genetics all coming together. Therefore, we propose that this amyloid peptide begins to aggregate, and as it does so, it causes damage to the other cells of the brain, ultimately leading to the loss of neurons. And that’s where the dementia comes from.” That has been and it became and remains to this day the basis of Alzheimer’s disease. Again, as a hypothesis, that’s a perfectly rational point of view, but it’s become dogma.

Paul Rand: The challenge, as you write in your book, is that defense of this hypothesis became the mission in and of itself. Tell me what you mean by that, and is that unusual in science research? Or is it that this got jumped to the front of the line, when it wasn’t ready to be put in the front of the line?

Karl Herrup: What I try and bring out in my book is that, if you go back to those early days, this Amyloid Cascade Hypothesis was quite innovative and very compelling. It isn’t often that we have this beautiful congruence between biochemistry, genetics, and neuropathology, but as my mathematics friends say, “The best thing about a good hypothesis is you can take it out in the backyard and throw things at it to try and break it.” That’s what never happened with the Amyloid Cascade Hypothesis, that, instead what happened the certainty of its proponents became so rigid that counter arguments were just dismissed as silly and a waste of time.

Paul Rand: What if that original hypothesis from 1906 was misguided? What if these plaques don’t cause Alzheimer’s, rather the disease itself that leads to the plaques? Or even more controversial, what if these plaques are just a much smaller piece of a larger puzzle than we’ve realized?

Karl Herrup: So do we know more than we did in the 1990s? Of course we do. Could we have moved farther towards treatments and possibly cures for Alzheimer’s? I would say absolutely.

Paul Rand: Okay, and you make the point that it becomes very difficult to get funding for any research that doesn’t put the Amyloid Plaques Cascade Hypothesis front and center of the proposal.

Karl Herrup: Well, that is the result.

Paul Rand: The evidence for why these amyloid plaques may be the wrong place to look and where Herrup thinks we should be looking after the break. Have you ever wondered who you are, but didn’t know who to ask? Well then, join Professor Eric Oliver as he poses the nine most essential questions for knowing yourself to some of humanity’s wisest and most interesting people. Nine Questions with Eric Oliver, part of the University of Chicago Podcast Network. For much of the time we’ve studied Alzheimer’s, researchers’ theories have remained the same, amyloid plaques causes Alzheimer’s.

Karl Herrup: But I say, okay, let’s take that on. If that’s true, then if I have plaques, I must have dementia or I must be on my way to it. And it turns out that nothing could be further from the truth. If we take a cross section of individuals who are, say, 75 years old or 80 years old, and we look in their brains, either with modern PET imaging or after they die with neuropathology, we will find that about 30% of them have plaque burdens that a pathologist would say looks like Alzheimer’s disease.

Paul Rand: But they don’t have the symptoms?

Karl Herrup: They have no symptoms. They’re cognitively normal.

Paul Rand: Very interesting.

Karl Herrup: That seems to me to need a lot of explanation. So the counter argument becomes, “Okay, well, they just haven’t shown the symptoms yet, but they’re on their way.” And in fact, five years after having been diagnosed as having all these plaques in your brain, only half of the people with that plaque burden are going to be developing dementia.

Paul Rand: Only half? Wow. Okay. Fascinating. All right, so another bit of evidence that comes out is there were anti-amyloid vaccines that were being tested.

Karl Herrup: What we’re getting at here is the other side of the coin. So I said, if you have plaques in your brain and you don’t have dementia, that’s really a challenge to the Amyloid Cascade Hypothesis. But then, an equally strong challenge would be, okay, if I take plaques out of your brain, at least I should stop the progress of the disease.

Paul Rand: And that’s exactly what these anti-amyloid vaccines do, they train the immune system to target amyloid beta, which stops the development of amyloid plaques.

Karl Herrup: So at the time the book was published, it was basically true that over three dozen clinical trials that had successfully lowered the plaque burden of the brain had either not made people better or actually made them worse. So you vaccinate people, and lo and behold, the level of amyloid in their brain goes down. The problem, of course, is that people start getting really aggressive encephalopathies, meaning their brain starts swelling and nothing good.

Paul Rand: These anti-amyloid vaccines have gotten a lot of attention in recent years, but still nothing has successfully been approved as a treatment. But there’s another treatment you might’ve heard of, monoclonal antibodies.

Karl Herrup: The new monoclonal antibodies, which have been hyped as breakthroughs and triumphs, sorry, I’m not buying.

Paul Rand: So explain what the monoclonal antibodies are and what you’re not buying.

Karl Herrup: Okay. So normally, when you make a vaccine, you’re challenging your immune system to make an antibody. So you show the immune system what a virus looks like or what amyloid looks like. The immune system says, “Aha, I’ll remember that if I ever see it again.” And so, the next time it comes up, it mounts what’s called an immune response, makes antibodies, and gets rid of the virus or amyloid. The other way you can do this is you can take the antibodies made in some other creature, engineer them, so they’re tolerated by a human, and put them back into a human through a process called infusion. Because your body isn’t actually making the antibody itself, you’re just receiving antibody that was engineered in the lab. But it’s marginally safer, because it doesn’t induce the really dramatic, at least for the amyloid antibodies, it doesn’t induce the dramatic inflammation that came along with the first studies. So there are now three drugs that are either approved or very close to being approved. They all very dramatically reduced amyloid plaques within the brain.

Paul Rand: So there’s a lot of noise about those being great breakthroughs.

Karl Herrup: I refuse as a scientist to say they’re breakthroughs, and the noise comes from the following observation, compared to the placebo, people who were getting the antibody had a 33% less cognitive decline. All right, but here’s where the marketing comes back in. Because it’s marketed as an improvement, and the problem is it’s not an improvement, it’s just a less decline. So everybody in the trial, people getting the antibodies and people not getting the antibodies, the placebo, everybody got worse. The people on the antibody just got worse a little bit slower.

Paul Rand: Is that worth celebrating?

Karl Herrup: No.

Paul Rand: No? Okay, tell me why.

Karl Herrup: Well, because if you look at the actual difference, now, not the relative difference, between the placebo arm and the treatment arm, if you look at the absolute difference, the cognitive scale, it’s clinically undetectable. So that difference is unlikely to be recognizable by the patient, by their family, or by their physician.

Paul Rand: Inside your book, you spend a good amount of time talking about the role that you think that government funding organizations, not-for-profit organizations, pharma companies, and so forth played in driving this really myopic focus on the amyloid plaques. And I wonder if you think what lessons that should hold for us and what evolution from that you would love to see.

Karl Herrup: So in the early days of a new institute at the National Institutes of Health, known as the National Institute on Aging, its founders quickly realized that they were going to get the short end of the stick when it came to NIH dollars to actually fund their research. And so, they decided they had to do something to attract the funding dollars, and they realized that the idea of aging is not really scary. None of us want to age, but it’s not frightening, not like cancer. So they decided they needed a boogeyman, and they decided that, of all the diseases of aging, dementia was probably the most feared. But they decided that they needed something more sort of tangible. And what was argued by the founders of the NIA was that, well, if you look at the brains of anyone with dementia, you can find these plaques.

And so, therefore, all late onset dementia must be Alzheimer’s disease. So that seriously inflated the definition of the disease, because now, what we used to call senile dementia, meaning age appropriate loss of cognitive capacity, now had a disease name, people could get terrified. And it worked, because the NIA ended up with a substantial budget to work with. I think one of the strongest lessons is that, in a very odd way, science and money don’t mix very well, and I don’t mean that in the sense of research support. So science needs infrastructure, science needs people, science needs resources to advance. What science doesn’t need is lobbying.

What science doesn’t need is, I don’t know how to put this, but having the ends drive the means, because the pharmaceutical industry stands to make so much money, if they can have proprietary rights on a drug that will prevent or improve somebody’s Alzheimer’s disease, they drive the show. They have an outsized influence on the funding agencies, both public and private. That works to our detriment. It’s good for their bottom line or at least it is at the end of the quarter, but it doesn’t help science. Am I being a bit parochial here? Of course, I’m a basic scientist. I’ve got a dog in this fight, as they say, but being as objective as I can, I’m always struck by how the most revolutionary scientific discoveries come out of left field.

Paul Rand: Your book came out, what, three years ago at this point. Now, what was reaction, not only to your work, to your book, because you’re challenging convention? And how did the community, the research community, that Alzheimer’s community, first, take to what you were proposing?

Karl Herrup: That’s a great question, and I like that question. And I’ll tell you later a question that I don’t like, but I like this question.

Paul Rand: Okay.

Karl Herrup: For most people who are new to the field, which is heavily slanted towards the younger people in the field, the reaction has been one of gratitude. “Thank you so much. I thought I was crazy. What you’re saying makes a lot of sense.”

Paul Rand: And so, anybody listening to this, that has been affected by it or otherwise, my back really goes up on this, because of course, if you’ve had the experience with it and then, realize there’s been opportunity squandered, it becomes pretty easy to get frustrated by that. The inevitable question that you and everybody else in the field, especially now with, I think, as you’re talking about, “Well, don’t do this, do this,” and where either this is going to be relief to people that are listening to it or they’re going to be frustrated as all get out, because we all want the magic pill or the magic vaccine or whatever the case is.

Karl Herrup: Remember I told you there was a question that I hated?

Paul Rand: Yes.

Karl Herrup: Well, you’re getting very close to that question. Because the question that I hate the most is, “Okay, wise guy, if it’s not amyloid, what is it?”

Paul Rand: Of course.

Karl Herrup: The problem with that question is that I have to be honest and say, “I don’t know.” My thinking is evolving, even since I’ve written the book, and honestly, I’m beginning to think of a dementia like Alzheimer’s as a multifactorial disease. And bear with me here, because it’s a little bit complicated, but I think, to me, as a scientist, it’s a satisfying explanation. I list a number of things we should talk about. There’s a reasonable biochemistry, reasonable cell biology, reasonable genetics, that suggests that an out of control inflammatory process is at work here. So inflammation is a big one. There’s good epidemiology that says, if you’ve been on anti-inflammatories, unfortunately, at very high doses, your lifetime risk of Alzheimer’s and dementia goes down.

Paul Rand: Goes down?

Karl Herrup: Yes. And I think, as I say in my book, do not go to the drugstore and buy ibuprofen and start popping pills, because you will rot out the internal lining of your stomach long before you do anything for your dementia.

Paul Rand: Okay. All right. Good to put that catch in there.

Karl Herrup: Fat management is another. And my favorite right now is the role of the myelin around neurons. Because it insulates, it actually saves the neuron an enormous amount of energy. Why it degenerates, as we get older, we don’t know, but there is no question that, as we age, we start to lose myelin and we start to lose it sort of in midlife, in our forties. And we lose it in precisely the areas that are most affected in the course of Alzheimer’s disease. Each of these, and let’s include amyloid in the list, that each of these contributes to an altered brain chemistry, and it’s actually that sum that is what we recognize as Alzheimer’s disease. And so, attacking each of them is valuable, and ultimately, we need to attack all of them.

It’s increasingly apparent that it is the non-pharmacological changes that have proven to be the most effective tools in the fight against Alzheimer’s. And they’re uninteresting. They’re not sexy, and there’s no intellectual property left. So the pharmaceutical companies are largely uninterested. But aggressively controlling blood pressure, I mentioned that in the book, keeping your blood pressure down. Exercise helps, mental exercise and physical. Recent study looking at just a daily dose of a multivitamin. Basically, things that are helpful in keeping your health up in the presence of an aging body, heart health, brain health, dietary health, that these are the most effective tools we have to fight Alzheimer’s disease. And it makes total sense, because as I described in the book, the biggest single risk factor for Alzheimer’s is age. I do think that this process of aging is what increases your risk of Alzheimer’s and a senile dementia.

But you see, because I don’t want your listeners to come away or you to come away with the idea that Alzheimer’s is just accelerated aging, I do think that there’s a disease process that something happens. There is a moment, a threshold, where the chemistry of our brains and our bodies kicks into a degenerative pattern. It’s as important to understand that biology as it is to understand the biology of normal aging, so we can help people in both ways. And just for your listeners and for you, I just urge patients, and I think that’s really important. Alzheimer’s, dementia, aging, these are extremely complicated problems, and their solution is not going to be quick and it’s not going to be simple. It’s going to be incremental. But despite everything I’ve said, I remain extremely optimistic that we’re going to crack this problem.

Matt Hodapp: Brains is a production of the University of Chicago Podcast Network. We are sponsored by the Graham School. Are you a lifelong learner with an insatiable curiosity? Access more than 50 open enrollment courses every quarter. Learn more at graham.uchicago.edu/bigbrains. If you like what you heard on our podcast, please leave us a rating and review. The show is hosted by Paul M. Rand and produced by Lea Ceasrine and me, Matt Hodapp. Thanks for listening.

Episode List

How to manifest your future using neuroscience, with James Doty (Ep. 135)

‘Mind Magic’ author explains the scientific research on how to train our brain to achieve our goals

Why we die—and how we can live longer, with Nobel laureate Venki Ramakrishnan (Ep. 134)

Nobel Prize-winning scientist explains how our quest to slow aging is becoming a reality

What dogs are teaching us about aging, with Daniel Promislow (Ep. 133)

World’s largest study of dogs finds clues in exercise, diet and loneliness

Why breeding millions of mosquitoes could help save lives, with Scott O’Neill (Ep. 131)

Nonprofit's innovative approach uses the bacteria Wolbachia to combat mosquito-borne diseases

Why shaming other countries often backfires, with Rochelle Terman (Ep. 130)

Scholar examines the geopolitical impacts of confronting human rights violations

Can Trump legally be president?, with William Baude (Ep. 129)

Scholar who ignited debate over 14th Amendment argument for disqualification examines upcoming Supreme Court case

What our hands reveal about our thoughts, with Susan Goldin-Meadow (Ep. 128)

Psychologist examines the secret conversations we have through gestures

Psychedelics without the hallucinations: A new mental health treatment? with David E. Olson (Ep. 127)

Scientist examines how non-hallucinogenic drugs could be used to treat depression, addiction and anxiety

Do we really have free will? with Robert Sapolsky (Ep. 126)

Renowned scholar argues that biology doesn’t shape our actions; it completely controls them

Subscribe to the Big Brains Insider newsletter

• 30 Most Unethical Psychology Human Experiments

Disturbing human experiments aren’t something the average person thinks too much about. Rather, the progress achieved in the last 150 years of human history is an accomplishment we’re reminded of almost daily. Achievements made in biomedicine and the f ield of psychology mean that we no longer need to worry about things like deadly diseases or masturbation as a form of insanity. For better or worse, we have developed more effective ways to gather information, treat skin abnormalities, and even kill each other. But what we are not constantly reminded of are the human lives that have been damaged or lost in the name of this progress. The following is a list of the 30 most disturbing human experiments in history.

30. The Tearoom Sex Study

Image Source Sociologist Laud Humphreys often wondered about the men who commit impersonal sexual acts with one another in public restrooms. He wondered why “tearoom sex” — fellatio in public restrooms — led to the majority of homosexual arrests in the United States. Humphreys decided to become a “watchqueen” (the person who keeps watch and coughs when a cop or stranger get near) for his Ph.D. dissertation at Washington University. Throughout his research, Humphreys observed hundreds of acts of fellatio and interviewed many of the participants. He found that 54% of his subjects were married, and 38% were very clearly neither bisexual or homosexual. Humphreys’ research shattered a number of stereotypes held by both the public and law enforcement.

29. Prison Inmates as Test Subjects

Image Source In 1951, Dr. Albert M. Kligman, a dermatologist at the University of Pennsylvania and future inventor of Retin-A, began experimenting on inmates at Philadelphia’s Holmesburg Prison. As Kligman later told a newspaper reporter, “All I saw before me were acres of skin. It was like a farmer seeing a field for the first time.” Over the next 20 years, inmates willingly allowed Kligman to use their bodies in experiments involving toothpaste, deodorant, shampoo, skin creams, detergents, liquid diets, eye drops, foot powders, and hair dyes. Though the tests required constant biopsies and painful procedures, none of the inmates experienced long-term harm.

28. Henrietta Lacks

Image Source In 1955, Henrietta Lacks, a poor, uneducated African-American woman from Baltimore, was the unwitting source of cells which where then cultured for the purpose of medical research. Though researchers had tried to grow cells before, Henrietta’s were the first successfully kept alive and cloned. Henrietta’s cells, known as HeLa cells, have been instrumental in the development of the polio vaccine, cancer research, AIDS research, gene mapping, and countless other scientific endeavors. Henrietta died penniless and was buried without a tombstone in a family cemetery. For decades, her husband and five children were left in the dark about their wife and mother’s amazing contribution to modern medicine.

27. Project QKHILLTOP

Image Source In 1954, the CIA developed an experiment called Project QKHILLTOP to study Chinese brainwashing techniques, which they then used to develop new methods of interrogation. Leading the research was Dr. Harold Wolff of Cornell University Medical School. After requesting that the CIA provide him with information on imprisonment, deprivation, humiliation, torture, brainwashing, hypnoses, and more, Wolff’s research team began to formulate a plan through which they would develop secret drugs and various brain damaging procedures. According to a letter he wrote, in order to fully test the effects of the harmful research, Wolff expected the CIA to “make available suitable subjects.”

26. Stateville Penitentiary Malaria Study

Image Source During World War II, malaria and other tropical diseases were impeding the efforts of American military in the Pacific. In order to get a grip, the Malaria Research Project was established at Stateville Penitentiary in Joliet, Illinois. Doctors from the University of Chicago exposed 441 volunteer inmates to bites from malaria-infected mosquitos. Though one inmate died of a heart attack, researchers insisted his death was unrelated to the study. The widely-praised experiment continued at Stateville for 29 years, and included the first human test of Primaquine, a medication still used in the treatment of malaria and Pneumocystis pneumonia.

25. Emma Eckstein and Sigmund Freud

Image Source Despite seeking the help of Sigmund Freud for vague symptoms like stomach ailments and slight depression, 27-year old Emma Eckstein was “treated” by the German doctor for hysteria and excessive masturbation, a habit then considered dangerous to mental health. Emma’s treatment included a disturbing experimental surgery in which she was anesthetized with only a local anesthetic and cocaine before the inside of her nose was cauterized. Not surprisingly, Emma’s surgery was a disaster. Whether Emma was a legitimate medical patient or a source of more amorous interest for Freud, as a recent movie suggests, Freud continued to treat Emma for three years.

24. Dr. William Beaumont and the Stomach

Image Source In 1822, a fur trader on Mackinac Island in Michigan was accidentally shot in the stomach and treated by Dr. William Beaumont. Despite dire predictions, the fur trader survived — but with a hole (fistula) in his stomach that never healed. Recognizing the unique opportunity to observe the digestive process, Beaumont began conducting experiments. Beaumont would tie food to a string, then insert it through the hole in the trader’s stomach. Every few hours, Beaumont would remove the food to observe how it had been digested. Though gruesome, Beaumont’s experiments led to the worldwide acceptance that digestion was a chemical, not a mechanical, process.

23. Electroshock Therapy on Children

Image Source In the 1960s, Dr. Lauretta Bender of New York’s Creedmoor Hospital began what she believed to be a revolutionary treatment for children with social issues — electroshock therapy. Bender’s methods included interviewing and analyzing a sensitive child in front of a large group, then applying a gentle amount of pressure to the child’s head. Supposedly, any child who moved with the pressure was showing early signs of schizophrenia. Herself the victim of a misunderstood childhood, Bender was said to be unsympathetic to the children in her care. By the time her treatments were shut down, Bender had used electroshock therapy on over 100 children, the youngest of whom was age three.

22. Project Artichoke

Image Source In the 1950s, the CIA’s Office of Scientific Intelligence ran a series of mind control projects in an attempt to answer the question “Can we get control of an individual to the point where he will do our bidding against his will and even against fundamental laws of nature?” One of these programs, Project Artichoke, studied hypnosis, forced morphine addiction, drug withdrawal, and the use of chemicals to incite amnesia in unwitting human subjects. Though the project was eventually shut down in the mid-1960s, the project opened the door to extensive research on the use of mind-control in field operations.

21. Hepatitis in Mentally Disabled Children

Image Source In the 1950s, Willowbrook State School, a New York state-run institution for mentally handicapped children, began experiencing outbreaks of hepatitis. Due to unsanitary conditions, it was virtually inevitable that these children would contract hepatitis. Dr. Saul Krugman, sent to investigate the outbreak, proposed an experiment that would assist in developing a vaccine. However, the experiment required deliberately infecting children with the disease. Though Krugman’s study was controversial from the start, critics were eventually silenced by the permission letters obtained from each child’s parents. In reality, offering one’s child to the experiment was oftentimes the only way to guarantee admittance into the overcrowded institution.

20. Operation Midnight Climax

Image Source Initially established in the 1950s as a sub-project of a CIA-sponsored, mind-control research program, Operation Midnight Climax sought to study the effects of LSD on individuals. In San Francisco and New York, unconsenting subjects were lured to safehouses by prostitutes on the CIA payroll, unknowingly given LSD and other mind-altering substances, and monitored from behind one-way glass. Though the safehouses were shut down in 1965, when it was discovered that the CIA was administering LSD to human subjects, Operation Midnight Climax was a theater for extensive research on sexual blackmail, surveillance technology, and the use of mind-altering drugs on field operations.

19. Study of Humans Accidentally Exposed to Fallout Radiation

Image Source The 1954 “Study of Response of Human Beings exposed to Significant Beta and Gamma Radiation due to Fall-out from High-Yield Weapons,” known better as Project 4.1, was a medical study conducted by the U.S. of residents of the Marshall Islands. When the Castle Bravo nuclear test resulted in a yield larger than originally expected, the government instituted a top secret study to “evaluate the severity of radiation injury” to those accidentally exposed. Though most sources agree the exposure was unintentional, many Marshallese believed Project 4.1 was planned before the Castle Bravo test. In all, 239 Marshallese were exposed to significant levels of radiation.

18. The Monster Study

Image Source In 1939, University of Iowa researchers Wendell Johnson and Mary Tudor conducted a stuttering experiment on 22 orphan children in Davenport, Iowa. The children were separated into two groups, the first of which received positive speech therapy where children were praised for speech fluency. In the second group, children received negative speech therapy and were belittled for every speech imperfection. Normal-speaking children in the second group developed speech problems which they then retained for the rest of their lives. Terrified by the news of human experiments conducted by the Nazis, Johnson and Tudor never published the results of their “Monster Study.”

17. Project MKUltra

Image Source Project MKUltra is the code name of a CIA-sponsored research operation that experimented in human behavioral engineering. From 1953 to 1973, the program employed various methodologies to manipulate the mental states of American and Canadian citizens. These unwitting human test subjects were plied with LSD and other mind-altering drugs, hypnosis, sensory deprivation, isolation, verbal and sexual abuse, and various forms of torture. Research occurred at universities, hospitals, prisons, and pharmaceutical companies. Though the project sought to develop “chemical […] materials capable of employment in clandestine operations,” Project MKUltra was ended by a Congress-commissioned investigation into CIA activities within the U.S.

16. Experiments on Newborns

Image Source In the 1960s, researchers at the University of California began an experiment to study changes in blood pressure and blood flow. The researchers used 113 newborns ranging in age from one hour to three days old as test subjects. In one experiment, a catheter was inserted through the umbilical arteries and into the aorta. The newborn’s feet were then immersed in ice water for the purpose of testing aortic pressure. In another experiment, up to 50 newborns were individually strapped onto a circumcision board, then tilted so that their blood rushed to their head and their blood pressure could be monitored.

15. The Aversion Project

Image Source In 1969, during South Africa’s detestable Apartheid era, thousands of homosexuals were handed over to the care of Dr. Aubrey Levin, an army colonel and psychologist convinced he could “cure” homosexuals. At the Voortrekkerhoogte military hospital near Pretoria, Levin used electroconvulsive aversion therapy to “reorientate” his patients. Electrodes were strapped to a patient’s upper arm with wires running to a dial calibrated from 1 to 10. Homosexual men were shown pictures of a naked man and encouraged to fantasize, at which point the patient was subjected to severe shocks. When Levin was warned that he would be named an abuser of human rights, he emigrated to Canada where he currently works at a teaching hospital.

14. Medical Experiments on Prison Inmates

Image Source Perhaps one benefit of being an inmate at California’s San Quentin prison is the easy access to acclaimed Bay Area doctors. But if that’s the case, then a downside is that these doctors also have easy access to inmates. From 1913 to 1951, Dr. Leo Stanley, chief surgeon at San Quentin, used prisoners as test subjects in a variety of bizarre medical experiments. Stanley’s experiments included sterilization and potential treatments for the Spanish Flu. In one particularly disturbing experiment, Stanley performed testicle transplants on living prisoners using testicles from executed prisoners and, in some cases, from goats and boars.

13. Sexual Reassignment

Image Source In 1965, Canadian David Peter Reimer was born biologically male. But at seven months old, his penis was accidentally destroyed during an unconventional circumcision by cauterization. John Money, a psychologist and proponent of the idea that gender is learned, convinced the Reimers that their son would be more likely to achieve a successful, functional sexual maturation as a girl. Though Money continued to report only success over the years, David’s own account insisted that he had never identified as female. He spent his childhood teased, ostracized, and seriously depressed. At age 38, David committed suicide by shooting himself in the head.

12. Effect of Radiation on Testicles

Image Source Between 1963 and 1973, dozens of Washington and Oregon prison inmates were used as test subjects in an experiment designed to test the effects of radiation on testicles. Bribed with cash and the suggestion of parole, 130 inmates willingly agreed to participate in the experiments conducted by the University of Washington on behalf of the U.S. government. In most cases, subjects were zapped with over 400 rads of radiation (the equivalent of 2,400 chest x-rays) in 10 minute intervals. However, it was much later that the inmates learned the experiments were far more dangerous than they had been told. In 2000, the former participants settled a $2.4 million class-action settlement from the University.

11. Stanford Prison Experiment

Image Source Conducted at Stanford University from August 14-20, 1971, the Stanford Prison Experiment was an investigation into the causes of conflict between military guards and prisoners. Twenty-four male students were chosen and randomly assigned roles of prisoners and guards. They were then situated in a specially-designed mock prison in the basement of the Stanford psychology building. Those subjects assigned to be guards enforced authoritarian measures and subjected the prisoners to psychological torture. Surprisingly, many of the prisoners accepted the abuses. Though the experiment exceeded the expectations of all of the researchers, it was abruptly ended after only six days.

10. Syphilis Experiments in Guatemala

Image Source From 1946 to 1948, the United States government, Guatemalan president Juan José Arévalo, and some Guatemalan health ministries, cooperated in a disturbing human experiment on unwitting Guatemalan citizens. Doctors deliberately infected soldiers, prostitutes, prisoners, and mental patients with syphilis and other sexually transmitted diseases in an attempt to track their untreated natural progression. Treated only with antibiotics, the experiment resulted in at least 30 documented deaths. In 2010, the United States made a formal apology to Guatemala for their involvement in these experiments.

9. Tuskegee Syphilis Study

Image Source In 1932, the U.S. Public Health Service began working with the Tuskegee Institute to track the natural progression of untreated syphilis. Six hundred poor, illiterate, male sharecroppers were found and hired in Macon County, Alabama. Of the 600 men, only 399 had previously contracted syphilis, and none were told they had a life threatening disease. Instead, they were told they were receiving free healthcare, meals, and burial insurance in exchange for participating. Even after Penicillin was proven an effective cure for syphilis in 1947, the study continued until 1972. In addition to the original subjects, victims of the study included wives who contracted the disease, and children born with congenital syphilis. In 1997, President Bill Clinton formally apologized to those affected by what is often called the “most infamous biomedical experiment in U.S. history.”

8. Milgram Experiment

In 1961, Stanley Milgram, a psychologist at Yale University, began a series of social psychology experiments that measured the willingness of test subjects to obey an authority figure. Conducted only three months after the start of the trial of German Nazi war criminal Adolf Eichmann, Milgram’s experiment sought to answer the question, “Could it be that Eichmann and his million accomplices in the Holocaust were just following orders?” In the experiment, two participants (one secretly an actor and one an unwitting test subject) were separated into two rooms where they could hear, but not see, each other. The test subject would then read a series of questions to the actor, punishing each wrong answer with an electric shock. Though many people would indicate their desire to stop the experiment, almost all subjects continued when they were told they would not be held responsible, or that there would not be any permanent damage.

7. Infected Mosquitos in Towns

In 1956 and 1957, the United States Army conducted a number of biological warfare experiments on the cities of Savannah, Georgia and Avon Park, Florida. In one such experiment, millions of infected mosquitos were released into the two cities, in order to see if the insects could spread yellow fever and dengue fever. Not surprisingly, hundreds of researchers contracted illnesses that included fevers, respiratory problems, stillbirths, encephalitis, and typhoid. In order to photograph the results of their experiments, Army researchers pretended to be public health workers. Several people died as a result of the research.

6. Human Experimentation in the Soviet Union

Beginning in 1921 and continuing for most of the 21st century, the Soviet Union employed poison laboratories known as Laboratory 1, Laboratory 12, and Kamera as covert research facilities of the secret police agencies. Prisoners from the Gulags were exposed to a number of deadly poisons, the purpose of which was to find a tasteless, odorless chemical that could not be detected post mortem. Tested poisons included mustard gas, ricin, digitoxin, and curare, among others. Men and women of varying ages and physical conditions were brought to the laboratories and given the poisons as “medication,” or part of a meal or drink.

5. Human Experimentation in North Korea

Image Source Several North Korean defectors have described witnessing disturbing cases of human experimentation. In one alleged experiment, 50 healthy women prisoners were given poisoned cabbage leaves — all 50 women were dead within 20 minutes. Other described experiments include the practice of surgery on prisoners without anesthesia, purposeful starvation, beating prisoners over the head before using the zombie-like victims for target practice, and chambers in which whole families are murdered with suffocation gas. It is said that each month, a black van known as “the crow” collects 40-50 people from a camp and takes them to an known location for experiments.

4. Nazi Human Experimentation

Image Source Over the course of the Third Reich and the Holocaust, Nazi Germany conducted a series of medical experiments on Jews, POWs, Romani, and other persecuted groups. The experiments were conducted in concentration camps, and in most cases resulted in death, disfigurement, or permanent disability. Especially disturbing experiments included attempts to genetically manipulate twins; bone, muscle, and nerve transplantation; exposure to diseases and chemical gasses; sterilization, and anything else the infamous Nazi doctors could think up. After the war, these crimes were tried as part of the Nuremberg Trial and ultimately led to the development of the Nuremberg Code of medical ethics.

3. Unit 731

Image Source From 1937 to 1945, the imperial Japanese Army developed a covert biological and chemical warfare research experiment called Unit 731. Based in the large city of Harbin, Unit 731 was responsible for some of the most atrocious war crimes in history. Chinese and Russian subjects — men, women, children, infants, the elderly, and pregnant women — were subjected to experiments which included the removal of organs from a live body, amputation for the study of blood loss, germ warfare attacks, and weapons testing. Some prisoners even had their stomachs surgically removed and their esophagus reattached to the intestines. Many of the scientists involved in Unit 731 rose to prominent careers in politics, academia, business, and medicine.

2. Radioactive Materials in Pregnant Women

Image Source Shortly after World War II, with the impending Cold War forefront on the minds of Americans, many medical researchers were preoccupied with the idea of radioactivity and chemical warfare. In an experiment at Vanderbilt University, 829 pregnant women were given “vitamin drinks” they were told would improve the health of their unborn babies. Instead, the drinks contained radioactive iron and the researchers were studying how quickly the radioisotope crossed into the placenta. At least seven of the babies later died from cancers and leukemia, and the women themselves experienced rashes, bruises, anemia, loss of hair and tooth, and cancer.

1. Mustard Gas Tested on American Military

Image Source In 1943, the U.S. Navy exposed its own sailors to mustard gas. Officially, the Navy was testing the effectiveness of new clothing and gas masks against the deadly gas that had proven so terrifying in the first World War. The worst of the experiments occurred at the Naval Research Laboratory in Washington. Seventeen and 18-year old boys were approached after eight weeks of boot camp and asked if they wanted to participate in an experiment that would help shorten the war. Only when the boys reached the Research Laboratory were they told the experiment involved mustard gas. The participants, almost all of whom suffered severe external and internal burns, were ignored by the Navy and, in some cases, threatened with the Espionage Act. In 1991, the reports were finally declassified and taken before Congress.

28. Prison Inmates as Test Subjects Henrietta Lacks 26. Project QKHILLTOP 25. Stateville Penitentiary Malaria Study Stateville Penitentiary Malaria Study: Primaquine 24. Emma Eckstein 23. Dr. William Beaumont Dr. William Beaumont 21. Electroshock Therapy on Children 21. Project Artichoke 20. Operation Midnight Climax 19. Study of Humans Accidentally Exposed to Fallout Radiation 18. The Monster Experiment 17. Project MKUltra 16. Experiments on Newborns 15. The Aversion Project 14. Medical Experiments on Prison Inmates 13. Sexual Reassignment 12. Effect of Radiation on Testicles 11. Stanford Prison Experiment 10. Syphilis Experiment in Guatemala 9. Tuskegee Syphilis Study 8. Milgram Experiment 7. Infected Mosquitos in Towns 6. Human Experimentation in the Soviet Union 5. Human Experimentation in North Korea 4. Nazi Human Experimentation 3. Unit 731 2. Radioactive Materials in Pregnant Women 1. Mustard Gas Tested on American Military

• Psychology Education
• Bachelors in Psychology
• Masters in Psychology
• Doctorate in Psychology
• Psychology Resources
• Psychology License
• Psychology Salary
• Psychology Career
• Psychology Major
• What is Psychology
• Up & Coming Programs
• Top 10 Up and Coming Undergraduate Psychology Programs in the South
• Top 10 Up and Coming Undergraduate Psychology Programs in the Midwest
• Top 10 Up and Coming Undergraduate Psychology Programs in the West
• Top 10 Up and Coming Undergraduate Psychology Programs in the East
• Best Psychology Degrees Scholarship Opportunity
• The Pursuit of Excellence in Psychology Scholarship is Now Closed
• Meet Gemma: Our First Psychology Scholarship Winner
• 50 Most Affordable Clinical Psychology Graduate Programs
• 50 Most Affordable Selective Small Colleges for a Psychology Degree
• The 50 Best Schools for Psychology: Undergraduate Edition
• 30 Great Small Colleges for a Counseling Degree (Bachelor’s)
• Top 10 Best Online Bachelors in Psychology Degree Programs
• Top 10 Online Child Psychology Degree Programs
• 10 Best Online Forensic Psychology Degree Programs
• Top 10 Online Master’s in Psychology Degree Programs
• Top 15 Most Affordable School Psychology Programs
• Top 20 Most Innovative Graduate Psychology Degree Programs
• Top 8 Online Sports Psychology Degree Programs
• Recent Posts
• Does Psychology Require Math? – Requirements for Psychology Majors
• 10 Classes You Will Take as a Psychology Major
• Top 15 Highest-Paying Jobs with a Master’s Degree in Psychology
• The Highest Paying Jobs with an Associate’s Degree in Psychology
• The Highest-Paying Jobs with a Bachelor’s in Psychology
• Should I Major in Psychology?
• How to Become a CBT Therapist
• What is a Social Psychologist?
• How to Become a Clinical Neuropsychologist
• MA vs. MS in Psychology: What’s the Difference?
• PsyD vs. PhD in Psychology: What’s the Difference?
• What Can You Do with a Master’s in Psychology?
• What Can You Do With A PhD in Psychology?
• Master’s in Child Psychology Guide
• Master’s in Counseling Psychology – A Beginner’s Guide
• Master’s in Forensic Psychology – A Beginner’s Guide
• 8 Reasons to Become a Marriage and Family Therapist
• What Do Domestic Violence & Abuse Counselors Do?
• What Training is Needed to Be a Psychologist for People of the LGBTQ Community?
• 15 Inspiring TED Talks on Intelligence and Critical Thinking
• The 30 Most Inspiring Personal Growth and Development Blogs
• 30 Most Prominent Psychologists on Twitter
• New Theory Discredits the Myth that Individuals with Asperger’s Syndrome Lack Empathy
• 10 Crazy Things Famous People Have Believed
• Psychology Infographics
• Top Infographics About Psychology
• The Birth Order Effect [Infographic]
• The Psychology of Dogs [Infographic]
• Can Going Green Improve Your Mental Health? [Infographic]
• Surprising Alternative Treatments for Mental Disorders [Infographic]
• What Can Humans Learn From Animals? [Infographic]

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• CAREER FEATURE
• 08 May 2024

Illuminating ‘the ugly side of science’: fresh incentives for reporting negative results

• Rachel Brazil 0

Rachel Brazil is a freelance journalist in London, UK.

You can also search for this author in PubMed   Google Scholar

The editor-in-chief of the Journal of Trial & Error , Sarahanne Field wants to publish the messy, null and negative results sitting in researchers’ file drawers. Credit: Sander Martens

Editor-in-chief Sarahanne Field describes herself and her team at the Journal of Trial & Error as wanting to highlight the “ugly side of science — the parts of the process that have gone wrong”.

She clarifies that the editorial board of the journal, which launched in 2020 , isn’t interested in papers in which “you did a shitty study and you found nothing. We’re interested in stuff that was done methodologically soundly, but still yielded a result that was unexpected.” These types of result — which do not prove a hypothesis or could yield unexplained outcomes — often simply go unpublished, explains Field, who is also an open-science researcher at the University of Groningen in the Netherlands. Along with Stefan Gaillard, one of the journal’s founders, she hopes to change that.

Calls for researchers to publish failed studies are not new. The ‘file-drawer problem’ — the stacks of unpublished, negative results that most researchers accumulate — was first described in 1979 by psychologist Robert Rosenthal . He argued that this leads to publication bias in the scientific record: the gap of missing unsuccessful results leads to overemphasis on the positive results that do get published.

Careers Collection: Publishing

Over the past 30 years, the proportion of negative results being published has decreased further. A 2012 study showed that, from 1990 to 2007, there was a 22% increase in positive conclusions in papers; by 2007, 85% of papers published had positive results 1 . “People fail to report [negative] results, because they know they won’t get published — and when people do attempt to publish them, they get rejected,” says Field. A 2022 survey of researchers in France in chemistry, physics, engineering and environmental sciences showed that, although 81% had produced relevant negative results and 75% were willing to publish them, only 12.5% had the opportunity to do so 2 .

One factor that is leading some researchers to revisit the problem is the growing use of predictive modelling using machine-learning tools in many fields. These tools are trained on large data sets that are often derived from published work, and scientists have found that the absence of negative data in the literature is hampering the process. Without a concerted effort to publish more negative results that artificial intelligence (AI) can be trained on, the promise of the technology could be stifled.

“Machine learning is changing how we think about data,” says chemist Keisuke Takahashi at Hokkaido University in Japan, who has brought the issue to the attention of the catalysis-research community . Scientists in the field have typically relied on a mixture of trial and error and serendipity in their experiments, but there is hope that AI could provide a new route for catalyst discovery. Takahashi and his colleagues mined data from 1,866 previous studies and patents to train a machine-learning model to predict the best catalyst for the reaction between methane and oxygen to form ethane and ethylene, both of which are important chemicals used in industry 3 . But, he says, “over the years, people have only collected the good data — if they fail, they don’t report it”. This led to a skewed model that, in some cases, enhanced the predicted performance of a material, rather than realistically assessing its properties.

Synthetic organic chemist Felix Strieth-Kalthoff found that published data were too heavily biased toward positive results to effectively train an AI model to optimize chemical reaction yields. Credit: Cindy Huang

Alongside the flawed training of AI models, the huge gap of negative results in the scientific record continues to be a problem across all disciplines. In areas such as psychology and medicine, publication bias is one factor exacerbating the ongoing reproducibility crisis — in which many published studies are impossible to replicate. Without sharing negative studies and data, researchers could be doomed to repeat work that led nowhere. Many scientists are calling for changes in academic culture and practice — be it the creation of repositories that include positive and negative data, new publication formats or conferences aimed at discussing failure. The solutions are varied, but the message is the same: “To convey an accurate picture of the scientific process, then at least one of the components should be communicating all the results, [including] some negative results,” says Gaillard, “and even where you don’t end up with results, where it just goes wrong.”

Science’s messy side

Synthetic organic chemist Felix Strieth-Kalthoff, who is now setting up his own laboratory at the University of Wuppertal, Germany, has encountered positive-result bias when using data-driven approaches to optimize the yields of certain medicinal-chemistry reactions. His PhD work with chemist Frank Glorius at the University of Münster, Germany, involved creating models that could predict which reactants and conditions would maximize yields. Initially, he relied on data sets that he had generated from high-throughput experiments in the lab, which included results from both high- and low-yield reactions, to train his AI model. “Our next logical step was to do that based on the literature,” says Strieth-Kalthoff. This would allow him to curate a much larger data set to be used for training.

But when he incorporated real data from the reactions database Reaxys into the training process, he says, “[it] turned out they don’t really work at all”. Strieth-Kalthoff concluded the errors were due the lack of low-yield reactions 4 ; “All of the data that we see in the literature have average yields of 60–80%.” Without learning from the messy ‘failed’ experiments with low yields that were present in the initial real-life data, the AI could not model realistic reaction outcomes.

Although AI has the potential to spot relationships in complex data that a researcher might not see, encountering negative results can give experimentalists a gut feeling, says molecular modeller Berend Smit at the Swiss Federal Institute of Technology Lausanne. The usual failures that every chemist experiences at the bench give them a ‘chemical intuition’ that AI models trained only on successful data lack.

Smit and his team attempted to embed something similar to this human intuition into a model tasked with designing a metal-organic framework (MOF) with the largest known surface area for this type of material. A large surface area allows these porous materials to be used as reaction supports or molecular storage reservoirs. “If the binding [between components] is too strong, it becomes amorphous; if the binding is too weak, it becomes unstable, so you need to find the sweet spot,” Smit says. He showed that training the machine-learning model on both successful and unsuccessful reaction conditions created better predictions and ultimately led to one that successfully optimized the MOF 5 . “When we saw the results, we thought, ‘Wow, this is the chemical intuition we’re talking about!’” he says.

According to Strieth-Kalthoff, AI models are currently limited because “the data that are out there just do not reflect all of our knowledge”. Some researchers have sought statistical solutions to fill the negative-data gap. Techniques include oversampling, which means supplementing data with several copies of existing negative data or creating artificial data points, for example by including reactions with a yield of zero. But, he says, these types of approach can introduce their own biases.

Computer scientist Ella Peltonen helped to organize the first International Workshop on Negative Results in Pervasive Computing in 2022 to give researchers an opportunity to discuss failed experiments. Credit: University of Oulu

Capturing more negative data is now a priority for Takahashi. “We definitely need some sort of infrastructure to share the data freely.” His group has created a website for sharing large amounts of experimental data for catalysis reactions . Other organizations are trying to collect and publish negative data — but Takahashi says that, so far, they lack coordination, so data formats aren’t standardized. In his field, Strieth-Kalthoff says, there are initiatives such as the Open Reaction Database , launched in 2021 to share organic-reaction data and enable training of machine-learning applications. But, he says, “right now, nobody’s using it, [because] there’s no incentive”.

Smit has argued for a modular open-science platform that would directly link to electronic lab notebooks to help to make different data types extractable and reusable . Through this process, publication of negative data in peer-reviewed journals could be skipped, but the information would still be available for researchers to use in AI training. Strieth-Kalthoff agrees with this strategy in theory, but thinks it’s a long way off in practice, because it would require analytical instruments to be coupled to a third-party source to automatically collect data — which instrument manufacturers might not agree to, he says.

Publishing the non-positive

In other disciplines, the emphasis is still on peer-reviewed journals that will publish negative results. Gaillard, a science-studies PhD student at Radboud University in Nijmegen, the Netherlands, co-founded the Journal of Trial & Error after attending talks on how science can be made more open. Gaillard says that, although everyone whom they approached liked the idea of the journal, nobody wanted to submit articles at first. He and the founding editorial team embarked on a campaign involving cold calls and publicity at open-science conferences. “Slowly, we started getting our first submissions, and now we just get people sending things in [unsolicited],” he says. Most years the journal publishes one issue of about 8–14 articles, and it is starting to publish more special issues. It focuses mainly on the life sciences and data-based social sciences.

In 2008, David Alcantara, then a chemistry PhD student at the University of Seville in Spain who was frustrated by the lack of platforms for sharing negative results, set up The All Results journals, which were aimed at disseminating results regardless of the outcome . Of the four disciplines included at launch, only the biology journal is still being published. “Attracting submissions has always posed a challenge,” says Alcantara, now president at the consultancy and training organization the Society for the Improvement of Science in Seville.

But Alcantara thinks there has been a shift in attitudes: “More established journals [are] becoming increasingly open to considering negative results for publication.” Gaillard agrees: “I’ve seen more and more journals, like PLoS ONE , for example, that explicitly mentioned that they also publish negative results.” ( Nature welcomes submissions of replication studies and those that include null results, as described in this 2020 editorial .)

Journals might be changing their publication preferences, but there are still significant disincentives that stop researchers from publishing their file-drawer studies. “The current academic system often prioritizes high-impact publications and ground-breaking discoveries for career advancement, grants and tenure,” says Alcantara, noting that negative results are perceived as contributing little to nothing to these endeavours. Plus, there is still a stigma associated with any kind of failure . “People are afraid that this will look negative on their CV,” says Gaillard. Smit describes reporting failed experiments as a no-win situation: “It’s more work for [researchers], and they don’t get anything in return in the short term.” And, jokes Smit, what’s worse is that they could be providing data for an AI tool to take over their role.

Ultimately, most researchers conclude that publishing their failed studies and negative data is just not worth the time and effort — and there’s evidence that they judge others’ negative research more harshly than positive outcomes. In a study published in August, 500 researchers from top economics departments around the world were randomized to two groups and asked to judge a hypothetical research paper. Half of the participants were told that the study had a null conclusion, and the other half were told the results were sizeably significant. The null results were perceived to be 25% less likely to be published, of lower quality and less important than were the statistically significant findings 6 .

Some researchers have had positive experiences sharing their unsuccessful findings. For example, in 2021, psychologist Wendy Ross at the London Metropolitan University published her negative results from testing a hypothesis about human problem-solving in the Journal of Trial & Error 7 , and says the paper was “the best one I have published to date”. She adds, “Understanding the reasons for null results can really test and expand our theoretical understanding.”

Fields forging solutions

The field of psychology has introduced one innovation that could change publication biases — registered reports (RRs). These peer-reviewed reports , first published in 2014, came about largely as a response to psychology’s replication crisis, which began in around 2011. RRs set out the methodology of a study before the results are known, to try to prevent selective reporting of positive results. Daniël Lakens, who studies science-reward structures at Eindhoven University of Technology in the Netherlands, says there is evidence that RRs increase the proportion of negative results in the psychology literature.

In a 2021 study, Lakens analysed the proportion of published RRs whose results eventually support the primary hypothesis. In a random sample of hypothesis-testing studies from the standard psychology literature, 96% of the results were positive. In RRs, this fell to only 44% 8 . Lakens says the study shows “that if you offer this as an option, many more null results enter the scientific literature, and that is a desirable thing”. At least 300 journals, including Nature , are now accepting RRs, and the format is spreading to journals in biology, medicine and some social-science fields.

Yet another approach has emerged from the field of pervasive computing, the study of how computer systems are integrated into physical surroundings and everyday life. About four years ago, members of the community started discussing reproducibility, says computer scientist Ella Peltonen at the University of Oulu in Finland. Peltonen says that researchers realized that, to avoid the repetition of mistakes, there was a need to discuss the practical problems with studies and failed results that don’t get published. So in 2022, Peltonen and her colleagues held the first virtual International Workshop on Negative Results in Pervasive Computing (PerFail) , in conjunction with the field’s annual conference, the International Conference on Pervasive Computing and Communications.

Peltonen explains that PerFail speakers first present their negative results and then have the same amount of time for discussion afterwards, during which participants tease out how failed studies can inform future work. “It also encourages the community to showcase that things require effort and trial and error, and there is value in that,” she adds. Now an annual event, the organizers invite students to attend so they can see that failure is a part of research and that “you are not a bad researcher because you fail”, says Peltonen.

In the long run, Alcantara thinks a continued effort to persuade scientists to share all their results needs to be coupled with policies at funding agencies and journals that reward full transparency. “Criteria for grants, promotions and tenure should recognize the value of comprehensive research dissemination, including failures and negative outcomes,” he says. Lakens thinks funders could be key to boosting the RR format, as well. Funders, he adds, should say, “We want the research that we’re funding to appear in the scientific literature, regardless of the significance of the finding.”

There are some positive signs of change about sharing negative data: “Early-career researchers and the next generation of scientists are particularly receptive to the idea,” says Alcantara. Gaillard is also optimistic, given the increased interest in his journal, including submissions for an upcoming special issue on mistakes in the medical domain. “It is slow, of course, but science is a bit slow.”

doi: https://doi.org/10.1038/d41586-024-01389-7

Fanelli, D. Scientometrics 90 , 891–904 (2012).

Article   Google Scholar  

Herbet, M.-E., Leonard, J., Santangelo, M. G. & Albaret, L. Learned Publishing 35 , 16–29 (2022).

Fujima, J., Tanaka, Y., Miyazato, I., Takahashi, L. & Takahashi, K. Reaction Chem. Eng. 5 , 903–911 (2020).

Strieth-Kalthoff, F. et al. Angew. Chem. Int. Edn 61 , e202204647 (2022).

Moosavi, S. M. et al. Nature Commun. 10 , 539 (2019).

Article   PubMed   Google Scholar  

Chopra, F., Haaland, I., Roth, C. & Stegmann, A. Econ. J. 134 , 193–219 (2024).

Ross, W. & Vallée-Tourangeau, F. J. Trial Error https://doi.org/10.36850/e4 (2021).

Scheel, A. M., Schijen, M. R. M. J. & Lakens, D. Adv. Methods Pract. Psychol. Sci . https://doi.org/10.1177/25152459211007467 (2021).

Download references

Related Articles

• Scientific community

How I fled bombed Aleppo to continue my career in science

Career Feature 08 MAY 24

Hunger on campus: why US PhD students are fighting over food

Career Feature 03 MAY 24

US National Academies report outlines barriers and solutions for scientist carers

Career News 02 MAY 24

Mount Etna’s spectacular smoke rings and more — April’s best science images

News 03 MAY 24

Plagiarism in peer-review reports could be the ‘tip of the iceberg’

Nature Index 01 MAY 24

How reliable is this research? Tool flags papers discussed on PubPeer

News 29 APR 24

Algorithm ranks peer reviewers by reputation — but critics warn of bias

Nature Index 25 APR 24

Researchers want a ‘nutrition label’ for academic-paper facts

Nature Index 17 APR 24

Assistant/Associate Professor in Sustainable Biobased Products Manufacturing

Lubbock, Texas

Texas Tech University

Professor and Center for Vector-borne and Zoonotic Diseases Director

Assistant scientist/professor in rare disease research, sanford research.

Assistant Scientist/Professor in Rare Disease Research, Sanford Research Sanford Research invites applications for full-time faculty at the rank of...

Sioux Falls, South Dakota

Sanford Research

Postdoctoral Fellow - Boyi Gan lab

New postdoctoral positions are open in a cancer research laboratory located within The University of Texas MD Anderson Cancer Center. The lab curre...

Houston, Texas (US)

The University of Texas MD Anderson Cancer Center - Experimental Radiation Oncology

Assistant Professor

Tenure-track Assistant Professor position in the Cell and Molecular Physiology Department at Loyola University Chicago Stritch School of Medicine.

Maywood, Illinois

Loyola University of Chicago - Cell and Molecular Physiology Department

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

• Skip to main content
• Keyboard shortcuts for audio player

• Your Health
• Treatments & Tests
• Health Inc.
• Public Health

Scientists welcome new rules on marijuana, but research will still face obstacles

Rhitu Chatterjee

For decades, researchers in the U.S. had to use only marijuana grown at a facility located in Oxford, Mississippi. A few other approved growers have been added in recent years. Brad Horrigan/Hartford Courant/Tribune News Service via Getty Images hide caption

For decades, researchers in the U.S. had to use only marijuana grown at a facility located in Oxford, Mississippi. A few other approved growers have been added in recent years.

As the Biden administration moves to reclassify marijuana as a less dangerous drug, scientists say the change will lift some of the restrictions on studying the drug.

But the change won't lift all restrictions, they say, neither will it decrease potential risks of the drug or help users better understand what those risks are.

Marijuana is currently classified as a Schedule I controlled substance , which is defined as a substance with no accepted medical use and a high potential for abuse. The Biden administration proposed this week to classify cannabis as a Schedule III controlled substance, a category that acknowledges it has some medical benefits.

The current Schedule I status imposes many regulations and restrictions on scientists' ability to study weed, even as state laws have made it increasingly available to the public.

"Cannabis as a Schedule I substance is associated with a number of very, very restrictive regulations," says neuroscientist Staci Gruber at McLean Hospital and Harvard Medical School. "You have very stringent requirements, for example, for storage and security and reporting all of these things."

These requirements are set by the Food and Drug Administration, the Drug Enforcement Administration, the Institutional Review Board and local authorities, she says. Scientists interested in studying the drug also have to register with the DEA and get a state and federal license to conduct research on the drug.

"It's a burdensome process and it is certainly a process that has prevented a number of young and rather invested researchers from pursuing [this kind of work]," says Gruber.

Reclassifying the drug as Schedule III puts it in the same category as ketamine and Tylenol with codeine. Substances in this category have accepted medical use in the United States, have less potential for abuse than in higher categories and abuse could lead to low to moderate levels of dependence on the drug.

This reclassification is "a very, very big paradigm shift," says Gruber. "I think that has a big trickle down effect in terms of the perspectives and the attitudes with regard to the actual sort of differences between studying Schedule III versus Schedule I substances."

Gruber welcomes the change, particularly for what it will mean for younger colleagues. "For researchers who are looking to get into the game, it will be easier. You don't have to have a Schedule I license," she says. "That's a big deal."

The rescheduling of cannabis will also "translate to more research on the benefits and risks of cannabis for the treatment of medical conditions," writes Dr. Andrew Monte in an email. He is associate director of Rocky Mountain Poison and Drug Safety and an emergency physician and toxicologist at the University of Colorado School of Medicine.

"This will also help improve the quality of the research since more researchers will be able to contribute," he adds.

Senate Democrats hold a press conference on Wednesday pitching new, less strict marijuana laws. From left are Senators Cory Booker of N.J., Majority Leader Chuck Schumer of N.Y., and Ron Wyden of Oregon. Tom Williams/CQ-Roll Call, Inc via Getty Imag hide caption

Senate Democrats hold a press conference on Wednesday pitching new, less strict marijuana laws. From left are Senators Cory Booker of N.J., Majority Leader Chuck Schumer of N.Y., and Ron Wyden of Oregon.

But the change in classification won't significantly expand the number of sources for the drug for researchers, says Gruber. For 50 years, researchers were allowed to use cannabis from only one source – a facility at the University of Mississippi. Then, in 2021, the DEA started to add a few more companies to that list of approved sources for medical and scientific research.

While she expects more sources to be added in time, she and many of the researchers she knows have yet to benefit from the recently added sources, as most have limited products available.

"And what we haven't seen is any ability for researchers –cannabis researchers, clinical researchers – to have the ability to study products that our patients and our recreational consumers or adult consumers are actually using," she adds. "That remains impossible."

Shots - Health News

Rare and mysterious vomiting illness linked to heavy marijuana use.

There is very little known information about what is in cannabis products on the market today. Some studies show that the level of THC, the main intoxicant in marijuana, being sold to consumers today is significantly higher than what was available decades ago, and high THC levels are known to pose more health risks.

And Monte cautions that the reclassification itself doesn't mean that cannabis has no health risks. Monte and his colleagues have been documenting some of those risks in Colorado by studying people who show up in the emergency room after consuming cannabis. Intoxication and cyclical vomiting ( cannabinoid hyperemesis syndrome ) and alarming psychiatric symptoms such as psychosis are among the top problems bringing some marijuana users to the hospital.

Research on cannabis has been lacking surveillance of these kinds of impacts for decades, he says. And rescheduling the drug will not fill that "gaping hole in risk surveillance," he writes.

• drug policy
• legal marijuana
• Scientific research
• biden administration

Appointments at Mayo Clinic

• Weight loss

Exercise for weight loss: Calories burned in 1 hour

Being active can help you lose weight and keep it off. Find out how much you need.

Being active is vital to losing weight and keeping it off. When active, the body uses more energy in the form of calories. And burning more calories than you take in leads to weight loss.

To lose weight, most people need to cut the number of calories they eat and move more. This is according to the 2020-2025 Dietary Guidelines for Americans. Most often, that means cutting daily calories by 500 to 750 to lose 1 1/2 pounds (0.7 kilograms) a week.

Other factors might be involved in losing weight. Because of changes to the body over time, you might need to cut calories more as you age to keep losing weight or to stay at the same weight.

Diet or exercise: Does one matter more?

Both are important. Diet affects weight loss more than physical activity does. Physical activity, including exercise, has a stronger effect in keeping weight from coming back after weight loss.

Losing weight with diet alone and without physical activity can make people weaker. This is because of age-related losses in bone density and muscle mass. Adding resistance training and aerobic exercise to a weight-loss program helps prevent the loss of bone and muscle.

These are the exercise guidelines for most healthy adults from the U.S. Department of Health and Human Services:

Aerobic activity. Get at least 150 minutes of moderate aerobic activity a week. Or get 75 minutes of vigorous aerobic activity a week. You also can get an equal mix of the two types.

Aim to exercise most days of the week. For even more health benefits, strive for 300 minutes a week or more of moderate aerobic activity or 150 minutes of vigorous activity. Exercising this much may help with weight loss or keeping off lost weight. But even small amounts of physical activity can be helpful. Being active for short periods of time during the day can add up and give you great health benefits.

• Strength training. Do strength training exercises for all major muscle groups at least two times a week. One set of each exercise is enough for health and fitness benefits. Use a weight or resistance level heavy enough to tire your muscles after about 12 to 15 repetitions.

Moderate aerobic exercise includes activities such as brisk walking, biking, swimming and mowing the lawn.

Vigorous aerobic exercise includes activities such as running, swimming hard laps, heavy yardwork and aerobic dancing.

Strength training can include use of weights or weight machines, your own body weight, resistance tubing, or activities such as rock climbing.

How much am I burning?

This list shows about how many calories are burned while doing certain exercises for one hour. This is based on a person who weighs 160 pounds (73 kilograms). The calories you burn depend on the exercise you do, how hard you do it, how much you weigh and other factors.

Based on Ainsworth BE, et al. 2011 compendium of physical activities: A second update of codes and MET values. Medicine & Science in Sports & Exercise. 2011;43:1575.

Remember, to lose weight or to keep weight from creeping up on you as you age, you need to eat less and move more. Moving more means adding more physical activity into your life.

There is a problem with information submitted for this request. Review/update the information highlighted below and resubmit the form.

From Mayo Clinic to your inbox

Sign up for free and stay up to date on research advancements, health tips, current health topics, and expertise on managing health. Click here for an email preview.

Error Email field is required

Error Include a valid email address

To provide you with the most relevant and helpful information, and understand which information is beneficial, we may combine your email and website usage information with other information we have about you. If you are a Mayo Clinic patient, this could include protected health information. If we combine this information with your protected health information, we will treat all of that information as protected health information and will only use or disclose that information as set forth in our notice of privacy practices. You may opt-out of email communications at any time by clicking on the unsubscribe link in the e-mail.

Thank you for subscribing!

You'll soon start receiving the latest Mayo Clinic health information you requested in your inbox.

Sorry something went wrong with your subscription

Please, try again in a couple of minutes

• Physical Activity Guidelines for Americans. 2nd ed. U.S. Department of Health and Human Services. https://health.gov/paguidelines/second-edition. Accessed March 13, 2024.
• Physical activity for a healthy weight. Centers for Disease Control and Prevention. https://www.cdc.gov/healthyweight/physical_activity/index.html. Accessed March 13, 2024.
• Ainsworth BE, et al. 2011 compendium of physical activities: A second update of codes and MET values. Medicine & Science in Sports & Exercise. 2011;43:1575.
• 2020-2025 Dietary Guidelines for Americans. U.S. Department of Health and Human Services and U.S. Department of Agriculture. https://www.dietaryguidelines.gov. Accessed March 13, 2024.
• Perreault L, et al. Obesity in adults: Role of physical activity and exercise. https://www.uptodate.com/contents/search. Accessed March 13, 2024.
• AskMayoExpert. Physical activity (adult). Mayo Clinic; 2022.

Products and Services

• A Book: The Mayo Clinic Diet Bundle
• The Mayo Clinic Diet Online
• Aerobic exercise
• Hate to exercise? Try these tips
• Strength training basics
• Walking for weight loss
• Walking for fitness

Mayo Clinic does not endorse companies or products. Advertising revenue supports our not-for-profit mission.

• Opportunities

Mayo Clinic Press

Check out these best-sellers and special offers on books and newsletters from Mayo Clinic Press .

• Mayo Clinic on Incontinence - Mayo Clinic Press Mayo Clinic on Incontinence
• The Essential Diabetes Book - Mayo Clinic Press The Essential Diabetes Book
• Mayo Clinic on Hearing and Balance - Mayo Clinic Press Mayo Clinic on Hearing and Balance
• FREE Mayo Clinic Diet Assessment - Mayo Clinic Press FREE Mayo Clinic Diet Assessment
• Mayo Clinic Health Letter - FREE book - Mayo Clinic Press Mayo Clinic Health Letter - FREE book
• Healthy Lifestyle
• Exercise for weight loss Calories burned in 1 hour

Your gift holds great power – donate today!

Make your tax-deductible gift and be a part of the cutting-edge research and care that's changing medicine.