research findings in education

• Our Mission

The 10 Most Significant Education Studies of 2021

From reframing our notion of “good” schools to mining the magic of expert teachers, here’s a curated list of must-read research from 2021.

It was a year of unprecedented hardship for teachers and school leaders. We pored through hundreds of studies to see if we could follow the trail of exactly what happened: The research revealed a complex portrait of a grueling year during which persistent issues of burnout and mental and physical health impacted millions of educators. Meanwhile, many of the old debates continued: Does paper beat digital? Is project-based learning as effective as direct instruction? How do you define what a “good” school is?

Other studies grabbed our attention, and in a few cases, made headlines. Researchers from the University of Chicago and Columbia University turned artificial intelligence loose on some 1,130 award-winning children’s books in search of invisible patterns of bias. (Spoiler alert: They found some.) Another study revealed why many parents are reluctant to support social and emotional learning in schools—and provided hints about how educators can flip the script.

1. What Parents Fear About SEL (and How to Change Their Minds)

When researchers at the Fordham Institute asked parents to rank phrases associated with social and emotional learning , nothing seemed to add up. The term “social-emotional learning” was very unpopular; parents wanted to steer their kids clear of it. But when the researchers added a simple clause, forming a new phrase—”social-emotional & academic learning”—the program shot all the way up to No. 2 in the rankings.

What gives?

Parents were picking up subtle cues in the list of SEL-related terms that irked or worried them, the researchers suggest. Phrases like “soft skills” and “growth mindset” felt “nebulous” and devoid of academic content. For some, the language felt suspiciously like “code for liberal indoctrination.”

But the study suggests that parents might need the simplest of reassurances to break through the political noise. Removing the jargon, focusing on productive phrases like “life skills,” and relentlessly connecting SEL to academic progress puts parents at ease—and seems to save social and emotional learning in the process.

2. The Secret Management Techniques of Expert Teachers

In the hands of experienced teachers, classroom management can seem almost invisible: Subtle techniques are quietly at work behind the scenes, with students falling into orderly routines and engaging in rigorous academic tasks almost as if by magic. 

That’s no accident, according to new research . While outbursts are inevitable in school settings, expert teachers seed their classrooms with proactive, relationship-building strategies that often prevent misbehavior before it erupts. They also approach discipline more holistically than their less-experienced counterparts, consistently reframing misbehavior in the broader context of how lessons can be more engaging, or how clearly they communicate expectations.

Focusing on the underlying dynamics of classroom behavior—and not on surface-level disruptions—means that expert teachers often look the other way at all the right times, too. Rather than rise to the bait of a minor breach in etiquette, a common mistake of new teachers, they tend to play the long game, asking questions about the origins of misbehavior, deftly navigating the terrain between discipline and student autonomy, and opting to confront misconduct privately when possible.

3. The Surprising Power of Pretesting

Asking students to take a practice test before they’ve even encountered the material may seem like a waste of time—after all, they’d just be guessing.

But new research concludes that the approach, called pretesting, is actually more effective than other typical study strategies. Surprisingly, pretesting even beat out taking practice tests after learning the material, a proven strategy endorsed by cognitive scientists and educators alike. In the study, students who took a practice test before learning the material outperformed their peers who studied more traditionally by 49 percent on a follow-up test, while outperforming students who took practice tests after studying the material by 27 percent.

The researchers hypothesize that the “generation of errors” was a key to the strategy’s success, spurring student curiosity and priming them to “search for the correct answers” when they finally explored the new material—and adding grist to a 2018 study that found that making educated guesses helped students connect background knowledge to new material.

Learning is more durable when students do the hard work of correcting misconceptions, the research suggests, reminding us yet again that being wrong is an important milestone on the road to being right.

4. Confronting an Old Myth About Immigrant Students

Immigrant students are sometimes portrayed as a costly expense to the education system, but new research is systematically dismantling that myth.

In a 2021 study , researchers analyzed over 1.3 million academic and birth records for students in Florida communities, and concluded that the presence of immigrant students actually has “a positive effect on the academic achievement of U.S.-born students,” raising test scores as the size of the immigrant school population increases. The benefits were especially powerful for low-income students.

While immigrants initially “face challenges in assimilation that may require additional school resources,” the researchers concluded, hard work and resilience may allow them to excel and thus “positively affect exposed U.S.-born students’ attitudes and behavior.” But according to teacher Larry Ferlazzo, the improvements might stem from the fact that having English language learners in classes improves pedagogy , pushing teachers to consider “issues like prior knowledge, scaffolding, and maximizing accessibility.”

5. A Fuller Picture of What a ‘Good’ School Is

It’s time to rethink our definition of what a “good school” is, researchers assert in a study published in late 2020.⁣ That’s because typical measures of school quality like test scores often provide an incomplete and misleading picture, the researchers found.

The study looked at over 150,000 ninth-grade students who attended Chicago public schools and concluded that emphasizing the social and emotional dimensions of learning—relationship-building, a sense of belonging, and resilience, for example—improves high school graduation and college matriculation rates for both high- and low-income students, beating out schools that focus primarily on improving test scores.⁣

“Schools that promote socio-emotional development actually have a really big positive impact on kids,” said lead researcher C. Kirabo Jackson in an interview with Edutopia . “And these impacts are particularly large for vulnerable student populations who don’t tend to do very well in the education system.”

The findings reinforce the importance of a holistic approach to measuring student progress, and are a reminder that schools—and teachers—can influence students in ways that are difficult to measure, and may only materialize well into the future.⁣

6. Teaching Is Learning

One of the best ways to learn a concept is to teach it to someone else. But do you actually have to step into the shoes of a teacher, or does the mere expectation of teaching do the trick?

In a 2021 study , researchers split students into two groups and gave them each a science passage about the Doppler effect—a phenomenon associated with sound and light waves that explains the gradual change in tone and pitch as a car races off into the distance, for example. One group studied the text as preparation for a test; the other was told that they’d be teaching the material to another student.

The researchers never carried out the second half of the activity—students read the passages but never taught the lesson. All of the participants were then tested on their factual recall of the Doppler effect, and their ability to draw deeper conclusions from the reading.

The upshot? Students who prepared to teach outperformed their counterparts in both duration and depth of learning, scoring 9 percent higher on factual recall a week after the lessons concluded, and 24 percent higher on their ability to make inferences. The research suggests that asking students to prepare to teach something—or encouraging them to think “could I teach this to someone else?”—can significantly alter their learning trajectories.

7. A Disturbing Strain of Bias in Kids’ Books

Some of the most popular and well-regarded children’s books—Caldecott and Newbery honorees among them—persistently depict Black, Asian, and Hispanic characters with lighter skin, according to new research .

Using artificial intelligence, researchers combed through 1,130 children’s books written in the last century, comparing two sets of diverse children’s books—one a collection of popular books that garnered major literary awards, the other favored by identity-based awards. The software analyzed data on skin tone, race, age, and gender.

Among the findings: While more characters with darker skin color begin to appear over time, the most popular books—those most frequently checked out of libraries and lining classroom bookshelves—continue to depict people of color in lighter skin tones. More insidiously, when adult characters are “moral or upstanding,” their skin color tends to appear lighter, the study’s lead author, Anjali Aduki,  told The 74 , with some books converting “Martin Luther King Jr.’s chocolate complexion to a light brown or beige.” Female characters, meanwhile, are often seen but not heard.

Cultural representations are a reflection of our values, the researchers conclude: “Inequality in representation, therefore, constitutes an explicit statement of inequality of value.”

8. The Never-Ending ‘Paper Versus Digital’ War

The argument goes like this: Digital screens turn reading into a cold and impersonal task; they’re good for information foraging, and not much more. “Real” books, meanwhile, have a heft and “tactility”  that make them intimate, enchanting—and irreplaceable.

But researchers have often found weak or equivocal evidence for the superiority of reading on paper. While a recent study concluded that paper books yielded better comprehension than e-books when many of the digital tools had been removed, the effect sizes were small. A 2021 meta-analysis further muddies the water: When digital and paper books are “mostly similar,” kids comprehend the print version more readily—but when enhancements like motion and sound “target the story content,” e-books generally have the edge.

Nostalgia is a force that every new technology must eventually confront. There’s plenty of evidence that writing with pen and paper encodes learning more deeply than typing. But new digital book formats come preloaded with powerful tools that allow readers to annotate, look up words, answer embedded questions, and share their thinking with other readers.

We may not be ready to admit it, but these are precisely the kinds of activities that drive deeper engagement, enhance comprehension, and leave us with a lasting memory of what we’ve read. The future of e-reading, despite the naysayers, remains promising.

9. New Research Makes a Powerful Case for PBL

Many classrooms today still look like they did 100 years ago, when students were preparing for factory jobs. But the world’s moved on: Modern careers demand a more sophisticated set of skills—collaboration, advanced problem-solving, and creativity, for example—and those can be difficult to teach in classrooms that rarely give students the time and space to develop those competencies.

Project-based learning (PBL) would seem like an ideal solution. But critics say PBL places too much responsibility on novice learners, ignoring the evidence about the effectiveness of direct instruction and ultimately undermining subject fluency. Advocates counter that student-centered learning and direct instruction can and should coexist in classrooms.

Now two new large-scale studies —encompassing over 6,000 students in 114 diverse schools across the nation—provide evidence that a well-structured, project-based approach boosts learning for a wide range of students.

In the studies, which were funded by Lucas Education Research, a sister division of Edutopia , elementary and high school students engaged in challenging projects that had them designing water systems for local farms, or creating toys using simple household objects to learn about gravity, friction, and force. Subsequent testing revealed notable learning gains—well above those experienced by students in traditional classrooms—and those gains seemed to raise all boats, persisting across socioeconomic class, race, and reading levels.

10. Tracking a Tumultuous Year for Teachers

The Covid-19 pandemic cast a long shadow over the lives of educators in 2021, according to a year’s worth of research.

The average teacher’s workload suddenly “spiked last spring,” wrote the Center for Reinventing Public Education in its January 2021 report, and then—in defiance of the laws of motion—simply never let up. By the fall, a RAND study recorded an astonishing shift in work habits: 24 percent of teachers reported that they were working 56 hours or more per week, compared to 5 percent pre-pandemic.

The vaccine was the promised land, but when it arrived nothing seemed to change. In an April 2021 survey  conducted four months after the first vaccine was administered in New York City, 92 percent of teachers said their jobs were more stressful than prior to the pandemic, up from 81 percent in an earlier survey.

It wasn’t just the length of the work days; a close look at the research reveals that the school system’s failure to adjust expectations was ruinous. It seemed to start with the obligations of hybrid teaching, which surfaced in Edutopia ’s coverage of overseas school reopenings. In June 2020, well before many U.S. schools reopened, we reported that hybrid teaching was an emerging problem internationally, and warned that if the “model is to work well for any period of time,” schools must “recognize and seek to reduce the workload for teachers.” Almost eight months later, a 2021 RAND study identified hybrid teaching as a primary source of teacher stress in the U.S., easily outpacing factors like the health of a high-risk loved one.

New and ever-increasing demands for tech solutions put teachers on a knife’s edge. In several important 2021 studies, researchers concluded that teachers were being pushed to adopt new technology without the “resources and equipment necessary for its correct didactic use.” Consequently, they were spending more than 20 hours a week adapting lessons for online use, and experiencing an unprecedented erosion of the boundaries between their work and home lives, leading to an unsustainable “always on” mentality. When it seemed like nothing more could be piled on—when all of the lights were blinking red—the federal government restarted standardized testing .

Change will be hard; many of the pathologies that exist in the system now predate the pandemic. But creating strict school policies that separate work from rest, eliminating the adoption of new tech tools without proper supports, distributing surveys regularly to gauge teacher well-being, and above all listening to educators to identify and confront emerging problems might be a good place to start, if the research can be believed.

Classroom Q&A

With larry ferlazzo.

In this EdWeek blog, an experiment in knowledge-gathering, Ferlazzo will address readers’ questions on classroom management, ELL instruction, lesson planning, and other issues facing teachers. Send your questions to [email protected]. Read more from this blog.

What Are the Most Important Education Research Findings in the Past 10 Years?

• Share article

(This is the first post in a two-part series.)

The new question-of-the-week is:

What do you think have been the most important education research findings from the past 10 years, and what areas are you hoping researchers focus on in the next 10 years?

There is so much education research out there, and much of it is inaccessible to K-12 teachers either because it’s written in arcane academic language or because it’s locked behind paywalls.

This series will try to highlight some of the most important findings that we teachers—and our students—can use.

Today, Beth M. Miller, Ph.D., and Jana Echevarria, Ph.D., share their reflections.

You might also be interested in many curated resources on ed. research at “Best” Lists o f the Week: Education Research .

Two ‘Streams’

Beth M. Miller, Ph.D., serves as the chief knowledge officer at EL Education. She leads the research, communications, and publications teams while mostly being in complete awe of the mad skills of her brilliant, compassionate, committed colleagues:

What happens in the learning process? Why do some students thrive at school and learn more than others, and why does this variation often reflect socially constructed racial and ethnic categories? In the last 10 years, two streams of research have vastly expanded our understanding of the answers to these complex but never-more-important questions.

Stream One: Research on How Students Learn

We now know, with greater clarity and evidence than ever, that learning is a social, emotional, and cognitive process. While early “brain research” findings were beginning to emerge 10 years ago (e.g., plasticity of the brain), in the past decade, this knowledge has converged in a growing science of learning and development (SoLD) with many important implications for instructional practices, school climate, and district policy.

Social-emotional learning (SEL) is deeply connected to academic achievement. We are increasingly learning that SEL can be developed in schools and that an integrated educational approach that deeply intertwines strands of social-emotional and academic development (versus teaching character as a siloed class on Tuesday mornings, for example) will be most effective.

Another key concept that has been developed through a body of evidence is the idea of mindset—how the student thinks of themself in relation to an idea or content will mediate their learning process and achievement. This insight from psychology, first developed by Carol Dweck, has resulted in a whole field of social psychology. Some of the short-term interventions have what seem like astounding results, because shifts in student mindset create a domino effect on motivation, self-efficacy, behavior, performance, and achievement.

For example, in several studies by David Yeager and his colleagues , teacher responses on a homework assignment communicating high expectations—and a belief that a student could reach these expectations—resulted in striking shifts in student academic performance over the course of a year. Teacher mindset also matters: When teachers who were trained on brain plasticity as it related to mathematics shifted their approach to teaching accordingly, doing so resulted in higher student achievement.

Stream Two: Research on the Impact of Racism in Education

Science of learning and development research can help to shift the dynamics of student experience and outcomes, but it is not enough to reach the goal we must attain: equitable learning opportunities and outcomes for all students. Another stream of research, less developed but equally imperative, is helping to uncover the ways that racism and other forms of marginalization create roadblocks to learning for millions of students and have throughout our history.

We can see this in the unequal financing of education between communities, the differences in teacher quality and facilities, and in the school experiences of millions of students. Despite the existence of brilliant students in every classroom and community, only some students will get the opportunity to develop to their full potential. In the last decade, research has highlighted how racism operates at every level of our education systems and, therefore, how to change it.

This body of research, often rooted in the theoretical work of scholars such as Gloria Ladson-Billings’ cconcept of “culturally relevant pedagogy” that she developed in the 1990s, includes ethnographic studies, correlational research, and quantitative large-scale studies, building a powerful body of evidence that racism and other forms of marginalization deeply and powerfully affect student achievement. Flipping the deficit-focused narrative of the “achievement gap” on its head, these researchers examine the resource gaps, opportunity gaps, racism, bias, and other processes and structures that drive differential experiences in school.

What we’ve learned might be a surprise to white people like me, but it only serves to expose the truth of what many people of color have experienced throughout their educational journey: Racism is deeply embedded in schools—by design, albeit often without conscious intention. Schools are a microcosm of our larger society. Without deep-seated, ongoing changes at multiple levels to shift that reality, racism remains a potent driver of school experiences and outcomes.

From research on the disproportionality of disciplinary practices to the impact on Black students of having even one Black teacher , we see racism—and other forms of marginalization—showing up anywhere we bring a lens to this study. We’ve learned a lot about the ways in which education policies, systems, and structures embed racism over the past decade. But that doesn’t mean individual teachers are off the hook: Multiple studies demonstrate the presence of negative perceptions and lower expectations of Black students on the part of many white teachers.

While deeply embedded policies and unconscious bias aren’t easy to shift, we are seeing evidence that it is not only possible to change these destructive dynamics, but also that this work significantly impacts student growth and learning. For example, a carefully designed training aimed at increasing teachers’ empathy for their students’ perspective by Jason Okonofua and colleagues shifted teachers from responding to behavior issues with punitive disciplinary practice to greater understanding and connection, leading to a 50 percent reduction in disciplinary actions. Other promising approaches, many rooted in culturally responsive education, from a community-center mathematics curriculum to the impact of ethnic - studies programs .

Where Do We Go From Here?

For the next 10 years, the most important work in education—whether in research studies or classrooms—will be in expanding the knowledge base where these two streams converge, i.e., combining what we know about how people learn, grow, and change with research that foregrounds the experiences and outcomes of historically marginalized students. After decades of education reforms that had little or no impact on the “stubborn” inequities in education, we have finally begun large-scale efforts to shift from measuring gaps to understanding why they exist and how we—not students—are the key to changing the dynamics. Some researchers, as well as organizations such as CASEL and the National Equity Project , are making progress, but we are in the early stages of this work. One thing we do know is that individual, incremental change will not create the equitable education system that our students deserve: Systemic changes in districts and charter networks will be needed, and we are only beginning the journey of creating the conditions at scale for all students to thrive.

One last note: We need to build on the current research base that demonstrates how disrupting racism benefits all students, including white students who will grow up in a diverse society. All students need the opportunity to experience what Rudine Sims Bishop coined “windows” as well as “mirrors” and deeply understand the multitude of experiences, histories, and perspectives we share in this country and around the world. Evidence that this learning matters—for all students—will help us create classrooms that enable us to build a better world.

English-Language Learners

Jana Echevarria, Ph.D., is professor emerita at California State University, Long Beach, where she was selected as Outstanding Professor. She is the co-developer of the SIOP Model of instruction for English-learners and the co-author of Making Content Comprehensible for English Learners: The SIOP Model and 99 Ideas and Activities for Teaching English Learners among other publications. Her blog is found at janaechevarria.com :

There are innumerable books, articles, and blogs written about what works with English-learners (ELs), but these resources don’t always reflect research-validated approaches and interventions. Empirical studies provide guidance for achieving desired outcomes that go beyond what intuitively seems like a good idea for teaching students in this population. The following areas of research are of particular importance in informing practice, especially for EL students.

Academic language . Cummins (1979) introduced the distinction between conversational language and academic language, and others more recently have discussed specific ways that academic language is challenging ( Scheppegrell, 2020 ), particularly for English - learners . Academic language is more formal and abstract than conversational language and uses complex sentence structure (e.g., embedded clauses and conjunctions), highly sophisticated, abstract vocabulary (e.g., representational democracy in social studies), and rhetorical forms (e.g., figurative language), and it is encountered almost exclusively in school.

Research has identified the critical relationship of academic language to reading comprehension, a cognitive and linguistic process needed to acquire and use knowledge in every academic-content area. As EL students become more proficient in English, they become more efficient readers and more similar to their English-speaking peers in their reading ability. Conversely, if EL students don’t become sufficiently proficient in English, they expend more cognitive effort, and their reading remains inefficient, which negatively affects achievement and motivation.

The importance of advancing academic-language development is clear. Findings verify that ELs don’t “pick up” academic language nor will the achievement gap close without explicit instruction in English-language development (ELD). A separate ELD time each day focusing on English-language instruction is critical but may not be sufficient for expediting English-language growth. In every content lesson, teaching key content vocabulary and exploiting teachable academic language-learning opportunities likely will enhance English proficiency.

Student assets . The idea that students come to school as empty vessels in need of filling has been dispelled. Indeed, students begin school with a minimum of five years of lived experiences, accumulated knowledge, and language development in their home language, and these continue to grow with each subsequent year. This treasure trove should be acknowledged and built upon as students learn academic content in school.

For English-learners, some lived experiences are culturally influenced, such as attending quinceañeras or receiving red envelopes as gifts, and others are common to their age group such as popular social media sites, video games, and sports. Linguistic knowledge in their home language can be used to bootstrap learning in English. Studies suggest that instructional routines that draw on students’ home language, their knowledge, and cultural assets support literacy development in English. Some examples of practices used in studies include previewing and reviewing materials in children’s home language, providing opportunities for students to engage in conversations around text with peers using their home language when needed, giving definitions for key vocabulary terms in both English and their home language, and introducing key concepts by connecting them to students’ knowledge or experience in the home and community.

Teachers who don’t speak the language of their students shouldn’t be apprehensive about using these types of practices. Many technologies assist in translating words and definitions, and peers can be used as supports by grouping students with a common home language together for discussions, then asking each group to summarize their discussion in English. Further, as teachers practice a dynamic interaction style with students, they will learn about students’ lived experiences which, in turn, can be used to connect lesson content to what students know and have experienced.

Capitalizing on students’ linguistic and experiential assets by linking them to content, materials, and activities has motivational and engagement benefits and contributes to EL students’ sense of belonging and well-being.

Reading foundations. Much has been written recently about the science of reading , a discussion that spans decades. However, little research specifically addresses English-learners and how teaching reading may or may not differ for this population. Goldenberg (2020) conducted a review of research on reading and English-learners. He summarizes the findings and draws several conclusions. First, learning to read is similar for English-learners and English-speaking students. EL students must learn the same foundational skills as English-proficient students. As Goldenberg says, “Full-fledged literacy certainly requires more, but there is a reason this group of skills is called foundational: It is required for the literacy edifice under construction. As with any building, if all you have is a foundation, you do not have much. Yet, a solid foundation is still essential” (p.133).

Secondly, along with foundational skills, additional supports are required for EL students so that instruction in English is made comprehensible to them. They need additional instruction in the vocabulary found in text, especially for beginning speakers who are learning to recognize new words as they are read. Also beneficial is additional repetition and rehearsal as well as opportunities to practice. Specifically, beginning readers need practice in developing oral language, primarily in the form of effective ELD instruction to boost English proficiency.

Lastly, as EL students advance through the grades, the academic language required to navigate grade-level texts and the disciplinary knowledge students need to comprehend texts become increasingly complex and demanding. Oral English-language instruction and support needs to match the level of challenge for these students, particularly in language-intensive subjects.

Future research

Developing English proficiency arguably has the greatest impact on success in school. Understanding and responding to the specific ways that academic language is most efficiently developed might offer ways for teaching ELD most effectively and result in accelerated English acquisition. Current studies show the importance of oral language for ELs to improve early literacy, but which components of the interventions were most impactful remain unknown.

Secondly, the effects of different instructional arrangements on EL students’ achievement should be explored. Debate continues around issues such as whether pullout or push-in services are more effective, the optimal amount of time devoted to ELD instruction, and whether to group ELs together or with English-speaking peers. These are areas of practice that warrant investigation.

Thanks to Beth and Jana for contributing their thoughts.

Consider contributing a question to be answered in a future post. You can send one to me at [email protected] . When you send it in, let me know if I can use your real name if it’s selected or if you’d prefer remaining anonymous and have a pseudonym in mind.

You can also contact me on Twitter at @Larryferlazzo .

Education Week has published a collection of posts from this blog, along with new material, in an e-book form. It’s titled Classroom Management Q&As: Expert Strategies for Teaching .

Just a reminder; you can subscribe and receive updates from this blog via email (The RSS feed for this blog, and for all Ed Week articles, has been changed by the new redesign—new ones are not yet available). And if you missed any of the highlights from the first 10 years of this blog, you can see a categorized list below.

• The 11 Most Popular Classroom Q&A Posts of the Year
• Race & Racism in Schools
• School Closures & the Coronavirus Crisis
• Classroom-Management Advice
• Best Ways to Begin the School Year
• Best Ways to End the School Year
• Student Motivation & Social-Emotional Learning
• Implementing the Common Core
• Challenging Normative Gender Culture in Education
• Teaching Social Studies
• Cooperative & Collaborative Learning
• Using Tech With Students
• Student Voices
• Parent Engagement in Schools
• Teaching English-Language Learners
• Reading Instruction
• Writing Instruction
• Education Policy Issues
• Differentiating Instruction
• Math Instruction
• Science Instruction
• Advice for New Teachers
• Author Interviews
• The Inclusive Classroom
• Learning & the Brain
• Administrator Leadership
• Teacher Leadership
• Relationships in Schools
• Professional Development
• Instructional Strategies
• Best of Classroom Q&A
• Professional Collaboration
• Classroom Organization
• Mistakes in Education
• Project-Based Learning

I am also creating a Twitter list including all contributors to this column .

The opinions expressed in Classroom Q&A With Larry Ferlazzo are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.

Sign Up for EdWeek Update

Edweek top school jobs.

Sign Up & Sign In

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals

Education articles from across Nature Portfolio

Outsourcing eureka moments to artificial intelligence

A two-stage learning algorithm is proposed to directly uncover the symbolic representation of rules for skill acquisition from large-scale training log data.

• Martijn Meeter

Latest Research and Reviews

Design and analysis of personalized serious games for information literacy: catering to introverted and extraverted individuals through game elements

• Phimphakan Thongthip
• Kannikar Intawong
• Kitti Puritat

Factors influencing school climate: an empirical study based on the TALIS principal survey

• Xiaodi Jiang

Defining social innovation for post-secondary educational institutions: a concept analysis

• K. M. Benzies
• D. B. Nicholas

What difference does one course make? Assessing the impact of content-based instruction on students’ sustainability literacy

• Inan Deniz Erguvan

Retinal macrocysts in an eye spectacle configuration

• Yunfei Yang
• Sreekala Burgula
• Francesco Sabatino

Saudi female students’ perceptions of the Community of Inquiry in online learning environments

• Tahani I. Aldosemani
• Craig E. Shepherd
• Doris U. Bolliger

News and Comment

Need a policy for using ChatGPT in the classroom? Try asking students

Students are the key users of AI chatbots in university settings, but their opinions are rarely solicited when crafting policies. That needs to change, says Maja Zonjić.

• Maja Zonjić

Chromatic inclusivity in chemistry

Reliance on colour-based experiments in the undergraduate laboratory is a considerable hurdle for those with colour vision deficiency. Designing course material that relies on interpretation and not perception creates a more accessible environment for all.

• Nicholas J. Roberts
• Jennifer L. MacDonald

Building trust and transparency in biomedical sciences through data walks

• Layla Fattah
• Jay Johnson
• Talia H. Swartz

Breaching boundaries: reflections on the journey towards a transdisciplinary arts and sciences undergraduate degree programme to address global challenges

This commentary reflects upon the progress, limitations, and some of the pitfalls of one UK London-based HE institution’s development of a trans-disciplinary arts and sciences undergraduate degree programme specifically designed to build knowledge and confidence in students to both reflect upon and effectively respond in constructive and just ways to some of the ‘global challenges’ facing society. It does not challenge the importance and necessity of specialist expertise but sees the potential of a trans-disciplinary approach to education as not just complementary but increasingly valuable to a wider range of graduates. Graduates needed to lead systems change and facilitate wider appreciation and practical understanding of multidimensional problem-solving, the importance of stakeholder engagement and more holistic systems thinking, something that should not be limited to those who have the opportunity and means to study Masters or PhD degrees. As one of a few UK universities that offer inter-disciplinary or trans-disciplinary undergraduate degrees and with some added insights from a former colleague who now works on University College London’s (UCL) interdisciplinary BASc, we offer the following suggestions and advice for those interested in working towards developing trans-disciplinary provision. This includes the development of a financial model that allows students and staff to work between departments or faculties; an administrative structure that promotes communication and information sharing between different departments without compromising the requirements of data protection; the buy-in and support of senior leaders who both understand and can advocate for the benefits of a trans-disciplinary approach and explicit university-wide recognition of the staff who work on such programmes in terms of career progression and support for the trans-disciplinary research they undertake.

• Mary E. Richards
• Mandekh Hussein
• Olwenn Martin

How medical schools can prepare students for new technologies

Patient educators and nurses can demonstrate the real-life use of health technologies.

• Chantal Mathieu

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

• Copy/Paste Link Link Copied

Using Research and Reason in Education: How Teachers Can Use Scientifically Based Research to Make Curricular & Instructional Decisions

Paula J. Stanovich and Keith E. Stanovich University of Toronto

Produced by RMC Research Corporation, Portsmouth, New Hampshire

This publication was produced under National Institute for Literacy Contract No. ED-00CO-0093 with RMC Research Corporation. Sandra Baxter served as the contracting officer's technical representative. The views expressed herein do not necessarily represent the policies of the National Institute for Literacy. No official endorsement by the National Institute for Literacy or any product, commodity, service, or enterprise is intended or should be inferred.

The National Institute for Literacy

Sandra Baxter, Interim Executive Director Lynn Reddy, Communications Director

To order copies of this booklet, contact the National Institute for Literacy at EdPubs, PO Box 1398, Jessup, MD 20794-1398. Call 800-228-8813 or email [email protected] .

The National Institute for Literacy, an independent federal organization, supports the development of high quality state, regional, and national literacy services so that all Americans can develop the literacy skills they need to succeed at work, at home, and in the community.

The Partnership for Reading, a project administered by the National Institute for Literacy, is a collaborative effort of the National Institute for Literacy, the National Institute of Child Health and Human Development, the U.S. Department of Education, and the U.S. Department of Health and Human Services to make evidence-based reading research available to educators, parents, policy makers, and others with an interest in helping all people learn to read well.

Editorial support provided by C. Ralph Adler and Elizabeth Goldman, and design/production support provided by Diane Draper and Bob Kozman, all of RMC Research Corporation.

Introduction

In the recent move toward standards-based reform in public education, many educational reform efforts require schools to demonstrate that they are achieving educational outcomes with students performing at a required level of achievement. Federal and state legislation, in particular, has codified this standards-based movement and tied funding and other incentives to student achievement.

At first, demonstrating student learning may seem like a simple task, but reflection reveals that it is a complex challenge requiring educators to use specific knowledge and skills. Standards-based reform has many curricular and instructional prerequisites. The curriculum must represent the most important knowledge, skills, and attributes that schools want their students to acquire because these learning outcomes will serve as the basis of assessment instruments. Likewise, instructional methods should be appropriate for the designed curriculum. Teaching methods should lead to students learning the outcomes that are the focus of the assessment standards.

Standards- and assessment-based educational reforms seek to obligate schools and teachers to supply evidence that their instructional methods are effective. But testing is only one of three ways to gather evidence about the effectiveness of instructional methods. Evidence of instructional effectiveness can come from any of the following sources:

• Demonstrated student achievement in formal testing situations implemented by the teacher, school district, or state;
• Published findings of research-based evidence that the instructional methods being used by teachers lead to student achievement; or
• Proof of reason-based practice that converges with a research-based consensus in the scientific literature. This type of justification of educational practice becomes important when direct evidence may be lacking (a direct test of the instructional efficacy of a particular method is absent), but there is a theoretical link to research-based evidence that can be traced.

Each of these methods has its pluses and minuses. While testing seems the most straightforward, it is not necessarily the clear indicator of good educational practice that the public seems to think it is. The meaning of test results is often not immediately clear. For example, comparing averages or other indicators of overall performance from tests across classrooms, schools, or school districts takes no account of the resources and support provided to a school, school district, or individual professional. Poor outcomes do not necessarily indict the efforts of physicians in Third World countries who work with substandard equipment and supplies. Likewise, objective evidence of below-grade or below-standard mean performance of a group of students should not necessarily indict their teachers if essential resources and supports (e.g., curriculum materials, institutional aid, parental cooperation) to support teaching efforts were lacking. However, the extent to which children could learn effectively even in under-equipped schools is not known because evidence-based practices are, by and large, not implemented. That is, there is evidence that children experiencing academic difficulties can achieve more educationally if they are taught with effective methods; sadly, scientific research about what works does not usually find its way into most classrooms.

Testing provides a useful professional calibrator, but it requires great contextual sensitivity in interpretation. It is not the entire solution for assessing the quality of instructional efforts. This is why research-based and reason-based educational practice are also crucial for determining the quality and impact of programs. Teachers thus have the responsibility to be effective users and interpreters of research. Providing a survey and synthesis of the most effective practices for a variety of key curriculum goals (such as literacy and numeracy) would seem to be a helpful idea, but no document could provide all of that information. (Many excellent research syntheses exist, such as the National Reading Panel, 2000; Snow, Burns, & Griffin, 1998; Swanson, 1999, but the knowledge base about effective educational practices is constantly being updated, and many issues remain to be settled.)

As professionals, teachers can become more effective and powerful by developing the skills to recognize scientifically based practice and, when the evidence is not available, use some basic research concepts to draw conclusions on their own. This paper offers a primer for those skills that will allow teachers to become independent evaluators of educational research.

The Formal Scientific Method and Scientific Thinking in Educational Practice

When you go to your family physician with a medical complaint, you expect that the recommended treatment has proven to be effective with many other patients who have had the same symptoms. You may even ask why a particular medication is being recommended for you. The doctor may summarize the background knowledge that led to that recommendation and very likely will cite summary evidence from the drug's many clinical trials and perhaps even give you an overview of the theory behind the drug's success in treating symptoms like yours.

All of this discussion will probably occur in rather simple terms, but that does not obscure the fact that the doctor has provided you with data to support a theory about your complaint and its treatment. The doctor has shared knowledge of medical science with you. And while everyone would agree that the practice of medicine has its "artful" components (for example, the creation of a healing relationship between doctor and patient), we have come to expect and depend upon the scientific foundation that underpins even the artful aspects of medical treatment. Even when we do not ask our doctors specifically for the data, we assume it is there, supporting our course of treatment.

Actually, Vaughn and Dammann (2001) have argued that the correct analogy is to say that teaching is in part a craft, rather than an art. They point out that craft knowledge is superior to alternative forms of knowledge such as superstition and folklore because, among other things, craft knowledge is compatible with scientific knowledge and can be more easily integrated with it. One could argue that in this age of education reform and accountability, educators are being asked to demonstrate that their craft has been integrated with science--that their instructional models, methods, and materials can be likened to the evidence a physician should be able to produce showing that a specific treatment will be effective. As with medicine, constructing teaching practice on a firm scientific foundation does not mean denying the craft aspects of teaching.

Architecture is another professional practice that, like medicine and education, grew from being purely a craft to a craft based firmly on a scientific foundation. Architects wish to design beautiful buildings and environments, but they must also apply many foundational principles of engineering and adhere to structural principles. If they do not, their buildings, however beautiful they may be, will not stand. Similarly, a teacher seeks to design lessons that stimulate students and entice them to learn--lessons that are sometimes a beauty to behold. But if the lessons are not based in the science of pedagogy, they, like poorly constructed buildings, will fail.

Education is informed by formal scientific research through the use of archival research-based knowledge such as that found in peer-reviewed educational journals. Preservice teachers are first exposed to the formal scientific research in their university teacher preparation courses (it is hoped), through the instruction received from their professors, and in their course readings (e.g., textbooks, journal articles). Practicing teachers continue their exposure to the results of formal scientific research by subscribing to and reading professional journals, by enrolling in graduate programs, and by becoming lifelong learners.

Scientific thinking in practice is what characterizes reflective teachers--those who inquire into their own practice and who examine their own classrooms to find out what works best for them and their students. What follows in this document is, first, a "short course" on how to become an effective consumer of the archival literature that results from the conduct of formal scientific research in education and, second, a section describing how teachers can think scientifically in their ongoing reflection about their classroom practice.

Being able to access mechanisms that evaluate claims about teaching methods and to recognize scientific research and its findings is especially important for teachers because they are often confronted with the view that "anything goes" in the field of education--that there is no such thing as best practice in education, that there are no ways to verify what works best, that teachers should base their practice on intuition, or that the latest fad must be the best way to teach, please a principal, or address local school reform. The "anything goes" mentality actually represents a threat to teachers' professional autonomy. It provides a fertile environment for gurus to sell untested educational "remedies" that are not supported by an established research base.

Teachers as independent evaluators of research evidence

One factor that has impeded teachers from being active and effective consumers of educational science has been a lack of orientation and training in how to understand the scientific process and how that process results in the cumulative growth of knowledge that leads to validated educational practice. Educators have only recently attempted to resolve educational disputes scientifically, and teachers have not yet been armed with the skills to evaluate disputes on their own.

Educational practice has suffered greatly because its dominant model for resolving or adjudicating disputes has been more political (with its corresponding factions and interest groups) than scientific. The field's failure to ground practice in the attitudes and values of science has made educators susceptible to the "authority syndrome" as well as fads and gimmicks that ignore evidence-based practice.

When our ancestors needed information about how to act, they would ask their elders and other wise people. Contemporary society and culture are much more complex. Mass communication allows virtually anyone (on the Internet, through self-help books) to proffer advice, to appear to be a "wise elder." The current problem is how to sift through the avalanche of misguided and uninformed advice to find genuine knowledge. Our problem is not information; we have tons of information. What we need are quality control mechanisms.

Peer-reviewed research journals in various disciplines provide those mechanisms. However, even with mechanisms like these in behavioral science and education, it is all too easy to do an "end run" around the quality control they provide. Powerful information dissemination outlets such as publishing houses and mass media frequently do not discriminate between good and bad information. This provides a fertile environment for gurus to sell untested educational "remedies" that are not supported by an established research base and, often, to discredit science, scientific evidence, and the notion of research-based best practice in education. As Gersten (2001) notes, both seasoned and novice teachers are "deluged with misinformation" (p. 45).

We need tools for evaluating the credibility of these many and varied sources of information; the ability to recognize research-based conclusions is especially important. Acquiring those tools means understanding scientific values and learning methods for making inferences from the research evidence that arises through the scientific process. These values and methods were recently summarized by a panel of the National Academy of Sciences convened on scientific inquiry in education (Shavelson & Towne, 2002), and our discussion here will be completely consistent with the conclusions of that NAS panel.

The scientific criteria for evaluating knowledge claims are not complicated and could easily be included in initial teacher preparation programs, but they usually are not (which deprives teachers from an opportunity to become more efficient and autonomous in their work right at the beginning of their careers). These criteria include:

• the publication of findings in refereed journals (scientific publications that employ a process of peer review),
• the duplication of the results by other investigators, and
• a consensus within a particular research community on whether there is a critical mass of studies that point toward a particular conclusion.

In their discussion of the evolution of the American Educational Research Association (AERA) conference and the importance of separating research evidence from opinion when making decisions about instructional practice, Levin and O'Donnell (2000) highlight the importance of enabling teachers to become independent evaluators of research evidence. Being aware of the importance of research published in peer-reviewed scientific journals is only the first step because this represents only the most minimal of criteria. Following is a review of some of the principles of research-based evaluation that teachers will find useful in their work.

Publicly verifiable research conclusions: Replication and Peer Review

Source credibility: the consumer protection of peer reviewed journals..

The front line of defense for teachers against incorrect information in education is the existence of peer-reviewed journals in education, psychology, and other related social sciences. These journals publish empirical research on topics relevant to classroom practice and human cognition and learning. They are the first place that teachers should look for evidence of validated instructional practices.

As a general quality control mechanism, peer review journals provide a "first pass" filter that teachers can use to evaluate the plausibility of educational claims. To put it more concretely, one ironclad criterion that will always work for teachers when presented with claims of uncertain validity is the question: Have findings supporting this method been published in recognized scientific journals that use some type of peer review procedure? The answer to this question will almost always separate pseudoscientific claims from the real thing.

In a peer review, authors submit a paper to a journal for publication, where it is critiqued by several scientists. The critiques are reviewed by an editor (usually a scientist with an extensive history of work in the specialty area covered by the journal). The editor then decides whether the weight of opinion warrants immediate publication, publication after further experimentation and statistical analysis, or rejection because the research is flawed or does not add to the knowledge base. Most journals carry a statement of editorial policy outlining their exact procedures for publication, so it is easy to check whether a journal is in fact, peer-reviewed.

Peer review is a minimal criterion, not a stringent one. Not all information in peer-reviewed scientific journals is necessarily correct, but it has at the very least undergone a cycle of peer criticism and scrutiny. However, it is because the presence of peer-reviewed research is such a minimal criterion that its absence becomes so diagnostic. The failure of an idea, a theory, an educational practice, behavioral therapy, or a remediation technique to have adequate documentation in the peer-reviewed literature of a scientific discipline is a very strong indication to be wary of the practice.

The mechanisms of peer review vary somewhat from discipline to discipline, but the underlying rationale is the same. Peer review is one way (replication of a research finding is another) that science institutionalizes the attitudes of objectivity and public criticism. Ideas and experimentation undergo a honing process in which they are submitted to other critical minds for evaluation. Ideas that survive this critical process have begun to meet the criterion of public verifiability. The peer review process is far from perfect, but it really is the only external consumer protection that teachers have.

The history of reading instruction illustrates the high cost that is paid when the peer-reviewed literature is ignored, when the normal processes of scientific adjudication are replaced with political debates and rhetorical posturing. A vast literature has been generated on best practices that foster children's reading acquisition (Adams, 1990; Anderson, Hiebert, Scott, & Wilkinson, 1985; Chard & Osborn, 1999; Cunningham & Allington, 1994; Ehri, Nunes, Stahl, & Willows, 2001; Moats, 1999; National Reading Panel, 2000; Pearson, 1993; Pressley, 1998; Pressley, Rankin, & Yokol, 1996; Rayner, Foorman, Perfetti, Pesetsky, & Seidenberg, 2002; Reading Coherence Initiative, 1999; Snow, Burns, & Griffin, 1998; Spear-Swerling & Sternberg, 2001). Yet much of this literature remains unknown to many teachers, contributing to the frustrating lack of clarity about accepted, scientifically validated findings and conclusions on reading acquisition.

Teachers should also be forewarned about the difference between professional education journals that are magazines of opinion in contrast to journals where primary reports of research, or reviews of research, are peer reviewed. For example, the magazines Phi Delta Kappan and Educational Leadership both contain stimulating discussions of educational issues, but neither is a peer-reviewed journal of original research. In contrast, the American Educational Research Journal (a flagship journal of the AERA) and the Journal of Educational Psychology (a flagship journal of the American Psychological Association) are both peer-reviewed journals of original research. Both are main sources for evidence on validated techniques of reading instruction and for research on aspects of the reading process that are relevant to a teacher's instructional decisions.

This is true, too, of presentations at conferences of educational organizations. Some are data-based presentations of original research. Others are speeches reflecting personal opinion about educational problems. While these talks can be stimulating and informative, they are not a substitute for empirical research on educational effectiveness.

Replication and the importance of public verifiability.

Research-based conclusions about educational practice are public in an important sense: they do not exist solely in the mind of a particular individual but have been submitted to the scientific community for criticism and empirical testing by others. Knowledge considered "special"--the province of the thought of an individual and immune from scrutiny and criticism by others--can never have the status of scientific knowledge. Research-based conclusions, when published in a peer reviewed journal, become part of the public realm, available to all, in a way that claims of "special expertise" are not.

Replication is the second way that science uses to make research-based conclusions concrete and "public." In order to be considered scientific, a research finding must be presented to other researchers in the scientific community in a way that enables them to attempt the same experiment and obtain the same results. When the same results occur, the finding has been replicated . This process ensures that a finding is not the result of the errors or biases of a particular investigator. Replicable findings become part of the converging evidence that forms the basis of a research-based conclusion about educational practice.

John Donne told us that "no man is an island." Similarly, in science, no researcher is an island. Each investigator is connected to the research community and its knowledge base. This interconnection enables science to grow cumulatively and for research-based educational practice to be built on a convergence of knowledge from a variety of sources. Researchers constantly build on previous knowledge in order to go beyond what is currently known. This process is possible only if research findings are presented in such a way that any investigator can use them to build on.

Philosopher Daniel Dennett (1995) has said that science is "making mistakes in public. Making mistakes for all to see, in the hopes of getting the others to help with the corrections" (p. 380). We might ask those proposing an educational innovation for the evidence that they have in fact "made some mistakes in public." Legitimate scientific disciplines can easily provide such evidence. For example, scientists studying the psychology of reading once thought that reading difficulties were caused by faulty eye movements. This hypothesis has been shown to be in error, as has another that followed it, that so-called visual reversal errors were a major cause of reading difficulty. Both hypotheses were found not to square with the empirical evidence (Rayner, 1998; Share & Stanovich, 1995). The hypothesis that reading difficulties can be related to language difficulties at the phonological level has received much more support (Liberman, 1999; National Reading Panel, 2000; Rayner, Foorman, Perfetti, Pesetsky, & Seidenberg, 2002; Shankweiler, 1999; Stanovich, 2000).

After making a few such "errors" in public, reading scientists have begun, in the last 20 years, to get it right. But the only reason teachers can have confidence that researchers are now "getting it right" is that researchers made it open, public knowledge when they got things wrong. Proponents of untested and pseudoscientific educational practices will never point to cases where they "got it wrong" because they are not committed to public knowledge in the way that actual science is. These proponents do not need, as Dennett says, "to get others to help in making the corrections" because they have no intention of correcting their beliefs and prescriptions based on empirical evidence.

Education is so susceptible to fads and unproven practices because of its tacit endorsement of a personalistic view of knowledge acquisition--one that is antithetical to the scientific value of the public verifiability of knowledge claims. Many educators believe that knowledge resides within particular individuals--with particularly elite insights--who then must be called upon to dispense this knowledge to others. Indeed, some educators reject public, depersonalized knowledge in social science because they believe it dehumanizes people. Science, however, with its conception of publicly verifiable knowledge, actually democratizes knowledge. It frees practitioners and researchers from slavish dependence on authority.

Subjective, personalized views of knowledge degrade the human intellect by creating conditions that subjugate it to an elite whose "personal" knowledge is not accessible to all (Bronowski, 1956, 1977; Dawkins, 1998; Gross, Levitt, & Lewis, 1997; Medawar, 1982, 1984, 1990; Popper, 1972; Wilson, 1998). Empirical science, by generating knowledge and moving it into the public domain, is a liberating force. Teachers can consult the research and decide for themselves whether the state of the literature is as the expert portrays it. All teachers can benefit from some rudimentary grounding in the most fundamental principles of scientific inference. With knowledge of a few uncomplicated research principles, such as control, manipulation, and randomization, anyone can enter the open, public discourse about empirical findings. In fact, with the exception of a few select areas such as the eye movement research mentioned previously, much of the work described in noted summaries of reading research (e.g., Adams, 1990; Snow, Burns, & Griffin, 1998) could easily be replicated by teachers themselves.

There are many ways that the criteria of replication and peer review can be utilized in education to base practitioner training on research-based best practice. Take continuing teacher education in the form of inservice sessions, for example. Teachers and principals who select speakers for professional development activities should ask speakers for the sources of their conclusions in the form of research evidence in peer-reviewed journals. They should ask speakers for bibliographies of the research evidence published on the practices recommended in their presentations.

The science behind research-based practice relies on systematic empiricism

Empiricism is the practice of relying on observation. Scientists find out about the world by examining it. The refusal by some scientists to look into Galileo's telescope is an example of how empiricism has been ignored at certain points in history. It was long believed that knowledge was best obtained through pure thought or by appealing to authority. Galileo claimed to have seen moons around the planet Jupiter. Another scholar, Francesco Sizi, attempted to refute Galileo, not with observations, but with the following argument:

There are seven windows in the head, two nostrils, two ears, two eyes and a mouth; so in the heavens there are two favorable stars, two unpropitious, two luminaries, and Mercury alone undecided and indifferent. From which and many other similar phenomena of nature such as the seven metals, etc., which it were tedious to enumerate, we gather that the number of planets is necessarily seven...ancient nations, as well as modern Europeans, have adopted the division of the week into seven days, and have named them from the seven planets; now if we increase the number of planets, this whole system falls to the ground...moreover, the satellites are invisible to the naked eye and therefore can have no influence on the earth and therefore would be useless and therefore do not exist. (Holton & Roller, 1958, p. 160)

Three centuries of the demonstrated power of the empirical approach give us an edge on poor Sizi. Take away those years of empiricism, and many of us might have been there nodding our heads and urging him on. In fact, the empirical approach is not necessarily obvious, which is why we often have to teach it, even in a society that is dominated by science.

Empiricism pure and simple is not enough, however. Observation itself is fine and necessary, but pure, unstructured observation of the natural world will not lead to scientific knowledge. Write down every observation you make from the time you get up in the morning to the time you go to bed on a given day. When you finish, you will have a great number of facts, but you will not have a greater understanding of the world. Scientific observation is termed systematic because it is structured so that the results of the observation reveal something about the underlying causal structure of events in the world. Observations are structured so that, depending upon the outcome of the observation, some theories of the causes of the outcome are supported and others rejected.

Teachers can benefit by understanding two things about research and causal inferences. The first is the simple (but sometimes obscured) fact that statements about best instructional practices are statements that contain a causal claim. These statements claim that one type of method or practice causes superior educational outcomes. Second, teachers must understand how the logic of the experimental method provides the critical support for making causal inferences.

Science addresses testable questions

Science advances by positing theories to account for particular phenomena in the world, by deriving predictions from these theories, by testing the predictions empirically, and by modifying the theories based on the tests (the sequence is typically theory -> prediction -> test -> theory modification). What makes a theory testable? A theory must have specific implications for observable events in the natural world.

Science deals only with a certain class of problem: the kind that is empirically solvable. That does not mean that different classes of problems are inherently solvable or unsolvable and that this division is fixed forever. Quite the contrary: some problems that are currently unsolvable may become solvable as theory and empirical techniques become more sophisticated. For example, decades ago historians would not have believed that the controversial issue of whether Thomas Jefferson had a child with his slave Sally Hemings was an empirically solvable question. Yet, by 1998, this problem had become solvable through advances in genetic technology, and a paper was published in the journal Nature (Foster, Jobling, Taylor, Donnelly, Deknijeff, Renemieremet, Zerjal, & Tyler-Smith, 1998) on the question.

The criterion of whether a problem is "testable" is called the falsifiability criterion: a scientific theory must always be stated in such a way that the predictions derived from it can potentially be shown to be false. The falsifiability criterion states that, for a theory to be useful, the predictions drawn from it must be specific. The theory must go out on a limb, so to speak, because in telling us what should happen, the theory must also imply that certain things will not happen. If these latter things do happen, it is a clear signal that something is wrong with the theory. It may need to be modified, or we may need to look for an entirely new theory. Either way, we will end up with a theory that is closer to the truth.

In contrast, if a theory does not rule out any possible observations, then the theory can never be changed, and we are frozen into our current way of thinking with no possibility of progress. A successful theory cannot posit or account for every possible happening. Such a theory robs itself of any predictive power.

What we are talking about here is a certain type of intellectual honesty. In science, the proponent of a theory is always asked to address this question before the data are collected: "What data pattern would cause you to give up, or at least to alter, this theory?" In the same way, the falsifiability criterion is a useful consumer protection for the teacher when evaluating claims of educational effectiveness. Proponents of an educational practice should be asked for evidence; they should also be willing to admit that contrary data will lead them to abandon the practice. True scientific knowledge is held tentatively and is subject to change based on contrary evidence. Educational remedies not based on scientific evidence will often fail to put themselves at risk by specifying what data patterns would prove them false.

Objectivity and intellectual honesty

Objectivity, another form of intellectual honesty in research, means that we let nature "speak for itself" without imposing our wishes on it--that we report the results of experimentation as accurately as we can and that we interpret them as fairly as possible. (The fact that this goal is unattainable for any single human being should not dissuade us from holding objectivity as a value.)

In the language of the general public, open-mindedness means being open to possible theories and explanations for a particular phenomenon. But in science it means that and something more. Philosopher Jonathan Adler (1998) teaches us that science values another aspect of open-mindedness even more highly: "What truly marks an open-minded person is the willingness to follow where evidence leads. The open-minded person is willing to defer to impartial investigations rather than to his own predilections...Scientific method is attunement to the world, not to ourselves" (p. 44).

Objectivity is critical to the process of science, but it does not mean that such attitudes must characterize each and every scientist for science as a whole to work. Jacob Bronowski (1973, 1977) often argued that the unique power of science to reveal knowledge about the world does not arise because scientists are uniquely virtuous (that they are completely objective or that they are never biased in interpreting findings, for example). It arises because fallible scientists are immersed in a process of checks and balances --a process in which scientists are always there to criticize and to root out errors. Philosopher Daniel Dennett (1999/2000) points out that "scientists take themselves to be just as weak and fallible as anybody else, but recognizing those very sources of error in themselvesÉthey have devised elaborate systems to tie their own hands, forcibly preventing their frailties and prejudices from infecting their results" (p. 42). More humorously, psychologist Ray Nickerson (1998) makes the related point that the vanities of scientists are actually put to use by the scientific process, by noting that it is "not so much the critical attitude that individual scientists have taken with respect to their own ideas that has given science its success...but more the fact that individual scientists have been highly motivated to demonstrate that hypotheses that are held by some other scientists are false" (p. 32). These authors suggest that the strength of scientific knowledge comes not because scientists are virtuous, but from the social process where scientists constantly cross-check each others' knowledge and conclusions.

The public criteria of peer review and replication of findings exist in part to keep checks on the objectivity of individual scientists. Individuals cannot hide bias and nonobjectivity by personalizing their claims and keeping them from public scrutiny. Science does not accept findings that have failed the tests of replication and peer review precisely because it wants to ensure that all findings in science are in the public domain, as defined above. Purveyors of pseudoscientific educational practices fail the test of objectivity and are often identifiable by their attempts to do an "end run" around the public mechanisms of science by avoiding established peer review mechanisms and the information-sharing mechanisms that make replication possible. Instead, they attempt to promulgate their findings directly to consumers, such as teachers.

The principle of converging evidence

The principle of converging evidence has been well illustrated in the controversies surrounding the teaching of reading. The methods of systematic empiricism employed in the study of reading acquisition are many and varied. They include case studies, correlational studies, experimental studies, narratives, quasi-experimental studies, surveys, epidemiological studies and many others. The results of many of these studies have been synthesized in several important research syntheses (Adams, 1990; Ehri et al., 2001; National Reading Panel, 2000; Pressley, 1998; Rayner et al., 2002; Reading Coherence Initiative, 1999; Share & Stanovich, 1995; Snow, Burns, & Griffin, 1998; Snowling, 2000; Spear-Swerling & Sternberg, 2001; Stanovich, 2000). These studies were used in a process of establishing converging evidence, a principle that governs the drawing of the conclusion that a particular educational practice is research-based.

The principle of converging evidence is applied in situations requiring a judgment about where the "preponderance of evidence" points. Most areas of science contain competing theories. The extent to which a particular study can be seen as uniquely supporting one particular theory depends on whether other competing explanations have been ruled out. A particular experimental result is never equally relevant to all competing theories. An experiment may be a very strong test of one or two alternative theories but a weak test of others. Thus, research is considered highly convergent when a series of experiments consistently supports a given theory while collectively eliminating the most important competing explanations. Although no single experiment can rule out all alternative explanations, taken collectively, a series of partially diagnostic experiments can lead to a strong conclusion if the data converge.

Contrast this idea of converging evidence with the mistaken view that a problem in science can be solved with a single, crucial experiment, or that a single critical insight can advance theory and overturn all previous knowledge. This view of scientific progress fits nicely with the operation of the news media, in which history is tracked by presenting separate, disconnected "events" in bite-sized units. This is a gross misunderstanding of scientific progress and, if taken too seriously, leads to misconceptions about how conclusions are reached about research-based practices.

One experiment rarely decides an issue, supporting one theory and ruling out all others. Issues are most often decided when the community of scientists gradually begins to agree that the preponderance of evidence supports one alternative theory rather than another. Scientists do not evaluate data from a single experiment that has finally been designed in the perfect way. They most often evaluate data from dozens of experiments, each containing some flaws but providing part of the answer.

Although there are many ways in which an experiment can go wrong (or become confounded ), a scientist with experience working on a particular problem usually has a good idea of what most of the critical factors are, and there are usually only a few. The idea of converging evidence tells us to examine the pattern of flaws running through the research literature because the nature of this pattern can either support or undermine the conclusions that we might draw.

For example, suppose that the findings from a number of different experiments were largely consistent in supporting a particular conclusion. Given the imperfect nature of experiments, we would evaluate the extent and nature of the flaws in these studies. If all the experiments were flawed in a similar way, this circumstance would undermine confidence in the conclusions drawn from them because the consistency of the outcome may simply have resulted from a particular, consistent flaw. On the other hand, if all the experiments were flawed in different ways, our confidence in the conclusions increases because it is less likely that the consistency in the results was due to a contaminating factor that confounded all the experiments. As Anderson and Anderson (1996) note, "When a conceptual hypothesis survives many potential falsifications based on different sets of assumptions, we have a robust effect." (p. 742).

Suppose that five different theoretical summaries (call them A, B, C, D, and E) of a given set of phenomena exist at one time and are investigated in a series of experiments. Suppose that one set of experiments represents a strong test of theories A, B, and C, and that the data largely refute theories A and B and support C. Imagine also that another set of experiments is a particularly strong test of theories C, D, and E, and that the data largely refute theories D and E and support C. In such a situation, we would have strong converging evidence for theory C. Not only do we have data supportive of theory C, but we have data that contradict its major competitors. Note that no one experiment tests all the theories, but taken together, the entire set of experiments allows a strong inference.

In contrast, if the two sets of experiments each represent strong tests of B, C, and E, and the data strongly support C and refute B and E, the overall support for theory C would be less strong than in our previous example. The reason is that, although data supporting theory C have been generated, there is no strong evidence ruling out two viable alternative theories (A and D). Thus research is highly convergent when a series of experiments consistently supports a given theory while collectively eliminating the most important competing explanations. Although no single experiment can rule out all alternative explanations, taken collectively, a series of partially diagnostic experiments can lead to a strong conclusion if the data converge in the manner of our first example.

Increasingly, the combining of evidence from disparate studies to form a conclusion is being done more formally by the use of the statistical technique termed meta-analysis (Cooper & Hedges, 1994; Hedges & Olkin, 1985; Hunter & Schmidt, 1990; Rosenthal, 1995; Schmidt, 1992; Swanson, 1999) which has been used extensively to establish whether various medical practices are research based. In a medical context, meta-analysis:

involves adding together the data from many clinical trials to create a single pool of data big enough to eliminate much of the statistical uncertainty that plagues individual trials...The great virtue of meta-analysis is that clear findings can emerge from a group of studies whose findings are scattered all over the map. (Plotkin,1996, p. 70)

The use of meta-analysis for determining the research validation of educational practices is just the same as in medicine. The effects obtained when one practice is compared against another are expressed in a common statistical metric that allows comparison of effects across studies. The findings are then statistically amalgamated in some standard ways (Cooper & Hedges, 1994; Hedges & Olkin, 1985; Swanson, 1999) and a conclusion about differential efficacy is reached if the amalgamation process passes certain statistical criteria. In some cases, of course, no conclusion can be drawn with confidence, and the result of the meta-analysis is inconclusive.

More and more commentators on the educational research literature are calling for a greater emphasis on meta-analysis as a way of dampening the contentious disputes about conflicting studies that plague education and other behavioral sciences (Kavale & Forness, 1995; Rosnow & Rosenthal, 1989; Schmidt, 1996; Stanovich, 2001; Swanson, 1999). The method is useful for ending disputes that seem to be nothing more than a "he-said, she-said" debate. An emphasis on meta-analysis has often revealed that we actually have more stable and useful findings than is apparent from a perusal of the conflicts in our journals.

The National Reading Panel (2000) found just this in their meta-analysis of the evidence surrounding several issues in reading education. For example, they concluded that the results of a meta-analysis of the results of 66 comparisons from 38 different studies indicated "solid support for the conclusion that systematic phonics instruction makes a bigger contribution to children's growth in reading than alternative programs providing unsystematic or no phonics instruction" (p. 2-84). In another section of their report, the National Reading Panel reported that a meta-analysis of 52 studies of phonemic awareness training indicated that "teaching children to manipulate the sounds in language helps them learn to read. Across the various conditions of teaching, testing, and participant characteristics, the effect sizes were all significantly greater than chance and ranged from large to small, with the majority in the moderate range. Effects of phonemic awareness training on reading lasted well beyond the end of training" (p. 2-5).

A statement by a task force of the American Psychological Association (Wilkinson, 1999) on statistical methods in psychology journals provides an apt summary for this section. The task force stated that investigators should not "interpret a single study's results as having importance independent of the effects reported elsewhere in the relevant literature" (p. 602). Science progresses by convergence upon conclusions. The outcomes of one study can only be interpreted in the context of the present state of the convergence on the particular issue in question.

The logic of the experimental method

Scientific thinking is based on the ideas of comparison, control, and manipulation . In a true experimental study, these characteristics of scientific investigation must be arranged to work in concert.

Comparison alone is not enough to justify a causal inference. In methodology texts, correlational investigations (which involve comparison only) are distinguished from true experimental investigations that warrant much stronger causal inferences because they involve comparison, control, and manipulation. The mere existence of a relationship between two variables does not guarantee that changes in one are causing changes in the other. Correlation does not imply causation.

There are two potential problems with drawing causal inferences from correlational evidence. The first is called the third-variable problem. It occurs when the correlation between the two variables does not indicate a direct causal path between them but arises because both variables are related to a third variable that has not even been measured.

The second reason is called the directionality problem. It creates potential interpretive difficulties because even if two variables have a direct causal relationship, the direction of that relationship is not indicated by the mere presence of the correlation. In short, a correlation between variables A and B could arise because changes in A are causing changes in B or because changes in B are causing changes in A. The mere presence of the correlation does not allow us to decide between these two possibilities.

The heart of the experimental method lies in manipulation and control. In contrast to a correlational study, where the investigator simply observes whether the natural fluctuation in two variables displays a relationship, the investigator in a true experiment manipulates the variable thought to be the cause (the independent variable) and looks for an effect on the variable thought to be the effect (the dependent variable ) while holding all other variables constant by control and randomization. This method removes the third-variable problem because, in the natural world, many different things are related. The experimental method may be viewed as a way of prying apart these naturally occurring relationships. It does so because it isolates one particular variable (the hypothesized cause) by manipulating it and holding everything else constant (control).

When manipulation is combined with a procedure known as random assignment (in which the subjects themselves do not determine which experimental condition they will be in but, instead, are randomly assigned to one of the experimental groups), scientists can rule out alternative explanations of data patterns. By using manipulation, experimental control, and random assignment, investigators construct stronger comparisons so that the outcome eliminates alternative theories and explanations.

The need for both correlational methods and true experiments

As strong as they are methodologically, studies employing true experimental logic are not the only type that can be used to draw conclusions. Correlational studies have value. The results from many different types of investigation, including correlational studies, can be amalgamated to derive a general conclusion. The basis for conclusion rests on the convergence observed from the variety of methods used. This is most certainly true in classroom and curriculum research. It is necessary to amalgamate the results from not only experimental investigations, but correlational studies, nonequivalent control group studies, time series designs, and various other quasi-experimental designs and multivariate correlational designs, All have their strengths and weaknesses. For example, it is often (but not always) the case that experimental investigations are high in internal validity, but limited in external validity, whereas correlational studies are often high in external validity, but low in internal validity.

Internal validity concerns whether we can infer a causal effect for a particular variable. The more a study employs the logic of a true experiment (i.e., includes manipulation, control, and randomization), the more we can make a strong causal inference. External validity concerns the generalizability of the conclusion to the population and setting of interest. Internal and external validity are often traded off across different methodologies. Experimental laboratory investigations are high in internal validity but may not fully address concerns about external validity. Field classroom investigations, on the other hand, are often quite high in external validity but because of the logistical difficulties involved in carrying them out, they are often quite low in internal validity. That is why we need to look for a convergence of results, not just consistency from one method. Convergence increases our confidence in the external and internal validity of our conclusions.

Again, this underscores why correlational studies can contribute to knowledge. First, some variables simply cannot be manipulated for ethical reasons (for instance, human malnutrition or physical disabilities). Other variables, such as birth order, sex, and age, are inherently correlational because they cannot be manipulated, and therefore the scientific knowledge concerning them must be based on correlational evidence. Finally, logistical difficulties in classroom and curriculum research often make it impossible to achieve the logic of the true experiment. However, this circumstance is not unique to educational or psychological research. Astronomers obviously cannot manipulate all the variables affecting the objects they study, yet they are able to arrive at conclusions.

Complex correlational techniques are essential in the absence of experimental research because complex correlational statistics such as multiple regression, path analysis, and structural equation modeling that allow for the partial control of third variables when those variables can be measured. These statistics allow us to recalculate the correlation between two variables after the influence of other variables is removed. If a potential third variable can be measured, complex correlational statistics can help us determine whether that third variable is determining the relationship. These correlational statistics and designs help to rule out certain causal hypotheses, even if they cannot demonstrate the true causal relation definitively.

Stages of scientific investigation: The Role of Case Studies and Qualitative Investigations

The educational literature includes many qualitative investigations that focus less on issues of causal explanation and variable control and more on thick description , in the manner of the anthropologist (Geertz, 1973, 1979). The context of a person's behavior is described as much as possible from the standpoint of the participant. Many different fields (e.g., anthropology, psychology, education) contain case studies where the focus is detailed description and contextualization of the situation of a single participant (or very few participants).

The usefulness of case studies and qualitative investigations is strongly determined by how far scientific investigation has advanced in a particular area. The insights gained from case studies or qualitative investigations may be quite useful in the early stages of an investigation of a certain problem. They can help us determine which variables deserve more intense study by drawing attention to heretofore unrecognized aspects of a person's behavior and by suggesting how understanding of behavior might be sharpened by incorporating the participant's perspective.

However, when we move from the early stages of scientific investigation, where case studies may be very useful, to the more mature stages of theory testing--where adjudicating between causal explanations is the main task--the situation changes drastically. Case studies and qualitative description are not useful at the later stages of scientific investigation because they cannot be used to confirm or disconfirm a particular causal theory. They lack the comparative information necessary to rule out alternative explanations.

Where qualitative investigations are useful relates strongly to a distinction in philosophy of science between the context of discovery and the context of justification . Qualitative research, case studies, and clinical observations support a context of discovery where, as Levin and O'Donnell (2000) note in an educational context, such research must be regarded as "preliminary/exploratory, observational, hypothesis generating" (p. 26). They rightly point to the essential importance of qualitative investigations because "in the early stages of inquiry into a research topic, one has to look before one can leap into designing interventions, making predictions, or testing hypotheses" (p. 26). The orientation provided by qualitative investigations is critical in such cases. Even more important, the results of quantitative investigations--which must sometimes abstract away some of the contextual features of a situation--are often contextualized by the thick situational description provided by qualitative work.

However, in the context of justification, variables must be measured precisely, large groups must be tested to make sure the conclusion generalizes and, most importantly, many variables must be controlled because alternative causal explanations must be ruled out. Gersten (2001) summarizes the value of qualitative research accurately when he says that "despite the rich insights they often provide, descriptive studies cannot be used as evidence for an intervention's efficacy...descriptive research can only suggest innovative strategies to teach students and lay the groundwork for development of such strategies" (p. 47). Qualitative research does, however, help to identify fruitful directions for future experimental studies.

Nevertheless, here is why the sole reliance on qualitative techniques to determine the effectiveness of curricula and instructional strategies has become problematic. As a researcher, you desire to do one of two things.

Objective A

The researcher wishes to make some type of statement about a relationship, however minimal. That is, you at least want to use terms like greater than, or less than, or equal to. You want to say that such and such an educational program or practice is better than another. "Better than" and "worse than" are, of course, quantitative statements--and, in the context of issues about what leads to or fosters greater educational achievement, they are causal statements as well . As quantitative causal statements, the support for such claims obviously must be found in the experimental logic that has been outlined above. To justify such statements, you must adhere to the canons of quantitative research logic.

Objective B

The researcher seeks to adhere to an exclusively qualitative path that abjures statements about relationships and never uses comparative terms of magnitude. The investigator desires to simply engage in thick description of a domain that may well prompt hypotheses when later work moves on to the more quantitative methods that are necessary to justify a causal inference.

Investigators pursuing Objective B are doing essential work. They provide quantitative information with suggestions for richer hypotheses to study. In education, however, investigators sometimes claim to be pursuing Objective B but slide over into Objective A without realizing they have made a crucial switch. They want to make comparative, or quantitative, statements, but have not carried out the proper types of investigation to justify them. They want to say that a certain educational program is better than another (that is, it causes better school outcomes). They want to give educational strictures that are assumed to hold for a population of students, not just to the single or few individuals who were the objects of the qualitative study. They want to condemn an educational practice (and, by inference, deem an alternative quantitatively and causally better). But instead of taking the necessary course of pursuing Objective A, they carry out their investigation in the manner of Objective B.

Let's recall why the use of single case or qualitative description as evidence in support of a particular causal explanation is inappropriate. The idea of alternative explanations is critical to an understanding of theory testing. The goal of experimental design is to structure events so that support of one particular explanation simultaneously disconfirms other explanations. Scientific progress can occur only if the data that are collected rule out some explanations. Science sets up conditions for the natural selection of ideas. Some survive empirical testing and others do not.

This is the honing process by which ideas are sifted so that those that contain the most truth are found. But there must be selection in this process: data collected as support for a particular theory must not leave many other alternative explanations as equally viable candidates. For this reason, scientists construct control or comparison groups in their experimentation. These groups are formed so that, when their results are compared with those from an experimental group, some alternative explanations are ruled out.

Case studies and qualitative description lack the comparative information necessary to prove that a particular theory or educational practice is superior, because they fail to test an alternative; they rule nothing out. Take the seminal work of Jean Piaget for example. His case studies were critical in pointing developmental psychology in new and important directions, but many of his theoretical conclusions and causal explanations did not hold up in controlled experiments (Bjorklund, 1995; Goswami, 1998; Siegler, 1991).

In summary, as educational psychologist Richard Mayer (2000) notes, "the domain of science includes both some quantitative and qualitative methodologies" (p. 39), and the key is to use each where it is most effective (see Kamil, 1995). Likewise, in their recent book on research-based best practices in comprehension instruction, Block and Pressley (2002) argue that future progress in understanding how comprehension works will depend on a healthy interaction between qualitative and quantitative approaches. They point out that getting an initial idea of the comprehension processes involved in hypertext and Web-based environments will involve detailed descriptive studies using think-alouds and assessments of qualitative decision making. Qualitative studies of real reading environments will set the stage for more controlled investigations of causal hypotheses.

The progression to more powerful methods

A final useful concept is the progression to more powerful research methods ("more powerful" in this context meaning more diagnostic of a causal explanation). Research on a particular problem often proceeds from weaker methods (ones less likely to yield a causal explanation) to ones that allow stronger causal inferences. For example, interest in a particular hypothesis may originally emerge from a particular case study of unusual interest. This is the proper role for case studies: to suggest hypotheses for further study with more powerful techniques and to motivate scientists to apply more rigorous methods to a research problem. Thus, following the case studies, researchers often undertake correlational investigations to verify whether the link between variables is real rather than the result of the peculiarities of a few case studies. If the correlational studies support the relationship between relevant variables, then researchers will attempt experiments in which variables are manipulated in order to isolate a causal relationship between the variables.

Summary of principles that support research-based inferences about best practice

Our sketch of the principles that support research-based inferences about best practice in education has revealed that:

• Science progresses by investigating solvable, or testable, empirical problems.
• To be testable, a theory must yield predictions that could possible be shown to be wrong.
• The concepts in the theories in science evolve as evidence accumulates. Scientific knowledge is not infallible knowledge, but knowledge that has at least passed some minimal tests. The theories behind research-based practice can be proven wrong, and therefore they contain a mechanism for growth and advancement.
• Theories are tested by systematic empiricism. The data obtained from empirical research are in the public domain in the sense that they are presented in a manner that allows replication and criticism by other scientists.
• Data and theories in science are considered in the public domain only after publication in peer-reviewed scientific journals.
• Empiricism is systematic because it strives for the logic of control and manipulation that characterizes a true experiment.
• Correlational techniques are helpful when the logic of an experiment cannot be approximated, but because these techniques only help rule out hypotheses, they are considered weaker than true experimental methods.
• Researchers use many different methods to arrive at their conclusions, and the strengths and weaknesses of these methods vary. Most often, conclusions are drawn only after a slow accumulation of data from many studies.

Scientific thinking in educational practice: Reason-based practice in the absence of direct evidence

Some areas in educational research, to date, lack a research-based consensus, for a number of reasons. Perhaps the problem or issue has not been researched extensively. Perhaps research into the issue is in the early stages of investigation, where descriptive studies are suggesting interesting avenues, but no controlled research justifying a causal inference has been completed. Perhaps many correlational studies and experiments have been conducted on the issue, but the research evidence has not yet converged in a consistent direction.

Even if teachers know the principles of scientific evaluation described earlier, the research literature sometimes fails to give them clear direction. They will have to fall back on their own reasoning processes as informed by their own teaching experiences. In those cases, teachers still have many ways of reasoning scientifically.

Tracing the link from scientific research to scientific thinking in practice

Scientific thinking in can be done in several ways. Earlier we discussed different types of professional publications that teachers can read to improve their practice. The most important defining feature of these outlets is whether they are peer reviewed. Another defining feature is whether the publication contains primary research rather than presenting opinion pieces or essays on educational issues. If a journal presents primary research, we can evaluate the research using the formal scientific principles outlined above.

If the journal is presenting opinion pieces about what constitutes best practice, we need to trace the link between those opinions and archival peer-reviewed research. We would look to see whether the authors have based their opinions on peer-reviewed research by reading the reference list. Do the authors provide a significant amount of original research citations (is their opinion based on more than one study)? Do the authors cite work other than their own (have the results been replicated)? Are the cited journals peer-reviewed? For example, in the case of best practice for reading instruction, if we came across an article in an opinion-oriented journal such as Intervention in School and Clinic, we might look to see if the authors have cited work that has appeared in such peer-reviewed journals as Journal of Educational Psychology , Elementary School Journal , Journal of Literacy Research , Scientific Studies of Reading , or the Journal of Learning Disabilities .

These same evaluative criteria can be applied to presenters at professional development workshops or papers given at conferences. Are they conversant with primary research in the area on which they are presenting? Can they provide evidence for their methods and does that evidence represent a scientific consensus? Do they understand what is required to justify causal statements? Are they open to the possibility that their claims could be proven false? What evidence would cause them to shift their thinking?

An important principle of scientific evaluation--the connectivity principle (Stanovich, 2001)--can be generalized to scientific thinking in the classroom. Suppose a teacher comes upon a new teaching method, curriculum component, or process. The method is advertised as totally new, which provides an explanation for the lack of direct empirical evidence for the method. A lack of direct empirical evidence should be grounds for suspicion, but should not immediately rule it out. The principle of connectivity means that the teacher now has another question to ask: "OK, there is no direct evidence for this method, but how is the theory behind it (the causal model of the effects it has) connected to the research consensus in the literature surrounding this curriculum area?" Even in the absence of direct empirical evidence on a particular method or technique, there could be a theoretical link to the consensus in the existing literature that would support the method.

For further tips on translating research into classroom practice, see Warby, Greene, Higgins, & Lovitt (1999). They present a format for selecting, reading, and evaluating research articles, and then importing the knowledge gained into the classroom.

Let's take an imaginary example from the domain of treatments for children with extreme reading difficulties. Imagine two treatments have been introduced to a teacher. No direct empirical tests of efficacy have been carried out using either treatment. The first, Treatment A, is a training program to facilitate the awareness of the segmental nature of language at the phonological level. The second, Treatment B, involves giving children training in vestibular sensitivity by having them walk on balance beams while blindfolded. Treatment A and B are equal in one respect--neither has had a direct empirical test of its efficacy, which reflects badly on both. Nevertheless, one of the treatments has the edge when it comes to the principle of connectivity. Treatment A makes contact with a broad consensus in the research literature that children with extraordinary reading difficulties are hampered because of insufficiently developed awareness of the segmental structure of language. Treatment B is not connected to any corresponding research literature consensus. Reason dictates that Treatment A is a better choice, even though neither has been directly tested.

Direct connections with research-based evidence and use of the connectivity principle when direct empirical evidence is absent give us necessary cross-checks on some of the pitfalls that arise when we rely solely on personal experience. Drawing upon personal experience is necessary and desirable in a veteran teacher, but it is not sufficient for making critical judgments about the effectiveness of an instructional strategy or curriculum. The insufficiency of personal experience becomes clear if we consider that the educational judgments--even of veteran teachers--often are in conflict. That is why we have to adjudicate conflicting knowledge claims using the scientific method.

Let us consider two further examples that demonstrate why we need controlled experimentation to verify even the most seemingly definitive personal observations. In the 1990s, considerable media and professional attention were directed at a method for aiding the communicative capacity of autistic individuals. This method is called facilitated communication. Autistic individuals who had previously been nonverbal were reported to have typed highly literate messages on a keyboard when their hands and arms were supported over the typewriter by a so-called facilitator. These startlingly verbal performances by autistic children who had previously shown very limited linguistic behavior raised incredible hopes among many parents of autistic children.

Unfortunately, claims for the efficacy of facilitated communication were disseminated by many media outlets before any controlled studies had been conducted. Since then, many studies have appeared in journals in speech science, linguistics, and psychology and each study has unequivocally demonstrated the same thing: the autistic child's performance is dependent upon tactile cueing from the facilitator. In the experiments, it was shown that when both child and facilitator were looking at the same drawing, the child typed the correct name of the drawing. When the viewing was occluded so that the child and the facilitator were shown different drawings, the child typed the name of the facilitator's drawing, not the one that the child herself was looking at (Beck & Pirovano, 1996; Burgess, Kirsch, Shane, Niederauer, Graham, & Bacon, 1998; Hudson, Melita, & Arnold, 1993; Jacobson, Mulick, & Schwartz, 1995; Wheeler, Jacobson, Paglieri, & Schwartz, 1993). The experimental studies directly contradicted the extensive case studies of the experiences of the facilitators of the children. These individuals invariably deny that they have inadvertently cued the children. Their personal experience, honest and heartfelt though it is, suggests the wrong model for explaining this outcome. The case study evidence told us something about the social connections between the children and their facilitators. But that is something different than what we got from the controlled experimental studies, which provided direct tests of the claim that the technique unlocks hidden linguistic skills in these children. Even if the claim had turned out to be true, the verification of the proof of its truth would not have come from the case studies or personal experiences, but from the necessary controlled studies.

Another example of the need for controlled experimentation to test the insights gleaned from personal experience is provided by the concept of learning styles--the idea that various modality preferences (or variants of this theme in terms of analytic/holistic processing or "learning styles") will interact with instructional methods, allowing teachers to individualize learning. The idea seems to "feel right" to many of us. It does seem to have some face validity, but it has never been demonstrated to work in practice. Its modern incarnation (see Gersten, 2001, Spear-Swerling & Sternberg, 2001) takes a particularly harmful form, one where students identified as auditory learners are matched with phonics instruction and visual and/or kinesthetic learners matched with holistic instruction. The newest form is particularly troublesome because the major syntheses of reading research demonstrate that many children can benefit from phonics-based instruction, not just "auditory" learners (National Reading Panel, 2000; Rayner et al., 2002; Stanovich, 2000). Excluding students identified as "visual/kinesthetic" learners from effective phonics instruction is a bad instructional practice--bad because it is not only not research based, it is actually contradicted by research.

A thorough review of the literature by Arter and Jenkins (1979) found no consistent evidence for the idea that modality strengths and weaknesses could be identified in a reliable and valid way that warranted differential instructional prescriptions. A review of the research evidence by Tarver and Dawson (1978) found likewise that the idea of modality preferences did not hold up to empirical scrutiny. They concluded, "This review found no evidence supporting an interaction between modality preference and method of teaching reading" (p. 17). Kampwirth and Bates (1980) confirmed the conclusions of the earlier reviews, although they stated their conclusions a little more baldly: "Given the rather general acceptance of this idea, and its common-sense appeal, one would presume that there exists a body of evidence to support it. UnfortunatelyÉno such firm evidence exists" (p. 598).

More recently, the idea of modality preferences (also referred to as learning styles, holistic versus analytic processing styles, and right versus left hemispheric processing) has again surfaced in the reading community. The focus of the recent implementations refers more to teaching to strengths, as opposed to remediating weaknesses (the latter being more the focus of the earlier efforts in the learning disabilities field). The research of the 1980s was summarized in an article by Steven Stahl (1988). His conclusions are largely negative because his review of the literature indicates that the methods that have been used in actual implementations of the learning styles idea have not been validated. Stahl concludes: "As intuitively appealing as this notion of matching instruction with learning style may be, past research has turned up little evidence supporting the claim that different teaching methods are more or less effective for children with different reading styles" (p. 317).

Obviously, such research reviews cannot prove that there is no possible implementation of the idea of learning styles that could work. However, the burden of proof in science rests on the investigator who is making a new claim about the nature of the world. It is not incumbent upon critics of a particular claim to show that it "couldn't be true." The question teachers might ask is, "Have the advocates for this new technique provided sufficient proof that it works?" Their burden of responsibility is to provide proof that their favored methods work. Teachers should not allow curricular advocates to avoid this responsibility by introducing confusion about where the burden of proof lies. For example, it is totally inappropriate and illogical to ask "Has anyone proved that it can't work?" One does not "prove a negative" in science. Instead, hypotheses are stated, and then must be tested by those asserting the hypotheses.

Reason-based practice in the classroom

Effective teachers engage in scientific thinking in their classrooms in a variety of ways: when they assess and evaluate student performance, develop Individual Education Plans (IEPs) for their students with disabilities, reflect on their practice, or engage in action research. For example, consider the assessment and evaluation activities in which teachers engage. The scientific mechanisms of systematic empiricism--iterative testing of hypotheses that are revised after the collection of data--can be seen when teachers plan for instruction: they evaluate their students' previous knowledge, develop hypotheses about the best methods for attaining lesson objectives, develop a teaching plan based on those hypotheses, observe the results, and base further instruction on the evidence collected.

This assessment cycle looks even more like the scientific method when teachers (as part of a multidisciplinary team) are developing and implementing an IEP for a student with a disability. The team must assess and evaluate the student's learning strengths and difficulties, develop hypotheses about the learning problems, select curriculum goals and objectives, base instruction on the hypotheses and the goals selected, teach, and evaluate the outcomes of that teaching. If the teaching is successful (goals and objectives are attained), the cycle continues with new goals. If the teaching has been unsuccessful (goals and objectives have not been achieved), the cycle begins again with new hypotheses. We can also see the principle of converging evidence here. No one piece of evidence might be decisive, but collectively the evidence might strongly point in one direction.

Scientific thinking in practice occurs when teachers engage in action research. Action research is research into one's own practice that has, as its main aim, the improvement of that practice. Stokes (1997) discusses how many advances in science came about as a result of "use-inspired research" which draws upon observations in applied settings. According to McNiff, Lomax, and Whitehead (1996), action research shares several characteristics with other types of research: "it leads to knowledge, it provides evidence to support this knowledge, it makes explicit the process of enquiry through which knowledge emerges, and it links new knowledge with existing knowledge" (p. 14). Notice the links to several important concepts: systematic empiricism, publicly verifiable knowledge, converging evidence, and the connectivity principle.

Teachers and Research Commonality in a "what works" epistemology

Many educational researchers have drawn attention to the epistemological commonalities between researchers and teachers (Gersten, Vaughn, Deshler, & Schiller, 1997; Stanovich, 1993/1994). A "what works" epistemology is a critical source of underlying unity in the world views of educators and researchers (Gersten & Dimino, 2001; Gersten, Chard, & Baker, 2000). Empiricism, broadly construed (as opposed to the caricature of white coats, numbers, and test tubes that is often used to discredit scientists) is about watching the world, manipulating it when possible, observing outcomes, and trying to associate outcomes with features observed and with manipulations. This is what the best teachers do. And this is true despite the grain of truth in the statement that "teaching is an art." As Berliner (1987) notes: "No one I know denies the artistic component to teaching. I now think, however, that such artistry should be research-based. I view medicine as an art, but I recognize that without its close ties to science it would be without success, status, or power in our society. Teaching, like medicine, is an art that also can be greatly enhanced by developing a close relationship to science (p. 4)."

In his review of the work of the Committee on the Prevention of Reading Difficulties for the National Research Council of the National Academy of Sciences (Snow, Burns, & Griffin, 1998), Pearson (1999) warned educators that resisting evaluation by hiding behind the "art of teaching" defense will eventually threaten teacher autonomy. Teachers need creativity, but they also need to demonstrate that they know what evidence is, and that they recognize that they practice in a profession based in behavioral science. While making it absolutely clear that he opposes legislative mandates, Pearson (1999) cautions:

We have a professional responsibility to forge best practice out of the raw materials provided by our most current and most valid readings of research...If professional groups wish to retain the privileges of teacher prerogative and choice that we value so dearly, then the price we must pay is constant attention to new knowledge as a vehicle for fine-tuning our individual and collective views of best practice. This is the path that other professions, such as medicine, have taken in order to maintain their professional prerogative, and we must take it, too. My fear is that if the professional groups in education fail to assume this responsibility squarely and openly, then we will find ourselves victims of the most onerous of legislative mandates (p. 245).

Those hostile to a research-based approach to educational practice like to imply that the insights of teachers and those of researchers conflict. Nothing could be farther from the truth. Take reading, for example. Teachers often do observe exactly what the research shows--that most of their children who are struggling with reading have trouble decoding words. In an address to the Reading Hall of Fame at the 1996 meeting of the International Reading Association, Isabel Beck (1996) illustrated this point by reviewing her own intellectual history (see Beck, 1998, for an archival version). She relates her surprise upon coming as an experienced teacher to the Learning Research and Development Center at the University of Pittsburgh and finding "that there were some people there (psychologists) who had not taught anyone to read, yet they were able to describe phenomena that I had observed in the course of teaching reading" (Beck, 1996, p. 5). In fact, what Beck was observing was the triangulation of two empirical approaches to the same issue--two perspectives on the same underlying reality. And she also came to appreciate how these two perspectives fit together: "What I knew were a number of whats--what some kids, and indeed adults, do in the early course of learning to read. And what the psychologists knew were some whys--why some novice readers might do what they do" (pp. 5-6).

Beck speculates on why the disputes about early reading instruction have dragged on so long without resolution and posits that it is due to the power of a particular kind of evidence--evidence from personal observation. The determination of whole language advocates is no doubt sustained because "people keep noticing the fact that some children or perhaps many children--in any event a subset of children--especially those who grow up in print-rich environments, don't seem to need much more of a boost in learning to read than to have their questions answered and to point things out to them in the course of dealing with books and various other authentic literacy acts" (Beck, 1996, p. 8). But Beck points out that it is equally true that proponents of the importance of decoding skills are also fueled by personal observation: "People keep noticing the fact that some children or perhaps many children--in any event a subset of children--don't seem to figure out the alphabetic principle, let alone some of the intricacies involved without having the system directly and systematically presented" (p. 8). But clearly we have lost sight of the basic fact that the two observations are not mutually exclusive--one doesn't negate the other. This is just the type of situation for which the scientific method was invented: a situation requiring a consensual view, triangulated across differing observations by different observers.

Teachers, like scientists, are ruthless pragmatists (Gersten & Dimino, 2001; Gersten, Chard, & Baker, 2000). They believe that some explanations and methods are better than others. They think there is a real world out there--a world in flux, obviously--but still one that is trackable by triangulating observations and observers. They believe that there are valid, if fallible, ways of finding out which educational practices are best. Teachers believe in a world that is predictable and controllable by manipulations that they use in their professional practice, just as scientists do. Researchers and educators are kindred spirits in their approach to knowledge, an important fact that can be used to forge a coalition to bring hard-won research knowledge to light in the classroom.

• Adams, M. J. (1990). Beginning to read: Thinking and learning about print . Cambridge, MA: MIT Press.
• Adler, J. E. (1998, January). Open minds and the argument from ignorance. Skeptical Inquirer , 22 (1), 41-44.
• Anderson, C. A., & Anderson, K. B. (1996). Violent crime rate studies in philosophical context: A destructive testing approach to heat and Southern culture of violence effects. Journal of Personality and Social Psychology , 70 , 740-756.
• Anderson, R. C., Hiebert, E. H., Scott, J., & Wilkinson, I. (1985). Becoming a nation of readers . Washington, D. C.: National Institute of Education.
• Arter, A. and Jenkins, J. (1979). Differential diagnosis-prescriptive teaching: A critical appraisal, Review of Educational Research , 49 , 517-555.
• Beck, A. R., & Pirovano, C. M. (1996). Facilitated communications' performance on a task of receptive language with children and youth with autism. Journal of Autism and Developmental Disorders , 26 , 497-512.
• Beck, I. L. (1996, April). Discovering reading research: Why I didn't go to law school . Paper presented at the Reading Hall of Fame, International Reading Association, New Orleans.
• Beck, I. (1998). Understanding beginning reading: A journey through teaching and research. In J. Osborn & F. Lehr (Eds.), Literacy for all: Issues in teaching and learning (pp. 11-31). New York: Guilford Press.
• Berliner, D. C. (1987). Knowledge is power: A talk to teachers about a revolution in the teaching profession. In D. C. Berliner & B. V. Rosenshine (Eds.), Talks to teachers (pp. 3-33). New York: Random House.
• Bjorklund, D. F. (1995). Children's thinking: Developmental function and individual differences (Second Edition) . Pacific Grove, CA: Brooks/Cole.
• Block, C. C., & Pressley, M. (Eds.). (2002). Comprehension instruction: Research-based best practices . New York: Guilford Press.
• Bronowski, J. (1956). Science and human values . New York: Harper & Row.
• Bronowski, J. (1973). The ascent of man . Boston: Little, Brown.
• Bronowski, J. (1977). A sense of the future . Cambridge: MIT Press.
• Burgess, C. A., Kirsch, I., Shane, H., Niederauer, K., Graham, S., & Bacon, A. (1998). Facilitated communication as an ideomotor response. Psychological Science , 9 , 71-74.
• Chard, D. J., & Osborn, J. (1999). Phonics and word recognition in early reading programs: Guidelines for accessibility. Learning Disabilities Research & Practice , 14 , 107-117.
• Cooper, H. & Hedges, L. V. (Eds.), (1994). The handbook of research synthesis . New York: Russell Sage Foundation.
• Cunningham, P. M., & Allington, R. L. (1994). Classrooms that work: They all can read and write . New York: HarperCollins.
• Dawkins, R. (1998). Unweaving the rainbow . Boston: Houghton Mifflin.
• Dennett, D. C. (1995). Darwin's dangerous idea: Evolution and the meanings of life . New York: Simon & Schuster.
• Dennett, D. C. (1999/2000, Winter). Why getting it right matters. Free Inquiry , 20 (1), 40-43.
• Ehri, L. C., Nunes, S., Stahl, S., & Willows, D. (2001). Systematic phonics instruction helps students learn to read: Evidence from the National Reading Panel's Meta-Analysis. Review of Educational Research , 71 , 393-447.
• Foster, E. A., Jobling, M. A., Taylor, P. G., Donnelly, P., Deknijff, P., Renemieremet, J., Zerjal, T., & Tyler-Smith, C. (1998). Jefferson fathered slave's last child. Nature , 396 , 27-28.
• Fraenkel, J. R., & Wallen, N. R. (1996). How to design and evaluate research in education (Third Edition). New York: McGraw-Hill.
• Geertz, C. (1973). The interpretation of cultures . New York: Basic Books.
• Geertz, C. (1979). From the native's point of view: On the nature of anthropological understanding. In P. Rabinow & W. Sullivan (Eds.), Interpretive social science (pp. 225-242). Berkeley: University of California Press.
• Gersten, R. (2001). Sorting out the roles of research in the improvement of practice. Learning Disabilities: Research & Practice , 16 (1), 45-50.
• Gersten, R., Chard, D., & Baker, S. (2000). Factors enhancing sustained use of research-based instructional practices. Journal of Learning Disabilities , 33 (5), 445-457.
• Gersten, R., & Dimino, J. (2001). The realities of translating research into classroom practice. Learning Disabilities: Research & Practice , 16 (2), 120-130.
• Gersten, R., Vaughn, S., Deshler, D., & Schiller, E. (1997).What we know about using research findings: Implications for improving special education practice. Journal of Learning Disabilities , 30 (5), 466-476.
• Goswami, U. (1998). Cognition in children . Hove, England: Psychology Press.
• Gross, P. R., Levitt, N., & Lewis, M. (1997). The flight from science and reason . New York: New York Academy of Science.
• Hedges, L. V., & Olkin, I. (1985). Statistical Methods for Meta-Analysis . New York: Academic Press.
• Holton, G., & Roller, D. (1958). Foundations of modern physical science . Reading, MA: Addison-Wesley.
• Hudson, A., Melita, B., & Arnold, N. (1993). A case study assessing the validity of facilitated communication. Journal of Autism and Developmental Disorders , 23 , 165-173.
• Hunter, J. E., & Schmidt, F. L. (1990). Methods of meta-analysis: Correcting error and bias in research findings . Newbury Park, CA: Sage.
• Jacobson, J. W., Mulick, J. A., & Schwartz, A. A. (1995). A history of facilitated communication: Science, pseudoscience, and antiscience. American Psychologist , 50 , 750-765.
• Kamil, M. L. (1995). Some alternatives to paradigm wars in literacy research. Journal of Reading Behavior , 27 , 243-261.
• Kampwirth, R., and Bates, E. (1980). Modality preference and teaching method: A review of the research, Academic Therapy , 15 , 597-605.
• Kavale, K. A., & Forness, S. R. (1995). The nature of learning disabilities: Critical elements of diagnosis and classification . Mahweh, NJ: Lawrence Erlbaum Associates.
• Levin, J. R., & O'Donnell, A. M. (2000). What to do about educational research's credibility gaps? Issues in Education: Contributions from Educational Psychology , 5 , 1-87.
• Liberman, A. M. (1999). The reading researcher and the reading teacher need the right theory of speech. Scientific Studies of Reading , 3 , 95-111.
• Magee, B. (1985). Philosophy and the real world: An introduction to Karl Popper . LaSalle, IL: Open Court.
• Mayer, R. E. (2000). What is the place of science in educational research? Educational Researcher , 29 (6), 38-39.
• McNiff, J.,Lomax, P., & Whitehead, J. (1996). You and your action research project . London: Routledge.
• Medawar, P. B. (1982). Pluto's republic . Oxford: Oxford University Press.
• Medawar, P. B. (1984). The limits of science . New York: Harper & Row.
• Medawar, P. B. (1990). The threat and the glory . New York: Harper Collins.
• Moats, L. (1999). Teaching reading is rocket science . Washington, DC: American Federation of Teachers.
• National Reading Panel: Reports of the Subgroups. (2000). Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction . Washington, DC.
• Nickerson, R. S. (1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology , 2 , 175-220.
• Pearson, P. D. (1993). Teaching and learning to read: A research perspective. Language Arts , 70 , 502-511.
• Pearson, P. D. (1999). A historically based review of preventing reading difficulties in young children. Reading Research Quarterly , 34 , 231-246.
• Plotkin, D. (1996, June). Good news and bad news about breast cancer. Atlantic Monthly , 53-82.
• Popper, K. R. (1972). Objective knowledge . Oxford: Oxford University Press.
• Pressley, M. (1998). Reading instruction that works: The case for balanced teaching . New York: Guilford Press.
• Pressley, M., Rankin, J., & Yokol, L. (1996). A survey of the instructional practices of outstanding primary-level literacy teachers. Elementary School Journal , 96 , 363-384.
• Rayner, K. (1998). Eye movements in reading and information processing: 20 Years of research. Psychological Bulletin , 124 , 372-422.
• Rayner, K., Foorman, B. R., Perfetti, C. A., Pesetsky, D., & Seidenberg, M. S. (2002, March). How should reading be taught? Scientific American , 286 (3), 84-91.
• Reading Coherence Initiative. (1999). Understanding reading: What research says about how children learn to read . Austin, TX: Southwest Educational Development Laboratory.
• Rosenthal, R. (1995). Writing meta-analytic reviews. Psychological Bulletin , 118 , 183-192.
• Rosnow, R. L., & Rosenthal, R. (1989). Statistical procedures and the justification of knowledge in psychological science. American Psychologist , 44 , 1276-1284.
• Shankweiler, D. (1999). Words to meaning. Scientific Studies of Reading , 3 , 113-127.
• Share, D. L., & Stanovich, K. E. (1995). Cognitive processes in early reading development: Accommodating individual differences into a model of acquisition. Issues in Education: Contributions from Educational Psychology , 1 , 1-57.
• Shavelson, R. J., & Towne, L. (Eds.) (2002). Scientific research in education . Washington, DC: National Academy Press.
• Siegler, R. S. (1991). Children's thinking (Second Edition) . Englewood Cliffs, NJ: Prentice Hall.
• Snow, C. E., Burns, M. S., & Griffin, P. (Eds.). (1998). Preventing reading difficulties in young children . Washington, DC: National Academy Press.
• Snowling, M. (2000). Dyslexia (Second Edition) . Oxford: Blackwell.
• Spear-Swerling, L., & Sternberg, R. J. (2001). What science offers teachers of reading. Learning Disabilities: Research & Practice , 16 (1), 51-57.
• Stahl, S. (December, 1988). Is there evidence to support matching reading styles and initial reading methods? Phi Delta Kappan , 317-327.
• Stanovich, K. E. (1993/1994). Romance and reality. The Reading Teacher , 47 (4), 280-291.
• Stanovich, K. E. (2000). Progress in understanding reading: Scientific foundations and new frontiers . New York: Guilford Press.
• Stanovich, K. E. (2001). How to think straight about psychology (Sixth Edition). Boston: Allyn & Bacon.
• Stokes, D. E. (1997). Pasteur's quadrant: Basic science and technological innovation . Washington, DC: Brookings Institution Press.
• Swanson, H. L. (1999). Interventions for students with learning disabilities: A meta-analysis of treatment outcomes . New York: Guilford Press.
• Tarver, S. G., & Dawson, E. (1978). Modality preference and the teaching of reading: A review, Journal of Learning Disabilities , 11, 17-29.
• Vaughn, S., & Dammann, J. E. (2001). Science and sanity in special education. Behavioral Disorders , 27, 21-29.
• Warby, D. B., Greene, M. T., Higgins, K., & Lovitt, T. C. (1999). Suggestions for translating research into classroom practices. Intervention in School and Clinic , 34 (4), 205-211.
• Wheeler, D. L., Jacobson, J. W., Paglieri, R. A., & Schwartz, A. A. (1993). An experimental assessment of facilitated communication. Mental Retardation , 31 , 49-60.
• Wilkinson, L. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist , 54 , 595-604.
• Wilson, E. O. (1998). Consilience: The unity of knowledge . New York: Knopf.

For additional copies of this document:

Contact the National Institute for Literacy at ED Pubs PO Box 1398, Jessup, Maryland 20794-1398

Phone 1-800-228-8813 Fax 301-430-1244 [email protected]

Date Published: 2003 Date Posted: March 2010

Using research to improve education under the Every Student Succeeds Act

Subscribe to the center for economic security and opportunity newsletter, mark dynarski mark dynarski owner - pemberton research, former brookings expert.

December 10, 2015

• 14 min read

Executive summary

The Every Student Succeeds Act, the new reauthorization of the federal program designed to support the education of disadvantaged students, requires that states and districts use evidence-based interventions to support school improvement. Researchers have studied the effectiveness of education programs for decades and that effort is now producing substantial gains in knowledge of what works and what doesn’t.  But educators note that this kind of research is not as useful as it could be for them because it is conducted in settings that differ from theirs. They are interested in research that fits their contexts.

Recently, another kind of research paradigm has emerged in which researchers work directly with educators to identify and implement paths for improvement within particular settings. This new kind of research—which has come to be known as improvement science—operates in local contexts of districts and schools. But it faces a capacity problem because there are relatively few researchers participating or able to participate in these efforts compared to the number of districts and schools that could benefit from more evidence-based programs and practices.

The two approaches need to be coordinated. In the first stage, research would identify effective programs and practices writ large. In the second stage, districts or schools not meeting targets or objectives would work with improvement-science teams to adapt those research-proven programs to local contexts.

The ‘Every Student Succeeds Act’ also creates a new program to support research on innovations in education. Using the existing infrastructure of the regional lab network can help identify priorities for this new research on innovations. The priorities should fill gaps in knowledge and proven programs that states and districts have identified as important to them. 

Scaling up effective education policies or programs is in everyone’s interests. Who argues that education should not improve? And findings from research often underscore where improvements are possible and where education can be more effective. But scaling up findings from research—having the findings lead to actions on a larger scale—is a challenge.

The issue is partly the size and dispersion of authority of the public-education enterprise, with its 15,000 districts, 65,000 schools, 4 million teachers, and 55 million students. For an improvement to find its way into even a fraction of this enterprise counts as progress and might be seen as miraculous.

How does the process of adoption appear to be working?  Do improvements identified by research find their way into the enterprise at all? Brian Jacob’s recent note here on Evidence Speaks commented on learning from research ‘failures,’ which arise when evidence emerges that a promising idea did not improve education outcomes when tested rigorously. A related question is whether the enterprise learns from research successes, when evidence emerges that a promising idea works. Do these successes become education practices?

The answer is hard to know because the extent to which research finds its way into schools and classrooms has not been measured. When educators are asked about research, however, they point to their perception of research having a “local perspective” as one reason for caution about using the research for programmatic decisions. They give more credence to findings that arise in contexts similar to their districts or schools compared to findings emerging elsewhere. [i]

This ‘localism,’ for lack of a better word, combined with limited avenues for research dissemination, has led to new forms of research in which researchers work directly with educators to develop local practices and programs. In the words of one of its foremost practitioners, this research takes a ‘design-engineering-development’ perspective, working from the ground up to tackle educators’ problems. [ii] This approach is now known as ‘improvement science.’ [iii]

This local approach sounds like an ideal way to move research closer to what educators value, program development and evidence about outcomes that occurs in their particular schools or districts and that results from a researcher-educator collaboration.  Maybe there will come a day in which most schools or school districts are sufficiently resourced to have their own program development and evaluation teams. But even then local improvement science would need to be a complement to research efforts to identify effective practices, not a substitute for them. Working with educators to promote more effective reading or math instruction, for example, needs to begin with sound research on reading and math instruction. Improvement science then can focus on encouraging and promoting the take-up of that sound research, working with educators to adapt what are believed to be key ingredients of the approaches. We’ll return to that two-stage approach below.

Identifying effective practices is not the same as implementing them

The dominant approach to studying a question of whether a practice or program improves an education outcome is known as ‘effectiveness’ research. Often experiments are used to answer the question—Are teachers who attend these workshops more effective? Does this dropout-prevention program keep students in school? Does using this software lead to stronger math skills? There are different kinds of methodologies that address effectiveness, including randomized controlled trials, regression-discontinuity designs, well-designed quasi-experimental studies, and single-case designs, but it is convenient to use ‘experiments’ as a term for all of these ways of measuring the effects of programs, practices, or policies.

Experiments were relatively uncommon in education, certainly compared to their use in medicine, until 2002, when Congress created the Institute of Education Sciences. Since 2002, IES has funded hundreds of experiments and disseminated their results, mostly through the web and through workshops, webinars, and social media channels. It also disseminates syntheses of research and appraisals of individual effectiveness studies through its ‘What Works Clearinghouse,’ by way of the web and through the other channels. The information reaches a large audience. Practice guides produced by the What Works Clearinghouse are downloaded about 22,000 times a month. One of the Clearinghouse’s most popular practice guides was downloaded nearly 90,000 times in its first month of release. [iv]

For disseminating research findings cheaply, it’s hard to top this model. In principle, every educator can hear about every finding of relevance to them for the cost of looking up a web page or watching a recorded webinar on YouTube. But whether the findings actually change educator practices is not known. Educators could be ignoring the findings and continuing to do what they are doing. The Government Accountability Office expressed concerns about this possible disconnect in its recent review of IES. [v]

Findings from experiments are information, but changing practices to do something with the findings is implementation. As Pfeffer and Sutton (2000) have written, knowing is a long way from doing. [vi] ‘Improvement science’ strives to close the gap between knowing and doing. At a risk of oversimplifying, improvement science poses a model—such as the ‘design-engineering-development’ one mentioned above—in which researchers work directly with educators in districts and schools. The focus is on using rapid tests of change and ‘Plan-Do-Study-Act’ cycles to learn by doing, and connecting participants (teachers, principals, administrators) through networks to expedite their learning.

For example, the Carnegie Foundation for the Advancement of Teaching is collaborating with community colleges to promote success in math, with urban school districts to improve skills of their new teachers, and with school districts and organizations to design classroom experiences that promote ‘academic mindsets’ and support students to develop their own learning strategies. [vii] Vanderbilt University is collaborating with school districts to enhance middle-school math curricula and create new kinds of professional development and teacher networks to improve math teaching. [viii]

By its nature, improvement science focuses intensively within districts. That focus is a plus because researchers and educators are at the table, but also a minus because there are not enough researchers to be at the many tables that need them. There are about four times more school districts than higher-education institutions, and many higher-education institutions have no one with the time or, in many cases, interest or expertise, to anchor the improvement science effort at a local school district. Unless improvement science can generate knowledge of how schools and districts can improve without researchers being involved in thousands of school districts, the limited number of researchers essentially precludes scaling up. And if the knowledge improvement science generates in a few districts has to be disseminated to many others, improvement science ends up being in the same place as effectiveness research: educators might not hear about the findings or might view them as too distant to be useful in their local areas.

So findings from experiments that are intended to produce generalizable results can be inexpensively disseminated but might not be used by educators, and improvement science may yield more local knowledge but cannot operate widely because of capacity and generalizability issues. The key is for effectiveness research to be more ‘localized’—more applicable to educators in their schools and districts — without it necessarily having to be produced locally.

Experiments can be more useful to educators

Depending on the intervention or policy being considered, a district or school needs to first learn of an improvement—for example, a relevant research finding that has been published, written about in the media, or passed along to local educators through word of mouth. Then, the educators’ questions become local. How much did staff in the study differ from staff in their schools? Did characteristics of students have a role in the experiment’s findings? Are the powers-that-be in the local education system, including its teachers, open to the type of program that generated the positive research findings? Does the state or district have the authority to implement the program that the research studied? Can school or district afford the out-of-pocket expense for licenses, fees, or materials, and the staff time to learn how to do the program?

These are a lot of judgments, and a significant gap opens up between how a researcher may view the findings—‘the study shows that the approach worked’—and how an educator might view the findings—‘it worked for somebody but I don’t know if it can work for me.’ What a researcher views as evidence becomes what an educator views as one variable in a risk equation when the risk itself is avoidable by sticking to the tried and true.

Move the approaches closer together

Both effectiveness research and improvement science can add to knowledge and both approaches can be useful. Both need to measure effects, provide information to prospective adopters about how to implement the program or approach, and be explicit about how much it will cost.

Scaling up should be the starting point in thinking about how to design experiments and improvement science efforts. If studies were designed from the view of scaling up, they would focus on developing information prospective adopters need: how large are effects, how can the program be implemented, and how much is it likely to cost? Approaches to effectiveness studies vary somewhat in how they measure effects, but they vary much more in how they study costs and implementation. Costs are rarely analyzed, and while some experiments report several hundred pages of detailed information about implementation, others describe implementation in a report chapter and many published papers simply do not mention it. [ix] The risks educators face in implementing programs shown by research to be effective would be mitigated if research on implementation focused on creating a manual on how to carry out the program. Researchers developed a process for ‘manualization’ more than a decade ago, but that process is rarely used in studies of education programs. [x]

Using a two-stage model for generating evidence on effective, implementable interventions will help put experiments and improvement science into balance. In the first stage, districts and schools would be able to learn about recent research on effective practices, how to implement those practices, and their costs. Currently there is no organization doing all that is envisioned here for the first stage of the model. The What Works Clearinghouse and Best Evidence Encyclopedia provide information on evidence of effects but little information about implementation and cost. [xi] This is not for lack of interest on their part; most research studies and reports provide too little information about implementation and cost, and standards are not in place for how to assess what is provided. Efforts to document implementation and cost need to be increased for this stage to be useful.

The second stage is improvement science. Districts that use evidence from the first stage but are not satisfied with the results or do not meet targets can work with improvement scientists to adapt interventions with evidence of effectiveness and monitor the results. The second stage needs only enough improvement-science capacity to work with districts that are committed to it. This may still be too many districts and not enough capacity, but starting from the total number of districts certainly overwhelms capacity, as noted above, whereas thinking of improvement science as targeted moves it into the realm of practicality.

The federal role in the two-stage model

The ‘ Every Student Succeeds Act’ gives states responsibility to develop accountability structures. These structures need to include ‘comprehensive support and improvement plans’ for schools that need improvement, and these plans must include evidence-based interventions. Using the two-stage approach—with districts and schools moving to the second stage if improvement targets are not met—is a sensible means to develop a pool of evidence-based interventions that meet state needs.

Schools also might move into the second stage if they fall in the 5 percent of schools for which Congress is requiring states to intervene. (States can choose to intervene in more schools, but not less than 5 percent.) The constellation of issues these schools face is an opportunity for educators and researchers to work together to identify improvements and implement new approaches. Having improvement-science teams working with schools that are most in need of improvement is a reasonable way to blend the strengths of the two approaches.

The two-stage model also can be connected to the new ‘Innovation Research’ section of the Act (section 4611). The section calls for the U.S. Department of Education to fund research to develop, test, and scale effective practices. The language does not indicate how funding priorities are to be identified.    

One way to do so is to ground priorities in expressed state and local needs. For example, if language acquisition is a need in rural areas of the Southwest, and research identified in the first stage is thought to be inadequate, some of the innovation grants could be used to fill that gap. Similarly, states and districts might express needs to bolster reading, math, or kindergarten readiness, or any number of other objectives. The regional lab network already has an infrastructure for assessing needs at state and district levels. It can have a role in tying these needs to innovation priorities and monitoring whether needs change over time. The Institute of Education Sciences within the department, which operates the labs under contract, is well positioned to work with the labs to identify innovation priorities emerging from local needs. Labs also can conduct research to meet needs that do not become innovation priorities.

Education research is sparsely funded and is unlikely to enjoy the resources of the National Institutes of Health any time soon. Effectiveness research and improvement science need to be deployed in concert to make the best use of these scarce resources.

[i] The most common theme that emerged from a survey of educators about hurdles in their use of evidence was ‘localism.’ See Nelson, S., J. Leffler, and Barbara Hansen. “Toward a Research Agenda for Undestanding and Improving the Use of Research Evidence. Accessed December 8, 2015: http://educationnorthwest.org/sites/default/files/toward-a-research-agenda.pdf .

[ii] See Bryk, A., and L Gomez. “Ruminations on Reinventing an R&D Capacity for Education Improvement.” Accessed December 8, 2015: http://cdn.carnegiefoundation.org/wp-content/uploads/2014/09/DED_paper.pdf .

[iii] See ‘Learning To Improve: How America’s Schools Can Get Better At Getting Better,’ by Anthony Bryk, Louis Gomez, Alicia Grumow, and Paul LeMahieu (2015), Bryk and Gomez’s 2008 paper , and the National Academies monograph laying out a design for the Strategic Education Research Partnership. Cohen-Vogel and her colleagues provide a succinct recounting of the emergence of improvement science against the backdrop of experiments. See L. Cohen-Vogel et al., “Implementing Educational Innovations At Scale: Transforming Researchers into Continuous Improvement Scientists.” Educational Policy, vol. 29(1), 257-277. Roots of improvement science can be found in the work of Donald Berwick in health-care, beginning in the eighties. Berwick, D. “The Science of Improvement.” Journal of the American Medical Association, 2008, 299(10), pp.1182-1184.

[iv] Full disclosure: I directed the What Works Clearinghouse from 2008 to 2010, when practice guides were first released, and I chaired a panel that produced one of the first guides. I continue to be involved with the Clearinghouse in various roles.

[v] IES recently funded two research centers to explore these topics but it will be several years before findings are known.

[vi] Jeffrey Pfeffer and Robert Sutton. ‘The Knowing-Doing Gap.’ Harvard Business School Press: 2000.

[vii] http://www.carnegiefoundation.org/in-action/student-agency-improvement-community/

[viii] http://peabody.vanderbilt.edu/departments/tl/teaching_and_learning_research/mist/index.php

[ix] For examples, see the Reading First study reported here , the study of teacher induction programs reported here , and the study of supplemental services reported here .

[x] For more on manualization, see http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4126244/ and Carroll KM, Nuro KF. One size cannot fit all: A stage model for psychotherapy manual development. Clinical Psychology: Science and Practice. 2002; 9(4): 396–406.

[xi] http://www.bestevidence.org/

Education Technology

Economic Studies

Center for Economic Security and Opportunity

Jing Liu, Cameron Conrad, David Blazar

May 1, 2024

Hannah C. Kistler, Shaun M. Dougherty

April 9, 2024

Darrell M. West, Joseph B. Keller

February 12, 2024

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List

Growing Brains, Nurturing Minds—Neuroscience as an Educational Tool to Support Students’ Development as Life-Long Learners

Associated data.

The study did not report any data.

Compared to other primates, humans are late bloomers, with exceptionally long childhood and adolescence. The extensive developmental period of humans is thought to facilitate the learning processes required for the growth and maturation of the complex human brain. During the first two and a half decades of life, the human brain is a construction site, and learning processes direct its shaping through experience-dependent neuroplasticity . Formal and informal learning, which generates long-term and accessible knowledge, is mediated by neuroplasticity to create adaptive structural and functional changes in brain networks. Since experience-dependent neuroplasticity is at full force during school years, it holds a tremendous educational opportunity. In order to fulfill this developmental and learning potential, educational practices should be human-brain-friendly and “ride” the neuroplasticity wave. Neuroscience can inform educators about the natural learning mechanisms of the brain to support student learning. This review takes a neuroscientific lens to explore central concepts in education (e.g., mindset, motivation, meaning-making, and attention) and suggests two methods of using neuroscience as an educational tool: teaching students about their brain (content level) and considering the neuro-mechanisms of learning in educational design (design level).

1. Educational Neuroscience (Teaching for the Brain and Teaching about the Brain)

Educational neuroscience is an interdisciplinary field exploring the effects of education on the human brain and promotes the translation of research findings to brain-based pedagogies and policies [ 1 ]. The brain is the target organ of education. Education is thought to influence brain development [ 2 , 3 ] and health, even as the brain ages [ 4 , 5 ]. Studying the dynamics between the brain and education can be instrumental in finding ways to better support learners across the lifespan.

Educational neuroscience research explores every possible relationship between the physiological, mental, and behavioral aspects of learning. Some studies have tried to identify the optimal physical conditions for neuroplasticity and learning. This stream of educational neuroscience research includes studies exploring the effects of sleep (or sleep deprivation), physical exercise, and environmental pollution on the brain and its cognitive performance [ 1 ]. While these studies focus on the effect of brain health on learning, other studies examine the effect of learning on brain health, assessing the long-term effects of learning/education on the human brain and exploring in what ways formal/informal education is associated with better aging of the human brain [ 2 , 3 , 4 ].

Some educational neuroscience studies take a developmental approach to study the relationship between cognitive and learning capacities across the lifespan. For example, multilevel measurements collected from adolescents (e.g., neuronal, hormonal, psychological, and behavioral) have advanced our understanding of how the massive neuronal changes that take place during adolescence promote cognitive development but also introduce immense neuronal and mental vulnerability (and the onset of most psychiatric disorders) [ 1 , 5 , 6 , 7 ]. Other studies in this line of research explore the factors supporting neuroplasticity in the mature brain—to support lifelong learning [ 8 ].

Educational neuroscience also explores the nature–nurture aspects of learning, for example, examining how learning environments interact with genetic conditions and what DNA variations predict differential learning abilities [ 9 ]. Environmental influences on learning include studies about the impacts of socio-economic status (SES) on the brain and cognitive developmental trajectory [ 10 ]. Furthermore, educational neuroscience seeks to understand the mechanisms that facilitate general learning abilities (such as executive control and social and emotional skills), discipline-specific learning abilities (such as literacy, numeracy, and science), the connections between these mechanisms, and the extent to which these learning skills are trainable [ 11 ].

As a developing, interdisciplinary research field, educational neuroscience faces challenges, limitations, and criticism, especially concerning the ability to generalize research findings in lab conditions to classroom learning, and its validity and transferability to larger scales, such as mass education systems. Other challenges stem from the fact that learning is one of the most basic yet complex brain functions that incorporates the entire brain and has a continuous effect. Furthermore, empirical studies in educational neuroscience are challenging and cumbersome due to the interdisciplinary nature of the field (education, psychology, and neuroscience); the need for repeated measures over time; and the young target population (school students), which imposes ethical restrictions on experimental designs. Finally, while still evolving as a research field, educational neuroscience is intriguing for many educational leaders who are enthusiastic about applying neuroscience in education practices. Unfortunately, the current gap between the high demand and limited supply may lead to misuse of neuroscience in pedagogy (e.g., neuromyths or the justification of educational methods based on limited to no evidence) [ 1 ].

While educational neuroscience is preliminary in forming evidence-based pedagogy, it can already offer valuable information and a much-needed bridge between educators and scientists in translating the research of learning into effective educational practices.

Neuroscience-informed educational design (teaching the way the brain learns) can promote learning motivation, high-level information processing, and knowledge retention. Moreover, neuroscience educational content (teaching about the brain) can inform students about their developing brains to promote scientific education and self-exploration.

1.1. Learning and Neuroplasticity

Human development is based on nature (genetics), nurture (physical and social environments), and their interactions (epigenetics) [ 12 , 13 ]. These factors play an essential role in learning processes and the reorganization of neuronal networks to create neuronal representations of new knowledge. Learning and training new knowledge or skills evoke specific and repeated activity patterns, and in the process of Hebbian neuroplasticity, neural pathways are reinforced by the strengthening of specific synapses, while less functional ones are eliminated [ 14 , 15 , 16 ].

Almost half a century ago, Vygotsky introduced the zone of proximal development (ZPD) [ 17 ] in education. According to the ZPD, learning and development depend on an optimal balance between support and challenge (see Figure 1 : the zone of proximal development and neuroplasticity), which should be tuned and tailored for each learner based on their specific developmental stage. The ZPD model was revolutionary, as it emphasized the importance of the educational environment (nurture) in unlocking the internal potential (nature) of students, and it placed the learning process (as opposed to the learning product ) as the central educational goal [ 17 ]. Some decades later, the biology of learning revealed a beautiful alignment with Vygotsky’s theory—with evidence showing that brain neuroplasticity is highly affected by environmental conditions and the balance between demands (challenge) and available resources (support) [ 18 ]. The impact of stressors on learning can be constructive or destructive depending on the intensity, duration, and accumulation of the stressors and the coping mechanisms and support that one has.

The zone of proximal development and neuroplasticity. An integrative approach between Vygotsky’s educational model and the neuroscience of learning. When learning and performance demands exceed the available support and resources, students are likely to be overwhelmed and resort to survival mode (stress zone). When learning and performance demands are significantly lower than the available support and resources, students are likely to be under stimulated and resort to static mode (comfort zone). When learning and performance demands match the available support and resources, students are likely to be appropriately challenged and work within their zone of proximal development, which promotes neuroplasticity and growth (stretch zone).

Neuroscience research suggests that experience-dependent neuroplasticity [ 19 ], which facilitates learning processes, benefits from several principles. The central one is that learning a skill or new knowledge requires the activation of relevant neuronal pathways. The research also points to the saliency, intensity, and repetition of the learned skill/knowledge as valuable strategies for enhancing neuroplastic changes [ 16 , 20 , 21 ]. Learners cannot be passive recipients of content but must be active participants in the learning process.

An enriched environment for enhanced neuroplasticity offers physiological integrity, cognitive challenge, and emotional safety. More specifically, an enriched environment includes adequate sleep and nutrition, sensory–motor and cognitive challenges, opportunities for exploration and novelty, and secured relationships that act like a safety net and enable learners to take on challenges [ 22 , 23 ]. Conversely, a lack of these conditions may slow down or decrease the level of neuroplasticity in the developing brain.

The social and cognitive safety net that enables learners to aim high while taking risks and to turn failure into resilience is rooted in safe relationships (with adults and peers) and in holding a growth mindset. A growth mindset is the belief that intelligence and learning potential are not fixed and can be developed [ 24 ]. Holding a growth mindset has been associated with academic success, emotional wellbeing, and motivation while reducing racial, gender, and social class achievement gaps [ 25 , 26 , 27 , 28 , 29 , 30 ]. While the impact of mindset interventions on academic performance is debatable regarding the general population [ 31 ], the literature is clear about the potential of growth mindset intervention in supporting the academic development of high-risk and economically disadvantaged students [ 26 , 27 , 31 , 32 ].

The notion of human potential as something dynamic resonates with the concept of the plastic brain. Moreover, teaching students about neuroplasticity and the dynamic potential of their brains has been shown to effectively reinforce a growth mindset [ 32 ].

1.1.1. Using Neuroplasticity as Educational Content

Teaching students about experience-based neuroplasticity and the dynamic changes in neuronal networks during learning provides strong evidence of their natural and powerful learning capacity. Furthermore, teaching students about neuroplasticity with explicit connections to the growth mindset and development creates a motivating premise for learners—according to which their learning potential is dynamic and depends significantly on their attitudes and learning practices.

The neuroplasticity rules of “use it or lose it” and “use it to improve it” mean that, while teachers should support and guide them, learning occurs by and within the students. This physiology-based realization can help build students’ responsibility and ownership over their learning.

Harnessing neuroplasticity and a growth mindset to motivate students can be especially important with neurodivergent learners, whose cognitive development and learning styles deviate from the typical range. Twenty percent of the population is neurodivergent, including students on the autistic spectrum (ASD), students with learning disabilities (e.g., dyslexia), attention disorders (e.g., ADHD), neurological disorders (e.g., epilepsy), and mental illness (e.g., PTSD). While neurodiversity and variations in neuronal and cognitive expressions hold many advantages [ 33 ], neurodivergent students face extra challenges navigating neurotypical-oriented school systems. Learning about neuroplasticity can be a potent form of validation for neurodivergent students, as neurodiversity is a natural result of experience-dependent neuroplasticity [ 19 ]. In addition, by fostering a growth mindset and neuroplasticity awareness, neurodivergent students can be motivated to participate in evidence-based interventions. For example, teaching students with dyslexia about the specific structural and functional brain changes associated with the reading interventions that they apply [ 34 , 35 , 36 ] can motivate them to endure the hard work before noticing visible results.

1.1.2. Using Neuroplasticity to Guide Learning Design

Organizing learning systems around conditions that promote neuroplasticity can enhance learners’ academic development and wellbeing. When a student accomplishes today what was not in their reach yesterday, it is the product of neuroplasticity through a growth mindset.

Educational environments that promote neuroplasticity include encouraging and modeling a healthy lifestyle (physical exercise, a balanced diet, sufficient sleep, and regulated stress), —for example, educating students about the counter-productiveness of sleep deprivation (e.g., “all-nighter” study marathons) on learning. In addition, learning systems should invest in intellectual stimulation (novelty and challenge) and the system’s social and emotional climate (human connections). Neuroplasticity and development are optimal in the stretch zone, where learners experience a motivating level of challenge and stimulation while feeling emotionally supported and socially safe. This ratio between support and challenge should be individualized (between learners and within learners over time).

Educating teachers about neuroplasticity can be powerful in understanding and supporting students that were affected by trauma. Childhood adversity hampers neuroplasticity duration and magnitude [ 37 ]; a surviving brain is not a learning brain. While neuroplasticity is compromised by early trauma, neuroplasticity is also the key to healing from trauma. Schools have a pivotal role in battling the damage of early trauma by creating enriched and safe learning environments that reinforce alternative neuronal pathways to reverse the effects of early adverse environments on child brain development [ 22 , 38 , 39 , 40 ].

1.2. Learning Motivation and Reward

Learning and adaptation are essential for surviving and thriving in dynamic environments. The brain evolved to make sense of information from our external and internal environments and to produce adaptive behaviors that promote survival. The brain is, therefore, a learning machine by nature, and learning does not require external initiation. However, learning is highly experience-dependent and can be directed and enhanced through education.

The brain reward system evolved to reinforce effortful behaviors that are essential for survival (e.g., foraging, reproduction, and caregiving). Such behaviors activate the dopaminergic system associated with reward and motivation [ 41 ]. The hormone/neurotransmitter dopamine is a central player in reward-motivated behavior and learning through the modulation of striatal and prefrontal functions [ 42 ]. The human brain reward system balances between (limbic) impulsive desire and (cortical) goal-directed wanting to guide flexible decision-making and adaptive motivational behaviors.

Psychologically, intrinsic motivation is driven by the need to experience a sense of competence, self-determination, and relatedness [ 43 , 44 , 45 , 46 , 47 ].

Competence refers to a perception of self-efficacy and confidence in one’s abilities to achieve a valuable outcome. Self-determination refers to the sense of autonomy and agency in the learning process. Relatedness refers to the drive to pursue goals that hold social value, which can be achieved by working collaboratively as part of a team or by creating something that resonates with others. Relatedness is a strong motivational driver, as it touches on a primary and primordial need to be part of a group and a higher spiritual and intellectual need for self-transcendence and impact.

Overall, these components are based on the human inclination to be valued and validated by the self and others. Biologically, they reflect basic survival needs that combine self-reliance (competence and ownership) and social reliance. Psychologically, these are all subjective perceptions that serve the need to maintain positive self-perception and self-integration. Finally, educationally, they reflect the natural human curiosity and tendency to learn and develop continuously.

The human brain reward system in the 21st century is an evolutionary mismatch. There is a discrepancy between the conditions that the reward system evolved to serve and those that it often faces in the 21st century. The reward system evolved over millions of years to motivate humans to work hard (invest time and energy) in maintaining their survival needs (e.g., nutrition, protection, reproduction, and the learning of new skills). However, this system is not designed for the abundance and immediacy of stimulation in the digital and instant reward era, which promotes the persistent release of dopamine that leads to an increased craving for reward (seeking behavior; wanting) and a decreased sense of pleasure and satisfaction (liking) [ 42 , 48 ].

Some of the most significant challenges of modern education systems relate to the massive changes in how people consume information and communicate in the digital era. Digital platforms have become dominant in information consumption and communication, which provide access to unlimited information and reinforce immediate rewards.

1.2.1. Using Neuroscience (of Reward and Motivation) as Educational Content

The science of human motivation, including its evolutionary mismatch, can be utilized to shed some light on students’ struggles with learning motivation. It can further provide a framework for students to explore their motivational (approach or avoid) tendencies regarding learning and academic challenges. Moreover, learning the neuroscience underlying motivation and reward can raise students’ awareness and proactivity in managing and protecting their reward system. Since adolescence is the peak time for the initiation of substance use, and early onset imposes a higher risk of mental health and substance abuse disorders persisting into adulthood [ 49 , 50 , 51 ], neuroscience knowledge about the reward system and its vulnerability (especially during brain development) is essential educational knowledge that can help in the prevention and mitigation of teen addiction.

1.2.2. Using Neuroscience to Guide Learning Design and Intrinsic Motivation

While students of the digital era are the most stimulation-flooded and attention-challenged in human history, learning is a process that takes time, selective attention, and perseverance. Therefore, learning designs that harness students’ intrinsic motivation for training and the development of stamina and grit (skills that might be hampered in the digital era) are precious for students’ health and success.

Motivational drivers include an adequate level of challenge that fits the student’s sense of competence and that creates optimal arousal levels, opportunities to expand social relatedness and impact, and balance between support and autonomy (see the ZPD, Figure 1 ).

Importantly, in classroom learning, educators are required to manage the attention, motivation, and reward system of not one but many students, which is a complex task. The typical classroom presents a broad spectrum of learners with diverse learning needs and stretch zones ( Figure 1 ). While the facilitation of autonomy and the sense of competence varies between learners and requires personalized support, the social norms that promote learning are more ubiquitous and apply to most learners. While educators do not always have the resources to support students’ motivation individually, harnessing the social aspects of classroom learning is a manageable, effective strategy to elevate students’ motivation. Learning environments that demonstrate empathy, inclusiveness, and psychological safety have shown positive results in students’ behavior, self-esteem, motivation, and academic success [ 52 , 53 , 54 , 55 , 56 ]. Social motivation has been shown to enhance the encoding of new information (even if the content is not social) [ 57 ]. Learning-for-teaching and peer tutoring (one student teaching another student) effectively encode information into memory. Beyond memory improvement, peer tutoring has many further benefits to both the tutor and the learner in academic achievements [ 58 , 59 ], motivation, and ownership over the learning process and results in a deep conceptual understanding of the material [ 60 ].

The teacher’s demeanor is another controllable factor with a high potential to affect students’ motivation. For example, the literature points to teachers’ immediacy (creating physical and psychological closeness with students) as an effective way to enhance students’ engagement, learning motivation, and performance (including memory retention) [ 61 , 62 , 63 , 64 , 65 ]. Immediacy can be demonstrated through verbal and non-verbal gestures that communicate interest and personal connection (relating to personal stories, using animated voice and body language, creating eye contact, and using humor).

The research also indicates that, when students perceive the content as being personally relevant, they are more motivated to study [ 66 ]. Therefore, educators can actively make the learning content more relevant by using stories and real-life examples, making explicit connections and demonstrations of how the content may be relevant/applicable to the students, and giving students opportunities to reflect and share their connections to the learning material.

In summary, physiological and psychological approaches point to primary motivational drivers that direct engagement and investment in the learning process. Not surprisingly, these drivers that are anchored around social and intellectual needs align with the conditions supporting neuroplasticity discussed in the first part of this review.

1.3. Intrinsic and Extrinsic Processing in Learning and Meaning-Making

As the environment provides more information than the brain can handle, survival depends on saliency detection and attention management to direct perception and behavior. The brain constantly selects and attends to relevant input while suppressing irrelevant or distracting information [ 67 ]. Information that is valuable or urgent for survival and prosperity receives attention. Attention capacities (e.g., alerting, orienting, and controlling attention) are managed by several brain systems that interact and coordinate [ 68 , 69 , 70 ]. Top–down, cognitive-driven attention that fosters a goal-directed thinking process is associated with the dorsal attention network (consisting of the intraparietal sulcus and the frontal eye fields) [ 71 ]. This mechanism enables students to read a paragraph, listen to a lecture, think about the teacher’s question, or write an essay. A second attention system is bottom–up and stimulus-driven, and it orients attention to unexpected and behaviorally relevant stimuli. This ventral attention network consists of the right temporoparietal junction and the ventral frontal cortex [ 71 ]. This attention-grabbing mechanism enables the individual to respond quickly to urgent environmental demands, for example, moving away to prevent a struck-by-object accident. Flexible attention control depends on dynamic interactions and switching between the two systems and involves the central executive network (CEN) [ 68 , 70 ].

The insula and anterior cingulate cortex comprise the core structures of the saliency network [ 72 ], another major player in attention altering to emotionally salient stimuli through the interaction of the sensory and cognitive influences that control attention [ 72 , 73 , 74 ].

In addition to outward-focused attention, the human brain is also invested in inward-focused processing. Functional brain imaging studies of the human brain show a robust functional anticorrelation between two large-scale systems, one highly extrinsic and the other deeply intrinsic [ 75 , 76 , 77 ]. The central executive network (CEN) is an externally driven system and is paramount for attention control, working memory, flexible thinking, and goal-directed behavior. The core components of the CEN are the dorsolateral PFC and the lateral posterior parietal cortex (hence, the frontoparietal network) [ 72 , 78 ]. When the human brain is not occupied with external tasks, the default mode network (DMN) is activated. This internally driven cognitive network includes the posterior cingulate cortex (PCC) and the medial prefrontal cortex (MPFC) as core components. The DMN is thought to facilitate reminiscing, contemplating, autobiographical memory, self-reflecting, and social cognition [ 79 ]. Conversely, the DMN is immediately suppressed when the brain is engaged in externally driven tasks and stimulation.

Resting-state brain imaging studies revealed that the activity in the DMN during resting awake states indicates the quality of subsequent neural and behavioral responses to environmental stimuli [ 72 , 80 ]. Moreover, a high connectivity between “intrinsic” (DMN) and “extrinsic” (CEN) brain networks, and specifically emotional saliency, attention (extrinsic), and reflection (intrinsic) networks is associated with better cognitive performance, meaning-making, and broad perspective thinking [ 75 , 76 , 81 ]. These networks function antagonistically but are highly connected and balance each other. Furthermore, the anticorrelation between their function is associated with better task performance and positive mental health [ 79 , 82 ]. Recent studies also suggest that a causal hierarchical architecture orchestrates this anticorrelation between externally and internally driven brain activities. More specifically, that regions of the saliency network and the dorsal attention network impose inhibition on the DMN. Conversely, the DMN exhibits an excitatory influence on the saliency and attention system [ 79 ].

1.3.1. The Neuroscience of Extrinsic and Intrinsic Processing as Educational Content

Teaching students about the dynamics of the default mode and executive control network can help them understand how their brain processes information, the importance of each process (e.g., extrinsic and intrinsic), and their integration for meaningful learning. This knowledge can be applied as students explore and experiment with ways to enhance their learning and memory by intentionally engaging both intrinsic and extrinsic processing and integrating the two.

1.3.2. Using the Neuroscience of Extrinsic and Intrinsic Processing to Guide Learning Design

Traditionally, instructional education is based on learning objectives that are externally dictated and is focused on outward attention (stimulus-driven lectures and assignments). Mind-wandering has become the enemy of classroom teachers, as it indicates students’ lack of attention and poor learning.

Nevertheless, neuroscience research indicates that meaning-making and cognitive performance benefit from the interplay between extrinsic and intrinsic oriented attention and processing [ 56 , 75 ].

Learning instructions should consider the different attention mechanisms, evoking adequate arousal levels and leading to goal-directed thinking. Furthermore, students will benefit from an educational design that stimulates the natural interplay between “intrinsic” (DMN) and “extrinsic” (CEN) brain networks by incorporating external stimulation (e.g., presenting content), allocating time and space for intrinsic reflection (e.g., guided reflection and journaling), and integrating the two (e.g., guided class discussion and insights sharing) [ 83 , 84 ].

2. Discussion

2.1. teaching students about their developing brains.

As far as we know, humankind is the only species with access to the underlying mechanisms of its perception, learning, and inner workings. In addition, the human brain is endowed with a lengthy developmental period of approximately 25 years [ 85 , 86 ]. Therefore, schooling years are the prime time for neuroplasticity, and students can learn about their brains while they are highly malleable and can utilize this to amplify their learning and growth.

While students were traditionally required to choose whether to focus on the humanities or science fields, an integrative view is becoming increasingly common in academic institutes. Multidisciplinary studies have been shown to promote students’ positive learning and professional outcomes [ 87 ]. Teaching neuroscience from a dual perspective, both scientific/objective and humanistic/subjective, is a novel but natural bridge between the humanities and science fields.

Studying neuroscience with explicit connections to the lived experience of brain development and its behavioral manifestation can be academically and personally transformative for students.

Shedding a scientific light on students’ experiences as they unfold can support significant developmental processes during those years, such as improvements in executive functions, emotional regulation skills, meta-cognition, and social cognition [ 88 ].

Among the topics and burning issues of teens and young adults that neuroscience can offer insights into are selective and leaky attention [ 89 ], the reward system and addiction [ 90 ], the PFC–limbic developmental mismatch during adolescence [ 91 ], neurodiversity and inclusion, emotion regulation, and mental health [ 92 ].

Moreover, adding a personal layer to neuroscience studies fits the notion that personal relatedness and relevance are essential for learning motivation. Teaching neuroscience from a dual (scientific and personal) perspective and connecting neuroscience knowledge to a deeper understanding of the self and others can elevate engagement and nurture students’ passion for science and their ability to integrate and transfer scientific knowledge across contexts. In addition, similar to the effect of physical education, educational neuroscience can promote the awareness of brain health and encourage students to be intentional about their education and developmental trajectory.

2.2. Teaching the Way(s) the Human Brain Learns Best

Teaching students in brain-friendly ways means implementing principles that align with how the human brain encodes, consolidates, and retrieves information. Educational neuroscience points to the importance of a holistic and integrated view of cognitive, emotional, and social aspects to support learning and development [ 52 , 75 , 93 , 94 ]. Maintaining physical health, cognitive challenge, and emotional safety are essential factors in creating an enriched environment that supports neuroplasticity and learning.

Assessing the learning progress rather than the end product can encourage students to move away from rote memorization to more meaningful learning that carries on beyond the final exam.

Meaningful learning can be promoted by learning designs that encourage students to take experimental and explorative approaches, take risks, and make mistakes without detrimental consequences to their grades.

Furthermore, assessments throughout the learning process and not only at the end of it, using multiple sample points and low-risk tasks, can provide information on the student’s learning curve and allow for personalized and timely feedback that students can apply to improve their learning on the go.

These methods not only promote psychological safety but also align with the evidence-based practices of building long-term and accessible knowledge by spreading out the learning concept across time (spacing), practicing information retrieval from memory (recall), and integrating and transferring knowledge (the application of knowledge in different contexts) [ 95 ].

3. Conclusions

Brain knowledge is brainpower; teaching students about their developing brain can support their academic and personal development by deepening their understanding of science and humanities, their mental capacity, and their self-identity. Educational neuroscience is a promising field in teaching students about their brains and teaching them in brain-friendly ways to support them in becoming lifelong learners.

Funding Statement

This research received no external funding.

Institutional Review Board Statement

The study did not require ethical approval.

Informed Consent Statement

Not applicable.

Data Availability Statement

Conflicts of interest.

The authors declare no conflict of interest.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Knowing What Students Know: The Science and Design of Educational Assessment (2001)

Chapter: 8 implications and recommendations for research, policy, and practice, 8 implications and recommendations for research, policy, and practice.

The Committee on the Foundations of Assessment produced this report, with the support of the National Science Foundation (NSF), to review and synthesize advances in the cognitive and measurement sciences and to explore the implications of those advances for improving educational assessment. Interest in the intersection of these two fields is not new. Prompted by calls for assessments that can better inform and support learning, a number of education researchers have put forth the potential benefits of merging modern knowledge in the areas of cognition and learning and methods of educational measurement (Baker, 1997; Cronbach and Gleser, 1965; Glaser, 1981; Glaser and Silver, 1994; Messick, 1984; Mislevy, 1994; National Academy of Education, 1996; Nichols, 1994; Pellegrino, Baxter, and Glaser, 1999; Snow and Lohman, 1993; Wilson and Adams, 1996).

Several decades of research in the cognitive sciences has advanced the knowledge base about how children develop understanding, how people reason and build structures of knowledge, which thinking processes are associated with competent performance, and how knowledge is shaped by social context (National Research Council [NRC], 1999b). These findings, presented in Chapter 3 , suggest directions for revamping assessment to provide better information about students’ levels of understanding, their thinking strategies, and the nature of their misunderstandings.

During this same period, there have been significant developments in measurement (psychometric) methods and theory. As presented in Chapter 4 , a wide array of statistical measurement methods are currently available to support the kinds of inferences that cognitive research suggests are important to pursue when assessing student achievement.

Meanwhile, computer and telecommunications technologies are making it possible to assess what students are learning at very fine levels of detail,

with vivid simulations of real-world situations, and in ways that are tightly integrated with instruction. Chapter 7 provides examples of how technology is making it feasible, for instance, for students to receive ongoing individualized feedback as they work with a computerized tutoring system—feedback more detailed than what a teacher could have provided a class of 30 students in the past.

This report describes a variety of promising assessment innovations that represent first steps in capitalizing on these opportunities. However, most of these examples have been limited to small-scale applications that have yet to affect mainstream assessment practice. In this final chapter, we discuss priorities for research, practice, and policy to enable the emergence of a “new science of assessment.” First, however, we summarize some of the main points from the preceding chapters by describing a vision for a future generation of educational assessments based on the merger of modern cognitive theory and methods of measurement.

A VISION FOR THE FUTURE OF ASSESSMENT

In the future envisioned by the committee, educational assessments will be viewed as a facilitator of high levels of student achievement. They will help students learn and succeed in school by making as clear as possible to them, their teachers, and other education stakeholders the nature of their accomplishments and the progress of their learning.

Teachers will assess students’ understanding frequently in the classroom to provide them with feedback and determine next steps for instruction. Their classroom practices will be grounded in principles of how students think and learn in content domains and of assessment as a process of reasoning from evidence. Teachers will use this knowledge to design assessments that provide students with feedback about particular qualities of their work and what they can do to improve.

Students will provide evidence of their understanding and thinking in a variety of ways—by responding to teachers’ questions, writing or producing projects, working with computerized tutoring systems, or attempting to explain concepts to other students. Teachers, in turn, will use this information to modify instruction for the class and for individuals on the basis of their understanding and thinking patterns.

Teachers will have a clear picture of the learning goals in subject domains, as well as typical learning pathways for reaching those goals. Ultimate and intermediate learning goals will be shared regularly with students as a part of instruction. Students will be engaged in activities such as peer and self-assessment to help them internalize the criteria for high-quality work and develop metacognitive skills.

Teachers will also use summative assessments for ongoing reflection and feedback about overall progress and for reporting of this information to others. External summative assessments, such as state tests, will reinforce the same ultimate goals and beliefs about learning that are operating in the classroom. Large-scale assessments will set valuable learning goals for students to pursue. Such assessments will broadly sample the desired outcomes for learning by using a variety of methods, such as on-demand assessment combined with a sampling of work produced during the course of instruction.

Policy makers, educators, and the public will come to expect more than the general comparisons and rankings that characterize current test results. Performance on large-scale assessments will be explicitly and publicly displayed so that students, parents, and teachers can see the concepts and processes entailed at different levels of competence. Assessments will be able to show, for instance, how a competent performer proceeds on a mathematics problem and forms an answer, in comparison with a student who is less proficient. Large-scale assessments will help show the different kinds of interpretations, procedural strategies, explanations, and products that differentiate among various levels or degrees of competence.

Within an education system, teachers, administrators, and policy makers will be working from a shared knowledge base about how students learn subject matter and what aspects of competence are important to assess. Resource materials that synthesize modern scientific understanding of how people learn in areas of the curriculum will serve as the basis for the design of classroom and large-scale assessments, as well as curriculum and instruction, so that all the system’s components work toward a coherent set of learning goals.

In many ways, this vision for assessment represents a significant departure from the types of assessments typically available today and from the ways in which such assessments are most commonly used. Current knowledge could serve as the basis for a number of improvements to the assessment design process (as described in Chapters 3 , 4 , and 5 of this report) to produce assessment information that would be more useful, valid, and fair. Full realization of the committee’s broader vision for educational assessment, however, will require more knowledge about how to design and use such assessments, as well as about the underlying fundamental properties of learning and measurement. Furthermore, the committee recognizes that the maximum potential of new forms of assessment cannot be realized unless educational practices and policies adapt in significant ways. Some of the constraints that currently limit assessment practice will need to be relaxed if the full benefits of a merger between the cognitive and measurement sciences are to be realized. The new kinds of assessment described in this report do not necessarily conform to the current mode of on-demand, pa-

per-and-pencil tests that students take individually at their desks under strictly standardized conditions. Furthermore, realizing the potential benefits of new forms of assessment will depend on making compatible changes in curriculum and instruction.

BRIDGING RESEARCH AND PRACTICE

Like other groups before us (NRC, 1999c; National Academy of Education, 1999), the committee recognizes that the bridge between research and practice takes time to build and that research and practice must proceed interactively. It is unlikely that the insights gained from current or new knowledge about cognition, learning, and measurement will be sufficient by themselves to bring about transformations in assessment such as those described in this report. As the NRC’s Committee on Learning Research and Educational Practice pointed out, research and practice need to be connected more directly through the building of a cumulative knowledge base that serves both sets of interests. In the context of this study, that knowledge base would focus on the development and use of theory-based assessment. Furthermore, it is essential to recognize that research impacts practice indirectly through the influence of the existing knowledge base on four important mediating arenas: educational tools and materials; teacher education and professional development; education policies; and public opinion and media coverage (NRC, 1999c). By affecting each of these arenas, an expanding knowledge base on the principles and practices of effective assessment can help change educational practice. And the study of changes in practice, in turn, can help in further developing the knowledge base. These organizing ideas regarding the connections between research and practice are illustrated in Figure 8–1 .

In this chapter we outline a proposed research and development agenda for expanding the knowledge base on the integration of cognition and measurement and consider the implications of such a knowledge base for each of the four mediating arenas that directly influence educational practice. In doing so we propose two general guidelines for how future work should proceed.

First, the committee advocates increased and sustained multidisciplinary collaboration around theoretical and practical matters of assessment. We apply this precept not only to the collaboration between researchers in the cognitive and measurement sciences, but also to the collaboration of these groups with teachers, curriculum specialists, and assessment developers. The committee believes the potential for an improved science and design of educational assessment lies in a mutually catalytic merger of the two foundational disciplines, especially as such knowledge is brought to bear on conceptual and pragmatic problems of assessment development and use.

FIGURE 8–1 Connections between research and practice.

SOURCE: Adapted from National Research Council (1999c, p. 34).

Second, the committee urges individuals in multiple communities, from research through practice and policy, to consider the conceptual scheme and language used in this report as a guide for stimulating further thinking and discussion about the many issues associated with the productive use of assessments in education. The assessment triangle set forth in Chapter 2 and summarized in Box 8–1 provides a conceptual framework for principled thinking about the assumptions and foundations underlying an assessment. In the next section of this chapter we consider some of the implications of our conceptual scheme for research that can contribute to the advancement of both theory and practice.

Before discussing specific implications for research and practice and presenting our recommendations in each of these areas, we would be remiss if we did not note our concern about continuing with the present system of educational assessment, including the pattern of increasing investment in large-scale assessment designs and practices that have serious limi-

tations and in some cases do more harm than good. This concern underlines the importance of seizing the opportunity that now exists to reshape the assessment landscape while simultaneously reinforcing many of the social and political reasons for investing in high-quality educational assessment materials, designs, and practices. That opportunity should not be lost just because every theoretical and operational detail has yet to be established for the design and implementation of assessments based on a merger of the cognitive and measurement sciences. There is much that can be done in the near term to improve assessment design and use on the basis of existing knowledge, while an investment is being made in the research and development needed to build assessments appropriate for the educational systems of the 21 st century.

IMPLICATIONS AND RECOMMENDATIONS FOR RESEARCH

The research needed to approach the new science of assessment envisioned by the committee needs to focus on those issues that lie at the intersection of cognitive and measurement science. In this section we present the committee’s recommendations for research organized into three broad categories: (1) synthesis of existing knowledge, (2) research to expand the current knowledge base, and (3) some initial steps for building the knowledge base.

For all the research recommendations presented below, we advocate a general approach to research and development that differs from conventional practices. In the traditional view of research, development, and implementation, scientists begin with basic research that involves gathering fundamental knowledge and developing theories about an area of inquiry. Other scientists and practitioners use this basic research, together with their experience, to design prototypes that apply the knowledge in practical settings. Still others then design ways to implement the prototypes on a larger scale.

The committee believes that, in the case of the assessments we envision, research should focus on design and implementation. The committee takes this position for two reasons. The first is strategic. As described throughout this report, some promising prototype assessments based on modern cognitive theory and measurement principles have already been developed. While the prototypes have been used effectively in selected classrooms and educational settings, there is generally limited experience with their application outside of relatively controlled settings or in large-scale contexts. In part this is because the new forms of assessment are often complex and have not been tailored for widespread practical use. In addition, there are issues involved in large-scale assessment that designers of classroom-based tools

have yet to confront. The committee takes the position that practical implementation should be studied to raise questions about fundamental science.

In his book Pasteur’s Quadrant, Stokes (1997) argues that the traditional dichotomy between “basic” and “applied” research is not always applicable. In many instances, research aimed at solving practical problems can test the validity and generality of fundamental principles and knowledge. Pasteur’s work is an archetype of this approach. By focusing on a very real practical problem—developing ways to combat harmful bacteria—Pasteur pursued “use-inspired strategic research” that not only helped solve the immediate problem, but also contributed greatly to enhancing fundamental knowledge about biology and biochemistry. Similarly, Hargreaves (1999) argues that research results cannot be applied directly to classroom practice, but must be transformed by practitioners; that is, teachers need to participate in creating new knowledge.

In a report to the National Education Research Policies and Priorities Board of the Office of Educational Research and Improvement, a panel of the National Academy of Education argues that federal agencies should fund research in Pasteur’s Quadrant as well as basic research (National Academy of Education, 1999). The panel states that “problem-solving research and development” (the equivalent of what Stokes describes as use-inspired strategic research) is characterized by four features:

Commitment to the improvement of complex systems.

Co-development by researchers and practitioners, with recognition of differences in expertise and authority.

Long-term engagement that involves continual refinement.

Commitment to theory and explanation.

The panel notes that this last feature would enable prototypes generated in one site or context of use to “travel” to other settings (the panel contrasts its view with the traditional notion of “dissemination”). To permit wider adoption, the research would have to generate principles to ensure that others would not simply replicate the surface features of an innovation. Also required would be consideration of tools that could help others apply the innovation faithfully, as well as people familiar with the design who could help others implement it. The committee is sympathetic to this argument and believes research that addresses ways to design assessments for use in either classrooms or large-scale settings can simultaneously enhance understanding of the design principles inherent in such assessments and improve basic knowledge about cognition and measurement.

We advocate that the research recommended below be funded by federal agencies and private foundations that currently support research on teaching and learning, as well as private-sector entities involved in commer-

cial assessment design and development. Among the salient federal agencies are the Department of Education, the NSF, and the National Institute of Child Health and Human Development. The research agenda is expansive in both scope and likely duration. It would be sensible for the funding of such work to be coordinated across agencies and, in many instances, pursued cooperatively with foundations and the private sector.

Synthesis of Existing Knowledge

Recommendation 1: Accumulated knowledge and ongoing advances from the merger of the cognitive and measurement sciences should be synthesized and made available in usable forms to multiple educational constituencies. These constituencies include educational researchers, test developers, curriculum specialists, teachers, and policy makers.

As discussed throughout this report, a great deal of the foundational research needed to move the science of assessment forward has already been conducted; however, it is not widely available or usable in synthetic form. This report is an initial attempt at such a synthesis, but the committee recognized from the start of its work that a comprehensive critique, synthesis, and extrapolation of all that is known was beyond the scope of a study such as this and remains a target for the future. Furthermore, there is an ongoing need to accumulate, synthesize, and disseminate existing knowledge—that is, to construct the cumulative knowledge base on assessment design and use that lies at the center of Figure 8–1 .

Expanding the Knowledge Base

Recommendation 2: Funding should be provided for a major program of research, guided by a synthesis of cognitive and measurement principles, focused on the design of assessments that yield more valid and fair inferences about student achievement. This research should be conducted collaboratively by multidisciplinary teams comprising both researchers and practitioners.

A priority should be the development of cognitive models of learning that can serve as the basis for assessment design for all areas of the school curriculum. Research on how students learn subject matter should be conducted in actual educational settings and with groups of learners representative of the diversity of the student population to be assessed.

Research on new statistical measurement models and their applicability should be tied to modern theories of cognition and learning. Work should be undertaken to better understand the fit between various types of cognitive theories and measurement models to determine which combinations work best together.

Research on assessment design should include exploration of systematic and fair methods for taking into account aspects of examinees’ instructional background when interpreting their responses to assessment tasks. This research should encompass careful examination of the possible consequences of such adaptations in high-stakes assessment contexts.

One priority for research is the development of cognitive models of learning for areas of the school curriculum. As noted in Chapter 3 , researchers have developed sophisticated models of student cognition in various areas of the curriculum, such as algebra and physics. However, an understanding of how people learn remains limited for many other areas. Moreover, even in subject domains for which characteristics of expertise have been identified, a detailed understanding of patterns of growth that would enable one to identify landmarks on the way to competence is often lacking. Such landmarks are essential for effective assessment design and implementation.

The development of models of learning should not be done exclusively by scientists in laboratory settings. As argued earlier, it would be more fruitful if such investigations were conducted, at least in part, in actual educational contexts by collaborative teams of researchers and practitioners. Such collaborations would help enhance both the quality and utility of the knowledge produced by the research.

To develop assessments that are fair—that are comparably valid across different groups of students—it is crucial that patterns of learning for different populations of students are studied. Much of the development of cognitive theories has been conducted with a restricted group of students (i.e., mostly middle-class whites). In many cases it is not clear whether current theories of learning apply equally well with diverse populations of students, including those who have been poorly served in the educational system, underrepresented minority students, English-language learners, and students with disabilities. There are typical learning pathways, but not a single pathway to competence. Furthermore, students will not necessarily respond in similar ways to assessment probes designed to diagnose knowledge and understanding. These kinds of natural variations among individuals need to

be better understood through empirical study and incorporated into the cognitive models of learning that serve as a basis for assessment design.

Sophisticated models of learning by themselves do not produce high-quality assessment information. Also needed are methods and tools both for eliciting appropriate and relevant data from students and for interpreting the data collected about student performance. As described in Chapter 4 , the measurement methods now available enable a much broader range of inferences to be drawn about student competence than many people realize. But research is needed to investigate the relative utility of existing and future statistical models for capturing critical aspects of learning specified in cognitive theories.

Most of the new measurement models have been applied only on a limited scale. Thus, there is a need to explore the utility and feasibility of the new models for a wider range of assessment applications and contexts. Within such a line of inquiry, a number of issues will need to be understood in more depth, including the level of detail at which models of student learning must be specified for implementing various types of classroom or large-scale assessments. Furthermore, there is a vital need for research on ways to make a broader range of measurement models usable by practitioners, rather than exclusively by measurement specialists. Many of the currently available measurement methods require complex statistical modeling that only people with highly specialized technical skills can use to advantage. If these tools are to be applied more widely, understandable interfaces will need to be built that rise above statistical complexity to enable widespread use, just as users of accounting and management programs need not understand all the calculations that go into each element of the software.

Another priority for assessment design is the exploration of new ways to address persisting issues of fairness and equity in testing. People often view fairness in testing in terms of ensuring that students are placed in test situations that are as similar or standardized as possible. But another way of approaching fairness is to take into account examinees’ histories of instruction or opportunities to learn the material being tested when interpreting their responses to assessment tasks. Ways of drawing such conditional inferences have been tried mainly on a small scale but hold promise for tackling persisting issues of equity in assessment.

Recommendation 3: Research should be conducted to explore how new forms of assessment can be made practical for use in classroom and large-scale contexts and how various new forms of assessment affect student learning, teacher practice, and educational decision making.

Research should explore ways in which teachers can be assisted in integrating new forms of assessment into their in-

structional practices. It is particularly important that such work be done in close collaboration with practicing teachers who have varying backgrounds and levels of teaching experience.

Also to be studied are ways in which school structures (e.g., length of time of classes, class size, and opportunity for teachers to work together) impact the feasibility of implementing new types of assessments and their effectiveness.

The committee firmly believes that the kinds of examples described in this report—all of which are currently being used in classrooms or large-scale contexts—represent positive steps toward the development of assessments that can not only inform but also improve learning. However, for these kinds of innovations to gain more widespread adoption, work is needed to make them practical for use in classroom and large-scale contexts, and evidence of their impact on student learning is needed.

Furthermore, the power offered by assessments to enhance learning in large numbers of classrooms depends on changes in the relationship between teacher and student, the types of lessons teachers use, the pace and structure of instruction, and many other factors. To take advantage of the new tools, many teachers will have to change their conception of their role in the classroom. They will have to shift toward placing much greater emphasis on exploring students’ understanding with the new tools and then undertaking a well-informed application of what has been revealed by use of the tools. This means teachers must be prepared to use feedback from classroom and external assessments to guide their students’ learning more effectively by modifying the classroom and its activities. In the process, teachers must guide their students to be more engaged actively in monitoring and managing their own learning—to assume the role of student as self-directed learner.

The power of new assessments depends on substantial changes not only in classroom practice, but also in the broader educational context in which assessments are conducted. For assessment to serve the goals of learning, there must be alignment among curriculum, instruction, and assessment. Furthermore, the existing structure and organization of schools may not easily accommodate the type of instruction users of the new assessments will need to employ. For instance, if teachers are going to gather more assessment information during the course of instruction, they will need time to assimilate that information. If these kinds of systemic and structural issues are not addressed, new forms of assessment will not live up to their full potential. This is a common fate for educational innovations. Many new techniques and procedures have failed to affect teaching and learning on a large scale because the innovators did not address all the factors that affect

teaching and learning (Elmore, 1996). Despite the promise of new procedures, most teachers tend to teach the way they have always taught, except in the “hothouse” settings where the innovations were designed.

Thus, if assessments based on the foundations of cognitive and measurement science are to be implemented on a broad scale, changes in school structures and practices will likely be needed. However, the precise nature of such changes is uncertain. As new assessments are implemented, researchers will need to examine the effects of such factors as class size and the length of the school day on the power of assessments to inform teachers and administrators about student learning. Also needed is a greater understanding of what structural changes are required for teachers to modify their practice in ways that will enable them to incorporate such assessments effectively.

Some Initial Steps for Building the Knowledge Base

Recommendation 4: Funding should be provided for in-depth analyses of the critical elements (cognition, observation, and interpretation) underlying the design of existing assessments that have attempted to integrate cognitive and measurement principles (including the multiple examples presented in this report). This work should also focus on better understanding the impact of such exemplars on student learning, teacher practice, and educational decision making.

The committee believes an ideal starting point for much of the research agenda is further study of the types of assessment examples provided in the preceding chapters, which represent initial attempts at synthesizing advances in the cognitive and measurement sciences. While these examples were presented to illustrate features of the committee’s proposed approach to assessment, the scope of this study did not permit in-depth analyses of all the design and operational features of each example or their impact on student learning, teacher practice, and educational decision making. Further analysis of these and other examples would help illuminate the principles and practices of assessment design and use described in this report. Several important and related directions of work need to be pursued.

First, to fully understand any assessment, one must carefully deconstruct and analyze it in terms of its underlying foundational assumptions. The assessment triangle provides a useful framework for analyzing the foundational elements of an assessment. Questions need to be asked and answered regarding the precise nature of the assumptions made about cognition, observation, and interpretation, including the degree to which they are in synchrony. Such an analysis should also consider ways in which current knowl-

edge from the cognitive and measurement sciences could be used to enhance the assessment in significant ways.

Second, once an assessment is well understood, its effectiveness as a tool for measurement and for support of learning must be explored and documented. The committee strongly believes that the examples in this report represent promising directions for further development, and where available, has presented empirical support for their effectiveness. However, there is a strong need for additional empirical studies aimed at exploring which tools are most effective and why, how they can best be used, and what costs and benefits they entail relative to current forms of assessment.

Third, while it is important to carefully analyze each of the examples as a separate instance of innovative design, they also need to be analyzed as a collective set of instances within a complex “design space.” The latter can be thought of as a multivariate environment expressing the important features that make specific instances simultaneously similar and different. This design space is only partially conceived and understood at the present time. Thus, analyses should be pursued that cut across effective exemplars with the goal of identifying and clarifying the underlying principles of the new science of assessment design. In this way, the principles described in this report can be refined and elaborated while additional principles and operational constructs are uncovered. If a new science of assessment grounded in concepts from cognitive and measurement science is to develop and mature, every attempt must be made to uncover the unique elements that emerge from the synthesis of the foundational sciences. This work can be stimulated by further in-depth analysis of promising design artifacts and the design space in which they exist.

Recommendation 5: Federal agencies and private-sector organizations concerned about issues of assessment should support the establishment of multidisciplinary discourse communities to facilitate cross-fertilization of ideas among researchers and assessment developers working at the intersection of cognitive theory and educational measurement.

Many of the innovative assessment practices described in this report were derived from projects funded by the NSF or the James S.McDonnell Foundation. These organizations have provided valuable opportunities for cross-fertilization of ideas, but more sharing of knowledge is needed. Many of the examples exist in relative isolation and are known only within limited circles of scientific research and/or educational practice. The committee believes there are enough good examples of assessments based on a merger of the cognitive and measurement sciences so that designers can start building from existing work. However, a discourse among multidisciplinary commu-

nities will need to be established to promote and sustain such efforts. As mentioned earlier, this report provides a language and conceptual base for discussing the ideas embedded in existing innovative assessment practices and for the broader sharing and critique of those ideas.

IMPLICATIONS AND RECOMMENDATIONS FOR POLICY AND PRACTICE

Research often does not directly affect educational practice, but it can effect educational change by influencing the four mediating arenas of the education system that do influence practice, shown previously in Figure 8–1 . For the earlier committee that identified these arenas, the question was how to bridge research on student learning and instructional practice in classrooms. The focus of the present committee is on a related part of the larger question: how to link research on the integration of cognition and measurement with actual assessment practice in schools and classrooms. By influencing and working through the four mediating arenas, the growing knowledge base on cognition and measurement can ultimately have an effect on assessment and instructional practice in classrooms and schools.

It is important to note that the path of influence does not flow only in one direction. Just as we believe that research on the integration of cognition and measurement should focus on use-inspired strategic research, we believe that practical matters involving educational tools and materials, teacher education and professional development, education policies, and public opinion and media coverage will influence the formulation of research questions that can further contribute to the development of a cumulative knowledge base. Research focused on these arenas will enhance understanding of practical matters related to how students learn and how learning can best be measured in a variety of school subjects.

Educational Tools and Materials

Recommendation 6: Developers of assessment instruments for classroom or large-scale use should pay explicit attention to all three elements of the assessment triangle (cognition, observation, and interpretation) and their coordination.

All three elements should be based on modern knowledge of how students learn and how such learning is best measured.

Considerable time and effort should be devoted to a theory-driven design and validation process before assessments are put into operational use.

When designing new tools for classroom or large-scale use, assessment developers are urged to use the assessment triangle as a guiding framework, as set forth and illustrated in Chapters 5 , 6 , and 7 . As discussed under Recommendation 1 above, a prerequisite for the development of new forms of assessment is that current knowledge derived from research be conveyed to assessment and curriculum developers in ways they can access and use.

A key feature of the approach to assessment development proposed in this report is that the effort should be guided by an explicit, contemporary cognitive model of learning that describes how people represent knowledge and develop competence in the subject domain, along with an interpretation model that is compatible with the cognitive model. Assessment tasks and procedures for evaluating responses should be designed to provide evidence of the characteristics of student understanding identified in the cognitive model of learning. The interpretation model must incorporate this evidence in the assessment results in a way that is consistent with the model of learning. Assessment designers should explore ways of using sets of tasks that work in combination to diagnose student understanding while at the same time maintaining high standards of reliability. The interpretation model must, in turn, reflect consideration of the complexity of such sets of tasks.

An important aspect of assessment validation often overlooked by assessment developers is the collection of evidence that tasks actually tap the intended cognitive content and processes. Starting with hypotheses about the cognitive demands of a task, a variety of research techniques, such as interviews, having students think aloud as they solve problems, and analysis of errors, can be used to explore the mental processes in which examinees actually engage during task performance. Conducting such analyses early in the assessment development process ensures that the assessments do, in fact, measure what they are intended to measure.

Recommendation 7: Developers of educational curricula and classroom assessments should create tools that will enable teachers to implement high-quality instructional and assessment practices, consistent with modern understanding of how students learn and how such learning can be measured.

Assessments and supporting instructional materials should interpret the findings from cognitive research in ways that are useful for teachers.

Developers are urged to take advantage of opportunities afforded by technology to assess what students are learning at fine levels of detail, with appropriate frequency, and in ways that are tightly integrated with instruction.

The committee believes a synthesis of cognitive and measurement principles has particularly significant potential for the design of high-quality tools for classroom assessment that can inform and improve learning. However, teachers should not be expected to devise on their own all the assessment tasks for students or ways of interpreting responses to those tasks. Some innovative classroom assessments that have emerged from this synthesis and are having a positive impact on learning have been described in preceding chapters. A key to the effectiveness of these tools is that they must be packaged in ways that are practical for use by teachers. As described in Chapter 7 , computer and telecommunications technologies offer a rich array of opportunities for providing teachers with sophisticated assessment tools that will allow them to present more complex cognitive tasks, capture and reply to students’ performances, share exemplars of competent performance, engage students in peer and self-reflection, and in the process gain critical information about student competence.

Recommendation 8: Large-scale assessments should sample the broad range of competencies and forms of student understanding that research shows are important aspects of student learning.

A variety of matrix sampling, curriculum-embedded, and other assessment approaches should be used to cover the breadth of cognitive competencies that are the goals of learning in a domain of the curriculum.

Large-scale assessment tools and supporting instructional materials should be developed so that clear learning goals and landmark performances along the way to competence are shared with teachers, students, and other education stakeholders. The knowledge and skills to be assessed and the criteria for judging the desired outcomes should be clearly specified and available to all potential examinees and other concerned individuals.

Assessment developers should pursue new ways of reporting assessment results that convey important differences in performance at various levels of competence in ways that are clear to different users, including educators, parents, and students.

Though further removed from day-to-day instruction than classroom assessments, large-scale assessments also have the potential to support instruction and learning if well designed and appropriately used. Deriving real benefits from the merger of cognitive and measurement theory in large-scale assessment requires finding ways to cover a broad range of competencies

and capture rich information about the nature of student understanding. Alternatives to the typical on-demand testing scenario—in which every student takes the same test at a specified time under strictly standardized conditions—should be considered to enable the collection of more diverse evidence of student achievement.

Large-scale assessments have an important role to play in providing dependable information for educational decision making by policy makers, school administrators, teachers, and parents. Large-scale assessments can also convey powerful messages about the kinds of learning valued by society and provide worthy goals to pursue. If such assessments are to serve these purposes, however, it is essential that externally set goals for learning be clearly communicated to teachers, students, and other education stakeholders.

Considerable resources should be devoted to producing materials for teachers and students that clearly present both the learning goals and landmark performances along the way to competence. Those performances can then be illustrated with samples of the work of learners at different levels of competence, accompanied by explanations of the aspects of cognitive competence exemplified by the work. These kinds of materials can foster valuable dialogue among teachers, students, and the public about what achievement in a domain of the curriculum looks like. The criteria by which student work will be judged on an assessment should also be made as explicit as possible. Curriculum materials should encourage the use of activities such as peer and self-assessment to help students internalize the criteria for high-quality work and foster metacognitive skills. All of these points are equally true for classroom assessments.

The use of assessments based on cognitive and measurement science will also necessitate different forms of reporting on student progress, both to parents and to administrators. The information gleaned from such assessments is far more nuanced than that obtainable from the assessments commonly used today, and teachers may want to provide more detail in reports to parents about the nature of their children’s understanding. In formulating reports based on new assessments, test developers, teachers, and school administrators should ensure that the reports include the information parents want and can appropriately use to support their children’s learning. Reports on student performance could also provide an important tool to assist administrators in their supervisory roles. Administrators could use such information to see how teachers are gauging their students’ learning and how they are responding to the students’ demonstration of understanding. Such information could help administrators determine where to focus resources for professional development. In general, for the information to be useful and meaningful, it will have to include a profile consisting of multiple elements and not just a single aggregate score.

Teacher Education and Professional Development

Recommendation 9: Instruction in how students learn and how learning can be assessed should be a major component of teacher preservice and professional development programs.

This training should be linked to actual experience in classrooms in assessing and interpreting the development of student competence.

To ensure that this occurs, state and national standards for teacher licensure and program accreditation should include specific requirements focused on the proper integration of learning and assessment in teachers’ educational experience.

Research on the integration of cognition and measurement also has major implications for teacher education. Teachers need training to understand how children learn subject matter and how assessment tools and practices can be used to obtain useful information about student competence. Both the initial preparation of teachers and their ongoing professional development can incorporate insights and examples from research on the integration of cognitive and measurement science and equip teachers with knowledge and skills they can use to employ high-quality assessments. At the same time, such learning opportunities can enable teachers to transform their practice in ways that will allow them to profit from those assessments.

If such assessments are to be used effectively, teacher education needs to equip beginning teachers with a deep understanding of many of the approaches students might take toward understanding a particular subject area, as well as ways to guide students at different levels toward understanding (Carpenter, Fennema, and Franke, 1996; Griffin and Case 1997). Teachers also need a much better understanding of the kinds of classroom environments that incorporate such knowledge (NRC, 1999b). Typically, teacher education programs provide very little preparation in assessment (Plake and Impara, 1997). Yet teaching in ways that integrate assessment with curriculum and instruction requires a strong understanding of methods of assessment and the uses of assessment data. This does not mean that all teachers need formal training in psychometries. However, teachers need to understand how to use tools that can yield valid inferences about student understanding and thinking, as well as methods of interpreting data derived from assessments.

In addition, school administrators need to provide teachers with ample opportunities to continue their learning about assessment throughout their professional practice. Professional development is increasingly seen as a vital element in improving of practice, for veteran as well as new teachers

(Cohen and Hill, 1998; Elmore and Burney, 1998). This continued learning should include the development of cognitive models of learning. Teachers’ professional development can be made more effective if it is tied closely to the work of teaching (e.g., National Academy of Education, 1999). The “lesson study” in which Japanese teachers engage offers one way to forge this link (Stigler and Hiebert, 1999). In that approach, teachers develop lessons on their own, based on a common curriculum. They try these lessons out in their classrooms and share their findings with fellow teachers. They then modify the lessons and try them again, collecting data as they implement the lessons and again working collaboratively with other teachers to polish them. The resulting lessons are often published and become widely used by teachers throughout the country.

Education Policies

Recommendation 10: Policy makers are urged to recognize the limitations of current assessments, and to support the development of new systems of multiple assessments that would improve their ability to make decisions about education programs and the allocation of resources.

Important decisions about individuals should not be based on a single test score. Policy makers should instead invest in the development of assessment systems that use multiple measures of student performance, particularly when high stakes are attached to the results.

Assessments at the classroom and large-scale levels should grow out of a shared knowledge base about the nature of learning. Policy makers should support efforts to achieve such coherence.

Policy makers should promote the development of assessment systems that measure growth or progress of students and the education system over time and that support multilevel analyses of the influences responsible for such change.

Recommendation 11: The balance of mandates and resources should be shifted from an emphasis on external forms of assessment to an increased emphasis on classroom formative assessment designed to assist learning.

Another arena through which research can influence practice is education policy. This is a particularly powerful arena in the case of assessment. Policy makers currently are putting great stock in large-scale assessments and using them for a variety of purposes. There is a good deal of evidence

that assessments used for policy purposes have had effects on educational practice, not all of them positive (e.g., Herman, 1992; NRC, 1999a; Koretz and Barron, 1998).

Research on the integration of cognition and measurement can affect practice through policy in several ways. Most directly, the research can enhance the assessments used for policy decisions. Furthermore, the decisions of policy makers could be better informed than is the case today by assessments that provide a broader picture of student learning. Since test developers respond to the marketplace, a demand from policy makers for new assessments would likely spur their development.

A range of assessment approaches should be used to provide a variety of evidence to support educational decision making. There is a need for comprehensive systems of assessment consisting of multiple measures, including those that rely on the professional judgments of teachers and that together meet high standards of validity and reliability. Single measures, while useful, are unlikely to tap all the dimensions of competence identified by learning goals. Multiple measures are essential in any system in which high-stakes decisions are made about individuals on the basis of assessment results (NRC, 1999a).

Currently, assessments at the classroom and large-scale levels often convey conflicting goals for learning. As argued in Chapter 6 , coherence is needed in the assessment system. A coherent assessment system supports learning for all students. If a state assessment were not designed from the same conceptual base as classroom assessments, the mismatch could undermine the potential for improved learning offered by a system of assessment based on the cognitive and measurement sciences.

To be sure, coherence in an educational system is easier to wish for than to achieve—particularly in an education system with widely dispersed authority such as that of the United States. In many ways, standards-based reform is a step toward achieving some of this coherence. But current content standards are not as useful as they could be. Cognitive research can contribute to the development of next-generation standards that are more effective for guiding curriculum, instruction, and assessment—standards that define not only the content to be learned, but also the ways in which subject matter understanding is acquired and develops. Classroom and large-scale assessments within a coherent system should grow from a shared knowledge base about how students think and learn in a domain of the curriculum. This kind of coherence could help all assessments support common learning goals.

Assessments should be aimed at improving learning by providing information needed by those at all levels of the education system on the aspects of schooling for which they are responsible. If properly conducted, assessments can also serve accountability purposes by providing valuable infor-

mation to teachers and administrators about the progress or growth of the education system over time. The committee refers to this feature as continuity. And if the assessments are instructionally sensitive—that is, if they show the effects of high-quality teaching—they can provide important information about the effectiveness of teaching practices as well (NRC, 1999d).

Developing and implementing a system of multiple assessments would likely be more costly than continuing with the array of tests now being used by states and school districts. Currently, states spend about $330 million for testing (Achieve, 2000). While this sum appears considerable, it represents less than one-tenth of 1 percent of the total amount spent on precollege education (National Center for Education Statistics, 2001). If used properly, the total spending for assessment should not be considered money for tests alone. Funds spent for teachers to score assessments, included in the cost of assessment, also serve an important professional development function. Moreover, spending on assessments that inform instruction represents an investment in teaching and learning, not just in system monitoring. Therefore, policy makers need to invest considerably more in assessment than is currently the case, presuming that the investment is in assessment systems of the type advocated in this report.

Public Opinion and Media Coverage

Recommendation 12: Programs for providing information to the public on the role of assessment in improving learning and on contemporary approaches to assessment should be developed in cooperation with the media. Efforts should be made to foster public understanding of basic principles of appropriate test interpretation and use.

A fourth arena in which research on the integration of cognitive and measurement science can affect practice is through public opinion and the media. Current interest among the public and the news media in testing and test results suggests that public opinion and media coverage can be a powerful arena for change. Information communicated to the public through the media can influence practice in at least two ways. First, the media influence the constituencies responsible for assessment development and practice, including teachers, school administrators, policy makers, and test developers. Perhaps of greater significance is recognition that the more the public is made aware of how assessment practice could be transformed to better serve the goals of learning, the greater will be the support that educators and policy makers have for the kinds of changes proposed in this volume.

Researchers should therefore undertake efforts to communicate with the media what student development toward competence looks like and how it

can best be measured; the media can, in turn, communicate those messages to the public. An attempt should also be made through the media and other avenues for communication with the public to foster understanding of basic principles of appropriate test interpretation and use. Assessment consumers, including the public, should understand that no test is a perfect measure, that more valid decisions are based on multiple indicators, and that the items on a particular assessment are only a sample from the larger domain of knowledge and skill identified as the targets of learning. As part of the design and delivery of such programs, research needs to be conducted on the public’s understanding of critical issues in assessment and the most effective ways to communicate outcomes from educational assessment.

CONCLUDING COMMENTS

As noted at the beginning of this report, educational assessment is an integral part of the quest for improved education. Through assessment, education stakeholders seek to determine how well students are learning and whether students and institutions are progressing toward the goals that have been set for educational systems. The problem is that the vital purposes of informing and improving education through assessment are being served only partially by present assessment practices.

The principles and practices of educational assessment have changed over the last century, but not sufficiently to keep pace with the substantial developments that have accrued in the understanding of learning and its measurement. It is time to harness the scientific knowledge of cognition and measurement to guide the principles and practices of educational assessment. There is already a substantial knowledge base about what better assessment means, what it looks like, and principled ways that can be used to build and use it. That knowledge base needs to be put into widespread practice, as well as continually expanded.

Educators, the public, and particularly parents should not settle for impoverished assessment information. They should be well informed about criteria for meaningful and helpful assessment. To do justice to the students in our schools and to support their learning, we need to recognize that the process of appraising them fairly and effectively requires multiple measures constructed to high standards. Useful and meaningful evidence includes profiling of multiple elements of proficiency, with less emphasis on overall aggregate scores. A central theme of this report is that it is essential to assess diverse aspects of knowledge and competence, including how students understand and explain concepts, reason with what they know, solve problems, are aware of their states of knowing, and can self-regulate their learning and performance.

Achieving these goals requires a strong connection between educational assessments and modern theories of cognition and learning. Without this connection, assessment results provide incomplete, and perhaps misleading, information about what has been learned and appropriate next steps for improvement. Creating better assessments should not be viewed as a luxury, but as a necessity.

Perhaps the greatest challenges to the new science and design of educational assessment relate to disciplinary boundaries and established practices. For instance, there is currently an implicit assumption that one can create good tasks or good assessments and then leave it up to technical people to figure out how to analyze and report the results. Instead, the assessment design process must be a truly multidisciplinary and collaborative activity, with educators, cognitive scientists, subject matter specialists, and psychometricians informing one another during the design process. Other obstacles to pursuing new approaches to assessment stem from existing social structures in which familiar assessment practices are now deeply embedded and thus difficult to change. Professional development and public education are needed to convey how assessment should be designed and how it can be used most effectively in the service of learning.

The investment required to improve educational assessment and further develop the knowledge base to support that effort is substantial. However, this investment in our children and their educational futures is a reasonable one given the public’s legitimate expectation that assessment should both inform and enhance student achievement.

Education is a hot topic. From the stage of presidential debates to tonight's dinner table, it is an issue that most Americans are deeply concerned about. While there are many strategies for improving the educational process, we need a way to find out what works and what doesn't work as well. Educational assessment seeks to determine just how well students are learning and is an integral part of our quest for improved education.

The nation is pinning greater expectations on educational assessment than ever before. We look to these assessment tools when documenting whether students and institutions are truly meeting education goals. But we must stop and ask a crucial question: What kind of assessment is most effective?

At a time when traditional testing is subject to increasing criticism, research suggests that new, exciting approaches to assessment may be on the horizon. Advances in the sciences of how people learn and how to measure such learning offer the hope of developing new kinds of assessments-assessments that help students succeed in school by making as clear as possible the nature of their accomplishments and the progress of their learning.

Knowing What Students Know essentially explains how expanding knowledge in the scientific fields of human learning and educational measurement can form the foundations of an improved approach to assessment. These advances suggest ways that the targets of assessment-what students know and how well they know it-as well as the methods used to make inferences about student learning can be made more valid and instructionally useful. Principles for designing and using these new kinds of assessments are presented, and examples are used to illustrate the principles. Implications for policy, practice, and research are also explored.

With the promise of a productive research-based approach to assessment of student learning, Knowing What Students Know will be important to education administrators, assessment designers, teachers and teacher educators, and education advocates.

READ FREE ONLINE

Welcome to OpenBook!

You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

Do you want to take a quick tour of the OpenBook's features?

Show this book's table of contents , where you can jump to any chapter by name.

...or use these buttons to go back to the previous chapter or skip to the next one.

Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

Switch between the Original Pages , where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

To search the entire text of this book, type in your search term here and press Enter .

Share a link to this book page on your preferred social network or via email.

View our suggested citation for this chapter.

Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

Get Email Updates

Do you enjoy reading reports from the Academies online for free ? Sign up for email notifications and we'll let you know about new publications in your areas of interest when they're released.

• Privacy Policy

Home » Research Findings – Types Examples and Writing Guide

Research Findings – Types Examples and Writing Guide

Table of Contents

Research Findings

Definition:

Research findings refer to the results obtained from a study or investigation conducted through a systematic and scientific approach. These findings are the outcomes of the data analysis, interpretation, and evaluation carried out during the research process.

Types of Research Findings

There are two main types of research findings:

Qualitative Findings

Qualitative research is an exploratory research method used to understand the complexities of human behavior and experiences. Qualitative findings are non-numerical and descriptive data that describe the meaning and interpretation of the data collected. Examples of qualitative findings include quotes from participants, themes that emerge from the data, and descriptions of experiences and phenomena.

Quantitative Findings

Quantitative research is a research method that uses numerical data and statistical analysis to measure and quantify a phenomenon or behavior. Quantitative findings include numerical data such as mean, median, and mode, as well as statistical analyses such as t-tests, ANOVA, and regression analysis. These findings are often presented in tables, graphs, or charts.

Both qualitative and quantitative findings are important in research and can provide different insights into a research question or problem. Combining both types of findings can provide a more comprehensive understanding of a phenomenon and improve the validity and reliability of research results.

Parts of Research Findings

Research findings typically consist of several parts, including:

• Introduction: This section provides an overview of the research topic and the purpose of the study.
• Literature Review: This section summarizes previous research studies and findings that are relevant to the current study.
• Methodology : This section describes the research design, methods, and procedures used in the study, including details on the sample, data collection, and data analysis.
• Results : This section presents the findings of the study, including statistical analyses and data visualizations.
• Discussion : This section interprets the results and explains what they mean in relation to the research question(s) and hypotheses. It may also compare and contrast the current findings with previous research studies and explore any implications or limitations of the study.
• Conclusion : This section provides a summary of the key findings and the main conclusions of the study.
• Recommendations: This section suggests areas for further research and potential applications or implications of the study’s findings.

How to Write Research Findings

Writing research findings requires careful planning and attention to detail. Here are some general steps to follow when writing research findings:

• Organize your findings: Before you begin writing, it’s essential to organize your findings logically. Consider creating an outline or a flowchart that outlines the main points you want to make and how they relate to one another.
• Use clear and concise language : When presenting your findings, be sure to use clear and concise language that is easy to understand. Avoid using jargon or technical terms unless they are necessary to convey your meaning.
• Use visual aids : Visual aids such as tables, charts, and graphs can be helpful in presenting your findings. Be sure to label and title your visual aids clearly, and make sure they are easy to read.
• Use headings and subheadings: Using headings and subheadings can help organize your findings and make them easier to read. Make sure your headings and subheadings are clear and descriptive.
• Interpret your findings : When presenting your findings, it’s important to provide some interpretation of what the results mean. This can include discussing how your findings relate to the existing literature, identifying any limitations of your study, and suggesting areas for future research.
• Be precise and accurate : When presenting your findings, be sure to use precise and accurate language. Avoid making generalizations or overstatements and be careful not to misrepresent your data.
• Edit and revise: Once you have written your research findings, be sure to edit and revise them carefully. Check for grammar and spelling errors, make sure your formatting is consistent, and ensure that your writing is clear and concise.

Research Findings Example

Following is a Research Findings Example sample for students:

Title: The Effects of Exercise on Mental Health

Sample : 500 participants, both men and women, between the ages of 18-45.

Methodology : Participants were divided into two groups. The first group engaged in 30 minutes of moderate intensity exercise five times a week for eight weeks. The second group did not exercise during the study period. Participants in both groups completed a questionnaire that assessed their mental health before and after the study period.

Findings : The group that engaged in regular exercise reported a significant improvement in mental health compared to the control group. Specifically, they reported lower levels of anxiety and depression, improved mood, and increased self-esteem.

Conclusion : Regular exercise can have a positive impact on mental health and may be an effective intervention for individuals experiencing symptoms of anxiety or depression.

Applications of Research Findings

Research findings can be applied in various fields to improve processes, products, services, and outcomes. Here are some examples:

• Healthcare : Research findings in medicine and healthcare can be applied to improve patient outcomes, reduce morbidity and mortality rates, and develop new treatments for various diseases.
• Education : Research findings in education can be used to develop effective teaching methods, improve learning outcomes, and design new educational programs.
• Technology : Research findings in technology can be applied to develop new products, improve existing products, and enhance user experiences.
• Business : Research findings in business can be applied to develop new strategies, improve operations, and increase profitability.
• Public Policy: Research findings can be used to inform public policy decisions on issues such as environmental protection, social welfare, and economic development.
• Social Sciences: Research findings in social sciences can be used to improve understanding of human behavior and social phenomena, inform public policy decisions, and develop interventions to address social issues.
• Agriculture: Research findings in agriculture can be applied to improve crop yields, develop new farming techniques, and enhance food security.
• Sports : Research findings in sports can be applied to improve athlete performance, reduce injuries, and develop new training programs.

When to use Research Findings

Research findings can be used in a variety of situations, depending on the context and the purpose. Here are some examples of when research findings may be useful:

• Decision-making : Research findings can be used to inform decisions in various fields, such as business, education, healthcare, and public policy. For example, a business may use market research findings to make decisions about new product development or marketing strategies.
• Problem-solving : Research findings can be used to solve problems or challenges in various fields, such as healthcare, engineering, and social sciences. For example, medical researchers may use findings from clinical trials to develop new treatments for diseases.
• Policy development : Research findings can be used to inform the development of policies in various fields, such as environmental protection, social welfare, and economic development. For example, policymakers may use research findings to develop policies aimed at reducing greenhouse gas emissions.
• Program evaluation: Research findings can be used to evaluate the effectiveness of programs or interventions in various fields, such as education, healthcare, and social services. For example, educational researchers may use findings from evaluations of educational programs to improve teaching and learning outcomes.
• Innovation: Research findings can be used to inspire or guide innovation in various fields, such as technology and engineering. For example, engineers may use research findings on materials science to develop new and innovative products.

Purpose of Research Findings

The purpose of research findings is to contribute to the knowledge and understanding of a particular topic or issue. Research findings are the result of a systematic and rigorous investigation of a research question or hypothesis, using appropriate research methods and techniques.

The main purposes of research findings are:

• To generate new knowledge : Research findings contribute to the body of knowledge on a particular topic, by adding new information, insights, and understanding to the existing knowledge base.
• To test hypotheses or theories : Research findings can be used to test hypotheses or theories that have been proposed in a particular field or discipline. This helps to determine the validity and reliability of the hypotheses or theories, and to refine or develop new ones.
• To inform practice: Research findings can be used to inform practice in various fields, such as healthcare, education, and business. By identifying best practices and evidence-based interventions, research findings can help practitioners to make informed decisions and improve outcomes.
• To identify gaps in knowledge: Research findings can help to identify gaps in knowledge and understanding of a particular topic, which can then be addressed by further research.
• To contribute to policy development: Research findings can be used to inform policy development in various fields, such as environmental protection, social welfare, and economic development. By providing evidence-based recommendations, research findings can help policymakers to develop effective policies that address societal challenges.

Characteristics of Research Findings

Research findings have several key characteristics that distinguish them from other types of information or knowledge. Here are some of the main characteristics of research findings:

• Objective : Research findings are based on a systematic and rigorous investigation of a research question or hypothesis, using appropriate research methods and techniques. As such, they are generally considered to be more objective and reliable than other types of information.
• Empirical : Research findings are based on empirical evidence, which means that they are derived from observations or measurements of the real world. This gives them a high degree of credibility and validity.
• Generalizable : Research findings are often intended to be generalizable to a larger population or context beyond the specific study. This means that the findings can be applied to other situations or populations with similar characteristics.
• Transparent : Research findings are typically reported in a transparent manner, with a clear description of the research methods and data analysis techniques used. This allows others to assess the credibility and reliability of the findings.
• Peer-reviewed: Research findings are often subject to a rigorous peer-review process, in which experts in the field review the research methods, data analysis, and conclusions of the study. This helps to ensure the validity and reliability of the findings.
• Reproducible : Research findings are often designed to be reproducible, meaning that other researchers can replicate the study using the same methods and obtain similar results. This helps to ensure the validity and reliability of the findings.

Advantages of Research Findings

Research findings have many advantages, which make them valuable sources of knowledge and information. Here are some of the main advantages of research findings:

• Evidence-based: Research findings are based on empirical evidence, which means that they are grounded in data and observations from the real world. This makes them a reliable and credible source of information.
• Inform decision-making: Research findings can be used to inform decision-making in various fields, such as healthcare, education, and business. By identifying best practices and evidence-based interventions, research findings can help practitioners and policymakers to make informed decisions and improve outcomes.
• Identify gaps in knowledge: Research findings can help to identify gaps in knowledge and understanding of a particular topic, which can then be addressed by further research. This contributes to the ongoing development of knowledge in various fields.
• Improve outcomes : Research findings can be used to develop and implement evidence-based practices and interventions, which have been shown to improve outcomes in various fields, such as healthcare, education, and social services.
• Foster innovation: Research findings can inspire or guide innovation in various fields, such as technology and engineering. By providing new information and understanding of a particular topic, research findings can stimulate new ideas and approaches to problem-solving.
• Enhance credibility: Research findings are generally considered to be more credible and reliable than other types of information, as they are based on rigorous research methods and are subject to peer-review processes.

Limitations of Research Findings

While research findings have many advantages, they also have some limitations. Here are some of the main limitations of research findings:

• Limited scope: Research findings are typically based on a particular study or set of studies, which may have a limited scope or focus. This means that they may not be applicable to other contexts or populations.
• Potential for bias : Research findings can be influenced by various sources of bias, such as researcher bias, selection bias, or measurement bias. This can affect the validity and reliability of the findings.
• Ethical considerations: Research findings can raise ethical considerations, particularly in studies involving human subjects. Researchers must ensure that their studies are conducted in an ethical and responsible manner, with appropriate measures to protect the welfare and privacy of participants.
• Time and resource constraints : Research studies can be time-consuming and require significant resources, which can limit the number and scope of studies that are conducted. This can lead to gaps in knowledge or a lack of research on certain topics.
• Complexity: Some research findings can be complex and difficult to interpret, particularly in fields such as science or medicine. This can make it challenging for practitioners and policymakers to apply the findings to their work.
• Lack of generalizability : While research findings are intended to be generalizable to larger populations or contexts, there may be factors that limit their generalizability. For example, cultural or environmental factors may influence how a particular intervention or treatment works in different populations or contexts.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Research Contribution – Thesis Guide

Research Objectives – Types, Examples and...

Ethical Considerations – Types, Examples and...

Survey Instruments – List and Their Uses

Informed Consent in Research – Types, Templates...

Institutional Review Board – Application Sample...

Dissemination of the Findings of Educational Research

• pp 1137–1150

Cite this chapter

• Grant J. Harman 10 &
• Kay Harman 10  

Part of the book series: Springer International Handbooks of Education ((SIHE,volume 11))

1906 Accesses

1 Citations

This article describes the organisation and recent expansion of educational research in the Asia-Pacific region and how findings are disseminated, particularly to educational practitioners and policy-makers as well as to other researchers. It also discusses the use and impact of research findings, and possible means to improve utilisation of the findings of educational research.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info
• Durable hardcover edition

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Unable to display preview.  Download preview PDF.

Asian Development Bank (1996). Case Studies in Educational Research and Policy . Manila: Asian Development Bank.

Google Scholar  

Anderson, D. S., & Biddle, B. J. (Eds.) (1991). Knowledge for Policy: Improving Education through Research . London: Falmer Press.

Australian Council for Educational Research (2001). 70th Annual Report 1999–2000 . Melbourne: Australian Council for Educational Research.

Bazargan, A. (2000). Internal evaluation as an approach to revitalise university systems: The case of the Islamic Republic of Iran. Higher Education Policy , 13, 173–180.

Article   Google Scholar  

Burgon, J., Philips, A., & Nana, P. (2000). Annual Research Report 1999–2000 . Wellington: Research Division, New Zealand Ministry of Education.

Centre for Educational Research and Innovation (1995). Educational Research and Development: Trends , Issues and Challenges . Paris: Organisation for Economic Co-operation and Development.

Figgis, J., Zubrich, A., & Alderson, A. (2000). Backtracking practice and policies in tesearch. In The Impact of Educational Research (pp. 279–374 ). Canberra: Higher Education Division, Department of Education, Training and Youth Affairs.

Harman, G. (2000). Research on tertiary education in Australia. In S. Schwarz., & U. Teichler (Eds.), The Institutional Basis of Higher Education Research: Experiences and Perspectives . Dordrecht: Kluwer Academic Publishers.

Hayden, M., & Parry, S. (1997). Research on higher education in Australia and New Zealand. In J. Sadak & P. G. Altbach (eds), Higher Education Research at the Turn of the New Century: Structures , Issues and Trends (pp. 163–188 ). Paris: UNESCO Publishing.

Holbrook, A. Ainley, J., Bourke, S., Owen, J., McKenzie, P., Misson, S., & Johnson, T. (2000). Mapping educational research and its impact on Australian schools. In The Impact of Educational Research (pp. 15–279 ). Canberra: Higher Education Division, Department of Education, Training and Youth Affairs.

Lakomski, G. (Ed.) (1991). Beyond paradigms: Coherence and holism in educational research. International Journal of Educational Research , 15(6) , 501–597.

McGaw, B., Boud, D., Poole, M., Warry, R., & McKenzie, P. (1992). Educational Research in Australia: Report of a Review Panel . Canberra: Australian Government Publishing Service.

McMeniman, M., Cumming, J., Wilson, J., Stevenson, J., & Sim, C. (2000). Teacher Knowledge in Action. In The Impact of Educational Research. Canberra: Higher Education Division, Department of Education, Training and Youth Affairs.

Nanzhao, Z., Mujue, Z., Jiguang, B., & Tienjun, Z. (1999). The relationship among educational research, information and policy-making: A case study of China. In W. Rokicka (Ed.), Educational Documentation , Research and Decision-Making: National Case Studies . Paris: UNESCO International Bureau of Educational Research.

National Institute for Educational Policy Research (2001). website: http://www.nier.go .jp/homepage/kyoutsuu/index.htm

Ordonez, V., & Maclean, R. (1997). Asia: The impact of educational research on decision making. Prospects, 27 (4), 645–654.

New Zealand Council for Educational Research (2000). Annual Report 1998–99. Wellington: New Zealand Council for Educational Research.

Phelan, T., Anderson, D. S., & Bourke, P. (2000). Educational research in Australia: A bibliometric analysis. In The Impact of Educational Research (pp. 573–671). Canberra: Higher Education Division, Department of Education, Training and Youth Affairs.

Selby Smith, C. (1997). The Relationship Between Research and Decision-Making in Education: An Empirical Investigation. In The Impact of Educational Research. Clayton: Monash University, CEET Working Paper.

The Impact of Educational Research (2000). Canberra: Higher Education Division, Department of Education, Training and Youth Affairs.

Yadav, M. S., & Lakshmi, T. K. S. (1998). Educational research: The Indian scene. Indian Educational Review, 33 (1), 1–15.

White, R. T. (1988). Indigenes and Exotics: Balance of Trade in Research between Australia and the United States. Melbourne: Paper presented to the annual conference of the Australian Association for Research in Education.

Download references

Author information

Authors and affiliations.

Centre for Higher Education Management and Policy, University of New England, Armidale, Australia

Grant J. Harman & Kay Harman

You can also search for this author in PubMed   Google Scholar

Editor information

Editors and affiliations.

Flinders University Institute of International Education, Australia

John P. Keeves

National Institute for Educational Policy Research of Japan, Tokyo, Japan

Ryo Watanabe

UNESCO-UNEVOC International Centre for Education, Bonn, Germany

Rupert Maclean

Griffith University, Southport, Australia

Peter D. Renshaw

University of Queensland, St Lucia, Queensland, Australia

Colin N. Power

New Zealand Council for Educational Research, Wellington, New Zealand

Robyn Baker

National Institute of Education, Nanyang Technological University, Singapore

S. Gopinathan  & Ho Wah Kam  & 

Hong Kong Institute of Education, Hong Kong

Yin Cheong Cheng

Institute of International Education, Stockholm University, Sweden

Albert C. Tuijnman

Rights and permissions

Reprints and permissions

Copyright information

© 2003 Springer Science+Business Media Dordrecht

About this chapter

Harman, G.J., Harman, K. (2003). Dissemination of the Findings of Educational Research. In: Keeves, J.P., et al. International Handbook of Educational Research in the Asia-Pacific Region. Springer International Handbooks of Education, vol 11. Springer, Dordrecht. https://doi.org/10.1007/978-94-017-3368-7_78

Download citation

DOI : https://doi.org/10.1007/978-94-017-3368-7_78

Publisher Name : Springer, Dordrecht

Print ISBN : 978-90-481-6167-6

Online ISBN : 978-94-017-3368-7

eBook Packages : Springer Book Archive

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Mitigating the Academic Impacts of Proximity to Homicide

Affiliated Researcher

Senior Research Analyst

Research Assistant

1. What is the extent, distribution, and impact of living in close geographical proximity to violence on CPS students’ performance in schools? How do proximity to violence and its impacts on young people vary geographically and for particular groups of students, specifically students of color and those living in communities with high levels of poverty?

2. To what extent is there evidence whether schools can insulate or protect students from the negative effects on academic and behavioral outcomes of living in close proximity to violence, so as to support students’ health and wellness?

3. What elements of school climate and organization are characteristic of schools that appear to protect students against the negative impacts of proximity to violence on academic and behavioral outcomes?

4. How do adults working in schools that mitigate the impact of living in close proximity to violence understand and describe their work?

Community violence can have traumatic effects on young people, presenting daunting challenges for families and school educators working to support students’ growth, development, and achievement in school. It is critical to understand its effects on students and consider what schools can do to mitigate those effects, while working to reduce the prevalence of homicide and gun violence in the broader society.

This report offers evidence that schools can, and do, mitigate the negative impacts of adversity that young people experience. At the same time, this is complex, resource-intensive, and emotionally-taxing work—requiring time, resources and intentional strategies.

Key Findings

• The experience of living in close geographical proximity to homicide varied considerably for students across Chicago. 
• Living in close geographical proximity to homicide negatively affected students’ academic performance. 
• Some schools mitigated typical negative effects of living in proximity to homicide on academic performance.
• Schools that mitigate the negative effects of living in close geographic proximity to homicide on students’ academic outcomes were characterized by strong, positive school climates across a range of measures, including engaging instruction and trusting, connected relationships among students and between students and adults.
• Systems, structures, and routines that coordinate the support adults provide, center students, and emphasize connection and relationship between adults and young people were vital tools for the educators, administrators, and school staff interviewed.

Key Considerations

• Greater public investment in addressing the epidemic of gun violence and the broader, longstanding historical disinvestment in communities of color throughout the city is needed for more educational equity.
• Intentional, coordinated, and sustained efforts of dedicated adults in schools can address harm to students and promote their resilience.
• Deep, sustained effort in building and sustaining strong, collaborative, and trusting relationships among adults in schools can help make schools more responsive and more effective at mitigating the negative impact of violence.
• Strong, supportive, and trusting relationships between educators and students are a crucial resource for protecting students from harm and promoting resilient school communities.
• Responsive, resilient school communities do not emerge from a single initiative, require substantial resources, and demand sustained and hard work in the face of immensely difficult circumstances.

Related Resources

• January 2022 Article How Strong Principals Succeed Improving Student Achievement in High-Poverty Urban Schools
• September 2015 Report Suspending Chicago's Students Differences in Discipline Practices Across Schools
• May 2011 Report Student and Teacher Safety in Chicago Public Schools The Roles of Community Context and School Social Organization

• Open access
• Published: 31 May 2024

The role of medical schools in UK students’ career intentions: findings from the AIMS study

• Tomas Ferreira 1 , 3 ,
• Alexander M. Collins 2 , 3 ,
• Arthur Handscomb 3 ,
• Dania Al-Hashimi 4 &

the AIMS Collaborative

BMC Medical Education volume  24 , Article number:  604 ( 2024 ) Cite this article

1348 Accesses

19 Altmetric

Metrics details

To investigate differences in students’ career intentions between UK medical schools.

Cross-sectional, mixed-methods online survey.

The primary study included all 44 UK medical schools, with this analysis comprising 42 medical schools.

Participants

Ten thousand four hundred eighty-six UK medical students.

Main outcome measures

Career intentions of medical students, focusing on differences between medical schools. Secondary outcomes included variation in medical students’ satisfaction with a prospective career in the NHS, by medical school.

2.89% of students intended to leave medicine altogether, with Cambridge Medical School having the highest proportion of such respondents. 32.35% of respondents planned to emigrate for practice, with Ulster medical students being the most likely. Of those intending to emigrate, the University of Central Lancashire saw the highest proportion stating no intentions to return. Cardiff Medical School had the greatest percentage of students intending to assume non-training clinical posts after completing FY2. 35.23% of participating medical students intended to leave the NHS within 2 years of graduating, with Brighton and Sussex holding the highest proportion of these respondents. Only 17.26% were satisfied with the prospect of working in the NHS, with considerable variation nationally; Barts and the London medical students had the highest rates of dissatisfaction.

Conclusions

This study reveals variability in students’ career sentiment across UK medical schools, emphasising the need for attention to factors influencing these trends. A concerning proportion of students intend to exit the NHS within 2 years of graduating, with substantial variation between institutions. Students’ intentions may be shaped by various factors, including curriculum focus and recruitment practices. It is imperative to re-evaluate these aspects within medical schools, whilst considering the wider national context, to improve student perceptions towards an NHS career. Future research should target underlying causes for these disparities to facilitate improvements to career satisfaction and retention.

Peer Review reports

Introduction

The rapidly changing dynamics of modern healthcare require a comprehensive understanding of the driving forces behind the career trajectories of doctors. As the landscape of patient care, healthcare policy, and medical technology continues to evolve, so too do the career choices of emerging doctors. These choices, as research increasingly demonstrates, are not solely the product of personal inclination or market demand but are deeply influenced by their experiences in medical school [ 1 ].

In recent years, the recruitment and retention of doctors within the United Kingdom’s (UK) National Health Service (NHS) have emerged as pressing concerns, requiring a detailed analysis of the factors influencing the career intentions of medical students [ 2 , 3 , 4 ]. To address this, the Ascertaining the career Intentions of Medical Students (AIMS) study — the largest ever UK medical student survey — delineated the career intentions and underlying motivations of students, highlighting a significant trend towards alternative careers or emigration, influenced predominantly by remuneration, work-life balance, and working conditions within the NHS [ 5 ].

Expanding upon the insights of the AIMS study, we seek to further explore the nuanced differences in career intentions among medical students, in relation to their institutional affiliations, and foster a dialogue concerning medical education and workforce planning in the UK, highlighting the role of medical schools in shaping career trajectories. It is posited that these educational institutions, with their diverse curricular designs and teaching philosophies, may play a pivotal role in shaping the prospective professional trajectories of their students. Furthermore, the distinct socio-economic and cultural environments in which these schools are situated, and those of the students they attract, may also contribute to the varied perspectives and career aspirations of students. Historically, the field of medical education has been subject to a variety of pedagogical philosophies, curricular reforms, and institutional priorities. These variations across medical schools, while often subtle, can result in significant differences in the way students perceive their roles, responsibilities, and opportunities within the broader healthcare ecosystem. Literature suggests that various elements including the culture of a medical school and its sociocultural context play a significant role in shaping the professional aspirations of its students [ 1 , 6 ].

This manuscript seeks to identify and characterise these differences, with a focused analysis on how various medical schools in the UK might be influencing the career preferences and intended paths of their students. These findings may hold significant implications for various stakeholders within the healthcare sector. Policymakers could find guidance for strategic investments and resource allocation to areas anticipated to experience shortages, while educationalists could gain an opportunity for reflection on the potential influence of their institutions on student aspirations, thereby considering necessary adjustments. Furthermore, it affords insights for improved recruitment strategies, critical to ensuring the NHS’s continued role in the UK.

Study design

The AIMS study was a national, cross-sectional, multi-centre study of medical students conducted according to its published protocol and extensively described in its main publication [ 5 , 7 ]. Participants from 44 UK medical schools recognised by the General Medical Council (GMC) were recruited through a non-random sampling method via a novel, self-administered, 71-item questionnaire. The survey was hosted on the Qualtrics survey platform (Provo, Utah, USA), a GDPR-compliant online platform that supports both mobile and desktop devices.

Participant recruitment and eligibility

In an attempt to minimise bias and increase the survey’s reach to promote representativeness, a network of approximately 200 collaborators was recruited across 42 medical schools – one collaborator per year group, per school – prior to the study launch to disseminate the study. All students were eligible to apply to become a collaborator. This approach aimed to obtain a representative sample and improve our findings’ generalisability. The survey was disseminated between 16 January 2023 and 27 March 2023, by the AIMS Collaborative via social media (including Instagram, Facebook, WhatsApp, and LinkedIn), word of mouth, medical student newsletters/bulletins, and medical school emailing lists.

Individuals were eligible to participate in the survey if they were actively enrolled in a UK medical school acknowledged by the GMC and listed by the Medical School Council (MSC). Certain new medical schools had received approval from the GMC but were yet to admit their inaugural cohort of students, so were excluded from the study.

Data processing and storage

To prevent data duplication, each response was restricted to a single institutional email address. Any replicated email entries were removed prior to data analysis. In cases where identical entries contained distinct responses, the most recent entry was kept. Responses for which valid institutional email addresses were missing were removed prior to data analysis to preserve the study’s integrity.

The findings of this subanalysis, and the AIMS study, were reported in accordance with the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) guidelines [ 8 ].

Quantitative data analysis

Descriptive analysis was carried out with Microsoft Excel (V.16.71) (Arlington, Virginia, USA), and statistical inference was performed using RStudio (V.4.2.1) (Boston, Massachusetts, USA). Tables and graphs were generated using GraphPad Prism (V.9.5.0) (San Diego, California, USA). ORs, CIs and p values were computed by fitting single-variable logistic regression models to explore the effect of various demographic characteristics on students’ career intentions. CIs were calculated at 95% level. We used p  < 0.05 to determine the statistical significance for all tests.

Study population and exclusion

All current students of all year groups at UK medical schools recognised by the GMC and the MSC were eligible for participation. Brunel Medical School and Kent and Medway Medical School were excluded from this current analysis due to the limited number of respondents from these institutions ( n  < 30), to avoid misrepresenting the career intentions and characteristics of their broader student populations.

Ethical approval

Ethical approval was granted by the University of Cambridge Research Ethics Committee (reference PRE.2022.124) on the 5th of January 2023. Prior to completing the survey, all participants provided informed consent. Participating medical schools were contacted prior to data collection to seek support and request permission to contact their students.

Demographics

In total, 10,486 students across all 44 UK medical schools participated in the survey. To enable comparison of students’ career intentions between medical schools, only 42 medical schools were considered due to the sample size gathered. The average number of responses per medical school was 244, with a median of 203 (IQR 135–281). Participants had a median age of 22 (IQR 20–23). Among the participants, 66.5% were female ( n  = 6977), 32.7% were male ( n  = 3429), 0.6% were non-binary ( n  = 64), and 16 individuals chose not to disclose their gender. A detailed breakdown of participant characteristics, including gender, ethnicity, previous schooling, and course type, is illustrated in Supplemental Figs.  1 a-d.

A total of 303/10,486 (2.89%, CI: 2.59, 3.23%) medical students intended to leave the profession entirely, either immediately after graduation ( n  = 104/303, 34.32%, CI: 29.20, 39.84%), after completion of FY1 ( n  = 132/303, 43.56%, CI: 38.1, 49.19%), or after completion of FY2 ( n  = 67/303, 22.11%, CI: 17.8, 27.12%). Figure  1 illustrates the distribution of these students throughout UK medical schools as a percentage of total response numbers per school. The medical schools of Cambridge, Oxford, and Imperial College medical schools had the highest proportion of students intending to leave the profession altogether.

Proportion of Medical Students Intending to Leave the Profession Across UK Medical Schools. The figure depicts the percentage of students at each UK medical school who intend to exit the medical field entirely. Percentages are calculated as a proportion of total respondents from each individual school

Furthermore, 32.35% of participating medical students ( n  = 3392/10,486, CI: 31.46, 33.25%) expressed intentions to emigrate to practise medicine, either immediately after graduation ( n  = 220/3292, 6.49%, CI: 5.71, 7.36%), after completion of FY1 ( n  = 1101/3292 32.46%, CI: 30.90, 34.05%) or after FY2 ( n  = 2071/3292, 61.06%, CI: 59.40, 62.68%). Figure  2 a demonstrates the distribution of these intentions across UK medical schools, relative to total response rates per school. Notably, Ulster University had the highest proportion of students considering emigration (45.45%), in contrast to Edge Hill, where 19.64% held similar intentions. Among students intending to emigrate, 49.56% ( n  = 1681, CI: 47.88, 51.24%) planned a return to the UK after a few years abroad, while 7.87% ( n  = 267, CI: 7.01, 8.83%) expected to return after completing their medical training abroad. The remaining 42.57% ( n  = 1444, CI: 40.92, 44.24%) expressed no plans to return to practise in the UK, as demonstrated in Fig.  2 b.

Proportion of Medical Students Intending to Emigrate Across UK Medical Schools (a) and Return Prospects (b). a illustrates the proportion of students from each UK medical school who intend to emigrate for medical practice, relative to total respondents from each school. b delineates the return prospects among students planning to emigrate

Of the 8806 respondents intending to complete both FY1 and FY2, 48.76% ( n  = 4294, CI: 47.72, 49.81%) planned to enter specialty training in the UK immediately thereafter; 21.11% ( n  = 1859, CI: 20.27, 21.98%) intended to enter a non-training clinical job in the UK (commonly comprising an ‘F3’ year, including a junior clinical fellowship or clinical teaching fellowship, or in locum roles). These ‘non-training’ roles, although valuable for gaining clinical experience, are largely standalone posts which do not contribute to accreditation within medical specialties. The school with the highest proportion of responses indicating plans to enter specialty training immediately after FY2 was Edge Hill (64.29%), whereas at Cardiff only 25.62% shared this intention. Cardiff students were also most likely to plan to enter non-training clinical posts after FY2, at 29.06%. Students from the University of Buckingham were, by far, the least likely to look to pursue non-training posts (2.70%). Figure  3 a and b present the distribution of these intentions across UK medical schools.

Distribution of Post-Foundation Programme Career Intentions Among UK Medical Students by School. a illustrates the proportion of students at each UK medical school intending to enter specialty training immediately following the Foundation Programme. b presents the proportion of students planning to enter non-training clinical roles (comprising ‘F3’ year roles, junior clinical fellowships, clinical teaching fellowships, or locum positions) in the UK after FY2

In total, 35.23% (3695/10,486) of medical students intend to leave the NHS within 2 years of graduating, either to practise abroad or leave medicine. Respondents from Brighton and Sussex Medical School expressed this intention most often (47.78%), whilst those from Aston Medical School were the least likely to do so (20.77%) (Fig.  4 ).

Proportion of UK Medical Students Intending to Leave the NHS Within 2 Years of Graduation, by School

To better ascertain the medical student population’s sentiments towards working in the NHS, respondents were asked to share their degree of satisfaction with several factors. Likert scale matrices were employed, with options ranging from ‘Very satisfied’ to ‘Not at all satisfied’. An important aspect was students’ overall satisfaction with the prospect of working within the NHS, with which only 17.26% of students were either satisfied or very satisfied. This figure varied substantially by institution as illustrated in Fig.  5 . Surveyed students from Barts and the London, Liverpool, and King’s College London GKT schools of medicine were the most dissatisfied, with dissatisfaction rates of 76.07, 72.48 and 66.84% respectively. Conversely, students from Aberdeen (43.27%), Buckingham (34.78%) and Ulster medical schools (33.33%) were those least dissatisfied with the prospect of working in the NHS.

Medical Students’ Overall Satisfaction with the Prospect of Working in the NHS, by School. The figure illustrates the variation in levels of career satisfaction across UK medical schools

Principal findings

This study identified considerable institutional variation in students’ career intentions and sentiment about their future careers.

Our results show that, in each UK medical school, over a fifth of participating medical students intend to leave the NHS within 2 years of graduation – and in some medical schools, this figure was approximately half. Nationally, this figure surpassed a third of surveyed medical students. Most would-be leavers plan to emigrate, many permanently, while a notable minority of respondents plan to leave the profession altogether. Here, we consider possible reasons for these trends, and offer potential means of adapting medical schools to avert the loss of these medics from the NHS workforce.

The levels of satisfaction among medical students concerning their prospective employment within the NHS displayed marked disparities, influenced potentially by institutional factors. In certain schools, up to 76% of students expressed dissatisfaction with the prospect of a career within the NHS, contrasted with the 48% recorded in others. The national average of 60% dissatisfaction is concerning and warrants further investigation to identify the underlying causes of this marked variability across different medical schools. Understanding the specific factors influencing medical students’ satisfaction levels could be critical in developing strategies to improve their perceptions of careers in the NHS.

Differing career sentiment between medical schools

Many differences exist between medical schools, some inherent or incidental, and others the result of decisions taken by medical faculties. Naturally, there is variation by geography, in the clinical environments and patient populations to which students are exposed, or in differences in the NHS between the UK’s devolved nations. The composition of the student body, in terms of various demographic characteristics also differs considerably between schools (Supplemental Figs.  1 a-d). Additionally, despite meeting minimum standards set by the GMC, medical schools are distinct in their curriculum delivery and priorities, culture, and other factors. This ‘hidden curriculum’ can be influential in students’ outlook towards medicine and their careers [ 9 ]. Medical schools’ autonomy extends to setting local recruitment practices, leading to differences in entry requirements and favoured attributes for which candidates are selected [ 10 ].

Curriculum focus and its influence

Certain faculties may favour students for academic potential or other attributes that may not necessarily correspond to their aptitude or interest in clinical medicine. At these schools, medical curricula may be more science-focused, such as by employing the ‘traditional’ model of medical education which firmly separates preclinical and clinical studies. During the early years of study, in which clinical exposure is low, students may find themselves detached from the medical field and begin considering alternative careers. This may be especially true where intercalated degrees form mandatory components of the curriculum – the receipt of which would enable pursuit of graduate roles or postgraduate degrees. Moreover, some institutions emphasising academic achievement may offer academic opportunities which could further distance those enrolled from the profession. For instance, previous graduates of MB/PhD programmes, an option to intercalate a PhD degree offered by only a limited number of universities, have gone onto careers in academia, industry, and business [ 11 , 12 ].

Recruitment practices

Despite the inherent importance of academic ability, it is important to recognise that a ‘good’ doctor requires a balance of various attributes including empathy, resilience, and communication skills. Furthermore, a clear understanding and realistic expectations of the profession are critical. The possible discrepancy between academic aptitude and the day-to-day reality of medical practice may be a contributing factor to the observed trends of students contemplating leaving the profession. Therefore, ensuring a balanced and holistic approach in selection processes could contribute to cultivating a workforce committed to pursuing medical practice in the NHS long term. Currently, prospective students undergo varying forms of interviews, which, due to their brevity and the substantial volume of applications, may not adequately capture a candidate’s realistic expectations and motivations towards a medical career. To increase the robustness of the selection process, medical schools should consider revisiting the structure of their interview processes, potentially incorporating methods to more accurately assess applicants’ understanding and enthusiasm for a medical career within the NHS more accurately. This approach could include comprehensive discussions focusing on the complexities and realities associated with a medical career [ 13 ]. Moreover, there are relevant differences in institutions’ selection criteria, with some valuing extracurricular activities, while some place greater emphasis on personal statements more, and others prioritise results achieved in admission exams [ 10 ]. Implementing such changes in the recruitment process can be a proactive step towards retaining talent within the NHS and encouraging more students to envisage a fulfilling career within the medical profession.

Institutional reputation

Respondents from institutions which place highly in national and international university rankings exhibited a greater propensity to consider leaving the profession [ 14 , 15 ]. Notably, the universities of Cambridge (8.59%), Oxford (8.26%), and Imperial College London (8.24%) led this trend. Attending these, and other, historically prestigious schools, may boost non-clinical career opportunities, so their students may be attracted to the perceived benefits of alternative careers over those in clinical practice. This institutional reputation may have initially attracted some students, for whom the career opportunities outside clinical practice now offer more compelling options compared to working in the NHS. This, coupled with growing reports of doctors looking to leave the health service, may partly explain the trend observed [ 3 ]. However, it is important to note that this phenomenon is neither new nor limited to the UK, with a 2001 study identifying growing numbers of medical students in the United States intending to pursue non-clinical, non-academic careers over time [ 16 ]. Notably, only four schools had 0% of students intending to leave the profession.

Demographic influences

Moreover, the composition of the student body, particularly in terms of demographic makeup may represent another potential influence on career intentions. For instance, if data indicate that students from certain demographics were more likely to pursue a certain career path, a school with a higher proportion of such students may appear to exhibit a similar inclination. It is important to note that these tendencies may be reflective of broader societal and demographic differences, rather than factors intrinsic to the respective institutions. A deeper analysis of demographic nuances may elucidate the intricate interplay of background and career choices, offering valuable insights for future policy and institutional strategies. Furthermore, it would be prudent to recognise that certain students, particularly those from widening participation backgrounds, may have limited agency regarding the career pathway they pursue. For some, this limitation may be financial in nature or due to caring responsibilities, while for others it may be more strongly related to the awarding gap [ 17 ].

Proposed solutions and future directions

Our findings underscore the need to explore the reasons for the observed disparities in students’ career sentiment across medical schools. Using this information, medical courses may be adapted to improve students’ feelings about their future medical careers in the NHS or otherwise. As students’ perspectives are guided by their educational experiences, undergraduate training they deem suboptimal could contribute to a diminished enthusiasm for a career in medicine. Higher standards of teaching may increase interest and engagement in the medical profession, while inadequate teaching quality could engender frustration and disillusionment. Unsatisfied students may opt to pursue alternative careers or relocate to destinations where they perceive education and training standards to be higher [ 18 ]. To substantiate this, further studies could endeavour to quantify perceptions towards teaching standards at medical school and the impact of teaching quality on students’ career choices, potentially guiding improvements in curriculum design and faculty development.

It is important to note that many respondents will have been studying medicine during the COVID-19 pandemic. During this period, medical schools had the difficult task of balancing infection risk with maintaining educational standards. Centres will have differed in their approach, and negative experiences - educational or otherwise - from this period may have adversely influenced students’ attitudes towards medicine [ 19 ].

Furthermore, the structure or variety of clinical placements used by some medical schools could more effectively convey a positive outlook of medical careers or the NHS. This is often contingent on the clinical environments in which medical students rotate. For instance, limited exposure to certain specialties or sub-specialties—only available at select centres—may inadvertently obscure potentially rewarding career paths. Similarly, limited opportunities in rural medicine, public health, or other non-hospital-based pathways may also achieve the same effect [ 20 ]. Spaces and learning opportunities may also be shared with increasing cohort sizes or, depending upon geography, with students from other medical schools, potentially diluting learning opportunities [ 21 ]. Staffing levels, workplace culture and health outcomes also vary geographically, both within and between the UK’s devolved nations [ 22 , 23 , 24 , 25 ]. These factors inform students’ perceptions of the career and may contribute to their decision-making. To mitigate this, medical faculties would benefit from establishing or expanding student feedback mechanisms. The objective is to identify factors affecting training experiences and to ensure equitable access across the UK, irrespective of the medical school attended.. Such engagement may also reveal which career paths are under-explored in individual medical curricula. In response to students’ views, or from faculties’ own understanding of where these deficits may lie, schools may consider offering means of addressing this, such as through optional specialty taster days.

Where higher proportions of students expressed interests in either relocating to work abroad or in leaving the profession entirely, there may be benefit in fostering a culture of mentorship and guidance around medical careers. Mentorship can support students to navigate systems used during applications for increasingly competitive specialty training programmes [ 26 , 27 ]. Guidance from medics acquainted with these processes can support students to pursue their preferred specialty and could consequently reduce attrition by improving their perceived career prospects.

Findings in context

The AIMS study highlighted a wide range of factors which contribute to medical students’ career sentiment and their intended career trajectory [ 5 ]. Here, we explored the role of medical schools in this complex equation and, although influential, this must be considered in that wider context. While national policy reform addressing factors such as remuneration and working conditions are required to reverse current trends in students’ career intentions, the strategies proposed in this manuscript may serve to address regional disparities.

Limitations

Despite the AIMS study constituting the largest ever study of UK medical students, due to the methods of dissemination, the number of students who saw the invitation to participate in the study is unknown, and therefore we are unable to calculate the response rate. Consequently, the sample may have been subject to selection bias, possibly driven by greater response rates among students with existing interests in this subject. Additionally, the questions in our survey instruct students to be definitive even when they might not yet have formulated their career plans, a not-improbable situation, particularly for those in the early years of medical school.

Moreover, being a cross-sectional study, it is not possible to comment on changes to medical students’ career sentiment with time. Although informed by their undergraduate training and experiences therefrom, at the time of participation, respondents had not yet worked as medical doctors. As such, their opinions may change once immersed in the career and working in the health service. In anticipation of this limitation, the questionnaire sought consent for a planned follow-up study, to which a 71.29% positive response rate was captured. It is hoped that this study’s findings may be validated by tracking changes in sentiment over time.

Importantly, there was also variability in the number of responses achieved from each medical school. This occurred despite recruitment of a large medical student collaborator network. This discrepancy might be attributed to various factors, including the approach of dissemination undertaken by university or medical school administrators, the design of clinical placements, or the presence and influence of local student societies, among other considerations. To avoid potential misrepresentation due to inadequate sample sizes, we opted to exclude data from the two medical schools that obtained fewer than 30 responses.

While the broader trends of medical students intending to leave the NHS within 2 years of graduating are concerning, the variation in career sentiment across UK medical schools requires consideration. This analysis implicates a complex interplay of factors—ranging from curriculum focus and cohort demographics to recruitment strategies, teaching quality, and clinical experience—in shaping these career intentions. Such variation in career sentiment between institutions may be indicative of deeper issues, possibly rooted in educational approaches and experiences at undergraduate level - on which the potential impact of the COVID-19 pandemic should be noted.

It is evident that approaches taken to recruitment, educational framework, and support within medical schools require reassessment. Subsequent investigations should examine the underlying causes of disparities in career sentiment by institution, aiming to cultivate resilience, dedication, and - critically - professional fulfilment among the future medical workforce in the UK.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request once all planned subsequent analyses are completed.

Ibrahim H, Nair SC, Shaban S, El-Zubeir M. Reducing the physician workforce crisis: career choice and graduate medical education reform in an emerging Arab country. Educ Health (Abingdon). 2016;29(2):82–8. https://doi.org/10.4103/1357-6283.188716 . PMID: 27549644.

Article   Google Scholar  

General Medical Council. The state of medical education and practice in the UK. The workforce report; 2022.

Google Scholar  

Waters A. A third of junior doctors plan to leave NHS to work abroad in next 12 months. BMJ. 2022;379:3066.

BMA. Catastrophic crisis facing NHS as nearly half of hospital consultants plan to leave in next year, WARNS. 2022. Available: https://www.bma.org.uk/bma-media-centre/bma-report-reveals-potentially-catastrophic-crisis-in-hospital-consultant-workforce-levels . Accessed 17 Apr 2024.

Ferreira T, Collins AM. Feng O the AIMS collaborative, et al career intentions of medical students in the UK: a national, cross-sectional study (AIMS study). BMJ Open. 2023;13:e075598. https://doi.org/10.1136/bmjopen-2023-075598 .

Rourke J. How can medical schools contribute to the education, recruitment and retention of rural physicians in their region? Bull World Health Organ. 2010;88(5):395–6. https://doi.org/10.2471/BLT.09.073072 .

Ferreira T, Collins AM, Horvath R. Ascertaining the career intentions of medical students (AIMS) in the United Kingdom post graduation: protocol for a mixed methods study. JMIR Res Protoc. 2023;12:e45992. https://doi.org/10.2196/45992 .

von EE, Altman DG, Egger M, et al. The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. BMJ. 2007;335:806–8.

Lempp H, Seale C. The hidden curriculum in undergraduate medical education: qualitative study of medical students’ perceptions of teaching. BMJ. 2004;329:770. https://doi.org/10.1136/bmj.329.7469.770 .

Medical Schools Council Medical school entry requirements for 2024 start [internet]. Medical Schools Council; [cited 2023 Sep 18]. Available from: https://www.medschools.ac.uk/studying-medicine/making-an-application/entry-requirements-for-2024-start#:~:text=On%20entry%2C%20applicants%20must%20havehigher%20than%20for%20those%20without .

Cox TM, Brimicombe J, Wood DF, Peters DK. The Cambridge bachelor of medicine (MB)/doctor of philosophy (PhD): graduate outcomes of the first MB/PhD programme in the UK. Clin Med (Lond). 2012;12(6):530–4. https://doi.org/10.7861/clinmedicine.12-6-530 . PMID: 23342406; PMCID: PMC5922592.

Barnett-Vanes A, Ho G, Cox TM. Clinician-scientist MB/PhD training in the UK: a nationwide survey of medical school policy. BMJ Open. 2015;5:e009852. https://doi.org/10.1136/bmjopen-2015-009852 .

Ferreira T. Beyond government accountability: the role of medical schools in addressing the NHS workforce crisis. J R Soc Med. 2023;116(11):395–8.

QS World University Rankings 2023. Top global universities [Internet]. Top Universities; 2023. [cited 2023 Sep 18]. Available from: https://www.topuniversities.com/university-rankings/world-university-rankings/2023?&tab=indicators .

Times Higher Education. World University Rankings 2023 [Internet]. 2023. [cited 2023 Sep 18]. Available from: https://www.timeshighereducation.com/world-university-rankings/2023/world-ranking .

Richard GV, Nakamoto DM, Lockwood JH. Medical Career Choices: Traditional and New Possibilities. JAMA. 2001;285(17):2249–50. https://doi.org/10.1001/jama.285.17.2249-JMS0502-3-1 .

Brown C, Goss C, Sam AH. Is the awarding gap at UK medical schools influenced by ethnicity and medical school attended? A retrospective cohort study. BMJ Open. 2023;13(12):e075945.

Gouda P, Kitt K, Evans DS, et al. Ireland’s medical brain drain: migration intentions of Irish medical students. Hum Resour Health. 2015;13:11. https://doi.org/10.1186/s12960-015-0003-9 .

Wilcha R. Effectiveness of virtual medical teaching during the COVID-19 crisis: systematic review. JMIR Med Educ. 2020;6(2):e20963 https://mededu.jmir.org/2020/2/e20963. 10.2196/20963 .

Pathman DE, Konrad TR, Ricketts TC 3rd. Medical education and the retention of rural physicians. Health Serv Res. 1994;29(1):39–58 PMID: 8163379; PMCID: PMC1069987.

Roberts N, Bolton P. Medical school places in England from September 2018. London: House of Commons Library; 2017.

NHS digital health. NHS workforce statistics. 2023. https://digital.nhs.uk/data-and-information/publications/statistical/nhs-workforce-statistics/february-2023 . Accessed 17 Apr 2024.

Dixon-Woods M, Baker R, Charles K, Dawson J, Jerzembek G, Martin G, McCarthy I, McKee L, Minion J, Ozieranski P, Willars J. Culture and behaviour in the English National Health Service: overview of lessons from a large multimethod study. BMJ Qual Saf. 2014;23(2):106–15.

Office for National Statistics. "Health in England: 2015 to 2020." 2022. Available from: https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/healthandwellbeing/bulletins/healthinengland/2015to2021 . Accessed 17 Apr 2024.

Report by the Comptroller and Auditor General. Healthcare across the UK: a comparison of the NHS in England, Scotland, Wales and Northern Ireland. London: National Audit Office; 2012. https://www.nao.org.uk/wp-content/uploads/2012/06/1213192es.pdf . Accessed 17 Apr 2024.

Best J. The growing bottlenecks in specialty training. BMJ. 2023;382:1732. https://doi.org/10.1136/bmj.p1732 .

Ferreira T. Escalating competition in NHS: implications for healthcare quality and workforce sustainability. Postgrad Med J. 2024:qgad131.

Download references

Acknowledgements

We would like to thank all students that participated in this study.

The AIMS Collaborative.

Tomas Ferreira 1 , Alexander M. Collins 2 , Rita Horvath 1 , Oliver Feng 4 , Richard J Samworth 4 , Mario K Teo 6 , Crispin C Wigfield 6 , Maeve K Mulchrone 7 , Alisha Pervaiz 8 , Heather A Lewis 7 , Anson Wong 7 , Buzz Gilks 1 , Charlotte Casteleyn 9 , Sara Kidher 10 , Erin Fitzsimons-West 1 , Tanzil Rujeedawa 1 , Meghna Sreekumar 1 , Eliza Wade 11 , Juel Choppy-Madeleine 8 , Yasemin Durmus 12 , Olivia King 10 , Yu Ning Ooi 8 , Malvi Shah 9 , Tan Jit Yih 13 , Samantha Burley 1 , Basma R Khan 4 , Emma Slack 1 , Rishik S Pilla 14 , Jenny Yang 1 , Vaishvi Dalal 8 , Brennan L Gibson 7 , Emma Westwood 9 , Brandon S H Low 6 , Sara R Sabur 9 , Wentin Chen 7 , Maryam A Malik 9 , Safa Razzaq 10 , Amardeep Sidki 10 , Giulia Cianci 15 , Felicity Greenfield 3 , Sajad Hussain 3 , Alexandra Thomas 11 , Annie Harrison 16 , Hugo Bernie 3 , Luke Dcaccia 11 , Linnuel J Pregil 13 , Olivia Rowe 11 , Ananya Jain 17 , Gregory K Anyaegbunam 8 , Syed Z Jafri 18 , Sudhanvita Arun 4 , Alfaiya Hashmi 19 , Ankith Pandian 15 , Joseph R Nicholson 20 , Hannah Layton-Joyce 21 , Kouther Mohsin 7 , Matilda Gardener 3 , Eunice C Y Kwan 18 , Emily R Finbow 4 , Sakshi Roy 22 , Zoe M Constantinou 13 , Mackenzie Garlick 3 , Clare L Carney 23 , Samantha Gold 24 , Bilal Qureshi 25 , Daniel Magee 26 , Grace Annetts 25 , Khyatee Shah 27 , Kholood T Munir 14 , Timothy Neill 22 , Gurpreet K Atwal 28 , Anesu Kusosa 18 , Anthony Vijayanathan 14 , Mia Mäntylä 8 , Momina Iqbal 27 , Sara Raja 29 , Tushar Rakhecha 3 , Muhammad H Shah 22 , Pranjil Pokharel 30 , Ashna Anil 31 , Kate Stenning 21 , Katie Appleton 18 , Keerthana Uthayakumar 28 , Rajan Panacer 32 , Yasmin Owadally 17 , Dilaxiha Rajendran 33 , Harsh S Modalavalasa 15 , Marta M Komosa 13 , Morea Turjaka 18 , Sruthi Saravanan 27 , Amelia Dickson 24 , Jack M Read 24 , Georgina Cooper 26 , Wing Chi Do 34 , Chiamaka Anthony-Okeke 35 , Daria M Bageac 24 , David C W Loh 28 , Rida Khan 19 , Ruth Omenyo 31 , Aidan Baker 34 , Imogen Milner 23 , Kavyesh Vivek 17 , Manon Everard 36 , Wajiha Rahman 14 , Denis Chen 26 , Michael E Bryan 34 , Shama Maliha 26 , Vera Onongaya 31 , Amber Dhoot 17 , Catherine L Otoibhi 35 , Harry Donkin-Everton 14 , Mia K Whelan 24 , Claudia S F Hobson 37 , Anthony Haynes 20 , Joshua Bayes-Green 35 , Mariam S Malik 28 , Subanki Srisakthivel 24 , Sophie Kidd 28 , Alan Saji 11 , Govind Dhillon 16 , Muhammed Asif 38 , Riya Patel 30 , Jessica L Marshall 20 , Nain T Raja 29 , Tawfique Rizwan 38 , Aleksandra Dunin- Borkowska 17 , James Brawn 23 , Karthig Thillaivasan 9 , Zainah Sindhoo 27 , Ayeza Akhtar 25 , Emma Hitchcock 36 , Kelly Fletcher 38 , Lok Pong Cheng 37 , Medha Pillaai 28 , Sakshi Garg 15 , Wajahat Khan 12 , Ben Sweeney 20 , Ria Bhatt 39 , Madison Slight 40 , Adan M I Chew 32 , Cameron Thurlow 41 , Kriti Yadav 39 , Niranjan Rajesh 39 , Nathan-Dhruv Mistry 16 , Alyssa Weissman 37 , Juan F E Jaramillo 30 , William Thompson 42 , Gregor W Abercromby 20 , Emily Gaskin 4 , Chloe Milton 43 , Matthew Kokkat 36 , Momina Hussain 26 , Nana A Ohene-Darkoh 39 , Syeda T Islam 33 , Anushruti Yadav 31 , Eve Richings 44 , Samuel Foxcroft 44 , Sukhdev Singh 32 , Vivek Sivadev 40 , Guilherme Movio 30 , Ellena Leigh 45 , Harriet Charlton 44 , James A Cairn 45 , Julia Shaaban 23 , Leah Njenje 43 , Mark J Bishop 44 , Humairaa Ismail 30 , Sarah L Henderson 44 , Daniel C Chalk 20 , Daniel J Mckenna 26 , Fizah Hasan 43 , Kanishka Saxena 32 , Iona E Gibson 44 and Saad Dosani 38 .

1 School of Clinical Medicine, University of Cambridge, Cambridge, United Kingdom.

2 School of Public Health, Faculty of Medicine, Imperial College London, London, United Kingdom.

3 Bristol Medical School, University of Bristol, Bristol, United Kingdom.

4 Sheffield Medical School, University of Sheffield, Sheffield, United Kingdom.

5 Statistical Laboratory, Centre for Mathematical Sciences, University of Cambridge, Cambridge, UK.

6 Department of Neurosurgery, Southmead Hospital, Bristol, UK.

7 School of Medicine, University of Birmingham, Birmingham.

8 School of Medicine, University of Glasgow, Glasgow.

9 UCL Medical School, University College London, London.

10 School of Medicine, University of Leicester, Leicester, UK.

11 School of Medicine, University of Southampton, Southampton, UK.

12 School of Medicine, University of Leeds, Leeds, UK.

13 School of Medicine and Dentistry, Queen Mary University of London, London, UK.

14 GKT School of Medical Education, King’s College London, London, UK.

15 School of Medicine, University of Nottingham, Nottingham, UK.

16 School of Medicine, University of Liverpool, Liverpool, UK.

17 Faculty of Medicine, Imperial College London, London, UK.

18 Norwich Medical School, University of East Anglia, Norwich, UK.

19 St George’s, University of London, London, UK.

20 Peninsula Schools of Medicine and Dentistry, Plymouth University, Plymouth, UK.

21 School of Medicine, University of Warwick, Coventry, UK.

22 School of Medicine, Queen’s University Belfast, Belfast, UK.

23 School of Medicine, Swansea University, Swansea, UK.

24 School of Medicine, University of Exeter, Exeter, UK.

25 Medical Sciences Division, University of Oxford, Oxford, UK.

26 School of Medicine, Keele University, Keele, UK.

27 Lincoln Medical School, University of Nottingham, Lincoln, UK.

28 School of Medicine, University of Dundee, Dundee, UK.

29 School of Medicine and Dentistry, University of Aberdeen, Aberdeen, UK.

30 School of Medicine, Lancaster University, Lancaster, UK.

31 School of Medicine, Cardiff University, Cardiff, UK.

32 School of Medicine, Aston University, Birmingham, UK.

33 School of Medicine, University of Sunderland, Sunderland, UK.

34 School of Medicine, The University of Edinburgh, Edinburgh, UK.

35 School of Medical Education, Newcastle University, Newcastle, UK.

36 Hull York Medical School, Hull and York, UK.

37 School of Medicine, University of Buckingham, Buckingham, UK.

38 School of Medicine, University of Manchester, Manchester, UK.

39 School of Medicine, University of Central Lancashire, Preston, UK.

40 School of Medicine, University of St Andrews, St Andrews, UK.

41 Brighton and Sussex Medical School, Brighton and Sussex, UK.

42 School of Medicine, Ulster University, Coleraine, UK.

43 School of Medicine, Anglia Ruskin University, Chelmsford, UK.

44 Scottish Graduate Entry Medicine (ScotGEM) Programme, Universities of St Andrews and Dundee, Scotland, UK.

45 School of Medicine, Edge Hill University, Ormskirk, UK.

TF is the guarantor.

Queens’ College, University of Cambridge. The institution has had no role in the design of the study, nor collection, analysis, and interpretation of data and in writing the manuscript.

Author information

Authors and affiliations.

School of Clinical Medicine, University of Cambridge, Cambridge, UK

• Tomas Ferreira

School of Public Health, Faculty of Medicine, Imperial College London, London, UK

Alexander M. Collins

Bristol Medical School, University of Bristol, Bristol, UK

Tomas Ferreira, Alexander M. Collins & Arthur Handscomb

Sheffield Medical School, University of Sheffield, Sheffield, UK

Dania Al-Hashimi

You can also search for this author in PubMed   Google Scholar

• , Alexander M. Collins
• , Rita Horvath
• , Oliver Feng
• , Richard J. Samworth
• , Mario K. Teo
• , Crispin C. Wigfield
• , Maeve K. Mulchrone
• , Alisha Pervaiz
• , Heather A. Lewis
• , Anson Wong
• , Buzz Gilks
• , Charlotte Casteleyn
• , Sara Kidher
• , Erin Fitzsimons-West
• , Tanzil Rujeedawa
• , Meghna Sreekumar
• , Eliza Wade
• , Juel Choppy-Madeleine
• , Yasemin Durmus
• , Olivia King
• , Yu Ning Ooi
• , Malvi Shah
• , Tan Jit Yih
• , Samantha Burley
• , Basma R. Khan
• , Emma Slack
• , Rishik S. Pilla
• , Jenny Yang
• , Vaishvi Dalal
• , Brennan L. Gibson
• , Emma Westwood
• , Brandon S. H. Low
• , Sara R. Sabur
• , Wentin Chen
• , Maryam A. Malik
• , Safa Razzaq
• , Amardeep Sidki
• , Giulia Cianci
• , Felicity Greenfield
• , Sajad Hussain
• , Alexandra Thomas
• , Annie Harrison
• , Hugo Bernie
• , Luke Dcaccia
• , Linnuel J. Pregil
• , Olivia Rowe
• , Ananya Jain
• , Gregory K. Anyaegbunam
• , Syed Z. Jafri
• , Sudhanvita Arun
• , Alfaiya Hashmi
• , Ankith Pandian
• , Joseph R. Nicholson
• , Hannah Layton-Joyce
• , Kouther Mohsin
• , Matilda Gardener
• , Eunice C. Y. Kwan
• , Emily R. Finbow
• , Sakshi Roy
• , Zoe M. Constantinou
• , Mackenzie Garlick
• , Clare L. Carney
• , Samantha Gold
• , Bilal Qureshi
• , Daniel Magee
• , Grace Annetts
• , Khyatee Shah
• , Kholood T. Munir
• , Timothy Neill
• , Gurpreet K. Atwal
• , Anesu Kusosa
• , Anthony Vijayanathan
• , Mia Mäntylä
• , Momina Iqbal
• , Sara Raja
• , Tushar Rakhecha
• , Muhammad H. Shah
• , Pranjil Pokharel
• , Ashna Anil
• , Kate Stenning
• , Katie Appleton
• , Keerthana Uthayakumar
• , Rajan Panacer
• , Yasmin Owadally
• , Dilaxiha Rajendran
• , Harsh S. Modalavalasa
• , Marta M. Komosa
• , Morea Turjaka
• , Sruthi Saravanan
• , Amelia Dickson
• , Jack M. Read
• , Georgina Cooper
• , Wing Chi Do
• , Chiamaka Anthony-Okeke
• , Daria M. Bageac
• , David C. W. Loh
• , Rida Khan
• , Ruth Omenyo
• , Aidan Baker
• , Imogen Milner
• , Kavyesh Vivek
• , Manon Everard
• , Wajiha Rahman
• , Denis Chen
• , Michael E. Bryan
• , Shama Maliha
• , Vera Onongaya
• , Amber Dhoot
• , Catherine L. Otoibhi
• , Harry Donkin-Everton
• , Mia K. Whelan
• , Claudia S. F. Hobson
• , Anthony Haynes
• , Joshua Bayes-Green
• , Mariam S. Malik
• , Subanki Srisakthivel
• , Sophie Kidd
• , Alan Saji
• , Govind Dhillon
• , Muhammed Asif
• , Riya Patel
• , Jessica L. Marshall
• , Nain T. Raja
• , Tawfique Rizwan
• , Aleksandra Dunin-Borkowska
• , James Brawn
• , Karthig Thillaivasan
• , Zainah Sindhoo
• , Ayeza Akhtar
• , Emma Hitchcock
• , Kelly Fletcher
• , Lok Pong Cheng
• , Medha Pillaai
• , Sakshi Garg
• , Wajahat Khan
• , Ben Sweeney
• , Ria Bhatt
• , Madison Slight
• , Adan M. I. Chew
• , Cameron Thurlow
• , Kriti Yadav
• , Niranjan Rajesh
• , Nathan-Dhruv Mistry
• , Alyssa Weissman
• , Juan F. E. Jaramillo
• , William Thompson
• , Gregor W. Abercromby
• , Emily Gaskin
• , Chloe Milton
• , Matthew Kokkat
• , Momina Hussain
• , Nana A. Ohene-Darkoh
• , Syeda T. Islam
• , Anushruti Yadav
• , Eve Richings
• , Samuel Foxcroft
• , Sukhdev Singh
• , Vivek Sivadev
• , Guilherme Movio
• , Ellena Leigh
• , Harriet Charlton
• , James A. Cairn
• , Julia Shaaban
• , Leah Njenje
• , Mark J. Bishop
• , Humairaa Ismail
• , Sarah L. Henderson
• , Daniel C. Chalk
• , Daniel J. Mckenna
• , Fizah Hasan
• , Kanishka Saxena
• , Iona E. Gibson
•  & Saad Dosani

Contributions

T.F. responsible for conceptualisation. T.F. responsible for obtaining funding and ethical approval. T.F. responsible for collaborator recruitment and management. T.F. responsible for project administration. All authors responsible for writing the manuscript. All authors responsible for editing and revising the manuscript. T.F. responsible for supervision. T.F. is the guarantor. All authors have read and approved the manuscript.

Corresponding author

Correspondence to Tomas Ferreira .

Ethics declarations

Ethics approval and consent to participate.

Ethical approval was granted by the University of Cambridge Research Ethics Committee (reference PRE.2022.124) on the 5th of January 2023. Informed consent was obtained from all participants.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., supplementary material 3., supplementary material 4., supplementary material 5., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Ferreira, T., Collins, A.M., Handscomb, A. et al. The role of medical schools in UK students’ career intentions: findings from the AIMS study. BMC Med Educ 24 , 604 (2024). https://doi.org/10.1186/s12909-024-05366-6

Download citation

Received : 08 October 2023

Accepted : 28 March 2024

Published : 31 May 2024

DOI : https://doi.org/10.1186/s12909-024-05366-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Medical students
• Career intentions
• Medical school
• Health policy

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]