conclusion of science education

Barriers and Opportunities for 2-Year and 4-Year STEM Degrees: Systemic Change to Support Students' Diverse Pathways (2016)

Chapter: 7 conclusions and recommendations, 7 conclusions and recommendations.

Students who enter college to earn a 2-year or 4-year degree in an area of science, technology, engineering, and mathematics (STEM) face many barriers in the multiple pathways to degree completion. The pathways that students are taking to earn STEM degrees are diverse and complex, with multiple entry and exit points and an increased tendency to earn credits from multiple institutions. The barriers students face differentially affect students from underrepresented minority groups and women, as shown by the lower rates of degree completion by black, Hispanic, and female students. The barriers are particularly difficult to overcome for students with limited experience with and knowledge of higher education in general and of STEM fields in particular, such as first-generation students and many of those who are eligible for Pell Grants. The undergraduate student population has undergone significant shifts, and undergraduates who aspire to earn STEM degrees are much different than their counterparts 25 years ago. The percentage of women and students from underrepresented backgrounds who are interested in STEM degrees has been on the rise ( National Science Board, 2014 ). The number of students attending undergraduate institutions who have previous work experience, have taken a semester or more away from college, and have families is also increasing ( National Center for Education Statistics, 2013 ). And as noted throughout this report, students interested in STEM degrees are navigating the undergraduate education system in far more complex ways than previously. Increasingly, students, including those seeking STEM degrees, are combining credits from multiple institutions to earn a degree, are transferring from 2-year to 4-year institutions (often without completing a degree or certificate program), are

transferring from 4-year to 2-year institutions, are enrolling at multiple institutions both simultaneously and sequentially, and are taking college credit in high school through dual enrollment and advanced placement courses (see Eagan et al., 2014 ; Salzman and Van Noy, 2014 ; Van Noy and Zeidenberg, 2014 ).

In the face of these changes in the student population, the committee found that—although there are some notable exceptions—postsecondary institutions, STEM departments, accrediting entities, and state and federal education policy have been slow to adapt. Although there are many small- and larger-scale efforts to remove the barriers that students face, we find that the underlying causes of these barriers need to be addressed much more deeply and systematically for widespread and sustainable reform to take hold. An important reason that institutions of higher education struggle to consistently deliver high-quality education experiences for STEM aspirants is that the institutions themselves and undergraduate education more generally were designed to serve much different student populations and to help them progress along much different education pathways than are typically being used today. In a sense, higher education institutions function more like a collection of discrete practices and policies, rather than being interconnected and synergistic.

There are many examples of unchanged policies and programs:

a “weed-out” culture in many STEM departments rather than a supportive environment;
graduation rates that are tracked on a 2-, 4-, or 6-year time clock, uninformed by data on median time to degree for different fields or the need to account for remediation time or the reality of part-time study;
recognition and rewards to institutions for the quantity of degrees awarded rather than the quality, relevance, and levels of learning that are expected of and provided to students; and
completion rates that are calculated on the basis of enrollment by first-time, full-time students and so discount part-time students and transfer students.

Several facts are worth noting. Institutions that take on the challenge of providing a high-quality STEM education to students from disadvantaged backgrounds often do so with fewer resources than elite institutions. Underrepresented minority students and first-generation students are more likely to enroll at a 2-year institution than a 4-year institution ( Van Noy and Zeidenberg, 2014 ). Historically black colleges and universities award about 20 percent of all of the STEM bachelor’s degrees earned by black students in fields other than psychology and social sciences, and about one-third of

black students who have earned a Ph.D. in these STEM fields attained a bachelor’s degree in STEM from historically black colleges and universities ( National Science Foundation, 2013 ).

Two overarching findings undergird our conclusions and recommendations:

The “STEM pipeline” metaphor focuses on the students who enter at one end of the education system and those who emerge with STEM degrees. The metaphor does not reflect the diverse ways that students now move across and within higher education institutions, the diversity of paths that lead students to STEM degrees, or the expanding range of careers for those with STEM degrees. The “STEM pathways” metaphor is a more comprehensive and inclusive way of examining how students progress through STEM degrees and the much broader kinds of supports that higher education needs to provide to enable these students to successfully complete a credential.
Undergraduate STEM reform efforts have been piecemeal and not institutional in nature, and those that do not attend to today’s students, their challenges or to the policy environments in which the institutions operate are likely to be short-lived and largely ineffective.

In the following three sections, we present our conclusions and recommendations related to today’s students, about the role of institutions in serving those students, and about the need for systemic and sustainable change. Our conclusions and recommendations are embedded in these sections. In addition, our recommendations are presented by stakeholder group in Box 7-1 .

TODAY’S STEM STUDENTS

CONCLUSION 1 There is an opportunity to expand and diversify the nation’s science, technology, engineering, and mathematics (STEM) workforce and STEM-skilled workers in all fields if there is a commitment to appropriately support students through degree completion and provide more opportunities to engage in high-quality STEM learning and experiences.

earning when they first begin their undergraduate studies, but also in the fact that one-third of students who begin with an undeclared major select a STEM discipline as a major ( Eagan et al., 2014 ).

The degree completion rates for all STEM aspirants is less than 50 percent, with the lowest completion rates found among students from underrepresented groups (blacks, Hispanics, and Native Americans). Three common threads among students from groups with low degree completion rates are that they have the greatest economic need, are more likely to require developmental courses, and have few if any immediate family members who completed college. Increasingly, students who aspire to earn STEM degrees are coming to college with a broad range of life experiences, are transferring among institutions at least once, and are more frequently stopping out. They are also likely to be working while attending college, especially 2-year colleges, and some are parents. Although the demographic

composition of students who are seeking STEM degrees is shifting, it remains true that on average, STEM aspirants arrive on campus better prepared and having achieved more academically than the student body as a whole. Yet only 40 percent of these students earn STEM degrees within 6 years.

degree due to the way that STEM is taught and the difficulty in attaining support. That support, such as tutoring, mentoring, authentic STEM experiences, or other supports, improves retention in STEM majors ( Estrada, 2014 ). In other words, students are dissuaded from studying STEM rather than being drawn into studying a different discipline. While some of the switching may be the result of considered choices based on opportunities to explore attractive alternatives, lack of a supportive environment in STEM likely contributes to those decisions.

Based on STEM persistence and completion rates, and research on why students leave, it seems clear that 2-year and 4-year institutions are not consistently providing all STEM degree seekers with a high-quality education experience and the supports that they need to succeed, especially in introductory and gateway courses.

CONCLUSION 2 Science, technology, engineering, and mathematics (STEM) aspirants increasingly navigate the undergraduate education system in new and complex ways. It takes students longer for completion of degrees, there are many patterns of student mobility within and across institutions, and the accommodation and management of student enrollment patterns can affect how quickly and even whether a student earns a STEM degree.

An increasing percentage of STEM aspirants and those who graduate with a STEM degree or certificate begin their college career at 2-year institutions. This is especially true among black, Hispanic, and American Indian students. In addition, the rate at which STEM aspirants and graduates transfer from a 4-year institution to a 2-year institution (reverse transfer) is also increasing ( Salzman and Van Noy, 2014 ). Likewise, there is increased availability of and enrollment in high school dual-enrollment programs and Advanced Placement and International Baccalaureate STEM courses, both of which provide students with college-level courses and are accepted for college credit and placement at many institutions. The increased movement of undergraduate STEM credential aspirants often leads to loss of credits earned (because some credits do not transfer), classes that may not count toward the degree requirements in a second institution, and difficulties in adjusting to new academic cultures. All of these factors influence the amount of time it takes STEM aspirants to graduate, even if they are consistently making progress toward their degree and doing well in their classes. Students who reverse transfer (from a 4-year to a 2-year institution) are substantially less likely to complete a STEM degree within 6 years. However, students who concurrently enroll in multiple institutions are only slightly less likely to complete a STEM degree in 6 years than those who attend only one institution. Students who need remedial classes also

need to take more credits, which often extends their time to graduation and increases the cost of their education. This is one reason that students with remedial needs often “time out” of federal financial aid.

CONCLUSION 3 National, state, and institutional undergraduate data systems often are not structured to gather information needed to understand how well the undergraduate education system and institutions of higher education are serving students.

Most large-scale data systems that include information on undergraduate students were built to track students in a pipeline model. Some systems focus primarily on gathering data on full-time or first-time students, while others do not account well for the swirling of students among institutions. These systems often rely on graduation rates as the sole metric of success for students and institutions: they do not systematically collect information on students’ goals, reasons for transferring or leaving institutions, progress toward a credential, nor do they provide access to evidence-based teaching practices or student support systems.

The limitations of the data systems make it difficult for the states and the federal government to understand how the postsecondary education system is serving students, if some students are being served better than others, and which institutions consistently do not meet the needs of their students. In addition, most faculty, departments, and institutions do not know when students encounter barriers to earning the degree they seek or what supports students may need to succeed.

RECOMMENDATION 1 Data collection systems should be adjusted to collect information to help departments and institutions better understand the nature of the student populations they serve and the pathways these students take to complete science, technology, engineering, and mathematics (STEM) degrees.

Colleges and universities need to more consistently leverage the information collected across their campuses (e.g., offices of institutional research, STEM departments, and student aid offices) to better understand who their students are, their movement among majors and institutions, the barriers they encounter in working toward their degrees, and the services or supports they need.
States and federal agencies should consider how the information they require institutions to collect might enable better tracking of students through pathways they take to earn a STEM degree within and especially across institutions. In addition, they should consider

expanding measures of success, which increasingly inform funding formulas, beyond graduation rates.

There are a growing number of institutions that are using the data collected across their institutions to support student learning and identify when and where students need support to continue with their work toward STEM degrees. More campuses are identifying difficult introductory courses to provide supplemental instruction or use evidence-based instructional strategies and track students with data dashboards to improve progress toward degrees; however, systematic collection and use of such data are not widespread. With a better understanding of what barriers students typically encounter, and when and why students typically encounter them, institutions can more efficiently provide individualized support to students.

Existing data on undergraduate students and institutions are limited in a number of ways. We were not able to ascertain the success of STEM students who transferred from community colleges without earning a credential, nor could we address questions related to what happens to students who “time out” of financial aid.

A vision of success that goes beyond graduation rates and time to completion has been emerging from definitions of success developed by various stakeholder groups, including the American Association of Community Colleges, the Aspen Institute, the Bill & Melinda Gates Foundation, the National Governors Association, and the Association of American Universities. These stakeholders have identified a broad set of academic indicators, such as success in remedial and first-year courses, course completion, credit accumulation, credits to degree, retention and transfer rates, degrees awarded, expanding access, and learning outcomes. Much work is needed by these and other stakeholders to develop a systematic, national data source on such factors.

RECOMMENDATION 2 Federal agencies, foundations, and other entities that fund research in undergraduate science, technology, engineering, and mathematics (STEM) education should prioritize research to assess whether enrollment mobility in STEM is a response to financial, institutional, individual, or other factors, both individually and collectively, and to improve understanding of how student progress in STEM, in comparison with other disciplines, is affected by enrollment mobility.

ultimate implications for student outcomes. While late decisions can force students to take more than the required number of credits for a major because many STEM programs are highly structured with various requirements, early decisions may not be possible or even desirable if students are unsure about their career paths and need time to discover their interests. These decisions may be influenced by institutional policies (e.g., on early deadlines to declare program entry), discipline-based professional societies, and accrediting bodies. Research is needed on:

what kinds of exploration students undertake as they decide to major (or not) in a STEM field and how they make their decisions,
why students enter STEM programs at different times,
the factors that attract them to STEM majors,
how institutional structures might facilitate or delay their entry into STEM, and
to what extent the identified problems may be associated with changing student demographics.

INSTITUTIONAL SUPPORT FOR TODAY’S STEM STUDENTS

CONCLUSION 4 Better alignment of science, technology, engineering, and mathematics (STEM) programs, instructional practices, and student supports is needed in institutions to meet the needs of the populations they serve. Programming and policies that address the climate of STEM departments and classrooms, the availability of instructional supports and authentic STEM experiences, and the implementation of effective teaching practices together can help students overcome key barriers to earning a STEM degree, including time to degree and the price of a STEM degree.

Substantial research in the last decade indicates that persistence in STEM is related to a host of factors that go beyond academic preparation of the individual student. Those factors include institutional practices and supports that reinforce student identities as scientists or engineers, recognition of talent, interaction with peers, and opportunities for authentic research experiences. Instructional practices that encourage active and interactive learning are keys to improving student learning and persistence in STEM. In addition, faculty behavior and attitudes inside and outside the classroom can provide cues that help students persist toward STEM degrees.

and increase their interaction with peers, faculty, and teaching assistants. The report also points to the slow adoption of these practices. Research has also shown increased effects of evidence-based teaching practices when paired with co-curricular supports.

Even when high-quality instructional practices are implemented, students often receive little guidance or support regarding how efficiently to navigate the vast array of undergraduate education options. This makes it difficult for students to know how to get from where they are academically to where they want to be or to help them explore options that they have not considered about current and future career opportunities. This situation may help explain the phenomena of students who take classes at multiple institutions, transfer between institutions, or take time off from college, but all of this “churning” is associated with lower rates of completion and longer times to degree. Time is the enemy of many undergraduate STEM students. As time to degree increases, the likelihood of graduating seems to decrease due to a host of factors, perhaps, most importantly, increasing student debt.

Long-term program evaluations of interventions now provide evidence of what can increase persistence and graduation rates among STEM students. The most promising interventions combine contact with faculty and a supportive peer group along with access to authentic STEM experiences. Undergraduate research experiences show positive effects for both persistence and intentions for graduate school, over and above faculty mentoring experiences (though the two are often combined in structured research programs). Co-curricular supports (e.g., research experiences, mentoring, summer bridge programs, and living and learning communities) have been shown to affect STEM student persistence and completion when they align with evidence-based practices in supporting student learning and interests.

The culture of STEM classrooms and departments also influences STEM student persistence. Many students interested in STEM degrees, especially those from underrepresented groups and women, decide to pursue other fields due to the instructional practices, the “weed out” culture of some introductory STEM courses, and the lack of opportunities to engage in authentic STEM experiences.

To train effective mentors and create a culture of inclusiveness, faculty need to be provided opportunities to become more aware of implicit bias and stereotyping as well as how to avoid them. Departments need to encourage greater student involvement in research and design experiences, as well as in clubs and organizations related to a discipline, which have been shown to improve retention in STEM ( Chang et al., 2014 ; Espinosa, 2011 ). The role of professional STEM clubs and organizations also points to the importance of local chapters as well as national student organizations and

the development or enhancement of professional society programs for undergraduates to sustaining interest and retention in STEM.

The need for and nature of student supports likely will differ by type of institution and student background. It would be useful for institutional leaders to collect the kind of data about students’ current interests and needs to better determine how they can offer a range of interventions that are most appropriate to the current and changing needs of their students.

In general, 2-year and 4-year institutions serve students with different backgrounds, goals, and educational preparation. Community colleges enroll more older, first-generation, and working students than 4-year colleges. They play a significant role in the pathways that a diverse population of students takes in earning STEM degrees and certificates. Science and engineering programs at 2-year institutions enrolled relatively high proportions of Hispanic, Asian, and female students but a lower proportion of black students, who were more likely to be enrolled in technical-level programs.

Although community college STEM students have relatively low completion rates, their high persistence rates are notable. Students who begin their undergraduate education at a 2-year institution often take more than 6 years to complete their degrees, due to part-time enrollment, interruptions in their enrollment, and loss of course credit when they transfer between institutions. Understanding the quality of the educational experiences provided by 2-year institutions is hampered by the existing data systems that do not provide clear information on students who transfer from 2-year institutions to 4-year institutions without earning a degree or certificate. In addition, the contribution of 2-year institutions to the degrees that transfer students receive at 4-year institutions is not tracked and so is not well understood. Although there is emerging evidence regarding the characteristics of departments that support the use of evidence-based pedagogy, we were unable to find data on the relative use of such pedagogy. In fact, we were unable to even find recent national data on who teaches STEM courses (full-time tenured faculty, adjunct, or other), the level of instructional training that instructors had received, or alignment of instructor practices with evidence-based practices.

RECOMMENDATION 3 Federal agencies, foundations, and other entities that support research in undergraduate science, technology, engineering, and mathematics education should support studies with multiple methodologies and approaches to better understand the effectiveness of various co-curricular programs.

There is a need for more studies that track students over time to assess both the short-term and long-term effects of program elements across academic pathways. Such studies should include data from similar cohorts of students who do not participate in the program as a comparison or control group. When possible and appropriate, participants should be randomly assigned to co-curricular program groups.

For these studies to be useful, co-curricular programs need to identify measurable outcomes such as retention, grades, knowledge, and degree conferment, and they should identify the discipline of study. In-depth case studies or focus groups with program participants and similar students to track experiences at time of participation and shortly after can add to the research. Studies should move beyond linear models of student progress to a credential to test models that are more reflective of the kind of decision making of students. In addition, studies of long-time co-curricular programs and the nature of the sites that house them are needed to better understand how to sustain successful programs.

RECOMMENDATION 4 Institutions, states, and federal policy makers should better align educational policies with the range of education goals of students enrolled in 2-year and 4-year institutions. Policies should account for the fact that many students take more than 6 years to graduate, and should reward 2- year and 4-year institutions for their contributions to the educational success of students they serve, which includes not only those who graduate.

The states and the federal government should revise undergraduate accountability policies so that systems of assessment, evaluation, and accountability give credit to and do not penalize (i.e., in-state funding formulas) institutions for supporting students’ progress toward their desired educational outcome. It is important that policies take into account the various ways that students are currently using different institutions in pursuit of a degree, certification, or technical skills.
The states and the federal government should extend financial aid eligibility to graduation for students making satisfactory progress toward a degree or certificate, so that students do not “time out” of financial aid eligibility.
Colleges and universities should shift their institutional policies toward a model in which all students who are admitted to a degree program are expected to complete that program and are provided the instruction, resources, and support they need to do so, rather than a model in which it is assumed that a large fraction of students will be unsuccessful and will leave science, technology, engineering,

and mathematics programs. This model can save money because the time to degree is shortened and the number of drops, failures, withdrawals, and repeating of courses is reduced.

Systems of accountability for undergraduate education need to better align to the pathways that students actually are taking to earn STEM degrees. To do so, more thought needs to go into how each institution can track students’ progression toward a degree or other outcome-—including gaining skills to upgrade current employment and earning a certificate while working toward an associate’s degree—recognizing the long time to degree completion among many STEM students.

STEM students are taking longer to earn degrees because of many factors, including transferring among institutions, changing majors, and the need to follow strict course sequencing. It is now uncommon for a student to complete a 2-year degree in 2 years or a 4-year degree in 4 years. The time frame of some current financial aid policies do not reflect what is now common and do not align with the pathways that students are taking to earn degrees. Providing financial aid on the basis of the number of semesters a student has spent in college has a differentially negative impact on students from underrepresented minority groups, who more frequently than other students need remedial courses due to weakness in their K-12 preparation, starting at 2-year institutions, and taking longer to graduate. Financial aid policies could recognize the current pathways by focusing on whether students are making adequate progress toward their academic goals.

The culture of many STEM courses and departments is undergirded by the belief that “natural” ability, gender, or ethnicity is a significant determinant of a student’s success in STEM . Related to this view is the tendency for introductory mathematics and science courses to be used as “gatekeeper” or “weeder” courses, which may discourage students from pursuing STEM degrees, through highly competitive classrooms and a lack of pedagogy that promotes active participation and emphasizes mastery and improvement. These courses often seek to select out and distinguish those with some perceived ability in STEM. The classroom and departmental culture needs to value diversity and be based on the understanding that all students aspiring to earn a STEM degree have the potential to succeed in STEM and provide all students the opportunity to make an informed decision about whether they want to continue pursuing STEM credentials.

RECOMMENDATION 5 Institutions of higher education, disciplinary societies, foundations, and federal agencies that fund undergraduate education should focus their efforts in a coordinated manner on critical issues to support science, technology, engineering, and mathematics

(STEM) strategies, programs, and policies that can improve STEM instruction.

Colleges and universities should adjust faculty reward systems to better promote high-quality instruction and provide support for faculty to integrate effective teaching strategies into their practice. They should encourage educators to learn about and implement effective teaching methods by supporting participation in workshops, professional meetings, campus-based faculty development programs, and other related opportunities. Instructional quality is a key aspect of a student’s undergraduate experience that could be addressed by providing incentives for more faculty members to align their classroom practices with evidence-based pedagogy.
Disciplinary and professional membership organizations should become more active in developing tools to support evidence-based teaching practices, and providing professional development in using these active pedagogies for new and potential faculty members and instructors.
The National Center for Education Statistics of the U.S. Department of Education should collect systematic data on tenured, tenure-track, and nontenure-track faculty and staff, as it previously did through the National Study of Postsecondary Faculty. Such data will make it possible to understand who is teaching STEM courses and whether they have participated in professional development programs to implement evidence-based instructional strategies. The Department of Education should support research on what supports are needed to allow all educators, including tenured, tenure-track faculty, full-time nontenured teaching faculty, adjunct faculty, and lecturers, to successfully implement such strategies.
Federal agencies, foundations, and other entities should invest in implementation research to better understand how to increase adoption of evidence-based instructional strategies.

Although a considerable body of research is emerging about the nature and effect of effective instructional practices, this awareness has not necessarily been translated into widespread implementation of such practices in STEM classrooms. More investment needs to be made in implementation research to determine how to support putting this knowledge into practice. There have been calls for working with postdoctoral scholars and graduate students during their education to ensure that professional development is available to them on effective teaching strategies. This requires departmental support and leadership across an institution, along with agreement that

future faculty should have mastered research-based teaching strategies as well as disciplinary research knowledge and skills.

RECOMMENDATION 6 Accrediting agencies, states, and institutions should take steps to support increased alignment of policies that can improve the transfer process for students.

Regional accrediting bodies should review student outcomes by participating colleges and require periodic updates of articulation agreements in response to those student outcomes.
States should encourage tracking transfer credits and using other metrics to measure the success of students who transfer.
Colleges and universities should work with other institutions in their regions to develop articulation agreements and student services that contribute to structured and supportive pathways for students seeking to transfer credits.

The pathways that students are taking to earn undergraduate STEM degrees have become increasingly complex, with greater numbers of students earning credits at more than one institution. Thus, issues of transfer and articulation are now relevant to an increasing proportion of STEM students, as well as students in other majors. The range of different regional, state, and institutional transfer and articulation policies that students encounter can be dizzying, and they can extend a student’s time to completion and increase the cost of college, as well as being stressful to navigate.

Regional accrediting agencies, states, and institutions can all take steps to remove the barriers that students currently face when transferring credits among institutions. Removing these barriers may require creative and collaborative solutions, but they have the potential not only to improve students’ educational experience, but also to make higher education institutions more efficient and effective.

RECOMMENDATION 7 State and federal agencies and accrediting bodies together should explore the efficacy and tradeoffs of different articulation agreements and transfer policies.

There is a need to better understand the efficacy of existing and new models of articulation agreements. Currently, it is not clear which types of agreements work for different types of students (including students from underrepresented groups and veterans), and for different types of transfers (vertical, reverse, and lateral). Research on the effects of articulation agreements needs to consider not only the policies that guide the transfer

of credits, but also the supports developed to make it easier for students to navigate the policies and adjust to their different academic environments.

SYSTEMIC AND SUSTAINABLE CHANGE IN STEM EDUCATION

CONCLUSION 5 There is no single approach that will improve the educational outcomes of all science, technology, engineering, and mathematics (STEM) aspirants. The nature of U.S. undergraduate STEM education will require a series of interconnected and evidence-based approaches to create systemic organizational change for student success.

From years of attempts to improve higher education for all, many lessons have been learned. Focusing narrowly on individuals rather than on the entire system is not effective because it leads to changes of minimal scale and sustainability. Failing to leverage the many actors in education—individuals, departments, institutions, disciplinary societies, business and industry, governments—in a systematic fashion is ineffective because different levels of the education system often operate in isolation and are often unaware of how their actions can both affect and be affected by other components of the system.

In addition, focusing narrowly on pedagogical and curricular changes and not considering other variables that are related to student success, such as institutional policies, articulation, faculty culture, and financial aid, limits the potential effects of such changes. It is not productive to focus on “silver bullets”: they often lead to “fixing the student” approaches rather than identifying problems throughout the system, from mathematics preparation, to science culture, to faculty teaching, to financial aid, to articulation and transfer. Finally, it is clear that such barriers to change as the nature of the incentive structure in colleges and universities remain largely unaddressed, and studies have not been conducted to determine if addressing such barriers would facilitate large-scale and sustainable change in institutions or education systems.

CONCLUSION 6 Improving undergraduate science, technology, engineering, and mathematics education for all students will require a more systemic approach to change that includes use of evidence to support institutional decisions, learning communities and faculty development networks, and partnerships across the education system.

preparation for college, especially in mathematics. Partnerships with employers can lead to better articulation of the skills and knowledge that are relevant for their workforces, as well as opportunities for internships and work-related experiences that may improve students’ understanding of and commitment to STEM education.

At the institutional level, program faculty and administrators need to recognize that successful improvements usually include strong leadership, including support for faculty to undertake the changes needed; awareness of how to overcome the barriers to adaptation and implementation of curricula that have been demonstrated to be effective; faculty who implement instructional practices developed through discipline-based education research; and data to monitor students’ progress and to hold departments accountable for losses and recognize and reward them for student success.

Strong, multi-institutional articulation agreements, including common general education, common introductory courses, common course numbering, and online, easily available student access to equivalencies, can improve the percentage of contributory credits transferred, shorten the time to degree, and increase completion rates.

Department-level leadership is critical for systematic change. It can drive changes in rigid course sequencing requirements, transfer credit policies, degree requirements, differential tuition policies, and classroom practices. It can build connections between the reform efforts in their department and broader efforts in their institutions, as well as connect to multi-institutional reform efforts supported by foundations and disciplinary associations. The training of STEM department chairs supported by a number of programs and professional organizations has yielded promising results for departmental programs and their students.

RECOMMENDATION 8 Institutions should consider how expanded and improved co-curricular supports for science, technology, engineering, and mathematics (STEM) students can be informed by and integrated into work on more systemic reforms in undergraduate STEM education to more equitably serve their student populations.

ways to the degrees and careers they want. Students seem to benefit most from such supports when they are paired with evidence-based instructional strategies and when three or more co-curricular supports are bundled together ( Estrada, 2014 ). Such efforts will be more sustainable and effective if they are integrated into more systemic reform efforts.

RECOMMENDATION 9 Disciplinary departments, institutions, university associations, disciplinary societies, federal agencies, and accrediting bodies should work together to support systemic and long-lasting changes to undergraduate science, technology, engineering, and mathematics education.

STEM departments and entire academic units should support learning communities and networks that can help change faculty belief systems and practices and develop sustainable changes.
Colleges and universities should offer instructor training and mentoring to graduate students and postdoctoral scholars. Participating in such efforts as The Center for the Integration of Research, Teaching, and Learning (funded by the National Science Foundation; see Chapter 3 ) can educate graduate students about the value of treating their teaching as a form of scholarship and to value use of evidence-based approaches to teaching.
University associations and organizations should continue to facilitate undergraduate STEM educational reforms in their member institutions, particularly by examining reward structures and barriers to change and providing resources for data collection on student success, as well as by providing resources for interventions, support programs, and ways to share effective practices.
Disciplinary societies should s upport the development of continuing and intensive national and regional faculty development programs, awards, and recognition to encourage use of evidence-based instructional practices.
Federal agencies that support undergraduate STEM education should consider giving greater priority to supporting large-scale transformation strategies that are conceptualized to include and extend beyond instructional reform, and they should support both implementation research and research on barriers to reform that can support success for all students. They should increase the percentage of undergraduate STEM reform efforts and projects that focus on multiple levels—department, institution, discipline, government, and business and industry.
Following the policies adopted by some disciplinary accrediting bodies (e.g., the Accreditation Board for Engineering and Technol-

ogy), regional and professional accrediting bodies should consider incorporating evidence-based instructional practices and faculty professional development efforts into their criteria and guidelines.

The nature of the challenges of removing the barriers to 2-year and 4-year STEM degree completion can only be addressed by a system of solutions that includes the commitment to transformation. Looking from the ground up, those who teach need to be enabled to adopt and engage in effective classroom practices; co-curricular supports need to be made available for students who begin college with interest in STEM but who may lack some of the skills necessary to be immediately successful in their pursuit of study in STEM.

Money still matters: strategies need to be explored for addressing financial need in ways that connect students to STEM (such as through STEM-related work-study programs and internships and co-ops) rather than distracting them from it. Providing quality advice about courses, fields of study, careers, and navigating the many college pathways in STEM—as well as supporting learning communities—can help avoid many of the pitfalls that can delay or prevent degree completion.

Looking across institutions, the policy barriers to articulation and alignment need to be addressed. Although some removal of barriers can be promoted locally through, for example, the active commitment of individuals, (e.g., chemistry faculty in 4-year institutions working directly with chemistry faculty in feeder 2-year institutions and high schools), a negatively structured policy environment can impede such interventions. There is a clear need to explore all the policy impediments that make navigation of the pathways to STEM degrees in and across institutional boundaries especially difficult, and there are examples in various states and institutions that can be considered to smooth STEM pathways.

Looking from the top down, leadership is needed at every level to support change. Institutional leaders need to be committed to providing the supportive infrastructure that can make grassroots pedagogical and administrative changes possible (including active classrooms, technology, co-curricular supports, data systems, and teaching-learning centers). Loss of state support has negatively affected the operational model of many public institutions, forcing increased costs to be passed through to students, which disproportionately affects those who can least afford to attend, extending time to degree and may affect students’ choices of major (e.g., when there is differential tuition for programs such as engineering). National accountability structures, though well intentioned, currently reward the most selective institutions while penalizing those with fewer resources, but the latter are the ones who often enroll and succeed in enrolling STEM students from disadvantaged and less selective backgrounds. The admonishment to “first,

do no harm” should lead to a national discussion of how to recognize and honor the work of such institutions. At the same time, highly resourced institutions can be challenged to better support their STEM students through programs of active retention rather than “weeding out.”

Finally, leadership is required from all constituents, including state and federal government, funders, business and industry, and both higher education and STEM professionals, both within and across those communities. Rather than relying on failed or unsustainable structures that serve only a few or push out students who aspire to and are capable of completing a STEM degree, they should seek solutions that connect the pathways to STEM degrees.

Chang, M., Sharkness, J., Hurtado, S., and Newman, C. (2014). What matters in college for retaining aspiring scientists and engineers from underrepresented racial groups. Journal of Research in Science Teaching, 51 (5), 555–580.

Eagan, K., Hurtado, S., Figueroa, T., and Hughes, B. (2014). Examining STEM Pathways among Students Who Begin College at Four-Year Institutions. Commissioned paper prepared for the Committee on Barriers and Opportunities in Completing 2- and 4-Year STEM Degrees, National Academy of Sciences, Washington, DC. Available: http://sites.nationalacademies.org/cs/groups/dbassesite/documents/webpage/dbasse_088834.pdf [April 2015].

Espinosa, L.L. (2011). Pathways and pipelines: Women of color in undergraduate STEM majors and the college experiences that contribute to persistence. Harvard Educational Review, 81 (2), 209–241.

Estrada, M. (2014). Ingredients for Improving the Culture of STEM Degree Attainment with Co-curricular Supports for Underrepresented Minority Students . Paper prepared for the Committee on Barriers and Opportunities in Completing 2- and 4-Year STEM Degrees. http://sites.nationalacademies.org/cs/groups/dbassesite/documents/webpage/dbasse_088832.pdf [April 2015].

National Center for Education Statistics (2013). Digest of Education Statistics 2013. Washington, DC: U.S. Department of Education.

National Research Council. (2012). Discipline-Based Education Research: Understanding and Improving Learning in Undergraduate Science and Engineering. Committee on the Status, Contributions, and Future Directions of Discipline-Based Education Research. S. Singer, N.R. Nielsen, and H.A. Schweingruber (Eds.). Board on Science Education, Division of Behavioral and Social Sciences and Education. Washington DC: The National Academies Press.

National Science Board. (2014). Science and Engineering Indicators 2014. NSB #14-01. Arlington VA: National Science Foundation.

National Science Foundation and National Center for Science and Engineering Statistics. (2013). Women, Minorities, and Persons with Disabilities in Science and Engineering: 2013 . Arlington, VA: National Science Foundation.

President’s Council of Advisors on Science and Technology. (2012). Report to the President. Engage to Excel: Producing One Million Additional College Graduates with Degrees in Science, Technology, Engineering and Mathematics. Available: http://www.whitehouse.gov/sites/default/files/microsites/ostp/pcast-engage-to-excel-final_feb.pdf [April 2015].

Salzman, H., and Van Noy, M. (2014). Crossing the Boundaries: STEM Students in Four-Year and Community Colleges. Paper prepared for the Committee on Barriers and Opportunities in Completing 2- and 4-Year STEM Degrees. Available: http://sites.nationalacademies.org/cs/groups/dbassesite/documents/webpage/dbasse_089924.pdf [April 2015].

Seymour, E., and Hewitt, N. (1997). Talking about Leaving: Why Undergraduates Leave the Sciences. Boulder, CO: Westview Press.

Van Noy, M., and Zeidenberg, M. (2014). Hidden STEM Knowledge Producers: Community Colleges’ Multiple Contributions to STEM Education and Workforce Development. Paper prepared for the Committee on Barriers and Opportunities in Completing 2- and 4-Year STEM Degrees. Available: http://sites.nationalacademies.org/cs/groups/dbassesite/documents/webpage/dbasse_088831.pdf [April 2015].

This page intentionally left blank.

Nearly 40 percent of the students entering 2- and 4-year postsecondary institutions indicated their intention to major in science, technology, engineering, and mathematics (STEM) in 2012. But the barriers to students realizing their ambitions are reflected in the fact that about half of those with the intention to earn a STEM bachelor's degree and more than two-thirds intending to earn a STEM associate's degree fail to earn these degrees 4 to 6 years after their initial enrollment. Many of those who do obtain a degree take longer than the advertised length of the programs, thus raising the cost of their education. Are the STEM educational pathways any less efficient than for other fields of study? How might the losses be "stemmed" and greater efficiencies realized? These questions and others are at the heart of this study.

Barriers and Opportunities for 2-Year and 4-Year STEM Degrees reviews research on the roles that people, processes, and institutions play in 2-and 4-year STEM degree production. This study pays special attention to the factors that influence students' decisions to enter, stay in, or leave STEM majors—quality of instruction, grading policies, course sequences, undergraduate learning environments, student supports, co-curricular activities, students' general academic preparedness and competence in science, family background, and governmental and institutional policies that affect STEM educational pathways.

Because many students do not take the traditional 4-year path to a STEM undergraduate degree, Barriers and Opportunities describes several other common pathways and also reviews what happens to those who do not complete the journey to a degree. This book describes the major changes in student demographics; how students, view, value, and utilize programs of higher education; and how institutions can adapt to support successful student outcomes. In doing so, Barriers and Opportunities questions whether definitions and characteristics of what constitutes success in STEM should change. As this book explores these issues, it identifies where further research is needed to build a system that works for all students who aspire to STEM degrees. The conclusions of this report lay out the steps that faculty, STEM departments, colleges and universities, professional societies, and others can take to improve STEM education for all students interested in a STEM degree.

READ FREE ONLINE

Welcome to OpenBook!

You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

Do you want to take a quick tour of the OpenBook's features?

Show this book's table of contents , where you can jump to any chapter by name.

...or use these buttons to go back to the previous chapter or skip to the next one.

Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

Switch between the Original Pages , where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

To search the entire text of this book, type in your search term here and press Enter .

Share a link to this book page on your preferred social network or via email.

View our suggested citation for this chapter.

Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

Get Email Updates

Do you enjoy reading reports from the Academies online for free ? Sign up for email notifications and we'll let you know about new publications in your areas of interest when they're released.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

View all journals
Explore content
About the journal
Publish with us
Sign up for alerts
03 October 2018

Science and technology education

Brian Owens 0

Contributing editor

You can also search for this author in PubMed Google Scholar

A strong background in science, technology, engineering and mathematics (STEM) is vital for more than budding scientists. Future jobs in a wide variety of areas will require skills in STEM subjects. This Outlook explores how science education is being modernized to prepare students for life in the twenty-first century.

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 51 print issues and online access

185,98 € per year

only 3,65 € per issue

Rent or buy this article

Prices vary by article type

Prices may be subject to local taxes which are calculated during checkout

Nature 562 , S1 (2018)

doi: https://doi.org/10.1038/d41586-018-06829-9

This article is part of Nature Outlook: Science and technology education , an editorially independent supplement produced with the financial support of third parties. About this content .

Brazil’s plummeting graduate enrolments hint at declining interest in academic science careers

Career News 21 MAY 24

How religious scientists balance work and faith

Career Feature 20 MAY 24

How to set up your new lab space

Career Column 20 MAY 24

Reading between the lines: application essays predict university success

Research Highlight 17 MAY 24

How to stop students cramming for exams? Send them to sea

News & Views 30 APR 24

Can mathematicians help to solve social-justice problems?

Career Feature 22 MAY 24

Why role-playing games can spur climate action

World View 22 MAY 24

Internet use and teen mental health: it’s about more than just screen time

Correspondence 21 MAY 24

Recruitment of Global Talent at the Institute of Zoology, Chinese Academy of Sciences (IOZ, CAS)

The Institute of Zoology (IOZ), Chinese Academy of Sciences (CAS), is seeking global talents around the world.

Beijing, China

Institute of Zoology, Chinese Academy of Sciences (IOZ, CAS)

Full Professorship (W3) in “Organic Environmental Geochemistry (f/m/d)

The Institute of Earth Sciences within the Faculty of Chemistry and Earth Sciences at Heidelberg University invites applications for a FULL PROFE...

Heidelberg, Brandenburg (DE)

Universität Heidelberg

Postdoc: deep learning for super-resolution microscopy

The Ries lab is looking for a PostDoc with background in machine learning.

Vienna, Austria

University of Vienna

Postdoc: development of a novel MINFLUX microscope

The Ries lab is developing super-resolution microscopy methods for structural cell biology. In this project we will develop a fast, simple, and robust

Postdoctoral scholarship in Structural biology of neurodegeneration

A 2-year fellowship in multidisciplinary project combining molecular, structural and cell biology approaches to understand neurodegenerative disease

Umeå, Sweden

Umeå University

Quick links

Explore articles by subject
Guide to authors
Editorial policies

Position paper
Open access
Published: 14 January 2020

Expanding our views of science education to address sustainable development, empowerment, and social transformation

William C. Kyle Jr. ORCID: orcid.org/0000-0002-0342-6504 1

Disciplinary and Interdisciplinary Science Education Research volume 2 , Article number: 2 ( 2020 ) Cite this article

6684 Accesses

18 Citations

2 Altmetric

Metrics details

On 25 September 2015, the UN General Assembly adopted a resolution, which took effect 1 January 2016, to transform the world to better meet human needs and the requirements of economic transformation, while protecting the environment, ensuring peace and realizing human rights. Since 1987, there have been several global initiatives oriented toward sustainable development, yet science educators have often remained silent with respect to ensuring the goals of science education are linked intrinsically to the central tenets of sustainable development. Why such silence? Where are the voices of science educators?

In this position statement, I offer a rationale for expanding our views of science education to address sustainable development, empowerment, and social transformation. Science education ought to be a primary vehicle for addressing the current and emerging global challenges facing humanity. All too often, science educators merely focus upon fostering awareness and concern for global challenges. Such an orientation falls short of the education discourse that ought to be oriented toward addressing the goals, aspirations, desires, and needs of youth, who presently number 1.8 billion and represent the largest segment of the global population being underserved. The active engagement of youth in sustainable development efforts is imperative to achieve the goals of the 2030 Agenda. Youth are not mere beneficiaries of the 2030 Agenda; they have a critical role in the implementation of the Sustainable Development Goals (SDGs).

I offer a rationale for why science educators ought to become active agents in facilitating the engagement of youth in addressing global challenges facing humanity. Youth are demanding action; science educators ought to enable learners and communities to transform and reinvent the world they are inheriting.

Herein I challenge the decades honored curricular focus upon universalism and standardization. The imposition of standards and accountability - within the context of science teaching and learning - represents the antithesis of what an education in the sciences ought to be. I wish to focus on the 2030 Agenda for Sustainable Development (United Nations, 2015a ), in the context of current and emerging global challenges, and the need to transform education. With this focus in mind, I draw attention to the fact that the present generation of youth (between the ages of 10 and 24) numbers 1.8 billion, which is approximately 24% of the global population (The Commonwealth, 2016 ; UNFPA, 2014a ). Footnote 1 In addition, the largest segment of the global population being underserved is youth (UNFPA, 2014a ), with 90% of the global youth population living in less developed countries (LDCs).

The importance of population dynamics for sustainable development is at the center of the post-2015 development agenda (UNFPA, 2014b ). It is imperative that national development plans consider shifts in youth population dynamics. By 2050, action is needed on environmental issues, climate change, and biodiversity; and, significant investment is needed in education / literacy, health care and nutrition, addressing poverty alleviation and hunger, decreasing under- and un-employment, and enhancing food production / productivity. The World Youth Report on Youth and the 2030 Agenda for Sustainable Development examines the mutually supportive roles of the new agenda and current youth development efforts (United Nations, 2018 ). Youth are not mere beneficiaries of the 2030 Agenda, rather they have a critical role in the implementation of the Sustainable Development Goals (SDGs). The active engagement of youth in sustainable development efforts will be imperative to achieving the goals of the 2030 Agenda.

Science educators ought to be at the forefront of ensuring the education discourse is oriented toward the goals, aspirations, desires, and needs of all 1.8 billion youth versus adhering to a bureaucratic characterization of science in which universal standards, goals, objectives, and accountability measures are imposed upon learners, teachers, and administrators by policy makers and politicians. All too often, as a result of universalism and standardization, learners experience an education in science disassociated from the contextual realities of life.

The United Nations Development Programme (UNDP), the United Nations Population Fund (UNFPA), the United Nations Children’s Fund (UNICEF), and the United Nations Entity for Gender Equality and the Empowerment of Women (UNWomen) all commit to collaborate to deliver on the 2030 Agenda for Sustainable Development (UNFPA, 2017 ). Their underlying principle is ‘leaving no one behind’ and ‘reaching the furthest behind’. In compliance with their respective mandates, they will focus upon such key areas as: “(a) Eradicating poverty; (b) Addressing climate change; (c) Improving adolescent and maternal health; (d) Achieving gender equality and the empowerment of women and girls; (e) Ensuring greater availability and use of disaggregated data for sustainable development” (p. iii).

Gro Harlem Brundtland, in the Prologue to the first quadrennial Global Sustainable Development Report (Independent Group of Scientists appointed by the Secretary-General, 2019 ), notes the adoption of the Sustainable Development Goals “was a key moment in building a consensus for urgent, inclusive action” (Brundtland, 2019 , p. xv). She states:

Today, faced with the imperative of tackling climate change and responding to radical, fast-paced shifts in global technology, consumption and population patterns, there is growing consensus that sustainable development is the only way that we can avert environmental and social disaster. (Brundtland, 2019 , p. xv)

Further, she asserts the implementation of the SDGs “offers a pathway to a world where poverty, inequality and conflict will not blight the life chances of millions of people who are currently denied the opportunity to enjoy their fundamental rights and freedoms” (Brundtland, 2019 , p. xv).

The Independent Group of Scientists appointed by the Secretary-General ( 2019 ) assert:

The challenge of achieving sustainable development is to secure human well-being in ways that are not only safe , in terms of not threatening the Earth system with irreversible change, but also just . Ultimately then, sustainable development should be pursued in the spirit of finding pathways that enable a good life for all, leaving no one behind, while safeguarding the environment for future generations and ensuring planetary justice. (p. 2)

The authors consider “how science can best accelerate the achievement of the Sustainable Development Goals” and they argue “in favour of a sustainability science as a new way for science to contribute directly to sustainable development (p. 2). The Report presents a scientific view on integrated ways to accomplish the transformation of the planet. The authors identify six essential entry points, where the interconnections across the SDGs and targets are suitable for accelerating the necessary transformation. The six entry points are:

Strengthening human well-being and capabilities;

Shifting towards sustainable and just economies;

Building sustainable food systems and healthy nutrition patterns;

Achieving energy decarbonization with universal access to energy;

Promoting sustainable urban and peri-urban development; and

Securing the global environmental commons.

Considering these global initiatives, where are the voices of science educators? I assert we must expand our views of science education to address sustainable development, empowerment, and social transformation, thereby ensuring an informed, ecologically / environmentally literate, thoughtful, and empathetic citizenry. Empowerment – particularly in LDCs - is correlated with poverty alleviation, addressing inequality, and economic growth. An education in science must be contextualized and connected to the life world experiences of learners, while taking into consideration issues of place-based locality, as well as social, civic, and cultural values.

Prior to the 1960s, the philosophy of science was dominated by the writings of the logical empiricists, whom Habermas ( 1972 ) regarded as presenting a ‘scientistic misconception’ of science. Historically, science educators perceived of themselves as being more aligned with science than with education. Science courses - from elementary school through undergraduate studies - were structured and taught from the perspective of an uncritical acceptance of logical positivism and, to a large extent, as a mastery of abstract concepts and principles, rarely connected to real life experiences (Kyle Jr., 2006 ; Onwu, 2000 ; Onwu & Kyle Jr., 2011 ). All too often, science educators neglected to acknowledge the differences in style, subject matter, rhetoric, and results between the natural sciences and the human studies (see Habermas, 1972 ).

By adopting an image of science teaching and learning focused upon a historical bent, it was expected students would learn about the great discoveries of the past, rather than the practices of present-day scientists. Early science educators failed to recognize how post-empiricist philosophy - that is, the repudiation of the idea that science and knowledge can be grounded in theory neutral observations - revealed just how closely traditional interpretations of knowledge were connected to an understanding of power and of the relation between power and knowledge (see Marsonet, 2016 , 2018 ; Oldroyd, 1986 ).

To the present day, the link between science and real-world experiences is almost always tenuous in the minds of learners. The lack of curricular connections between science and learners’ day-to-day lived experiences obscures and diminishes the relevance of science in their lives. Scientific practices are political in ways central to their epistemic success (see Brown & Malone, 2004 ). Fischer ( 1998 ) notes sociological research has documented the extent to which science is as much a socio-cultural activity as a technical enterprise. He asserts full understanding of scientific findings is incomprehensible apart from the socio-cultural settings, which offer purpose and meaning. Thus, students - and citizens alike - have been denied access to this essential feature of science; they have been led to reconstruct the development of science as a steady accumulation of results with the supporting evidence. In general, students – and ultimately the general citizenry - have been deprived of the opportunity to experience the shifts in interests that have marked the history of science. Devoid of the social and political processes of science, the science curriculum epitomized a single, collective, consistent account of the progression of science. This is true with respect to environmental science as well, an interdisciplinary academic field that learners seldom experience in the context of their school-based science education.

The origin of environmental science, and subsequently environmental education, can be traced to the 1960s. The environmental movement and environmental education (EE) arose as a result of public awareness. In the US, and many developed nations, Rachel Carson’s ( 1962 ) Silent Spring inspired the public’s interest and engagement with environmental issues. These fields emerged based upon the need for interdisciplinary studies to analyze emerging environmental issues and concerns. During this same period, primarily in Western developed countries, environmental laws and protections were being passed, leading to a heightened awareness among the public to such issues. The laws were wide ranging, focusing upon such issues as: air and water quality; waste management and contaminant cleanup; water, mineral, and forest resource management; and biodiversity protection. Environmental laws are part of the fabric of most nations, as well as the basis of international law and treaties.

It should be noted, however, the current Trump Administration in the US was the first to not name a science advisor since 1941. It was nearly 2 years into the Administration before the US Senate confirmed a director of the White House Office of Science and Technology Policy on 2 Jan 2019. The Administration’s distrust of academic, peer-reviewed science and science advisors imperils domestic US policy, as well as the ability of the US to engage internationally on science-related matters of global importance, especially regarding issues related to sustainable development and the environment. With respect to environmental issues, the Trump Administration consistently places politics ahead of public health and survival of the planet (Kyle Jr, 2019 ). This is evidenced by the Administration pulling out of international accords, such as the Paris Climate Agreement, an agreement within the United Nations Framework Convention on Climate Change (UNFCCC) that focuses upon greenhouse-gas-emissions mitigation, adaptation, and finance; or through the reversal of more than 80 environmental rules and regulations (see Harvard Law School Environmental & Energy Law Program, 2019 ; Sabin Center for Climate Change Law, 2019 ).

Youth should not be disenfranchised in their educational opportunities due to poor political leadership. Rather, science educators ought to facilitate ways for youth to express their political agency. O’Brien, Selboe, and Hayward ( 2018 ) highlight diverse ways in which youth are challenging power relationships and political interests to promote climate-resilient futures.

Sustainable development, environmental education and global challenges

The focus herein is upon sustainable development – inclusive of, but not limited to, environmental education and environmental issues. For me, the broader concept of sustainable development addresses the global consideration of the perspective of education for the development of responsible societies (see Sauvé, 1996 ). Sauvé notes “responsible development, which must be defined contextually, becomes the guarantee of a type of sustainability deliberately chosen by the community” (p. 29). It is this place-based orientation that enables educators to focus upon sustainable development, empowerment and social transformation in ways meaningful and relevant to the current generation of youth. The 17 Global Goals for Sustainable Development (United Nations, 2015a ) offer a starting point for educators to begin to collaborate with youth, schools, and communities and initiate a research agenda that should extend well beyond 2030 in order to ensure progress is made toward addressing and achieving the SDGs. Learning opportunities must be transformed to ensure the active engagement of youth and communities in the context of experiential learning (see Eyler, 2009 ; Kolb, 1984 ; Shulman, 2002 ). Educators ought to be purposefully engaging with learners in direct experience and focused reflection to increase knowledge, develop skills, clarify values, and develop the capacity of learners to contribute to their communities. Experiential education methodologies include, but are not limited to, informal/free-choice learning, service learning, internships, field experiences, and project-based/problem-based learning.

An orientation focusing upon the SDGs would be more relevant for the many cultures that do not possess a term for the environment (Strathern, 1980 ). Many local languages do not have a word for the phenomenon environment; or, for such issues as climate change and biodiversity. DeLoughrey, Didur, and Carrigan ( 2015 ) note such cultures “ethical and philosophical codes are not simply assimilable to the binaries of western knowledge configurations” (p. 11). In addition, EE often promotes the dominant Western cultural values of an idealized nature (Low, Taplin, & Scheld, 2005 ). The dominant cultural narrative espouses a universal conception of how individuals ought to interact with the environment and fails to reflect social and cultural diversity. The processes of cultural hegemony – the dominance of one cultural group’s ideology and values over another’s – in the context of EE, encourages a paradigm of pro-environmental behaviors (PEBs). PEBs is defined by Kollmuss and Agyeman ( 2002 ) as “behavior that consciously seeks to minimize the negative impact of one’s actions on the natural and built world” (p. 240). However, identifying behaviors to change and evaluating the effects of interventions, the focus of much scholarship in environmental psychology (see Steg & Vlek, 2009 ), fails to address the totality of the societal transformations necessary to address sustainable development and fails to acknowledge the need for the place-based contextual reality of such transformations. Kurisu ( 2015 ) advances the field of PEBs, from the origins in developed countries focused upon limited target behaviors, to address issues in developing countries, offering practical academic tools for analyzing environmental behaviors.

The most comprehensive compilation of work in the field of environmental education is the International Handbook of Research on Environmental Education , edited by Stevenson, Brody, Dillon, and Wals ( 2013 ). The Handbook illuminates the important understandings developed by EE research, critically examines the ways in which the field has changed over the decades, articulates the current debates and controversies, explicates what is still missing from the EE research agenda, and foreshadows where the agenda might and could be headed in the future. Stevenson, Wals, Brody, and Dillon ( 2013 ) note EE:

Has received considerably more attention in recent years as contested notions of environment and sustainability have become common topics of conversation among the public, the subject of media interest, and the focus of much political debate and legislation. Systemic linkages between environment, health, climate, poverty, development, and education have become more widely accepted as the years have passed. (p. 8)

However, despite the history of EE, it is not embedded or woven into the typical school curricula. EE is often avoided in school-based settings due to negative emotions and the overwhelming sense of hopelessness students and educators often feel as a result of immersing into such issues. Such perceptions are evidenced in the political discourse and media-covered hot topics; skepticism of science is rampant in conjunction with negative feelings constructed from science education. Hope and empowerment seem to be drivers for connecting environmental issues with environmental responsibility (Wilks & Harris, 2016 ).

David Suzuki, an outspoken leader in ecological sustainability, summarized the difference between transformative environmental education within science education in an interview with Farley Mowat ( 1990 ) as follows:

My sense of injustice at what human beings were doing to the living world didn’t suddenly happen. It was a gradual understanding that science is fundamentally flawed because scientists focus on parts of nature and study these in isolation from the rest. (pp. 173–174)

In other words, aspects of science education focus on facts that compartmentalize the scientist from the big picture and the daily lived experiences of students. In the typical classroom, students are often immersed with facts, vocabulary and laboratory activities, without the opportunity to connect their learning to the potential impact of daily choices to the environment. Thus, EE may inadvertently continue to separate new findings and science knowledge from its application in everyday life, sometimes even ignoring or rejecting the critical need for assimilation of knowledge into behavior changes (Chinn & Brewer, 1993 ). This is particularly true with respect to the global challenges confronting the present and future generations of youth.

Presently, the global population uses resources at a rate 40% faster than the planet can regenerate in a calendar year. As recently as around 1980, humanity’s demand for ecological resources – the Ecological Footprint - was congruent with the planet’s biocapacity – the amount of ecological resources Earth is able to generate that year (Earth Overshoot Day, 2019 , About Earth Overshoot Day, section ¶1). In essence, in the course of about 40 years, we have seen a shift to a situation where we increasingly overspend the ecological resources at a faster and faster rate (the status in 1980 did not imply equitable consumption of resources, as some nations used a lot less and some used a lot more; this is certainly true today as well). In 2019, Earth Overshoot Day was reached on July 29th. If we continue with a business-as-usual lifestyle and do not begin to make significant changes, then around the time children born in 2019 graduate from high school Earth Overshoot Day will arrive well before July 1st. What this means is in the mid-2030s it would take 2+ years for Earth to regenerate what is used in one year. Reaching this level of ecological deficit spending may be physically impossible (Ewing et al., 2008 ; Wackernagel, 2008 ).

Thus, over the course of the next 15 years, between now and the mid-2030s, a different kind of community of practice in science classrooms is going to have to emerge. The world is facing almost insurmountable challenges. These challenges transcend national boundaries. The Millennium Project identified 15 Global Challenges Facing Humanity that “provide a framework to assess the global and local prospects for humanity” (Glenn, Gordon, & Florescu, 2009 , p. 10). Our ability to provide life’s essentials, for an ever-expanding human population and within the carrying capacity of supporting ecosystems, will require major advances in science and technology and a scientifically literate citizenry. Glenn, Gordon, and Florescu ( 2011 ) assert:

The world has the resources to address its challenges. What is not clear is whether the world will make good decisions fast enough and on the scale necessary to really address the global challenges. Hence, the world is in a race between implementing ever-increasing ways to improve the human condition and the seemingly ever-increasing complexity and scale of global problems. (p. 2)

Transforming our vision of education

Given the urgent need for humanity to generate and implement effective responses to current challenges, there is recognition among governments that fundamental reordering of global priorities is needed in order to implement the goals of sustainable development. The term sustainable development was initially conceptualized in a report entitled Our Common Future , referred to as the Brundtland Report, from the United Nations World Commission on Environment and Development (WCED, 1987 ). The document states:

Sustainable development is development that meets the needs of the present without compromising the ability of future generations to meet their own needs. It contains within it two key concepts: the concept of ‘needs’, in particular the essential needs of the world’s poor, to which overriding priority should be given; and the idea of limitations imposed by the state of technology and social organization on the environment’s ability to meet present and future needs. (The Concept of Sustainable Development section, ¶1)

We have witnessed 30+ years of UN declarations, agreements and reports, in which successes and gaps in achieving goals are reported and calls for future action are asserted. A brief overview of the history of these initiatives follows, accompanied by my own personal assertion that after over 30 years it is imperative to transform our vision of education.

The notion of fundamentally reordering global priorities was first enshrined in the Declaration of the Earth Summit in Rio de Janeiro (Brazil) in 1992. Ten years later, when the World Summit on Sustainable Development (WSSD) convened in Johannesburg (South Africa), it was hardly a point of dispute acknowledging not much progress had been made at the level of local communities for most global environmental issues (United Nations, 2002 ). With poverty deepening and becoming more widespread, and environmental degradation of essential ecosystems worsening, questions arose whether the subsequent actions and recommendations of the World Summit would be able to contribute in meaningful and realistic ways to achieving sustainable development.

In 2000, the Millennium Summit put forth an agreement to help developing countries attain what were later codified as the Millennium Development Goals (MDGs). The eight MDGs became the overarching framework for developing countries for 15 years. In the summative report (United Nations, 2015b ), the dual reality of 15 years of development efforts was characterized as follows: “unprecedented efforts have resulted in profound achievements” (p. 4) and “despite many successes, the poorest and most vulnerable people are being left behind” (p. 8).

On 25 September 2015, the UN General Assembly adopted a resolution, which took effect 1 January 2016, to “stimulate action over the next 15 years in areas of critical importance for humanity and the planet” (United Nations, 2015a , Preamble, ¶4). Entitled Transforming our world: The 2030 Agenda for Sustainable Development offers a plan to transform the world to better meet human needs and the requirements of economic transformation, while protecting the environment, ensuring peace and realizing human rights. The agenda includes 17 Sustainable Development Goals (SDGs) and 169 targets, which build upon the MDGs and strive to complete what was not achieved in the previous 15 years. The 17 SDGs are not legally binding; countries should assume ownership and establish a national framework for achieving the goals (see, http://www.un.org/sustainabledevelopment/sustainable-development-goals/ ). While the MDGs were intended for action in developing countries only, the 17 SDGs apply to all countries.

Since the inception of the concept of sustainable development 30+ years ago, many governments, agencies, NGOs, and citizens have been engaged in efforts to improve the lives of people and protect the planet. Yet, I raise the following questions: Why have there been so few efforts to transform schooling to ensure the goals of science education are linked to the central tenets of sustainable development? Why are the 15 Global Challenges Facing Humanity, the 8 Millennium Development Goals, the 17 Sustainable Development Goals, and the notion of Earth Overshoot Day not a part of the discourse of every citizen on the planet? If we hope citizens will engage in deliberation and action-taking around the most significant issues confronting humanity, then we should expand our views of the goals of education to address sustainable development, empowerment, and social transformation.

Sachs ( 2015 ) asserts “sustainable development is a way to understand the world as a complex interaction of economic, social, environmental, and political systems” (p. 11). I pose the following questions: Where in the standard curriculum do students construct such understandings? In what ways should education be more intrinsically linked to issues of sustainable development? I suggest a response to these questions ought to begin by ensuring education is more relevant to the needs of learners, communities, and society. Sachs further asserts sustainable development is “a way to define the objectives of a well-functioning society, one that delivers wellbeing for its citizens today and for future generations” (p. 11).

Traditionally, literacy is framed in terms of students’ learning disciplinary knowledge from the past. Such a perspective fails to capture dynamic aspects of the emergence / disappearance of new literacies. Among science educators, van Eijck ( 2009 ) offers the notion of scientific literacy as an emergent feature of collective praxis. This notion is “grounded in a conception of knowledge as a collective and distributed cognitive entity” (p. 255). He notes “grounding the concept of scientific literacy in a cultural-historical perspective allows the articulation of what being scientifically literate means” (p. 256). Stevenson and Dillon ( 2010 ) emphasize the importance of engaging learners as active agents. Meaningful learning about and informed action on environmental issues requires critical inquiry and reflection, as well as imagination to generate possibilities for creating more sustainable socio-ecological practices, and action to ameliorate current environmental concerns. They highlight the challenges and complexity of engaging youth and adults in meaningful learning.

Onwu and Kyle Jr. ( 2011 ) state if we wish to integrate the goals of sustainable development into science education, then there is a need to expand our view of the goals of science education beyond the content and process aims of science teaching and learning. What is needed is a shift of emphasis of science education from one bound by disciplines and subject matter headings - from learning science as a body of knowledge - to learning science linked to contextual realities of life and living. There is a need to recognize the challenge of sustainable development is not universal, but rather context dependent. And, as noted above, youth should no longer be viewed as mere beneficiaries of education; they ought to be viewed as having a critical role in the transformation of education and society, as well as in the implementation of the SDGs. The 1.8 billion youth / adolescents represent the future; a future that offers new opportunities for:

education, entrepreneurial, and skill-development initiatives;

community development and social transformation;

equitable and sustainable economic growth; as well as

opportunities to address the many global challenges facing humanity.

Greene ( 1995 ) states emphatically the main point of education in the context of a lived life is “to enable a human being to become increasingly mindful with regard to his or her lived situation - and its untapped possibilities” (p. 182). Science is a human activity. The values of science are therefore human values. As Bronowski ( 1956 /65) posits, the strengths of science and its safeguards rest predominantly on principles of freedom; notably, free inquiry, free thought, free speech and tolerance, all of which are the hallmarks of respect of human rights, freedom, and democracy. Learners ought to be afforded the opportunity to exercise creativity, debate, and dissent in the process of learning science. Through experiencing such an education in science, youth may acquire important insights into social change, systems change, citizenship, and democracy that many education systems are currently failing to provide (Hayward, 2012 ).

Science educators must begin to regard education as a primary means of investing in human resources. The youth of today must be able to address complex everyday issues yet unforeseen. This is not a modest goal. We must ensure all learners have access to an equitable education. There is a need to bridge the divide and facilitate dialogue between formal and informal / free-choice educators, as well as disciplinary and interdisciplinary science education researchers. Today’s youth recognize the implications of failing to transform toward a more sustainable future are profound (IPCC, 2014 ).

Ideally learners will be afforded the opportunity to experience a more progressive education (Dewey 1990 /1900) in the sciences, oriented toward real-world, experiential, context-based approaches to teaching and learning. In addition to challenging the notion of universalism and standardization, progressive science education will require a different form of assessment oriented toward performance observations and active assessment of learning. The goal of assessment ought to be oriented toward self- and social empowerment, action-taking, and transformation.

Throughout this article, I have raised the following questions:

Why have there been so few efforts to transform schooling to ensure the goals of science education are linked to the central tenets of sustainable development?

Why are the 15 Global Challenges Facing Humanity, the 8 Millennium Development Goals, the 17 Sustainable Development Goals, and the notion of Earth Overshoot Day not a part of the discourse of every citizen on the planet?

Where in the standard curriculum do students construct the understanding that sustainable development is a way to understand the world as a complex interaction of economic, social, environmental, and political systems?

In what ways should education be more intrinsically linked to issues of sustainable development?

On 6 May 2019, the Intergovernmental Science-Policy Platform on Biodiversity and Ecosystem Services (IPBES) released a report titled, Nature’s Dangerous Decline ‘Unprecedented’;

Species Extinction Rates ‘Accelerating’ . The report assesses changes over the past five decades and asserts nature is declining globally at rates unprecedented in human history - and the rate of species extinctions is accelerating, with grave impacts on people around the world now likely. Sir Robert Watson, IPBES Chair, stated:

The overwhelming evidence of the IPBES Global Assessment, from a wide range of different fields of knowledge, presents an ominous picture. The health of ecosystems on which we and all other species depend is deteriorating more rapidly than ever. We are eroding the very foundations of our economies, livelihoods, food security, health and quality of life worldwide.

Watson further notes the report tells us:

It is not too late to make a difference, but only if we start now at every level from local to global. Through ‘transformative change’, nature can still be conserved, restored and used sustainably – this is also key to meeting most other global goals. By transformative change, we mean a fundamental, system-wide reorganization across technological, economic and social factors, including paradigms, goals and values.

Fundamental transformation is imperative to achieve the sustainable future articulated in the 2030 Agenda. The youth of today – and tomorrow – have the potential to transform the planet. As educators, our role ought to be to enable learners and communities to change and reinvent the world they are inheriting. We ought to strive to enhance the ability of youth and communities to work collectively toward a better society.

Availability of data and materials

Not applicable. Data sharing is not applicable to this article as no datasets were generated or analyzed during the current study.

Organizations use varying age ranges for the global youth population. For example, The United Nations reports there are 1.2 billion youth aged 15 to 24 years, accounting for 16% of the global population (United Nations, 2018 ). While the age ranges considered to be youth vary by organization, the population and percentage of the global population are commensurate with the age ranges.

Abbreviations

Environmental Education

Intergovernmental Science-Policy Platform on Biodiversity and Ecosystem Services

Intergovernmental Panel on Climate Change

Less Developed Countries

Millennium Development Goals

Non-Government Organizations

Pro-Environmental Behaviors

Sustainable Development Goals

The United Nations Development Programme

United Nations Framework Convention on Climate Change

United Nations Population Fund

United Nations Children’s Fund

United Nations Entity for Gender Equality and the Empowerment of Women

World Commission on Environment and Development

World Summit on Sustainable Development

Bronowski, J. (1956/1965). Science and human values , (Rev. ed., ). New York: Harper & Row.

Google Scholar

Brown, R., & Malone, E. (2004). Reason, politics, and the politics of truth: How science is both autonomous and dependent. Sociological Theory , 22 (1), 106–122.

Article Google Scholar

Brundtland, G. H. (2019). Prologue. Independent Group of Scientists appointed by the secretary-general. Global Sustainable Development Report 2019: The Future is Now – Science for Achieving Sustainable Development (pp. xv – xvii). New York: United Nations.

Carson, R. (1962). Silent spring . New York: Houghton-Mifflin.

Chinn, C. A., & Brewer, W. F. (1993). The role of anomalous data in knowledge acquisition: A theoretical framework and implications for science instruction. Review of Educational Research , 63 (1), 1–49.

DeLoughrey, E., Didur, J., & Carrigan, A. (Eds.) (2015). Global ecologies and the environmental humanities: Post-colonial approaches . New York: Routledge.

Dewey, J. (1990/1900). The school and society. Chicago: University of Chicago Press. (Originally published in 1900)

Earth Overshoot Day (2019). About Earth overshoot day. Retrieved December 18, 2019, from. https://www.overshootday.org/about-earth-overshoot-day/

Ewing, B., Goldfinger, S., Wackernagel, M., Stechbart, M., Rizk, S. M., Reed, A., & Kitzes, J. (2008). The Ecological footprint atlas 2008 . Oakland: Global Footprint Network.

Eyler, J. (2009). The power of experiential education. Liberal Education , 95 (4), 22–26.

Fischer, F. (1998). Beyond empiricism: Policy inquiry in post positivist perspective. Policy Studies Journal , 26 (1), 129–146.

Glenn, J. C., Gordon, T. J., & Florescu, E. (2009). 2009 state of the future . Washington, DC: The Millennium Project.

Glenn, J. C., Gordon, T. J., & Florescu, E. (2011). 2011 state of the future . Washington, DC: The Millennium Project.

Greene, M. (1995). Releasing the imagination: Essays on education, the arts, and social change . San Francisco: Jossey-Bass.

Habermas, J. (1972). Knowledge and human interests (J. J. Shapiro, Trans.). London: Heinemann (original work published in 1968).

Harvard Law School Environmental & Energy Law Program. (2019, November). Regulatory rollback tracker . https://eelp.law.harvard.edu/regulatory-rollback-tracker/

Hayward, B. (2012). Children, citizenship and environment: Nurturing a democratic imagination in a changing world . London: Routledge.

Book Google Scholar

Independent Group of Scientists appointed by the Secretary-General (2019). Global sustainable development report 2019: The future is now – Science for achieving sustainable development . New York: United Nations.

Intergovernmental Panel on Climate Change (IPPC). (2014). Climate change 2014 synthesis report: Summary for policy makers . http://www.ipcc.ch/pdf/assessment-report/ar5/syr/AR5_SYR_FINAL_SPM.pdf

IPBES. (2019). Nature’s dangerous decline ‘unprecedented’; Species extinction rates ‘accelerating’ . IPBES Global Assessment Report on Biodiversity and Ecosystem Services. https://www.ipbes.net/news/Media-Release-Global-Assessment

Kolb, D. (1984). Experiential learning: Experience as the source of learning and development . Englewood Cliffs: Prentice-Hall.

Kollmuss, A., & Agyeman, J. (2002). Mind the gap: Why do people act environmentally and what are the barriers to pro-environmental behavior? Environmental Education Research , 8 (3), 239–260.

Kurisu, K. (2015). Pro-environmental behaviors . New York: Springer.

Kyle Jr., W. C. (2006). The road from Rio to Johannesburg: Where are the footpaths to/from science education? International Journal of Science and Mathematics Education , 4 , 1–18.

Kyle Jr., W. C. (2019, November). The marginalization of science in the Trump era [Paper presentation]. Critical Questions in Education (CQiE) conference. Chicago, United States.

Low, S., Taplin, D., & Scheld, S. (2005). Rethinking urban parks: Public space and cultural diversity . Austin: University of Texas Press.

Marsonet, M. (2016). Science and different images of the world. Academicus: International Scientific Journal , 14 , 14–27.

Marsonet, M. (2018). Post-empiricism and philosophy of science. Academicus: International Scientific Journal , 16 , 26–33.

Mowat, F. (1990). Rescue the earth! Conversations with green crusaders . Toronto: McCelland & Stewart.

O’Brien, K., Selboe, E., & Hayward, B. M. (2018). Exploring youth activism on climate change: Dutiful, disruptive, and dangerous dissent. Ecology and Society , 23 (3), 42. https://doi.org/10.5751/ES-10287-230342 .

Oldroyd, D. (1986). The arch of knowledge: An introductory study of the history, philosophy, and methodology of science . New York: Methuen.

Onwu, G. O. (2000). How should we educate science teachers for a changing society? South African Journal of Higher Education , 14 (3), 43–50.

Onwu, G. O., & Kyle Jr., W. C. (2011). Increasing the socio-cultural relevance of science education for sustainable development. African Journal of Research in Mathematics, Science and Technology Education , 15 (3), 5–26.

Sabin Center for Climate Change Law. (2019, November). Climate deregulation tracker . https://climate.law.columbia.edu/climate-deregulation-tracker

Sachs, J. D. (2015). The age of sustainable development . New York: Columbia University Press.

Sauvé, L. (1996). Environmental education and sustainable development: A further appraisal. Canadian Journal of Environmental Education , 1 , 7–34.

Shulman, L. (2002). Making differences: A table of learning . Palo Alto: The Carnegie Foundation for the Advancement of Teaching.

Steg, L., & Vlek, C. (2009). Encouraging pro-environmental behaviour: An integrative review and research agenda. Journal of Environmental Psychology , 29 , 309–317.

Stevenson, R. B., & Dillon, J. (Eds.) (2010). Engaging environmental education: Learning, culture and agency . Rotterdam: Sense Publishers.

Stevenson, R. B., Brody, M., Dillon, J., & Wals, A. E. J. (Eds.) (2013). International handbook of research on environmental education. New York: Routledge.

Stevenson, R. B., Wals, A. E. J., Brody, M., & Dillon, J. (2013). Introduction: An orientation to environmental education and the handbook. In R. B. Stevenson, M. Brody, J. Dillon, & A. E. J. Wals (Eds.), International handbook of research on Environmental education, (pp. 1–6). New York: Routledge.

Chapter Google Scholar

Strathern, M. (1980). No nature, no culture: The Hagan case. In C. MacCormack, & M. Strathern (Eds.), Nature, culture, and gender , (pp. 174–222). Cambridge: Cambridge University Press.

The Commonwealth (2016). Global youth development index and report 2016 . London: Commonwealth Secretariet.

UNFPA (2014a). State of world population – The power of 1.8 billion: Adolescents, youth and the transformation of the future . New York: United Nations Population Fund.

UNFPA (2014b). Population dynamics in the post-2015 development agenda: Report of the global thematic consultation on population dynamics . New York: United Nations Population Fund.

UNFPA (2017). UNFPA strategic plan, 2018–2021 . New York: United Nations Population Fund.

United Nations. (2002). Commission on sustainable development. The Johannesburg summit test: What will change? Available at www.johannesburgsummit.org

United Nations. (2015a). Transforming our world: The 2030 Agenda for sustainable development . Resolution A/Res/70/1. Retrieved from: http://www.un.org/ga/search/view_doc.asp?symbol=A/RES/70/1&Lang=E

United Nations. (2015b). The millennium development goals report 2015 . New York: United Nations. Retrieved from http://www.un.org/millenniumgoals/2015_MDG_Report/pdf/MDG%202015%20rev%20(July%201).pdf

United Nations (2018). Youth and the 2030 agenda for sustainable development . New York: United Nations.

van Eijck, M. (2009). Scientific literacy: Past research, present conceptions, and future developments. In W.-M. Roth, & K. Tobin (Eds.), Handbook of research in North America , (pp. 245–258). Rotterdam: Sense Publishers.

Wackernagel, M. (2008). Forward. In B. Ewing, S. Goldfinger, M. Wackernagel, M. Stechbart, S. M. Rizk, A. Reed, & J. Kitzes (Eds.), The ecological footprint atlas 2008 , (pp. 1–2). Oakland: Global Footprint Network.

WCED (World Commission on Environment and Development). (1987, December 11). Our common future. Oxford: Oxford University Press. Retrieved from: http://habitat.igc.org/open-gates/wced-ocf.htm , The Concept of Sustainable Development section, 1.

Wilks, L., & Harris, N. (2016). Examining the conflict and interconnectedness of young people’s ideas about environmental issues, responsibility and action. Environmental Education Research. , 22 (5), 683–696.

Download references

There were no funding sources for this article.

Author information

Authors and affiliations.

E. Desmond Lee Family Professor of Science Education, College of Education, University Missouri - St. Louis, St. Louis, MO, 63121-4499, USA

William C. Kyle Jr.

You can also search for this author in PubMed Google Scholar

Contributions

The author is the sole author. The author read and approved the final manuscript.

Corresponding author

Correspondence to William C. Kyle Jr. .

Ethics declarations

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Cite this article.

Kyle, W.C. Expanding our views of science education to address sustainable development, empowerment, and social transformation. Discip Interdscip Sci Educ Res 2 , 2 (2020). https://doi.org/10.1186/s43031-019-0018-5

Download citation

Received : 25 May 2019

Accepted : 21 November 2019

Published : 14 January 2020

DOI : https://doi.org/10.1186/s43031-019-0018-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Sustainable development
Global challenges
Environmental education

The Open University
Guest user / Sign out
Study with The Open University

My OpenLearn Profile

Personalise your OpenLearn profile, save your favourite content and get recognition for your learning

About this free course

Become an ou student, download this course, share this free course.

Start this free course now. Just create an account and sign in. Enrol and complete the course for a free statement of participation or digital badge if available.

In this free course, Teaching secondary science , you have looked at the images and conceptions of science, and how these impact on school science education. Teachers need to have these views in mind and employ strategies to make science accessible and appealing to all. This involves teaching students about the nature of science through purposefully designed activities, as well as taking opportunities presented through practical work. Above all, teachers must consider what students will learn through practical activity and be aware of the hidden messages that might be inadvertently communicated. By engaging with the stories and lives of past scientists, as well as the controversial science issues affecting society, science can be brought to life as a human endeavour that has relevance and meaning for them and their future.

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Kate Billingsley

introduction

Technology is the application of scientific knowledge for practical purposes, especially in industry. Technology is a tool that can be used to solve real-world problems. The field of Science, Technology, and Society (STS) “seeks to promote cross-disciplinary integration, civic engagement, and critical thinking” of concepts in the worlds of science and technology ( Harvard University, n.d.). As an aspect of everyday life, technology is continuously evolving to ensure that humanity can be productive, efficient, and follow the path of globalization . STS is a concept that encompasses countless fields of study. “Scientists, engineers, and medical professionals swim (as they must) in the details of their technical work: experiments, inventions, treatments and cures. “promotes cross-disciplinary integration, civic engagement, and critical thinking” It’s an intense and necessary focus” ( Stanford University , n.d.). On the opposite side of the spectrum is STS, which “draws attention to the water: the social, political, legal, economic, and cultural environment that shapes research and invention, supports or inhibits it — and is in turn shaped by evolving science and technology” ( Stanford University , n.d.). Technology is a crucial part of life that is constantly developing to fit the changing needs of society and aiding humanity in simplifying the demands of everyday life.

According to Oberdan (2010), science and technology share identical goals. “At first glance, they seem to provide a deep and thorough going division between the two but, as the discussion progresses, it will become clear that there are, indeed, areas of overlap, too” (Oberdan, 25). Philosophers believe that for a claim to be considered knowledge, it must first be justified, like a hypothesis, and true. Italian astronomer, physicist, and engineer, Galileo Galilei , was incredibly familiar with the obstacles involved with proving something to be a fact or a theory within the scientific world. Galileo was condemned by the Roman Catholic church for his beliefs that contradicted existing church doctrine (Coyne, 2013). Galileo’s discoveries, although denounced by the church were incredibly innovative and progressive for their time, and are still seen as the basis for modern astronomy today. Nearly 300 years later, Galileo was eventually forgiven by the church, and to this day he is seen as one of the most well known and influential astronomers of all time. Many new innovations and ideas often receive push back before becoming revolutionary and universal practices.

INNOVATION IN TECHNOLOGY

Flash forward to modern time where we can see that innovation is happening even more around us. Look no further than what could be considered the culmination of modern technological innovation: the mobile phone. Cell phone technology has developed exponentially since the invention of the first mobile phone in 1973 ( Seward , 2013). Although there was a period for roughly 20 years in which cell phones were seen as unnecessary and somewhat impractical, as society’s needs changed and developed in the late 1990s, there was a large spike in consumer purchases of mobile phones. Now, cell phones are an entity that can be seen virtually anywhere, which is in large part due to their practicality. Cell phones, specifically smartphones such as Apple’s iPhone , have changed the way society uses technology. Smartphone technology has eliminated the need for people to have a separate cell phone, MP3 player, GPS, mobile video gaming systems, and more. Consumers may fail to realize how many aspects of modern technological advancement are involved in the use of their mobile phones. Cell phones use wifi to browse the internet, use google, access social media, and more. Although these technologies are beneficial, they also allow consumers locations to be traced and phone conversations to be recorded. Modern cell phone technologies collect data on consumers, and many people are unsure how this information is being used. Additionally, mobile phones come equipped with virus protection which brings the field of cybersecurity into smartphone usage. The technological advances that have been made in the market for mobile phones have been targeted towards the changing needs of consumers and society. As proven by the rise in cell phones, with advancements in the field of STS comes new unforeseen obstacles and ethical dilemmas.

Technology is changing the way we live in this world. Innovations in the scientific world are becoming increasingly more advanced to help conserve earth’s resources and aid in the reduction of pollutants . Transportation is a field that has changed greatly in recent years due to modernization in science and technology, as well as an increased awareness of environmental concerns. The transportation industry continues to be a large producer of pollution

due to emissions from cars, trains, and other modes of transportation. As a result, cars have changed a great deal in recent years. A frontrunner in creating environmentally friendly luxury cars is Tesla, lead by CEO Elon Musk. Although nearly every brand of car has an electric option that either runs completely gas free, or uses significantly less fuel than standard cars, Tesla has taken this one step further and created a zero emissions vehicle. However, some believe that Tesla has taken their innovations in the transportation market a bit too far, specifically with their release of driverless cars.

“The recent reset of expectations on driverless cars is a leading indicator for other types of AI-enabled systems as well,” says David A. Mindell, professor of aeronautics and astronautics, and the Dibner Professor of the History of Engineering and Manufacturing at MIT. “These technologies hold great promise, but it takes time to understand the optimal combination of people and machines. And the timing of adoption is crucial for understanding the impact on workers” ( Dizikes , 2019).

As the earth becomes more and more polluted, consumers are seeking to find new ways to cut down on their negative impacts on the earth. Eco-friendly cars are a simple yet effective way in which consumers can cut back on their pollution within their everyday lives.

THE INTERSECTION OF SCIENCE AND TECHNOLOGY

The way in which energy is generated has changed greatly to benefit consumers and the environment. Energy production has followed a rather linear path over time, and is a prime example of how new innovations stem from old technologies. In the early 1800s, the steam engine acted as the main form of creating energy. It wasn’t until the mid-late 1800s that the combustion engine was invented. This invention was beneficial because it was more efficient than its predecessor, and became a form of energy that was streamlined to be used in countless applications. As time has progressed, this linear path of innovation has continued. As new energy creating technologies have emerged, machinery that was once seen as efficient and effective have been phased out. Today, largely due to the increased demand for clean energy sources, the linear path has split and consumers are faced with numerous options for clean, environmentally friendly energy sources. Over time, scientists and engineers have come to realize that these forms of energy pollute and damage the earth. Solar power, a modern form of clean energy, was once seen as an expensive and impractical way of turning the sun’s energy into usable energy. Now, it is common to see newly built homes with solar panels already built in. Since technology develops to fit the needs of society, scientists have worked to improve solar panels to make them cheaper and easier to access. A total of 173,000 terawatts (trillions of watts) of solar energy strikes the Earth continuously, which is more than 10,000 times of the world’s total energy use ( Chandler , 2011). This information may seem staggering, but is crucial in understanding the importance, as well as the large influence that modern forms of energy can have on society.

Technology has become a crucial part of our society. Without technological advancements, so much of our everyday lives would be drastically different. As technology develops, it strives to fulfill the changing needs of society. Technology progresses as society evolves. That being said, progress comes at a price. This price is different for each person, and varies based on how much people value technological and scientific advancements in their own lives. Thomas Parke Hughes’s Networks of Power “compared how electric power systems developed in America, England, and Germany, showing that they required not only electrical but social ‘engineering’ to create the necessary legal frameworks, financing, standards, political support, and organizational designs” ( Stanford University ). In other words, the scientific invention and production of a new technology does not ensure its success. Technology’s success is highly dependent on society’s acceptance or rejection of a product, as well as whether or not any path dependence is involved. Changing technologies benefit consumers in countless aspects of their lives including in the workforce, in communications, in the use of natural resources, and so much more. These innovations across numerous different markets aid society by making it easier to complete certain tasks. Innovation will never end; rather, it will continue to develop at increasing rates as science and technological fields becomes more and more cutting edge.

Chapter Questions

True or False: Improvements in science and technology always benefit society
Multiple Choice : Technology is: A. The application of scientific knowledge for practical purposes, especially in industry B. Tools and machines that may be used to solve real-world problems C. Something that does not change D. Both A and B
Short Answer: Discuss ways in which technological progression over time is related and how this relationship has led to the creation of new innovation.

Chandler, D. (2011). Shining brightly: Vast amounts of solar energy radiate to the Earth constantly, but tapping that energy cost-effectively remains a challenge. MIT News. http://news.mit.edu/2011/energy-scale-part3-1026

Coyne, SJ, G. V. (2013). Science meets biblical exegesis in the Galileo affair. Zygon® , 48 (1), 221-229. https://doi-org.libproxy.clemson.edu/10.1111/j.1467-9744.2012.01324.x

Dizikes, P., & MIT News Office. (2019). MIT report examines how to make technology work for society. http://news.mit.edu/2019/work-future-report-technology-jobs-society-0904

Florez, D., García-Duque, C. E., & Osorio, J. C. (2019). Is technology (still) applied science? Technology in Society. Technology in Society, 59. doi: 10.1016/j.techsoc.2019.101193

Groce, J. E., Farrelly, M. A., Jorgensen, B. S., & Cook, C. N. (2019). Using social‐network research to improve outcomes in natural resource management. Conservation biology , 33 (1), 53-65. https://conbio.onlinelibrary.wiley.com/doi/epdf/10.1111/cobi.13127

Harvard University. (n.d.) What is STS? . http://sts.hks.harvard.edu/about/whatissts.html .

Union of Concerned Scientists. (2018). How Do Battery Electric Cars Work? https://www.ucsusa.org/clean-vehicles/electric-vehicles/how-do-battery-electric-cars-work .

Oberdan, T. (2010). Science, Technology, and the Texture of Our Lives. Tavenner Publishing Company.

Seward, Z. M. (2013). The First Mobile Phone Call Was Made 40 Years Ago Today . The Atlantic. https://www.theatlantic.com/technology/archive/2013/04/the-first- mobile-phone-call-was-made-40-years-ago-today/274611/ .

Stanford University. (n.d.). What is the Study of STS? . https://sts.stanford.edu/about/what-study-sts .

Wei, R., & Lo, V.-H. (2006). Staying connected while on the move: Cell phone use and social connectedness. New Media & Society, 8 (1), 53–72. https://doi.org/10.1177/1461444806059870

Winston, B. (2006). Media Technology and Society: A History From the Telegraph to the Internet . London: Routledge.

Images & Videos

“Tesla Model 3 Monaco” is licensed under CC BY-NC-SA 4.0

Building bridges between science and society for a better future. | Nadine Bongaerts | TEDxSaclay

“Tesla Model 3 Monaco” is licensed under CC BY-NC-SA 4.0

To the extent possible under law, Kate Billingsley has waived all copyright and related or neighboring rights to Science, Technology, & Society: A Student-Led Exploration , except where otherwise noted.

Share This Book

Tutorial Review
Open access
Published: 24 January 2018

Teaching the science of learning

Yana Weinstein ORCID: orcid.org/0000-0002-5144-968X 1 ,
Christopher R. Madan 2 , 3 &
Megan A. Sumeracki 4

Cognitive Research: Principles and Implications volume 3 , Article number: 2 ( 2018 ) Cite this article

246k Accesses

90 Citations

765 Altmetric

Metrics details

The science of learning has made a considerable contribution to our understanding of effective teaching and learning strategies. However, few instructors outside of the field are privy to this research. In this tutorial review, we focus on six specific cognitive strategies that have received robust support from decades of research: spaced practice, interleaving, retrieval practice, elaboration, concrete examples, and dual coding. We describe the basic research behind each strategy and relevant applied research, present examples of existing and suggested implementation, and make recommendations for further research that would broaden the reach of these strategies.

Significance

Education does not currently adhere to the medical model of evidence-based practice (Roediger, 2013 ). However, over the past few decades, our field has made significant advances in applying cognitive processes to education. From this work, specific recommendations can be made for students to maximize their learning efficiency (Dunlosky, Rawson, Marsh, Nathan, & Willingham, 2013 ; Roediger, Finn, & Weinstein, 2012 ). In particular, a review published 10 years ago identified a limited number of study techniques that have received solid evidence from multiple replications testing their effectiveness in and out of the classroom (Pashler et al., 2007 ). A recent textbook analysis (Pomerance, Greenberg, & Walsh, 2016 ) took the six key learning strategies from this report by Pashler and colleagues, and found that very few teacher-training textbooks cover any of these six principles – and none cover them all, suggesting that these strategies are not systematically making their way into the classroom. This is the case in spite of multiple recent academic (e.g., Dunlosky et al., 2013 ) and general audience (e.g., Dunlosky, 2013 ) publications about these strategies. In this tutorial review, we present the basic science behind each of these six key principles, along with more recent research on their effectiveness in live classrooms, and suggest ideas for pedagogical implementation. The target audience of this review is (a) educators who might be interested in integrating the strategies into their teaching practice, (b) science of learning researchers who are looking for open questions to help determine future research priorities, and (c) researchers in other subfields who are interested in the ways that principles from cognitive psychology have been applied to education.

While the typical teacher may not be exposed to this research during teacher training, a small cohort of teachers intensely interested in cognitive psychology has recently emerged. These teachers are mainly based in the UK, and, anecdotally (e.g., Dennis (2016), personal communication), appear to have taken an interest in the science of learning after reading Make it Stick (Brown, Roediger, & McDaniel, 2014 ; see Clark ( 2016 ) for an enthusiastic review of this book on a teacher’s blog, and “Learning Scientists” ( 2016c ) for a collection). In addition, a grassroots teacher movement has led to the creation of “researchED” – a series of conferences on evidence-based education (researchED, 2013 ). The teachers who form part of this network frequently discuss cognitive psychology techniques and their applications to education on social media (mainly Twitter; e.g., Fordham, 2016 ; Penfound, 2016 ) and on their blogs, such as Evidence Into Practice ( https://evidenceintopractice.wordpress.com/ ), My Learning Journey ( http://reflectionsofmyteaching.blogspot.com/ ), and The Effortful Educator ( https://theeffortfuleducator.com/ ). In general, the teachers who write about these issues pay careful attention to the relevant literature, often citing some of the work described in this review.

These informal writings, while allowing teachers to explore their approach to teaching practice (Luehmann, 2008 ), give us a unique window into the application of the science of learning to the classroom. By examining these blogs, we can not only observe how basic cognitive research is being applied in the classroom by teachers who are reading it, but also how it is being misapplied, and what questions teachers may be posing that have gone unaddressed in the scientific literature. Throughout this review, we illustrate each strategy with examples of how it can be implemented (see Table 1 and Figs. 1 , 2 , 3 , 4 , 5 , 6 and 7 ), as well as with relevant teacher blog posts that reflect on its application, and draw upon this work to pin-point fruitful avenues for further basic and applied research.

Spaced practice schedule for one week. This schedule is designed to represent a typical timetable of a high-school student. The schedule includes four one-hour study sessions, one longer study session on the weekend, and one rest day. Notice that each subject is studied one day after it is covered in school, to create spacing between classes and study sessions. Copyright note: this image was produced by the authors

a Blocked practice and interleaved practice with fraction problems. In the blocked version, students answer four multiplication problems consecutively. In the interleaved version, students answer a multiplication problem followed by a division problem and then an addition problem, before returning to multiplication. For an experiment with a similar setup, see Patel et al. ( 2016 ). Copyright note: this image was produced by the authors. b Illustration of interleaving and spacing. Each color represents a different homework topic. Interleaving involves alternating between topics, rather than blocking. Spacing involves distributing practice over time, rather than massing. Interleaving inherently involves spacing as other tasks naturally “fill” the spaces between interleaved sessions. Copyright note: this image was produced by the authors, adapted from Rohrer ( 2012 )

Concept map illustrating the process and resulting benefits of retrieval practice. Retrieval practice involves the process of withdrawing learned information from long-term memory into working memory, which requires effort. This produces direct benefits via the consolidation of learned information, making it easier to remember later and causing improvements in memory, transfer, and inferences. Retrieval practice also produces indirect benefits of feedback to students and teachers, which in turn can lead to more effective study and teaching practices, with a focus on information that was not accurately retrieved. Copyright note: this figure originally appeared in a blog post by the first and third authors ( http://www.learningscientists.org/blog/2016/4/1-1 )

Illustration of “how” and “why” questions (i.e., elaborative interrogation questions) students might ask while studying the physics of flight. To help figure out how physics explains flight, students might ask themselves the following questions: “How does a plane take off?”; “Why does a plane need an engine?”; “How does the upward force (lift) work?”; “Why do the wings have a curved upper surface and a flat lower surface?”; and “Why is there a downwash behind the wings?”. Copyright note: the image of the plane was downloaded from Pixabay.com and is free to use, modify, and share

Three examples of physics problems that would be categorized differently by novices and experts. The problems in ( a ) and ( c ) look similar on the surface, so novices would group them together into one category. Experts, however, will recognize that the problems in ( b ) and ( c ) both relate to the principle of energy conservation, and so will group those two problems into one category instead. Copyright note: the figure was produced by the authors, based on figures in Chi et al. ( 1981 )

Example of how to enhance learning through use of a visual example. Students might view this visual representation of neural communications with the words provided, or they could draw a similar visual representation themselves. Copyright note: this figure was produced by the authors

Example of word properties associated with visual, verbal, and motor coding for the word “SPOON”. A word can evoke multiple types of representation (“codes” in dual coding theory). Viewing a word will automatically evoke verbal representations related to its component letters and phonemes. Words representing objects (i.e., concrete nouns) will also evoke visual representations, including information about similar objects, component parts of the object, and information about where the object is typically found. In some cases, additional codes can also be evoked, such as motor-related properties of the represented object, where contextual information related to the object’s functional intention and manipulation action may also be processed automatically when reading the word. Copyright note: this figure was produced by the authors and is based on Aylwin ( 1990 ; Fig. 2 ) and Madan and Singhal ( 2012a , Fig. 3 )

Spaced practice

The benefits of spaced (or distributed) practice to learning are arguably one of the strongest contributions that cognitive psychology has made to education (Kang, 2016 ). The effect is simple: the same amount of repeated studying of the same information spaced out over time will lead to greater retention of that information in the long run, compared with repeated studying of the same information for the same amount of time in one study session. The benefits of distributed practice were first empirically demonstrated in the 19 th century. As part of his extensive investigation into his own memory, Ebbinghaus ( 1885/1913 ) found that when he spaced out repetitions across 3 days, he could almost halve the number of repetitions necessary to relearn a series of 12 syllables in one day (Chapter 8). He thus concluded that “a suitable distribution of [repetitions] over a space of time is decidedly more advantageous than the massing of them at a single time” (Section 34). For those who want to read more about Ebbinghaus’s contribution to memory research, Roediger ( 1985 ) provides an excellent summary.

Since then, hundreds of studies have examined spacing effects both in the laboratory and in the classroom (Kang, 2016 ). Spaced practice appears to be particularly useful at large retention intervals: in the meta-analysis by Cepeda, Pashler, Vul, Wixted, and Rohrer ( 2006 ), all studies with a retention interval longer than a month showed a clear benefit of distributed practice. The “new theory of disuse” (Bjork & Bjork, 1992 ) provides a helpful mechanistic explanation for the benefits of spacing to learning. This theory posits that memories have both retrieval strength and storage strength. Whereas retrieval strength is thought to measure the ease with which a memory can be recalled at a given moment, storage strength (which cannot be measured directly) represents the extent to which a memory is truly embedded in the mind. When studying is taking place, both retrieval strength and storage strength receive a boost. However, the extent to which storage strength is boosted depends upon retrieval strength, and the relationship is negative: the greater the current retrieval strength, the smaller the gains in storage strength. Thus, the information learned through “cramming” will be rapidly forgotten due to high retrieval strength and low storage strength (Bjork & Bjork, 2011 ), whereas spacing out learning increases storage strength by allowing retrieval strength to wane before restudy.

Teachers can introduce spacing to their students in two broad ways. One involves creating opportunities to revisit information throughout the semester, or even in future semesters. This does involve some up-front planning, and can be difficult to achieve, given time constraints and the need to cover a set curriculum. However, spacing can be achieved with no great costs if teachers set aside a few minutes per class to review information from previous lessons. The second method involves putting the onus to space on the students themselves. Of course, this would work best with older students – high school and above. Because spacing requires advance planning, it is crucial that the teacher helps students plan their studying. For example, teachers could suggest that students schedule study sessions on days that alternate with the days on which a particular class meets (e.g., schedule review sessions for Tuesday and Thursday when the class meets Monday and Wednesday; see Fig. 1 for a more complete weekly spaced practice schedule). It important to note that the spacing effect refers to information that is repeated multiple times, rather than the idea of studying different material in one long session versus spaced out in small study sessions over time. However, for teachers and particularly for students planning a study schedule, the subtle difference between the two situations (spacing out restudy opportunities, versus spacing out studying of different information over time) may be lost. Future research should address the effects of spacing out studying of different information over time, whether the same considerations apply in this situation as compared to spacing out restudy opportunities, and how important it is for teachers and students to understand the difference between these two types of spaced practice.

It is important to note that students may feel less confident when they space their learning (Bjork, 1999 ) than when they cram. This is because spaced learning is harder – but it is this “desirable difficulty” that helps learning in the long term (Bjork, 1994 ). Students tend to cram for exams rather than space out their learning. One explanation for this is that cramming does “work”, if the goal is only to pass an exam. In order to change students’ minds about how they schedule their studying, it might be important to emphasize the value of retaining information beyond a final exam in one course.

Ideas for how to apply spaced practice in teaching have appeared in numerous teacher blogs (e.g., Fawcett, 2013 ; Kraft, 2015 ; Picciotto, 2009 ). In England in particular, as of 2013, high-school students need to be able to remember content from up to 3 years back on cumulative exams (General Certificate of Secondary Education (GCSE) and A-level exams; see CIFE, 2012 ). A-levels in particular determine what subject students study in university and which programs they are accepted into, and thus shape the path of their academic career. A common approach for dealing with these exams has been to include a “revision” (i.e., studying or cramming) period of a few weeks leading up to the high-stakes cumulative exams. Now, teachers who follow cognitive psychology are advocating a shift of priorities to spacing learning over time across the 3 years, rather than teaching a topic once and then intensely reviewing it weeks before the exam (Cox, 2016a ; Wood, 2017 ). For example, some teachers have suggested using homework assignments as an opportunity for spaced practice by giving students homework on previous topics (Rose, 2014 ). However, questions remain, such as whether spaced practice can ever be effective enough to completely alleviate the need or utility of a cramming period (Cox, 2016b ), and how one can possibly figure out the optimal lag for spacing (Benney, 2016 ; Firth, 2016 ).

There has been considerable research on the question of optimal lag, and much of it is quite complex; two sessions neither too close together (i.e., cramming) nor too far apart are ideal for retention. In a large-scale study, Cepeda, Vul, Rohrer, Wixted, and Pashler ( 2008 ) examined the effects of the gap between study sessions and the interval between study and test across long periods, and found that the optimal gap between study sessions was contingent on the retention interval. Thus, it is not clear how teachers can apply the complex findings on lag to their own classrooms.

A useful avenue of research would be to simplify the research paradigms that are used to study optimal lag, with the goal of creating a flexible, spaced-practice framework that teachers could apply and tailor to their own teaching needs. For example, an Excel macro spreadsheet was recently produced to help teachers plan for lagged lessons (Weinstein-Jones & Weinstein, 2017 ; see Weinstein & Weinstein-Jones ( 2017 ) for a description of the algorithm used in the spreadsheet), and has been used by teachers to plan their lessons (Penfound, 2017 ). However, one teacher who found this tool helpful also wondered whether the more sophisticated plan was any better than his own method of manually selecting poorly understood material from previous classes for later review (Lovell, 2017 ). This direction is being actively explored within personalized online learning environments (Kornell & Finn, 2016 ; Lindsey, Shroyer, Pashler, & Mozer, 2014 ), but teachers in physical classrooms might need less technologically-driven solutions to teach cohorts of students.

It seems teachers would greatly appreciate a set of guidelines for how to implement spacing in the curriculum in the most effective, but also the most efficient manner. While the cognitive field has made great advances in terms of understanding the mechanisms behind spacing, what teachers need more of are concrete evidence-based tools and guidelines for direct implementation in the classroom. These could include more sophisticated and experimentally tested versions of the software described above (Weinstein-Jones & Weinstein, 2017 ), or adaptable templates of spaced curricula. Moreover, researchers need to evaluate the effectiveness of these tools in a real classroom environment, over a semester or academic year, in order to give pedagogically relevant evidence-based recommendations to teachers.

Interleaving

Another scheduling technique that has been shown to increase learning is interleaving. Interleaving occurs when different ideas or problem types are tackled in a sequence, as opposed to the more common method of attempting multiple versions of the same problem in a given study session (known as blocking). Interleaving as a principle can be applied in many different ways. One such way involves interleaving different types of problems during learning, which is particularly applicable to subjects such as math and physics (see Fig. 2 a for an example with fractions, based on a study by Patel, Liu, & Koedinger, 2016 ). For example, in a study with college students, Rohrer and Taylor ( 2007 ) found that shuffling math problems that involved calculating the volume of different shapes resulted in better test performance 1 week later than when students answered multiple problems about the same type of shape in a row. This pattern of results has also been replicated with younger students, for example 7 th grade students learning to solve graph and slope problems (Rohrer, Dedrick, & Stershic, 2015 ). The proposed explanation for the benefit of interleaving is that switching between different problem types allows students to acquire the ability to choose the right method for solving different types of problems rather than learning only the method itself, and not when to apply it.

Do the benefits of interleaving extend beyond problem solving? The answer appears to be yes. Interleaving can be helpful in other situations that require discrimination, such as inductive learning. Kornell and Bjork ( 2008 ) examined the effects of interleaving in a task that might be pertinent to a student of the history of art: the ability to match paintings to their respective painters. Students who studied different painters’ paintings interleaved at study were more successful on a later identification test than were participants who studied the paintings blocked by painter. Birnbaum, Kornell, Bjork, and Bjork ( 2013 ) proposed the discriminative-contrast hypothesis to explain that interleaving enhances learning by allowing the comparison between exemplars of different categories. They found support for this hypothesis in a set of experiments with bird categorization: participants benefited from interleaving and also from spacing, but not when the spacing interrupted side-by-side comparisons of birds from different categories.

Another type of interleaving involves the interleaving of study and test opportunities. This type of interleaving has been applied, once again, to problem solving, whereby students alternate between attempting a problem and viewing a worked example (Trafton & Reiser, 1993 ); this pattern appears to be superior to answering a string of problems in a row, at least with respect to the amount of time it takes to achieve mastery of a procedure (Corbett, Reed, Hoffmann, MacLaren, & Wagner, 2010 ). The benefits of interleaving study and test opportunities – rather than blocking study followed by attempting to answer problems or questions – might arise due to a process known as “test-potentiated learning”. That is, a study opportunity that immediately follows a retrieval attempt may be more fruitful than when that same studying was not preceded by retrieval (Arnold & McDermott, 2013 ).

For problem-based subjects, the interleaving technique is straightforward: simply mix questions on homework and quizzes with previous materials (which takes care of spacing as well); for languages, mix vocabulary themes rather than blocking by theme (Thomson & Mehring, 2016 ). But interleaving as an educational strategy ought to be presented to teachers with some caveats. Research has focused on interleaving material that is somewhat related (e.g., solving different mathematical equations, Rohrer et al., 2015 ), whereas students sometimes ask whether they should interleave material from different subjects – a practice that has not received empirical support (Hausman & Kornell, 2014 ). When advising students how to study independently, teachers should thus proceed with caution. Since it is easy for younger students to confuse this type of unhelpful interleaving with the more helpful interleaving of related information, it may be best for teachers of younger grades to create opportunities for interleaving in homework and quiz assignments rather than putting the onus on the students themselves to make use of the technique. Technology can be very helpful here, with apps such as Quizlet, Memrise, Anki, Synap, Quiz Champ, and many others (see also “Learning Scientists”, 2017 ) that not only allow instructor-created quizzes to be taken by students, but also provide built-in interleaving algorithms so that the burden does not fall on the teacher or the student to carefully plan which items are interleaved when.

An important point to consider is that in educational practice, the distinction between spacing and interleaving can be difficult to delineate. The gap between the scientific and classroom definitions of interleaving is demonstrated by teachers’ own writings about this technique. When they write about interleaving, teachers often extend the term to connote a curriculum that involves returning to topics multiple times throughout the year (e.g., Kirby, 2014 ; see “Learning Scientists” ( 2016a ) for a collection of similar blog posts by several other teachers). The “interleaving” of topics throughout the curriculum produces an effect that is more akin to what cognitive psychologists call “spacing” (see Fig. 2 b for a visual representation of the difference between interleaving and spacing). However, cognitive psychologists have not examined the effects of structuring the curriculum in this way, and open questions remain: does repeatedly circling back to previous topics throughout the semester interrupt the learning of new information? What are some effective techniques for interleaving old and new information within one class? And how does one determine the balance between old and new information?

Retrieval practice

While tests are most often used in educational settings for assessment, a lesser-known benefit of tests is that they actually improve memory of the tested information. If we think of our memories as libraries of information, then it may seem surprising that retrieval (which happens when we take a test) improves memory; however, we know from a century of research that retrieving knowledge actually strengthens it (see Karpicke, Lehman, & Aue, 2014 ). Testing was shown to strengthen memory as early as 100 years ago (Gates, 1917 ), and there has been a surge of research in the last decade on the mnemonic benefits of testing, or retrieval practice . Most of the research on the effectiveness of retrieval practice has been done with college students (see Roediger & Karpicke, 2006 ; Roediger, Putnam, & Smith, 2011 ), but retrieval-based learning has been shown to be effective at producing learning for a wide range of ages, including preschoolers (Fritz, Morris, Nolan, & Singleton, 2007 ), elementary-aged children (e.g., Karpicke, Blunt, & Smith, 2016 ; Karpicke, Blunt, Smith, & Karpicke, 2014 ; Lipko-Speed, Dunlosky, & Rawson, 2014 ; Marsh, Fazio, & Goswick, 2012 ; Ritchie, Della Sala, & McIntosh, 2013 ), middle-school students (e.g., McDaniel, Thomas, Agarwal, McDermott, & Roediger, 2013 ; McDermott, Agarwal, D’Antonio, Roediger, & McDaniel, 2014 ), and high-school students (e.g., McDermott et al., 2014 ). In addition, the effectiveness of retrieval-based learning has been extended beyond simple testing to other activities in which retrieval practice can be integrated, such as concept mapping (Blunt & Karpicke, 2014 ; Karpicke, Blunt, et al., 2014 ; Ritchie et al., 2013 ).

A debate is currently ongoing as to the effectiveness of retrieval practice for more complex materials (Karpicke & Aue, 2015 ; Roelle & Berthold, 2017 ; Van Gog & Sweller, 2015 ). Practicing retrieval has been shown to improve the application of knowledge to new situations (e.g., Butler, 2010 ; Dirkx, Kester, & Kirschner, 2014 ); McDaniel et al., 2013 ; Smith, Blunt, Whiffen, & Karpicke, 2016 ); but see Tran, Rohrer, and Pashler ( 2015 ) and Wooldridge, Bugg, McDaniel, and Liu ( 2014 ), for retrieval practice studies that showed limited or no increased transfer compared to restudy. Retrieval practice effects on higher-order learning may be more sensitive than fact learning to encoding factors, such as the way material is presented during study (Eglington & Kang, 2016 ). In addition, retrieval practice may be more beneficial for higher-order learning if it includes more scaffolding (Fiechter & Benjamin, 2017 ; but see Smith, Blunt, et al., 2016 ) and targeted practice with application questions (Son & Rivas, 2016 ).

How does retrieval practice help memory? Figure 3 illustrates both the direct and indirect benefits of retrieval practice identified by the literature. The act of retrieval itself is thought to strengthen memory (Karpicke, Blunt, et al., 2014 ; Roediger & Karpicke, 2006 ; Smith, Roediger, & Karpicke, 2013 ). For example, Smith et al. ( 2013 ) showed that if students brought information to mind without actually producing it (covert retrieval), they remembered the information just as well as if they overtly produced the retrieved information (overt retrieval). Importantly, both overt and covert retrieval practice improved memory over control groups without retrieval practice, even when feedback was not provided. The fact that bringing information to mind in the absence of feedback or restudy opportunities improves memory leads researchers to conclude that it is the act of retrieval – thinking back to bring information to mind – that improves memory of that information.

The benefit of retrieval practice depends to a certain extent on successful retrieval (see Karpicke, Lehman, et al., 2014 ). For example, in Experiment 4 of Smith et al. ( 2013 ), students successfully retrieved 72% of the information during retrieval practice. Of course, retrieving 72% of the information was compared to a restudy control group, during which students were re-exposed to 100% of the information, creating a bias in favor of the restudy condition. Yet retrieval led to superior memory later compared to the restudy control. However, if retrieval success is extremely low, then it is unlikely to improve memory (e.g., Karpicke, Blunt, et al., 2014 ), particularly in the absence of feedback. On the other hand, if retrieval-based learning situations are constructed in such a way that ensures high levels of success, the act of bringing the information to mind may be undermined, thus making it less beneficial. For example, if a student reads a sentence and then immediately covers the sentence and recites it out loud, they are likely not retrieving the information but rather just keeping the information in their working memory long enough to recite it again (see Smith, Blunt, et al., 2016 for a discussion of this point). Thus, it is important to balance success of retrieval with overall difficulty in retrieving the information (Smith & Karpicke, 2014 ; Weinstein, Nunes, & Karpicke, 2016 ). If initial retrieval success is low, then feedback can help improve the overall benefit of practicing retrieval (Kang, McDermott, & Roediger, 2007 ; Smith & Karpicke, 2014 ). Kornell, Klein, and Rawson ( 2015 ), however, found that it was the retrieval attempt and not the correct production of information that produced the retrieval practice benefit – as long as the correct answer was provided after an unsuccessful attempt, the benefit was the same as for a successful retrieval attempt in this set of studies. From a practical perspective, it would be helpful for teachers to know when retrieval attempts in the absence of success are helpful, and when they are not. There may also be additional reasons beyond retrieval benefits that would push teachers towards retrieval practice activities that produce some success amongst students; for example, teachers may hesitate to give students retrieval practice exercises that are too difficult, as this may negatively affect self-efficacy and confidence.

In addition to the fact that bringing information to mind directly improves memory for that information, engaging in retrieval practice can produce indirect benefits as well (see Roediger et al., 2011 ). For example, research by Weinstein, Gilmore, Szpunar, and McDermott ( 2014 ) demonstrated that when students expected to be tested, the increased test expectancy led to better-quality encoding of new information. Frequent testing can also serve to decrease mind-wandering – that is, thoughts that are unrelated to the material that students are supposed to be studying (Szpunar, Khan, & Schacter, 2013 ).

Practicing retrieval is a powerful way to improve meaningful learning of information, and it is relatively easy to implement in the classroom. For example, requiring students to practice retrieval can be as simple as asking students to put their class materials away and try to write out everything they know about a topic. Retrieval-based learning strategies are also flexible. Instructors can give students practice tests (e.g., short-answer or multiple-choice, see Smith & Karpicke, 2014 ), provide open-ended prompts for the students to recall information (e.g., Smith, Blunt, et al., 2016 ) or ask their students to create concept maps from memory (e.g., Blunt & Karpicke, 2014 ). In one study, Weinstein et al. ( 2016 ) looked at the effectiveness of inserting simple short-answer questions into online learning modules to see whether they improved student performance. Weinstein and colleagues also manipulated the placement of the questions. For some students, the questions were interspersed throughout the module, and for other students the questions were all presented at the end of the module. Initial success on the short-answer questions was higher when the questions were interspersed throughout the module. However, on a later test of learning from that module, the original placement of the questions in the module did not matter for performance. As with spaced practice, where the optimal gap between study sessions is contingent on the retention interval, the optimum difficulty and level of success during retrieval practice may also depend on the retention interval. Both groups of students who answered questions performed better on the delayed test compared to a control group without question opportunities during the module. Thus, the important thing is for instructors to provide opportunities for retrieval practice during learning. Based on previous research, any activity that promotes the successful retrieval of information should improve learning.

Retrieval practice has received a lot of attention in teacher blogs (see “Learning Scientists” ( 2016b ) for a collection). A common theme seems to be an emphasis on low-stakes (Young, 2016 ) and even no-stakes (Cox, 2015 ) testing, the goal of which is to increase learning rather than assess performance. In fact, one well-known charter school in the UK has an official homework policy grounded in retrieval practice: students are to test themselves on subject knowledge for 30 minutes every day in lieu of standard homework (Michaela Community School, 2014 ). The utility of homework, particularly for younger children, is often a hotly debated topic outside of academia (e.g., Shumaker, 2016 ; but see Jones ( 2016 ) for an opposing viewpoint and Cooper ( 1989 ) for the original research the blog posts were based on). Whereas some research shows clear links between homework and academic achievement (Valle et al., 2016 ), other researchers have questioned the effectiveness of homework (Dettmers, Trautwein, & Lüdtke, 2009 ). Perhaps amending homework to involve retrieval practice might make it more effective; this remains an open empirical question.

One final consideration is that of test anxiety. While retrieval practice can be very powerful at improving memory, some research shows that pressure during retrieval can undermine some of the learning benefit. For example, Hinze and Rapp ( 2014 ) manipulated pressure during quizzing to create high-pressure and low-pressure conditions. On the quizzes themselves, students performed equally well. However, those in the high-pressure condition did not perform as well on a criterion test later compared to the low-pressure group. Thus, test anxiety may reduce the learning benefit of retrieval practice. Eliminating all high-pressure tests is probably not possible, but instructors can provide a number of low-stakes retrieval opportunities for students to help increase learning. The use of low-stakes testing can serve to decrease test anxiety (Khanna, 2015 ), and has recently been shown to negate the detrimental impact of stress on learning (Smith, Floerke, & Thomas, 2016 ). This is a particularly important line of inquiry to pursue for future research, because many teachers who are not familiar with the effectiveness of retrieval practice may be put off by the implied pressure of “testing”, which evokes the much maligned high-stakes standardized tests (e.g., McHugh, 2013 ).

Elaboration

Elaboration involves connecting new information to pre-existing knowledge. Anderson ( 1983 , p.285) made the following claim about elaboration: “One of the most potent manipulations that can be performed in terms of increasing a subject’s memory for material is to have the subject elaborate on the to-be-remembered material.” Postman ( 1976 , p. 28) defined elaboration most parsimoniously as “additions to nominal input”, and Hirshman ( 2001 , p. 4369) provided an elaboration on this definition (pun intended!), defining elaboration as “A conscious, intentional process that associates to-be-remembered information with other information in memory.” However, in practice, elaboration could mean many different things. The common thread in all the definitions is that elaboration involves adding features to an existing memory.

One possible instantiation of elaboration is thinking about information on a deeper level. The levels (or “depth”) of processing framework, proposed by Craik and Lockhart ( 1972 ), predicts that information will be remembered better if it is processed more deeply in terms of meaning, rather than shallowly in terms of form. The leves of processing framework has, however, received a number of criticisms (Craik, 2002 ). One major problem with this framework is that it is difficult to measure “depth”. And if we are not able to actually measure depth, then the argument can become circular: is it that something was remembered better because it was studied more deeply, or do we conclude that it must have been studied more deeply because it is remembered better? (See Lockhart & Craik, 1990 , for further discussion of this issue).

Another mechanism by which elaboration can confer a benefit to learning is via improvement in organization (Bellezza, Cheesman, & Reddy, 1977 ; Mandler, 1979 ). By this view, elaboration involves making information more integrated and organized with existing knowledge structures. By connecting and integrating the to-be-learned information with other concepts in memory, students can increase the extent to which the ideas are organized in their minds, and this increased organization presumably facilitates the reconstruction of the past at the time of retrieval.

Elaboration is such a broad term and can include so many different techniques that it is hard to claim that elaboration will always help learning. There is, however, a specific technique under the umbrella of elaboration for which there is relatively strong evidence in terms of effectiveness (Dunlosky et al., 2013 ; Pashler et al., 2007 ). This technique is called elaborative interrogation, and involves students questioning the materials that they are studying (Pressley, McDaniel, Turnure, Wood, & Ahmad, 1987 ). More specifically, students using this technique would ask “how” and “why” questions about the concepts they are studying (see Fig. 4 for an example on the physics of flight). Then, crucially, students would try to answer these questions – either from their materials or, eventually, from memory (McDaniel & Donnelly, 1996 ). The process of figuring out the answer to the questions – with some amount of uncertainty (Overoye & Storm, 2015 ) – can help learning. When using this technique, however, it is important that students check their answers with their materials or with the teacher; when the content generated through elaborative interrogation is poor, it can actually hurt learning (Clinton, Alibali, & Nathan, 2016 ).

Students can also be encouraged to self-explain concepts to themselves while learning (Chi, De Leeuw, Chiu, & LaVancher, 1994 ). This might involve students simply saying out loud what steps they need to perform to solve an equation. Aleven and Koedinger ( 2002 ) conducted two classroom studies in which students were either prompted by a “cognitive tutor” to provide self-explanations during a problem-solving task or not, and found that the self-explanations led to improved performance. According to the authors, this approach could scale well to real classrooms. If possible and relevant, students could even perform actions alongside their self-explanations (Cohen, 1981 ; see also the enactment effect, Hainselin, Picard, Manolli, Vankerkore-Candas, & Bourdin, 2017 ). Instructors can scaffold students in these types of activities by providing self-explanation prompts throughout to-be-learned material (O’Neil et al., 2014 ). Ultimately, the greatest potential benefit of accurate self-explanation or elaboration is that the student will be able to transfer their knowledge to a new situation (Rittle-Johnson, 2006 ).

The technical term “elaborative interrogation” has not made it into the vernacular of educational bloggers (a search on https://educationechochamberuncut.wordpress.com , which consolidates over 3,000 UK-based teacher blogs, yielded zero results for that term). However, a few teachers have blogged about elaboration more generally (e.g., Hobbiss, 2016 ) and deep questioning specifically (e.g., Class Teaching, 2013 ), just without using the specific terminology. This strategy in particular may benefit from a more open dialog between researchers and teachers to facilitate the use of elaborative interrogation in the classroom and to address possible barriers to implementation. In terms of advancing the scientific understanding of elaborative interrogation in a classroom setting, it would be informative to conduct a larger-scale intervention to see whether having students elaborate during reading actually helps their understanding. It would also be useful to know whether the students really need to generate their own elaborative interrogation (“how” and “why”) questions, versus answering questions provided by others. How long should students persist to find the answers? When is the right time to have students engage in this task, given the levels of expertise required to do it well (Clinton et al., 2016 )? Without knowing the answers to these questions, it may be too early for us to instruct teachers to use this technique in their classes. Finally, elaborative interrogation takes a long time. Is this time efficiently spent? Or, would it be better to have the students try to answer a few questions, pool their information as a class, and then move to practicing retrieval of the information?

Concrete examples

Providing supporting information can improve the learning of key ideas and concepts. Specifically, using concrete examples to supplement content that is more conceptual in nature can make the ideas easier to understand and remember. Concrete examples can provide several advantages to the learning process: (a) they can concisely convey information, (b) they can provide students with more concrete information that is easier to remember, and (c) they can take advantage of the superior memorability of pictures relative to words (see “Dual Coding”).

Words that are more concrete are both recognized and recalled better than abstract words (Gorman, 1961 ; e.g., “button” and “bound,” respectively). Furthermore, it has been demonstrated that information that is more concrete and imageable enhances the learning of associations, even with abstract content (Caplan & Madan, 2016 ; Madan, Glaholt, & Caplan, 2010 ; Paivio, 1971 ). Following from this, providing concrete examples during instruction should improve retention of related abstract concepts, rather than the concrete examples alone being remembered better. Concrete examples can be useful both during instruction and during practice problems. Having students actively explain how two examples are similar and encouraging them to extract the underlying structure on their own can also help with transfer. In a laboratory study, Berry ( 1983 ) demonstrated that students performed well when given concrete practice problems, regardless of the use of verbalization (akin to elaborative interrogation), but that verbalization helped students transfer understanding from concrete to abstract problems. One particularly important area of future research is determining how students can best make the link between concrete examples and abstract ideas.

Since abstract concepts are harder to grasp than concrete information (Paivio, Walsh, & Bons, 1994 ), it follows that teachers ought to illustrate abstract ideas with concrete examples. However, care must be taken when selecting the examples. LeFevre and Dixon ( 1986 ) provided students with both concrete examples and abstract instructions and found that when these were inconsistent, students followed the concrete examples rather than the abstract instructions, potentially constraining the application of the abstract concept being taught. Lew, Fukawa-Connelly, Mejí-Ramos, and Weber ( 2016 ) used an interview approach to examine why students may have difficulty understanding a lecture. Responses indicated that some issues were related to understanding the overarching topic rather than the component parts, and to the use of informal colloquialisms that did not clearly follow from the material being taught. Both of these issues could have potentially been addressed through the inclusion of a greater number of relevant concrete examples.

One concern with using concrete examples is that students might only remember the examples – especially if they are particularly memorable, such as fun or gimmicky examples – and will not be able to transfer their understanding from one example to another, or more broadly to the abstract concept. However, there does not seem to be any evidence that fun relevant examples actually hurt learning by harming memory for important information. Instead, fun examples and jokes tend to be more memorable, but this boost in memory for the joke does not seem to come at a cost to memory for the underlying concept (Baldassari & Kelley, 2012 ). However, two important caveats need to be highlighted. First, to the extent that the more memorable content is not relevant to the concepts of interest, learning of the target information can be compromised (Harp & Mayer, 1998 ). Thus, care must be taken to ensure that all examples and gimmicks are, in fact, related to the core concepts that the students need to acquire, and do not contain irrelevant perceptual features (Kaminski & Sloutsky, 2013 ).

The second issue is that novices often notice and remember the surface details of an example rather than the underlying structure. Experts, on the other hand, can extract the underlying structure from examples that have divergent surface features (Chi, Feltovich, & Glaser, 1981 ; see Fig. 5 for an example from physics). Gick and Holyoak ( 1983 ) tried to get students to apply a rule from one problem to another problem that appeared different on the surface, but was structurally similar. They found that providing multiple examples helped with this transfer process compared to only using one example – especially when the examples provided had different surface details. More work is also needed to determine how many examples are sufficient for generalization to occur (and this, of course, will vary with contextual factors and individual differences). Further research on the continuum between concrete/specific examples and more abstract concepts would also be informative. That is, if an example is not concrete enough, it may be too difficult to understand. On the other hand, if the example is too concrete, that could be detrimental to generalization to the more abstract concept (although a diverse set of very concrete examples may be able to help with this). In fact, in a controversial article, Kaminski, Sloutsky, and Heckler ( 2008 ) claimed that abstract examples were more effective than concrete examples. Later rebuttals of this paper contested whether the abstract versus concrete distinction was clearly defined in the original study (see Reed, 2008 , for a collection of letters on the subject). This ideal point along the concrete-abstract continuum might also interact with development.

Finding teacher blog posts on concrete examples proved to be more difficult than for the other strategies in this review. One optimistic possibility is that teachers frequently use concrete examples in their teaching, and thus do not think of this as a specific contribution from cognitive psychology; the one blog post we were able to find that discussed concrete examples suggests that this might be the case (Boulton, 2016 ). The idea of “linking abstract concepts with concrete examples” is also covered in 25% of teacher-training textbooks used in the US, according to the report by Pomerance et al. ( 2016 ); this is the second most frequently covered of the six strategies, after “posing probing questions” (i.e., elaborative interrogation). A useful direction for future research would be to establish how teachers are using concrete examples in their practice, and whether we can make any suggestions for improvement based on research into the science of learning. For example, if two examples are better than one (Bauernschmidt, 2017 ), are additional examples also needed, or are there diminishing returns from providing more examples? And, how can teachers best ensure that concrete examples are consistent with prior knowledge (Reed, 2008 )?

Dual coding

Both the memory literature and folk psychology support the notion of visual examples being beneficial—the adage of “a picture is worth a thousand words” (traced back to an advertising slogan from the 1920s; Meider, 1990 ). Indeed, it is well-understood that more information can be conveyed through a simple illustration than through several paragraphs of text (e.g., Barker & Manji, 1989 ; Mayer & Gallini, 1990 ). Illustrations can be particularly helpful when the described concept involves several parts or steps and is intended for individuals with low prior knowledge (Eitel & Scheiter, 2015 ; Mayer & Gallini, 1990 ). Figure 6 provides a concrete example of this, illustrating how information can flow through neurons and synapses.

In addition to being able to convey information more succinctly, pictures are also more memorable than words (Paivio & Csapo, 1969 , 1973 ). In the memory literature, this is referred to as the picture superiority effect , and dual coding theory was developed in part to explain this effect. Dual coding follows from the notion of text being accompanied by complementary visual information to enhance learning. Paivio ( 1971 , 1986 ) proposed dual coding theory as a mechanistic account for the integration of multiple information “codes” to process information. In this theory, a code corresponds to a modal or otherwise distinct representation of a concept—e.g., “mental images for ‘book’ have visual, tactual, and other perceptual qualities similar to those evoked by the referent objects on which the images are based” (Clark & Paivio, 1991 , p. 152). Aylwin ( 1990 ) provides a clear example of how the word “dog” can evoke verbal, visual, and enactive representations (see Fig. 7 for a similar example for the word “SPOON”, based on Aylwin, 1990 (Fig. 2 ) and Madan & Singhal, 2012a (Fig. 3 )). Codes can also correspond to emotional properties (Clark & Paivio, 1991 ; Paivio, 2013 ). Clark and Paivio ( 1991 ) provide a thorough review of dual coding theory and its relation to education, while Paivio ( 2007 ) provides a comprehensive treatise on dual coding theory. Broadly, dual coding theory suggests that providing multiple representations of the same information enhances learning and memory, and that information that more readily evokes additional representations (through automatic imagery processes) receives a similar benefit.

Paivio and Csapo ( 1973 ) suggest that verbal and imaginal codes have independent and additive effects on memory recall. Using visuals to improve learning and memory has been particularly applied to vocabulary learning (Danan, 1992 ; Sadoski, 2005 ), but has also shown success in other domains such as in health care (Hartland, Biddle, & Fallacaro, 2008 ). To take advantage of dual coding, verbal information should be accompanied by a visual representation when possible. However, while the studies discussed all indicate that the use of multiple representations of information is favorable, it is important to acknowledge that each representation also increases cognitive load and can lead to over-saturation (Mayer & Moreno, 2003 ).

Given that pictures are generally remembered better than words, it is important to ensure that the pictures students are provided with are helpful and relevant to the content they are expected to learn. McNeill, Uttal, Jarvin, and Sternberg ( 2009 ) found that providing visual examples decreased conceptual errors. However, McNeill et al. also found that when students were given visually rich examples, they performed more poorly than students who were not given any visual example, suggesting that the visual details can at times become a distraction and hinder performance. Thus, it is important to consider that images used in teaching are clear and not ambiguous in their meaning (Schwartz, 2007 ).

Further broadening the scope of dual coding theory, Engelkamp and Zimmer ( 1984 ) suggest that motor movements, such as “turning the handle,” can provide an additional motor code that can improve memory, linking studies of motor actions (enactment) with dual coding theory (Clark & Paivio, 1991 ; Engelkamp & Cohen, 1991 ; Madan & Singhal, 2012c ). Indeed, enactment effects appear to primarily occur during learning, rather than during retrieval (Peterson & Mulligan, 2010 ). Along similar lines, Wammes, Meade, and Fernandes ( 2016 ) demonstrated that generating drawings can provide memory benefits beyond what could otherwise be explained by visual imagery, picture superiority, and other memory enhancing effects. Providing convergent evidence, even when overt motor actions are not critical in themselves, words representing functional objects have been shown to enhance later memory (Madan & Singhal, 2012b ; Montefinese, Ambrosini, Fairfield, & Mammarella, 2013 ). This indicates that motoric processes can improve memory similarly to visual imagery, similar to memory differences for concrete vs. abstract words. Further research suggests that automatic motor simulation for functional objects is likely responsible for this memory benefit (Madan, Chen, & Singhal, 2016 ).

When teachers combine visuals and words in their educational practice, however, they may not always be taking advantage of dual coding – at least, not in the optimal manner. For example, a recent discussion on Twitter centered around one teacher’s decision to have 7 th Grade students replace certain words in their science laboratory report with a picture of that word (e.g., the instructions read “using a syringe …” and a picture of a syringe replaced the word; Turner, 2016a ). Other teachers argued that this was not dual coding (Beaven, 2016 ; Williams, 2016 ), because there were no longer two different representations of the information. The first teacher maintained that dual coding was preserved, because this laboratory report with pictures was to be used alongside the original, fully verbal report (Turner, 2016b ). This particular implementation – having students replace individual words with pictures – has not been examined in the cognitive literature, presumably because no benefit would be expected. In any case, we need to be clearer about implementations for dual coding, and more research is needed to clarify how teachers can make use of the benefits conferred by multiple representations and picture superiority.

Critically, dual coding theory is distinct from the notion of “learning styles,” which describe the idea that individuals benefit from instruction that matches their modality preference. While this idea is pervasive and individuals often subjectively feel that they have a preference, evidence indicates that the learning styles theory is not supported by empirical findings (e.g., Kavale, Hirshoren, & Forness, 1998 ; Pashler, McDaniel, Rohrer, & Bjork, 2008 ; Rohrer & Pashler, 2012 ). That is, there is no evidence that instructing students in their preferred learning style leads to an overall improvement in learning (the “meshing” hypothesis). Moreover, learning styles have come to be described as a myth or urban legend within psychology (Coffield, Moseley, Hall, & Ecclestone, 2004 ; Hattie & Yates, 2014 ; Kirschner & van Merriënboer, 2013 ; Kirschner, 2017 ); skepticism about learning styles is a common stance amongst evidence-informed teachers (e.g., Saunders, 2016 ). Providing evidence against the notion of learning styles, Kraemer, Rosenberg, and Thompson-Schill ( 2009 ) found that individuals who scored as “verbalizers” and “visualizers” did not perform any better on experimental trials matching their preference. Instead, it has recently been shown that learning through one’s preferred learning style is associated with elevated subjective judgements of learning, but not objective performance (Knoll, Otani, Skeel, & Van Horn, 2017 ). In contrast to learning styles, dual coding is based on providing additional, complementary forms of information to enhance learning, rather than tailoring instruction to individuals’ preferences.

Genuine educational environments present many opportunities for combining the strategies outlined above. Spacing can be particularly potent for learning if it is combined with retrieval practice. The additive benefits of retrieval practice and spacing can be gained by engaging in retrieval practice multiple times (also known as distributed practice; see Cepeda et al., 2006 ). Interleaving naturally entails spacing if students interleave old and new material. Concrete examples can be both verbal and visual, making use of dual coding. In addition, the strategies of elaboration, concrete examples, and dual coding all work best when used as part of retrieval practice. For example, in the concept-mapping studies mentioned above (Blunt & Karpicke, 2014 ; Karpicke, Blunt, et al., 2014 ), creating concept maps while looking at course materials (e.g., a textbook) was not as effective for later memory as creating concept maps from memory. When practicing elaborative interrogation, students can start off answering the “how” and “why” questions they pose for themselves using class materials, and work their way up to answering them from memory. And when interleaving different problem types, students should be practicing answering them rather than just looking over worked examples.

But while these ideas for strategy combinations have empirical bases, it has not yet been established whether the benefits of the strategies to learning are additive, super-additive, or, in some cases, incompatible. Thus, future research needs to (a) better formalize the definition of each strategy (particularly critical for elaboration and dual coding), (b) identify best practices for implementation in the classroom, (c) delineate the boundary conditions of each strategy, and (d) strategically investigate interactions between the six strategies we outlined in this manuscript.

Aleven, V. A., & Koedinger, K. R. (2002). An effective metacognitive strategy: learning by doing and explaining with a computer-based cognitive tutor. Cognitive Science, 26 , 147–179.

Article Google Scholar

Anderson, J. R. (1983). A spreading activation theory of memory. Journal of Verbal Learning and Verbal Behavior, 22 , 261–295.

Arnold, K. M., & McDermott, K. B. (2013). Test-potentiated learning: distinguishing between direct and indirect effects of tests. Journal of Experimental Psychology: Learning, Memory, and Cognition, 39 , 940–945.

PubMed Google Scholar

Aylwin, S. (1990). Imagery and affect: big questions, little answers. In P. J. Thompson, D. E. Marks, & J. T. E. Richardson (Eds.), Imagery: Current developments . New York: International Library of Psychology.

Google Scholar

Baldassari, M. J., & Kelley, M. (2012). Make’em laugh? The mnemonic effect of humor in a speech. Psi Chi Journal of Psychological Research, 17 , 2–9.

Barker, P. G., & Manji, K. A. (1989). Pictorial dialogue methods. International Journal of Man-Machine Studies, 31 , 323–347.

Bauernschmidt, A. (2017). GUEST POST: two examples are better than one. [Blog post]. The Learning Scientists Blog . Retrieved from http://www.learningscientists.org/blog/2017/5/30-1 . Accessed 25 Dec 2017.

Beaven, T. (2016). @doctorwhy @FurtherEdagogy @doc_kristy Right, I thought the whole point of dual coding was to use TWO codes: pics + words of the SAME info? [Tweet]. Retrieved from https://twitter.com/TitaBeaven/status/807504041341308929 . Accessed 25 Dec 2017.

Bellezza, F. S., Cheesman, F. L., & Reddy, B. G. (1977). Organization and semantic elaboration in free recall. Journal of Experimental Psychology: Human Learning and Memory, 3 , 539–550.

Benney, D. (2016). (Trying to apply) spacing in a content heavy subject [Blog post]. Retrieved from https://mrbenney.wordpress.com/2016/10/16/trying-to-apply-spacing-in-science/ . Accessed 25 Dec 2017.

Berry, D. C. (1983). Metacognitive experience and transfer of logical reasoning. Quarterly Journal of Experimental Psychology, 35A , 39–49.

Birnbaum, M. S., Kornell, N., Bjork, E. L., & Bjork, R. A. (2013). Why interleaving enhances inductive learning: the roles of discrimination and retrieval. Memory & Cognition, 41 , 392–402.

Bjork, R. A. (1999). Assessing our own competence: heuristics and illusions. In D. Gopher & A. Koriat (Eds.), Attention and peformance XVII. Cognitive regulation of performance: Interaction of theory and application (pp. 435–459). Cambridge, MA: MIT Press.

Bjork, R. A. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe & A. Shimamura (Eds.), Metacognition: Knowing about knowing (pp. 185–205). Cambridge, MA: MIT Press.

Bjork, R. A., & Bjork, E. L. (1992). A new theory of disuse and an old theory of stimulus fluctuation. From learning processes to cognitive processes: Essays in honor of William K. Estes, 2 , 35–67.

Bjork, E. L., & Bjork, R. A. (2011). Making things hard on yourself, but in a good way: creating desirable difficulties to enhance learning. Psychology and the real world: Essays illustrating fundamental contributions to society , 56–64.

Blunt, J. R., & Karpicke, J. D. (2014). Learning with retrieval-based concept mapping. Journal of Educational Psychology, 106 , 849–858.

Boulton, K. (2016). What does cognitive overload look like in the humanities? [Blog post]. Retrieved from https://educationechochamberuncut.wordpress.com/2016/03/05/what-does-cognitive-overload-look-like-in-the-humanities-kris-boulton-2/ . Accessed 25 Dec 2017.

Brown, P. C., Roediger, H. L., & McDaniel, M. A. (2014). Make it stick . Cambridge, MA: Harvard University Press.

Book Google Scholar

Butler, A. C. (2010). Repeated testing produces superior transfer of learning relative to repeated studying. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36 , 1118–1133.

Caplan, J. B., & Madan, C. R. (2016). Word-imageability enhances association-memory by recruiting hippocampal activity. Journal of Cognitive Neuroscience, 28 , 1522–1538.

Article PubMed Google Scholar

Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006). Distributed practice in verbal recall tasks: a review and quantitative synthesis. Psychological Bulletin, 132 , 354–380.

Cepeda, N. J., Vul, E., Rohrer, D., Wixted, J. T., & Pashler, H. (2008). Spacing effects in learning a temporal ridgeline of optimal retention. Psychological Science, 19 , 1095–1102.

Chi, M. T., De Leeuw, N., Chiu, M. H., & LaVancher, C. (1994). Eliciting self-explanations improves understanding. Cognitive Science, 18 , 439–477.

Chi, M. T., Feltovich, P. J., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5 , 121–152.

CIFE. (2012). No January A level and other changes. Retrieved from http://www.cife.org.uk/cife-general-news/no-january-a-level-and-other-changes/ . Accessed 25 Dec 2017.

Clark, D. (2016). One book on learning that every teacher, lecturer & trainer should read (7 reasons) [Blog post]. Retrieved from http://donaldclarkplanb.blogspot.com/2016/03/one-book-on-learning-that-every-teacher.html . Accessed 25 Dec 2017.

Clark, J. M., & Paivio, A. (1991). Dual coding theory and education. Educational Psychology Review, 3 , 149–210.

Class Teaching. (2013). Deep questioning [Blog post]. Retrieved from https://classteaching.wordpress.com/2013/07/12/deep-questioning/ . Accessed 25 Dec 2017.

Clinton, V., Alibali, M. W., & Nathan, M. J. (2016). Learning about posterior probability: do diagrams and elaborative interrogation help? The Journal of Experimental Education, 84 , 579–599.

Coffield, F., Moseley, D., Hall, E., & Ecclestone, K. (2004). Learning styles and pedagogy in post-16 learning: a systematic and critical review . London: Learning & Skills Research Centre.

Cohen, R. L. (1981). On the generality of some memory laws. Scandinavian Journal of Psychology, 22 , 267–281.

Cooper, H. (1989). Synthesis of research on homework. Educational Leadership, 47 , 85–91.

Corbett, A. T., Reed, S. K., Hoffmann, R., MacLaren, B., & Wagner, A. (2010). Interleaving worked examples and cognitive tutor support for algebraic modeling of problem situations. In Proceedings of the Thirty-Second Annual Meeting of the Cognitive Science Society (pp. 2882–2887).

Cox, D. (2015). No stakes testing – not telling students their results [Blog post]. Retrieved from https://missdcoxblog.wordpress.com/2015/06/06/no-stakes-testing-not-telling-students-their-results/ . Accessed 25 Dec 2017.

Cox, D. (2016a). Ditch revision. Teach it well [Blog post]. Retrieved from https://missdcoxblog.wordpress.com/2016/01/09/ditch-revision-teach-it-well/ . Accessed 25 Dec 2017.

Cox, D. (2016b). ‘They need to remember this in three years time’: spacing & interleaving for the new GCSEs [Blog post]. Retrieved from https://missdcoxblog.wordpress.com/2016/03/25/they-need-to-remember-this-in-three-years-time-spacing-interleaving-for-the-new-gcses/ . Accessed 25 Dec 2017.

Craik, F. I. (2002). Levels of processing: past, present… future? Memory, 10 , 305–318.

Craik, F. I., & Lockhart, R. S. (1972). Levels of processing: a framework for memory research. Journal of Verbal Learning and Verbal Behavior, 11 , 671–684.

Danan, M. (1992). Reversed subtitling and dual coding theory: new directions for foreign language instruction. Language Learning, 42 , 497–527.

Dettmers, S., Trautwein, U., & Lüdtke, O. (2009). The relationship between homework time and achievement is not universal: evidence from multilevel analyses in 40 countries. School Effectiveness and School Improvement, 20 , 375–405.

Dirkx, K. J., Kester, L., & Kirschner, P. A. (2014). The testing effect for learning principles and procedures from texts. The Journal of Educational Research, 107 , 357–364.

Dunlosky, J. (2013). Strengthening the student toolbox: study strategies to boost learning. American Educator, 37 (3), 12–21.

Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning with effective learning techniques: promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14 , 4–58.

Ebbinghaus, H. (1913). Memory (HA Ruger & CE Bussenius, Trans.). New York: Columbia University, Teachers College. (Original work published 1885) . Retrieved from http://psychclassics.yorku.ca/Ebbinghaus/memory8.htm . Accessed 25 Dec 2017.

Eglington, L. G., & Kang, S. H. (2016). Retrieval practice benefits deductive inference. Educational Psychology Review , 1–14.

Eitel, A., & Scheiter, K. (2015). Picture or text first? Explaining sequential effects when learning with pictures and text. Educational Psychology Review, 27 , 153–180.

Engelkamp, J., & Cohen, R. L. (1991). Current issues in memory of action events. Psychological Research, 53 , 175–182.

Engelkamp, J., & Zimmer, H. D. (1984). Motor programme information as a separable memory unit. Psychological Research, 46 , 283–299.

Fawcett, D. (2013). Can I be that little better at……using cognitive science/psychology/neurology to plan learning? [Blog post]. Retrieved from http://reflectionsofmyteaching.blogspot.com/2013/09/can-i-be-that-little-better-atusing.html . Accessed 25 Dec 2017.

Fiechter, J. L., & Benjamin, A. S. (2017). Diminishing-cues retrieval practice: a memory-enhancing technique that works when regular testing doesn’t. Psychonomic Bulletin & Review , 1–9.

Firth, J. (2016). Spacing in teaching practice [Blog post]. Retrieved from http://www.learningscientists.org/blog/2016/4/12-1 . Accessed 25 Dec 2017.

Fordham, M. [mfordhamhistory]. (2016). Is there a meaningful distinction in psychology between ‘thinking’ & ‘critical thinking’? [Tweet]. Retrieved from https://twitter.com/mfordhamhistory/status/809525713623781377 . Accessed 25 Dec 2017.

Fritz, C. O., Morris, P. E., Nolan, D., & Singleton, J. (2007). Expanding retrieval practice: an effective aid to preschool children’s learning. The Quarterly Journal of Experimental Psychology, 60 , 991–1004.

Gates, A. I. (1917). Recitation as a factory in memorizing. Archives of Psychology, 6.

Gick, M. L., & Holyoak, K. J. (1983). Schema induction and analogical transfer. Cognitive Psychology, 15 , 1–38.

Gorman, A. M. (1961). Recognition memory for nouns as a function of abstractedness and frequency. Journal of Experimental Psychology, 61 , 23–39.

Hainselin, M., Picard, L., Manolli, P., Vankerkore-Candas, S., & Bourdin, B. (2017). Hey teacher, don’t leave them kids alone: action is better for memory than reading. Frontiers in Psychology , 8 .

Harp, S. F., & Mayer, R. E. (1998). How seductive details do their damage. Journal of Educational Psychology, 90 , 414–434.

Hartland, W., Biddle, C., & Fallacaro, M. (2008). Audiovisual facilitation of clinical knowledge: A paradigm for dispersed student education based on Paivio’s dual coding theory. AANA Journal, 76 , 194–198.

Hattie, J., & Yates, G. (2014). Visible learning and the science of how we learn . New York: Routledge.

Hausman, H., & Kornell, N. (2014). Mixing topics while studying does not enhance learning. Journal of Applied Research in Memory and Cognition, 3 , 153–160.

Hinze, S. R., & Rapp, D. N. (2014). Retrieval (sometimes) enhances learning: performance pressure reduces the benefits of retrieval practice. Applied Cognitive Psychology, 28 , 597–606.

Hirshman, E. (2001). Elaboration in memory. In N. J. Smelser & P. B. Baltes (Eds.), International encyclopedia of the social & behavioral sciences (pp. 4369–4374). Oxford: Pergamon.

Chapter Google Scholar

Hobbiss, M. (2016). Make it meaningful! Elaboration [Blog post]. Retrieved from https://hobbolog.wordpress.com/2016/06/09/make-it-meaningful-elaboration/ . Accessed 25 Dec 2017.

Jones, F. (2016). Homework – is it really that useless? [Blog post]. Retrieved from http://www.learningscientists.org/blog/2016/4/5-1 . Accessed 25 Dec 2017.

Kaminski, J. A., & Sloutsky, V. M. (2013). Extraneous perceptual information interferes with children’s acquisition of mathematical knowledge. Journal of Educational Psychology, 105 (2), 351–363.

Kaminski, J. A., Sloutsky, V. M., & Heckler, A. F. (2008). The advantage of abstract examples in learning math. Science, 320 , 454–455.

Kang, S. H. (2016). Spaced repetition promotes efficient and effective learning policy implications for instruction. Policy Insights from the Behavioral and Brain Sciences, 3 , 12–19.

Kang, S. H. K., McDermott, K. B., & Roediger, H. L. (2007). Test format and corrective feedback modify the effects of testing on long-term retention. European Journal of Cognitive Psychology, 19 , 528–558.

Karpicke, J. D., & Aue, W. R. (2015). The testing effect is alive and well with complex materials. Educational Psychology Review, 27 , 317–326.

Karpicke, J. D., Blunt, J. R., Smith, M. A., & Karpicke, S. S. (2014). Retrieval-based learning: The need for guided retrieval in elementary school children. Journal of Applied Research in Memory and Cognition, 3 , 198–206.

Karpicke, J. D., Lehman, M., & Aue, W. R. (2014). Retrieval-based learning: an episodic context account. In B. H. Ross (Ed.), Psychology of Learning and Motivation (Vol. 61, pp. 237–284). San Diego, CA: Elsevier Academic Press.

Karpicke, J. D., Blunt, J. R., & Smith, M. A. (2016). Retrieval-based learning: positive effects of retrieval practice in elementary school children. Frontiers in Psychology, 7 .

Kavale, K. A., Hirshoren, A., & Forness, S. R. (1998). Meta-analytic validation of the Dunn and Dunn model of learning-style preferences: a critique of what was Dunn. Learning Disabilities Research & Practice, 13 , 75–80.

Khanna, M. M. (2015). Ungraded pop quizzes: test-enhanced learning without all the anxiety. Teaching of Psychology, 42 , 174–178.

Kirby, J. (2014). One scientific insight for curriculum design [Blog post]. Retrieved from https://pragmaticreform.wordpress.com/2014/05/05/scientificcurriculumdesign/ . Accessed 25 Dec 2017.

Kirschner, P. A. (2017). Stop propagating the learning styles myth. Computers & Education, 106 , 166–171.

Kirschner, P. A., & van Merriënboer, J. J. G. (2013). Do learners really know best? Urban legends in education. Educational Psychologist, 48 , 169–183.

Knoll, A. R., Otani, H., Skeel, R. L., & Van Horn, K. R. (2017). Learning style, judgments of learning, and learning of verbal and visual information. British Journal of Psychology, 108 , 544-563.

Kornell, N., & Bjork, R. A. (2008). Learning concepts and categories is spacing the “enemy of induction”? Psychological Science, 19 , 585–592.

Kornell, N., & Finn, B. (2016). Self-regulated learning: an overview of theory and data. In J. Dunlosky & S. Tauber (Eds.), The Oxford Handbook of Metamemory (pp. 325–340). New York: Oxford University Press.

Kornell, N., Klein, P. J., & Rawson, K. A. (2015). Retrieval attempts enhance learning, but retrieval success (versus failure) does not matter. Journal of Experimental Psychology: Learning, Memory, and Cognition, 41 , 283–294.

Kraemer, D. J. M., Rosenberg, L. M., & Thompson-Schill, S. L. (2009). The neural correlates of visual and verbal cognitive styles. Journal of Neuroscience, 29 , 3792–3798.

Article PubMed PubMed Central Google Scholar

Kraft, N. (2015). Spaced practice and repercussions for teaching. Retrieved from http://nathankraft.blogspot.com/2015/08/spaced-practice-and-repercussions-for.html . Accessed 25 Dec 2017.

Learning Scientists. (2016a). Weekly Digest #3: How teachers implement interleaving in their curriculum [Blog post]. Retrieved from http://www.learningscientists.org/blog/2016/3/28/weekly-digest-3 . Accessed 25 Dec 2017.

Learning Scientists. (2016b). Weekly Digest #13: how teachers implement retrieval in their classrooms [Blog post]. Retrieved from http://www.learningscientists.org/blog/2016/6/5/weekly-digest-13 . Accessed 25 Dec 2017.

Learning Scientists. (2016c). Weekly Digest #40: teachers’ implementation of principles from “Make It Stick” [Blog post]. Retrieved from http://www.learningscientists.org/blog/2016/12/18-1 . Accessed 25 Dec 2017.

Learning Scientists. (2017). Weekly Digest #54: is there an app for that? Studying 2.0 [Blog post]. Retrieved from http://www.learningscientists.org/blog/2017/4/9/weekly-digest-54 . Accessed 25 Dec 2017.

LeFevre, J.-A., & Dixon, P. (1986). Do written instructions need examples? Cognition and Instruction, 3 , 1–30.

Lew, K., Fukawa-Connelly, T., Mejí-Ramos, J. P., & Weber, K. (2016). Lectures in advanced mathematics: Why students might not understand what the mathematics professor is trying to convey. Journal of Research in Mathematics Education, 47 , 162–198.

Lindsey, R. V., Shroyer, J. D., Pashler, H., & Mozer, M. C. (2014). Improving students’ long-term knowledge retention through personalized review. Psychological Science, 25 , 639–647.

Lipko-Speed, A., Dunlosky, J., & Rawson, K. A. (2014). Does testing with feedback help grade-school children learn key concepts in science? Journal of Applied Research in Memory and Cognition, 3 , 171–176.

Lockhart, R. S., & Craik, F. I. (1990). Levels of processing: a retrospective commentary on a framework for memory research. Canadian Journal of Psychology, 44 , 87–112.

Lovell, O. (2017). How do we know what to put on the quiz? [Blog Post]. Retrieved from http://www.ollielovell.com/olliesclassroom/know-put-quiz/ . Accessed 25 Dec 2017.

Luehmann, A. L. (2008). Using blogging in support of teacher professional identity development: a case study. The Journal of the Learning Sciences, 17 , 287–337.

Madan, C. R., Glaholt, M. G., & Caplan, J. B. (2010). The influence of item properties on association-memory. Journal of Memory and Language, 63 , 46–63.

Madan, C. R., & Singhal, A. (2012a). Motor imagery and higher-level cognition: four hurdles before research can sprint forward. Cognitive Processing, 13 , 211–229.

Madan, C. R., & Singhal, A. (2012b). Encoding the world around us: motor-related processing influences verbal memory. Consciousness and Cognition, 21 , 1563–1570.

Madan, C. R., & Singhal, A. (2012c). Using actions to enhance memory: effects of enactment, gestures, and exercise on human memory. Frontiers in Psychology, 3 .

Madan, C. R., Chen, Y. Y., & Singhal, A. (2016). ERPs differentially reflect automatic and deliberate processing of the functional manipulability of objects. Frontiers in Human Neuroscience, 10 .

Mandler, G. (1979). Organization and repetition: organizational principles with special reference to rote learning. In L. G. Nilsson (Ed.), Perspectives on Memory Research (pp. 293–327). New York: Academic Press.

Marsh, E. J., Fazio, L. K., & Goswick, A. E. (2012). Memorial consequences of testing school-aged children. Memory, 20 , 899–906.

Mayer, R. E., & Gallini, J. K. (1990). When is an illustration worth ten thousand words? Journal of Educational Psychology, 82 , 715–726.

Mayer, R. E., & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist, 38 , 43–52.

McDaniel, M. A., & Donnelly, C. M. (1996). Learning with analogy and elaborative interrogation. Journal of Educational Psychology, 88 , 508–519.

McDaniel, M. A., Thomas, R. C., Agarwal, P. K., McDermott, K. B., & Roediger, H. L. (2013). Quizzing in middle-school science: successful transfer performance on classroom exams. Applied Cognitive Psychology, 27 , 360–372.

McDermott, K. B., Agarwal, P. K., D’Antonio, L., Roediger, H. L., & McDaniel, M. A. (2014). Both multiple-choice and short-answer quizzes enhance later exam performance in middle and high school classes. Journal of Experimental Psychology: Applied, 20 , 3–21.

McHugh, A. (2013). High-stakes tests: bad for students, teachers, and education in general [Blog post]. Retrieved from https://teacherbiz.wordpress.com/2013/07/01/high-stakes-tests-bad-for-students-teachers-and-education-in-general/ . Accessed 25 Dec 2017.

McNeill, N. M., Uttal, D. H., Jarvin, L., & Sternberg, R. J. (2009). Should you show me the money? Concrete objects both hurt and help performance on mathematics problems. Learning and Instruction, 19 , 171–184.

Meider, W. (1990). “A picture is worth a thousand words”: from advertising slogan to American proverb. Southern Folklore, 47 , 207–225.

Michaela Community School. (2014). Homework. Retrieved from http://mcsbrent.co.uk/homework-2/ . Accessed 25 Dec 2017.

Montefinese, M., Ambrosini, E., Fairfield, B., & Mammarella, N. (2013). The “subjective” pupil old/new effect: is the truth plain to see? International Journal of Psychophysiology, 89 , 48–56.

O’Neil, H. F., Chung, G. K., Kerr, D., Vendlinski, T. P., Buschang, R. E., & Mayer, R. E. (2014). Adding self-explanation prompts to an educational computer game. Computers In Human Behavior, 30 , 23–28.

Overoye, A. L., & Storm, B. C. (2015). Harnessing the power of uncertainty to enhance learning. Translational Issues in Psychological Science, 1 , 140–148.

Paivio, A. (1971). Imagery and verbal processes . New York: Holt, Rinehart and Winston.

Paivio, A. (1986). Mental representations: a dual coding approach . New York: Oxford University Press.

Paivio, A. (2007). Mind and its evolution: a dual coding theoretical approach . Mahwah: Erlbaum.

Paivio, A. (2013). Dual coding theory, word abstractness, and emotion: a critical review of Kousta et al. (2011). Journal of Experimental Psychology: General, 142 , 282–287.

Paivio, A., & Csapo, K. (1969). Concrete image and verbal memory codes. Journal of Experimental Psychology, 80 , 279–285.

Paivio, A., & Csapo, K. (1973). Picture superiority in free recall: imagery or dual coding? Cognitive Psychology, 5 , 176–206.

Paivio, A., Walsh, M., & Bons, T. (1994). Concreteness effects on memory: when and why? Journal of Experimental Psychology: Learning, Memory, and Cognition, 20 , 1196–1204.

Pashler, H., McDaniel, M., Rohrer, D., & Bjork, R. (2008). Learning styles: concepts and evidence. Psychological Science in the Public Interest, 9 , 105–119.

Pashler, H., Bain, P. M., Bottge, B. A., Graesser, A., Koedinger, K., McDaniel, M., & Metcalfe, J. (2007). Organizing instruction and study to improve student learning. IES practice guide. NCER 2007–2004. National Center for Education Research .

Patel, R., Liu, R., & Koedinger, K. (2016). When to block versus interleave practice? Evidence against teaching fraction addition before fraction multiplication. In Proceedings of the 38th Annual Meeting of the Cognitive Science Society, Philadelphia, PA .

Penfound, B. (2017). Journey to interleaved practice #2 [Blog Post]. Retrieved from https://fullstackcalculus.com/2017/02/03/journey-to-interleaved-practice-2/ . Accessed 25 Dec 2017.

Penfound, B. [BryanPenfound]. (2016). Does blocked practice/learning lessen cognitive load? Does interleaved practice/learning provide productive struggle? [Tweet]. Retrieved from https://twitter.com/BryanPenfound/status/808759362244087808 . Accessed 25 Dec 2017.

Peterson, D. J., & Mulligan, N. W. (2010). Enactment and retrieval. Memory & Cognition, 38 , 233–243.

Picciotto, H. (2009). Lagging homework [Blog post]. Retrieved from http://blog.mathedpage.org/2013/06/lagging-homework.html . Accessed 25 Dec 2017.

Pomerance, L., Greenberg, J., & Walsh, K. (2016). Learning about learning: what every teacher needs to know. Retrieved from http://www.nctq.org/dmsView/Learning_About_Learning_Report . Accessed 25 Dec 2017.

Postman, L. (1976). Methodology of human learning. In W. K. Estes (Ed.), Handbook of learning and cognitive processes (Vol. 3). Hillsdale: Erlbaum.

Pressley, M., McDaniel, M. A., Turnure, J. E., Wood, E., & Ahmad, M. (1987). Generation and precision of elaboration: effects on intentional and incidental learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 13 , 291–300.

Reed, S. K. (2008). Concrete examples must jibe with experience. Science, 322 , 1632–1633.

researchED. (2013). How it all began. Retrieved from http://www.researched.org.uk/about/our-story/ . Accessed 25 Dec 2017.

Ritchie, S. J., Della Sala, S., & McIntosh, R. D. (2013). Retrieval practice, with or without mind mapping, boosts fact learning in primary school children. PLoS One, 8 (11), e78976.

Rittle-Johnson, B. (2006). Promoting transfer: effects of self-explanation and direct instruction. Child Development, 77 , 1–15.

Roediger, H. L. (1985). Remembering Ebbinghaus. [Retrospective review of the book On Memory , by H. Ebbinghaus]. Contemporary Psychology, 30 , 519–523.

Roediger, H. L. (2013). Applying cognitive psychology to education translational educational science. Psychological Science in the Public Interest, 14 , 1–3.

Roediger, H. L., & Karpicke, J. D. (2006). The power of testing memory: basic research and implications for educational practice. Perspectives on Psychological Science, 1 , 181–210.

Roediger, H. L., Putnam, A. L., & Smith, M. A. (2011). Ten benefits of testing and their applications to educational practice. In J. Mester & B. Ross (Eds.), The psychology of learning and motivation: cognition in education (pp. 1–36). Oxford: Elsevier.

Roediger, H. L., Finn, B., & Weinstein, Y. (2012). Applications of cognitive science to education. In Della Sala, S., & Anderson, M. (Eds.), Neuroscience in education: the good, the bad, and the ugly . Oxford, UK: Oxford University Press.

Roelle, J., & Berthold, K. (2017). Effects of incorporating retrieval into learning tasks: the complexity of the tasks matters. Learning and Instruction, 49 , 142–156.

Rohrer, D. (2012). Interleaving helps students distinguish among similar concepts. Educational Psychology Review, 24(3), 355–367.

Rohrer, D., Dedrick, R. F., & Stershic, S. (2015). Interleaved practice improves mathematics learning. Journal of Educational Psychology, 107 , 900–908.

Rohrer, D., & Pashler, H. (2012). Learning styles: Where’s the evidence? Medical Education, 46 , 34–35.

Rohrer, D., & Taylor, K. (2007). The shuffling of mathematics problems improves learning. Instructional Science, 35 , 481–498.

Rose, N. (2014). Improving the effectiveness of homework [Blog post]. Retrieved from https://evidenceintopractice.wordpress.com/2014/03/20/improving-the-effectiveness-of-homework/ . Accessed 25 Dec 2017.

Sadoski, M. (2005). A dual coding view of vocabulary learning. Reading & Writing Quarterly, 21 , 221–238.

Saunders, K. (2016). It really is time we stopped talking about learning styles [Blog post]. Retrieved from http://martingsaunders.com/2016/10/it-really-is-time-we-stopped-talking-about-learning-styles/ . Accessed 25 Dec 2017.

Schwartz, D. (2007). If a picture is worth a thousand words, why are you reading this essay? Social Psychology Quarterly, 70 , 319–321.

Shumaker, H. (2016). Homework is wrecking our kids: the research is clear, let’s ban elementary homework. Salon. Retrieved from http://www.salon.com/2016/03/05/homework_is_wrecking_our_kids_the_research_is_clear_lets_ban_elementary_homework . Accessed 25 Dec 2017.

Smith, A. M., Floerke, V. A., & Thomas, A. K. (2016). Retrieval practice protects memory against acute stress. Science, 354 , 1046–1048.

Smith, M. A., Blunt, J. R., Whiffen, J. W., & Karpicke, J. D. (2016). Does providing prompts during retrieval practice improve learning? Applied Cognitive Psychology, 30 , 784–802.

Smith, M. A., & Karpicke, J. D. (2014). Retrieval practice with short-answer, multiple-choice, and hybrid formats. Memory, 22 , 784–802.

Smith, M. A., Roediger, H. L., & Karpicke, J. D. (2013). Covert retrieval practice benefits retention as much as overt retrieval practice. Journal of Experimental Psychology: Learning, Memory, and Cognition, 39 , 1712–1725.

Son, J. Y., & Rivas, M. J. (2016). Designing clicker questions to stimulate transfer. Scholarship of Teaching and Learning in Psychology, 2 , 193–207.

Szpunar, K. K., Khan, N. Y., & Schacter, D. L. (2013). Interpolated memory tests reduce mind wandering and improve learning of online lectures. Proceedings of the National Academy of Sciences, 110 , 6313–6317.

Thomson, R., & Mehring, J. (2016). Better vocabulary study strategies for long-term learning. Kwansei Gakuin University Humanities Review, 20 , 133–141.

Trafton, J. G., & Reiser, B. J. (1993). Studying examples and solving problems: contributions to skill acquisition . Technical report, Naval HCI Research Lab, Washington, DC, USA.

Tran, R., Rohrer, D., & Pashler, H. (2015). Retrieval practice: the lack of transfer to deductive inferences. Psychonomic Bulletin & Review, 22 , 135–140.

Turner, K. [doc_kristy]. (2016a). My dual coding (in red) and some y8 work @AceThatTest they really enjoyed practising the technique [Tweet]. Retrieved from https://twitter.com/doc_kristy/status/807220355395977216 . Accessed 25 Dec 2017.

Turner, K. [doc_kristy]. (2016b). @FurtherEdagogy @doctorwhy their work is revision work, they already have the words on a different page, to compliment not replace [Tweet]. Retrieved from https://twitter.com/doc_kristy/status/807360265100599301 . Accessed 25 Dec 2017.

Valle, A., Regueiro, B., Núñez, J. C., Rodríguez, S., Piñeiro, I., & Rosário, P. (2016). Academic goals, student homework engagement, and academic achievement in elementary school. Frontiers in Psychology, 7 .

Van Gog, T., & Sweller, J. (2015). Not new, but nearly forgotten: the testing effect decreases or even disappears as the complexity of learning materials increases. Educational Psychology Review, 27 , 247–264.

Wammes, J. D., Meade, M. E., & Fernandes, M. A. (2016). The drawing effect: evidence for reliable and robust memory benefits in free recall. Quarterly Journal of Experimental Psychology, 69 , 1752–1776.

Weinstein, Y., Gilmore, A. W., Szpunar, K. K., & McDermott, K. B. (2014). The role of test expectancy in the build-up of proactive interference in long-term memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 40 , 1039–1048.

Weinstein, Y., Nunes, L. D., & Karpicke, J. D. (2016). On the placement of practice questions during study. Journal of Experimental Psychology: Applied, 22 , 72–84.

Weinstein, Y., & Weinstein-Jones, F. (2017). Topic and quiz spacing spreadsheet: a planning tool for teachers [Blog Post]. Retrieved from http://www.learningscientists.org/blog/2017/5/11-1 . Accessed 25 Dec 2017.

Weinstein-Jones, F., & Weinstein, Y. (2017). Topic spacing spreadsheet for teachers [Excel macro]. Zenodo. http://doi.org/10.5281/zenodo.573764 . Accessed 25 Dec 2017.

Williams, D. [FurtherEdagogy]. (2016). @doctorwhy @doc_kristy word accompanying the visual? I’m unclear how removing words benefit? Would a flow chart better suit a scientific exp? [Tweet]. Retrieved from https://twitter.com/FurtherEdagogy/status/807356800509104128 . Accessed 25 Dec 2017.

Wood, B. (2017). And now for something a little bit different….[Blog post]. Retrieved from https://justateacherstandinginfrontofaclass.wordpress.com/2017/04/20/and-now-for-something-a-little-bit-different/ . Accessed 25 Dec 2017.

Wooldridge, C. L., Bugg, J. M., McDaniel, M. A., & Liu, Y. (2014). The testing effect with authentic educational materials: a cautionary note. Journal of Applied Research in Memory and Cognition, 3 , 214–221.

Young, C. (2016). Mini-tests. Retrieved from https://colleenyoung.wordpress.com/revision-activities/mini-tests/ . Accessed 25 Dec 2017.

Download references

Acknowledgements

Not applicable.

YW and MAS were partially supported by a grant from The IDEA Center.

Availability of data and materials

Author information, authors and affiliations.

Department of Psychology, University of Massachusetts Lowell, Lowell, MA, USA

Yana Weinstein

Department of Psychology, Boston College, Chestnut Hill, MA, USA

Christopher R. Madan

School of Psychology, University of Nottingham, Nottingham, UK

Department of Psychology, Rhode Island College, Providence, RI, USA

Megan A. Sumeracki

You can also search for this author in PubMed Google Scholar

Contributions

YW took the lead on writing the “Spaced practice”, “Interleaving”, and “Elaboration” sections. CRM took the lead on writing the “Concrete examples” and “Dual coding” sections. MAS took the lead on writing the “Retrieval practice” section. All authors edited each others’ sections. All authors were involved in the conception and writing of the manuscript. All authors gave approval of the final version.

Corresponding author

Correspondence to Yana Weinstein .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

YW and MAS run a blog, “The Learning Scientists Blog”, which is cited in the tutorial review. The blog does not make money. Free resources on the strategies described in this tutorial review are provided on the blog. Occasionally, YW and MAS are invited by schools/school districts to present research findings from cognitive psychology applied to education.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Cite this article.

Weinstein, Y., Madan, C.R. & Sumeracki, M.A. Teaching the science of learning. Cogn. Research 3 , 2 (2018). https://doi.org/10.1186/s41235-017-0087-y

Download citation

Received : 20 December 2016

Accepted : 02 December 2017

Published : 24 January 2018

DOI : https://doi.org/10.1186/s41235-017-0087-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Community and Engagement
Honors and Awards
Give Now

Why Is It Important for Students to Understand How Scientific Decisions are Made? ‘If You Don’t Understand How Scientists Decide What Makes One Claim More Believable, Then It’s Actually Very Hard to Understand Science,’ Says STEM Education Department Head William Sandoval

For William Sandoval, head of the Department of STEM Education in the NC State College of Education, when preparing K-12 students to engage with real-world science, developing the skills to become career scientists is not nearly as important as helping them to engage with the science that will occur all around them in their everyday lives.

To help facilitate this, Sandoval’s research has focused on how kids understand scientific argumentation and how scientists make the case for why people should believe the theories they’ve developed based on causal claims and evidence.

The concept is known as epistemic cognition, or thinking about how people know what they know.

“If we want kids to understand how science works, they have to understand the standards that scientists use to evaluate competing claims. If you don’t understand how scientists decide what makes one claim more believable, then it’s actually very hard to understand science,” Sandoval said. “Every person who gets a science education through high school is eventually going to encounter some scientific issue in their adult lives that they didn’t learn about in school because science moves really fast. If you never thought about the way the scientific community makes choices, it’s very hard to make sense of it all as an individual citizen.”

How Do Kids Typically Think About Science?

Broadly speaking, research shows that, oftentimes, kids’ initially find many scientific concepts to be implausible because so many of the causal agents behind these theories are invisible and insensible.

“You can’t feel your genes; you can’t feel an atom, so there are these causal agents in science that are very far away from our sensory experience, which makes them hard to understand,” Sandoval said.

Despite this, research shows that children, even from a young age, believe that it’s much better to have evidence for a claim than to not have evidence. Therefore, teachers can draw on this by helping students understand the evidence behind the science they are learning through engagement in research, data collection or experimentation in the classroom.

How Can Teachers Help Students Understand How Scientific Decisions Are Made?

At its core, Sandoval said, science is all about evaluating competing claims about how the world works.

One of the best ways to help students understand how to do this is to have them engage in scientific argumentation. Sandoval shared the following steps to help students in this endeavor:

Identify topics within the curriculum that can elicit disagreement ; this can vary from asking elementary students how plants get water to how they inherit genetic traits from their parents: “Identify for everybody when there is disagreement and try to clarify what the nature of the disagreement is. Then, discuss with students how they are going to decide [which claim is valid] and what it will take for everyone to agree.”
Give students an opportunity to come up with their own evidence-based claims: “Our current national standards want kids to be doing investigations of various kinds, so these are really good opportunities for kids to engage in these epistemic considerations about how we’re going to decide what we’re going to believe about this thing that we’re studying. Because, if they’re getting data themselves, they can argue about not just what the data show but how you got the data, whether you did it in a reasonable way or not and whether you’ve interpreted the data in a reasonable way.”
Have students with opposing claims engage in discussion: “We found that when this is public, that works better and part of the reason is that this holds kids accountable to each other. They also have to be held accountable to standards of evidence. So, that’s a really important role for the teacher to play is to kind of push kids to talk about what evidence they have for this claim over that claim.”
Come to a consensus: “Talk about the standards or the criteria that you’d use to resolve your disagreement. It’s not enough that I believe one thing and someone else believes another thing. You’ve got to push us to agree, so pushing to consensus seems to be the thing that really helps kids work through their understanding of the criteria.”

For teachers who want to engage their students in this type of scientific thinking, Sandoval recommends using free resources provided by the NGSX professional learning system along with the inquiryHub ,a research-practice partnership that develops materials, tools and processes to promote STEM learning.

Research and Impact
Ask The Expert
homepage-news
resources for educators
science education
STEM Ed Research
William Sandoval

More From College of Education News

Catherine Hartman Named Assistant Professor of Community College Leadership

Leading the Way

An Unforgettable Adventure: Meet Reginald Simon, MSA ‘24

Along with Stanford news and stories, show me:

Student information
Faculty/Staff information

We want to provide announcements, events, leadership messages and resources that are relevant to you. Your selection is stored in a browser cookie which you can remove at any time using “Clear all personalization” below.

Image credit: Claire Scully

New advances in technology are upending education, from the recent debut of new artificial intelligence (AI) chatbots like ChatGPT to the growing accessibility of virtual-reality tools that expand the boundaries of the classroom. For educators, at the heart of it all is the hope that every learner gets an equal chance to develop the skills they need to succeed. But that promise is not without its pitfalls.

“Technology is a game-changer for education – it offers the prospect of universal access to high-quality learning experiences, and it creates fundamentally new ways of teaching,” said Dan Schwartz, dean of Stanford Graduate School of Education (GSE), who is also a professor of educational technology at the GSE and faculty director of the Stanford Accelerator for Learning . “But there are a lot of ways we teach that aren’t great, and a big fear with AI in particular is that we just get more efficient at teaching badly. This is a moment to pay attention, to do things differently.”

For K-12 schools, this year also marks the end of the Elementary and Secondary School Emergency Relief (ESSER) funding program, which has provided pandemic recovery funds that many districts used to invest in educational software and systems. With these funds running out in September 2024, schools are trying to determine their best use of technology as they face the prospect of diminishing resources.

Here, Schwartz and other Stanford education scholars weigh in on some of the technology trends taking center stage in the classroom this year.

AI in the classroom

In 2023, the big story in technology and education was generative AI, following the introduction of ChatGPT and other chatbots that produce text seemingly written by a human in response to a question or prompt. Educators immediately worried that students would use the chatbot to cheat by trying to pass its writing off as their own. As schools move to adopt policies around students’ use of the tool, many are also beginning to explore potential opportunities – for example, to generate reading assignments or coach students during the writing process.

AI can also help automate tasks like grading and lesson planning, freeing teachers to do the human work that drew them into the profession in the first place, said Victor Lee, an associate professor at the GSE and faculty lead for the AI + Education initiative at the Stanford Accelerator for Learning. “I’m heartened to see some movement toward creating AI tools that make teachers’ lives better – not to replace them, but to give them the time to do the work that only teachers are able to do,” he said. “I hope to see more on that front.”

He also emphasized the need to teach students now to begin questioning and critiquing the development and use of AI. “AI is not going away,” said Lee, who is also director of CRAFT (Classroom-Ready Resources about AI for Teaching), which provides free resources to help teach AI literacy to high school students across subject areas. “We need to teach students how to understand and think critically about this technology.”

Immersive environments

The use of immersive technologies like augmented reality, virtual reality, and mixed reality is also expected to surge in the classroom, especially as new high-profile devices integrating these realities hit the marketplace in 2024.

The educational possibilities now go beyond putting on a headset and experiencing life in a distant location. With new technologies, students can create their own local interactive 360-degree scenarios, using just a cell phone or inexpensive camera and simple online tools.

“This is an area that’s really going to explode over the next couple of years,” said Kristen Pilner Blair, director of research for the Digital Learning initiative at the Stanford Accelerator for Learning, which runs a program exploring the use of virtual field trips to promote learning. “Students can learn about the effects of climate change, say, by virtually experiencing the impact on a particular environment. But they can also become creators, documenting and sharing immersive media that shows the effects where they live.”

Integrating AI into virtual simulations could also soon take the experience to another level, Schwartz said. “If your VR experience brings me to a redwood tree, you could have a window pop up that allows me to ask questions about the tree, and AI can deliver the answers.”

Gamification

Another trend expected to intensify this year is the gamification of learning activities, often featuring dynamic videos with interactive elements to engage and hold students’ attention.

“Gamification is a good motivator, because one key aspect is reward, which is very powerful,” said Schwartz. The downside? Rewards are specific to the activity at hand, which may not extend to learning more generally. “If I get rewarded for doing math in a space-age video game, it doesn’t mean I’m going to be motivated to do math anywhere else.”

Gamification sometimes tries to make “chocolate-covered broccoli,” Schwartz said, by adding art and rewards to make speeded response tasks involving single-answer, factual questions more fun. He hopes to see more creative play patterns that give students points for rethinking an approach or adapting their strategy, rather than only rewarding them for quickly producing a correct response.

Data-gathering and analysis

The growing use of technology in schools is producing massive amounts of data on students’ activities in the classroom and online. “We’re now able to capture moment-to-moment data, every keystroke a kid makes,” said Schwartz – data that can reveal areas of struggle and different learning opportunities, from solving a math problem to approaching a writing assignment.

But outside of research settings, he said, that type of granular data – now owned by tech companies – is more likely used to refine the design of the software than to provide teachers with actionable information.

The promise of personalized learning is being able to generate content aligned with students’ interests and skill levels, and making lessons more accessible for multilingual learners and students with disabilities. Realizing that promise requires that educators can make sense of the data that’s being collected, said Schwartz – and while advances in AI are making it easier to identify patterns and findings, the data also needs to be in a system and form educators can access and analyze for decision-making. Developing a usable infrastructure for that data, Schwartz said, is an important next step.

With the accumulation of student data comes privacy concerns: How is the data being collected? Are there regulations or guidelines around its use in decision-making? What steps are being taken to prevent unauthorized access? In 2023 K-12 schools experienced a rise in cyberattacks, underscoring the need to implement strong systems to safeguard student data.

Technology is “requiring people to check their assumptions about education,” said Schwartz, noting that AI in particular is very efficient at replicating biases and automating the way things have been done in the past, including poor models of instruction. “But it’s also opening up new possibilities for students producing material, and for being able to identify children who are not average so we can customize toward them. It’s an opportunity to think of entirely new ways of teaching – this is the path I hope to see.”

Exploring the effect of directive and reflective feedback on elementary school students’ scientific conceptual understanding, epistemological beliefs, and inquiry performance in online inquiry activities

Published: 23 May 2024

Cite this article

Yafeng Zheng 1 ,
Shebing Sun 2 ,
Yang Yang 1 &
Chang Xu ORCID: orcid.org/0000-0002-0166-3143 3

This study explored the differences in the effects of directive versus reflective feedback on elementary school students’ conceptual understanding and epistemological beliefs in a simulation-based online science inquiry environment, by comparing more fine-grained differences in the inquiry behaviors of students who received the two types of feedback during the online scientific inquiry activity. Seventy-one 5th graders from a public elementary school in China participated in the study (38 students receiving directive feedback and 33 receiving reflective feedback). The results showed no significant difference between the two types of feedback on the students’ conceptual understanding and epistemological beliefs. However, reflective feedback outperformed directive feedback on the subdimension of source of epistemological beliefs, and directive feedback outperformed reflective feedback on the argumentation subdimension. In addition, the average inquiry time of the students receiving reflective feedback during the review session was significantly longer than that of the students receiving directive feedback. These results provide important insights for elementary teachers to implement feedback in science classrooms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Akaygun, S., & Adadan, E. (2019). Revisiting the understanding of redox reactions through critiquing animations in variance. In Research and Practice in Chemistry Education: Advances from the 25th IUPAC International Conference on Chemistry Education 2018 (pp. 7–29). Springer Singapore. https://doi.org/10.1007/978-981-13-6998-8_2 (Reprinted).

Al-Balushi, S. M., Al-Musawi, A. S., Ambusaidi, A. K., & Al-Hajri, F. H. (2017). The effectiveness of interacting with scientific animations in chemistry using mobile devices on grade 12 students’ spatial ability and scientific reasoning skills. Journal of Science Education and Technology , 26 (1), 70–81. https://doi.org/10.1007/s10956-016-9652-2 .

Article Google Scholar

Andujar, J. M., Mejías, A., & Márquez, M. A. (2011). Augmented reality for the improvement of remote laboratories: An augmented remote laboratory. IEEE Transactions on Education , 54 (3), 492–500. https://doi.org/10.1109/TE.2010.2085047 .

Attali, Y., & Powers, D. (2010). Immediate feedback and opportunity to revise answers to open-ended questions. Educational and Psychological Measurement , 70 (1), 22–35. https://doi.org/10.1177/0013164409332231 .

Beneroso, D., & Robinson, J. (2021). A tool for assessing and providing personalised formative feedback at scale within a second in engineering courses. Education for Chemical Engineers , 36 (1), 38–45. https://doi.org/10.1016/j.ece.2021.02.002 .

Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles Policy & Practice , 5 (1), 7–74. https://doi.org/10.1080/0969595980050102 .

Boramy, S. (2010). Using directive and facilitative feedback to improve student writing: A case study of a higher education setting in Cambodia. Language Education in Asia , 1 (1), 23–47. https://doi.org/10.5746/LEiA/10/V1/A04/Sou .

Chen, J. A., & Pajares, F. (2010). Implicit theories of ability of Grade 6 science students: Relation to epistemological beliefs and academic motivation and achievement in science. Contemporary Educational Psychology , 35 (1), 75–87. https://doi.org/10.1016/j.cedpsych.2009.10.003 .

Chen, C. M., & Wang, W. F. (2020). Mining effective learning behaviors in a web-based inquiry science environment. Journal of Science Education and Technology , 29 (4), 519–535. https://doi.org/10.1007/s10956-020-09833-9 .

Chen, J., Wang, M., Kirschner, P. A., & Tsai, C. C. (2018). The role of collaboration, computer use, learning environments, and supporting strategies in CSCL: A meta-analysis. Review of Educational Research , 88 (6), 799–843. https://doi.org/10.3102/0034654318791584 .

Chen, C. H., Huang, K., & Liu, J. H. (2020a). Inquiry-enhanced digital game-based learning: Effects on secondary students’ conceptual understanding in science, game performance, and behavioral patterns. The Asia-Pacific Education Researcher , 29 (4), 319–330. https://doi.org/10.1007/s40299-019-00486-w .

Chi, S., Wang, Z., & Liu, X. (2021). Moderating effects of teacher feedback on the associations among inquiry-based science practices and students’ science-related attitudes and beliefs. International Journal of Science Education , 43 (14), 2426–2456. https://doi.org/10.1080/09500693.2021.1968532 .

Chin, C., & Osborne, J. (2010). Students’ questions and discursive interaction: Their impact on argumentation during collaborative group discussions in science. Journal of Research in Science Teaching , 47 (7), 883–908. https://doi.org/10.1002/tea.20385 .

Conley, A. M., Pintrich, P. R., Vekiri, I., & Harrison, D. (2004). Changes in epistemological beliefs in elementary science students. Contemporary Educational Psychology , 29 (2), 186–204. https://doi.org/10.1016/j.cedpsych.2004.01.004 .

Demastes, S. S., Good, R. G., & Peebles, P. (1995). Students’ conceptual ecologies and the process of conceptual change in evolution. Science Education , 79 (6), 637–666. https://doi.org/10.1002/sce.3730790605 .

Dunn, J., & Ramnarain, U. (2020). The effect of simulation-supported inquiry on South African natural sciences learners’ understanding of atomic and molecular structures. Education Sciences , 10 (10), 280. https://doi.org/10.3390/educsci10100280 .

Efstathiou, C., Hovardas, T., Xenofontos, N. A., Zacharia, Z. C., deJong, T., Anjewierden, A., & van Riesen, S. A. (2018). Providing guidance in virtual lab experimentation: The case of an experiment design tool. Educational Technology Research and Development , 66 (3), 767–791. https://doi.org/10.1007/s11423-018-9576-z .

Furtak, E. M., Seidel, T., Iverson, H., & Briggs, D. C. (2012). Experimental and quasi-experimental studies of inquiry-based science teaching: A meta-analysis. Review of Educational Research , 82 (3), 300–329. https://doi.org/10.3102/0034654312457206 .

Gobert, J. D., Pedro, S., Raziuddin, M., J., & Baker, R. S. (2013). From log files to assessment metrics: Measuring students’ science inquiry skills using educational data mining. Journal of the Learning Sciences , 22 (4), 521–563. https://doi.org/10.1080/10508406.2013.837391 .

Green, N. L. (2013). Towards formative feedback on student arguments . AIED Workshops.

Heradio, R., De La Torre, L., Galan, D., Cabrerizo, F. J., Herrera-Viedma, E., & Dormido, S. (2016). Virtual and remote labs in education: A bibliometric analysis. Computers & Education , 98 (1), 14–38. https://doi.org/10.1016/j.compedu.2016.03.010 .

Higgins, S., & Simpson, A. (2011). Visible learning: A synthesis of over 800 Meta-analyses relating to achievement. By John A.C. Hattie. British Journal of Educational Studies , 59 (2), 197–201. https://doi.org/10.1080/00071005.2011.584660 .

Jansen, M., Lüdtke, O., & Schroeders, U. (2016). Evidence for a positive relation between interest and achievement: Examining between- person and within-person variation in five domains. Contemporary Educational Psychology , 46 , 116–127. https://doi.org/10.1016/j.cedpsy .

Keselman, H. J., Huberty, C. J., Lix, L. M., Olejnik, S., Cribbie, R. A., Donahue, B., Kowalchuk, R. K., Lowman, L. L., Petoskey, M. D., Keselman, J. C., & Levin, J. R. (1998). Statistical practices of educational researchers: An analysis of their ANOVA, MANOVA, and ANCOVA analyses. Review of Educational Research , 68 (3), 350–386. https://doi.org/10.3102/00346543068003350 .

Khalaf, B. K., & Mohammed Zin, Z. B. (2018). Traditional and inquiry-based learning pedagogy: A systematic critical review. International Journal of Instruction , 11 (4), 545–564. https://doi.org/10.12973/iji.2018.11434a .

Kizilgunes, B., Tekkaya, C., & Sungur, S. (2009). Modeling the relations among students’ epistemological beliefs, motivation, learning approach, and achievement. The Journal of Educational Research , 102 (4), 243–256. https://doi.org/10.3200/JOER.102.4.243-256 .

Knapp, T. R., & Schafer, W. D. (2009). From gain score t to ANCOVA F (and vice versa). Practical Assessment Research and Evaluation , 14 (1), 6. https://doi.org/10.7275/yke1-k937 .

Koedinger, K. R., McLaughlin, E. A., & Heffernan, N. T. (2010). A quasi-experimental evaluation of an on-line formative assessment and tutoring system. Journal of Educational Computing Research , 43 (4), 489–510. https://doi.org/10.2190/EC.43.4.d .

Lazonder, A. W., & Egberink, A. (2014). Children’s acquisition and use of the control-of-variables strategy: Effects of explicit and implicit instructional guidance. Instructional Science , 42 , 291–304. https://doi.org/10.1007/s11251-013-9284-3 .

Lee, H. S., Pallant, A., Pryputniewicz, S., Lord, T., Mulholland, M., & Liu, O. L. (2019). Automated text scoring and real-time adjustable feedback: Supporting revision of scientific arguments involving uncertainty. Science Education , 103 (3), 590–622. https://doi.org/10.1002/sce.21504 .

Liang, J. C., & Tsai, C. C. (2010). Relational analysis of college science-major students’ epistemological beliefs toward science and conceptions of learning science. International Journal of Science Education , 32 (17), 2273–2289. https://doi.org/10.1080/09500690903397796 .

Maier, U., Wolf, N., & Randler, C. (2016). Effects of a computer-assisted formative assessment intervention based on multiple-tier diagnostic items and different feedback types. Computers & Education , 95 , 85–98. https://doi.org/10.1016/j.compedu.2015.12.002 .

McElhaney, K. W., Chang, H. Y., Chiu, J. L., & Linn, M. C. (2015). Evidence for effective uses of dynamic visualisations in science curriculum materials. Studies in Science Education , 51 (1), 49–85. https://doi.org/10.1080/03057267.2014.984506 .

OECD (2016). PISA 2015 assessment and analytical framework: Science, reading, mathematic and financial literacy. Organisation for Economic Co-operation and Development. https://www.oecd-ilibrary.org/education/pisa-2015-assessment-and-analytical-framework_978926 4255425-en.

Özbay, H. E., & Köksal, M. S. (2021). Middle school students’ scientific epistemological beliefs, achievements in science and intellectual risk-taking. Science & Education , 30 (5), 1233–1252. https://doi.org/10.1007/s11191-021-00217-y .

Ryoo, K., & Linn, M. C. (2016). Designing automated guidance for concept diagrams in inquiry instruction. Journal of Research in Science Teaching , 53 (7), 1003–1035. https://doi.org/10.1002/tea.21321 .

Sadi, Ö., & Dağyar, M. (2015). High school students’ epistemological beliefs, conceptions of learning, and self-efficacy for learning biology: A study of their structural models. EURASIA Journal of Mathematics Science and Technology Education , 11 (5), 1061–1079. https://doi.org/10.12973/eurasia.2015.1375a .

Sandoval, W. A. (2005). Understanding students’ practical epistemologies and their influence on learning through inquiry. Science Education , 89 (4), 634–656. https://doi.org/10.1002/sce.20065 .

Sandoval, W. A., & Morrison, K. (2003). High school students’ ideas about theories and theory change after a biological inquiry unit. Journal of Research in Science Teaching , 40 (4), 369–392. https://doi.org/10.1002/tea.10081 .

Sari, R. P., Hasibuan, M. P., & Sulastri, S. (2021). The Effect of Feedback on the application of based Chemistry Practicum modules on local content-students’ science process skills. Jurnal Pendidikan Dan Pengajaran , 54 (1), 18–27. https://doi.org/10.23887/jpp.v54i1 .

Scalise, K. (2012). Science learning and instruction: Taking advantage of technology to promote knowledge integration. Science Education , 96 (6), 1136–1138. https://doi.org/10.1002/sce.21025 .

Scheiter, K., & Gerjets, P. (2007). Learner control in hypermedia environments. Educational Psychology Review , 19 (3), 285–307. https://doi.org/10.1007/s10648-007-9046-3 .

Schoppek, W., & Tulis, M. (2010). Enhancing arithmetic and word-problem solving skills efficiently by individualized computer-assisted practice. The Journal of Educational Research , 103 (4), 239–252. https://doi.org/10.1080/00220670903382962 .

Segedy, J. R., Kinnebrew, J. S., & Biswas, G. (2013). The effect of contextualized conversational feedback in a complex open-ended learning environment. Educational Technology Research and Development , 61 (1), 71–89. https://doi.org/10.1007/s11423-012-9275-0 .

Smith, C. L., Maclin, D., Houghton, C., & Hennessey, M. G. (2000). Sixth-grade students’ epistemologies of science: The impact of school science experiences on epistemological development. Cognition & Instruction , 18 (3), 349–422.

Sortkær, B. (2019). Feedback for everybody? Exploring the relationship between students’ perceptions of feedback and students’ socioeconomic status. British Educational Research Journal , 45 (4), 717–735. https://doi.org/10.1002/berj.3522 .

Srisawasdi, N., & Panjaburee, P. (2015). Exploring effectiveness of simulation-based inquiry learning in science with integration of formative assessment. Journal of Computers in Education , 2 (3), 323–352. https://doi.org/10.1007/s40692-015-0037-y .

Stathopoulou, C., & Vosniadou, S. (2007). Exploring the relationship between physics-related epistemological beliefs and physics understanding. Contemporary Educational Psychology , 32 (3), 255–281. https://doi.org/10.1016/j.cedpsych.2005.12.002 .

Sulisworo, D., Handayani, T., & Kusumaningtyas, D. A. (2019). The critical thinking effect of the computer simulation in the physics teaching and learning. Journal of Physics: Conference Peries , 1157 (3), 1–7. https://doi.org/10.1088/1742-6596/1157/3/032003 .

Tansomboon, C., Gerard, L. F., Vitale, J. M., & Linn, M. C. (2017). Designing automated guidance to promote productive revision of science explanations. International Journal of Artificial Intelligence in Education , 27 (4), 729–757. https://doi.org/10.1007/s40593-017-0145-0 .

Van der Kleij, F. M., Feskens, R. C., & Eggen, T. J. (2015). Effects of feedback in a computer-based learning environment on students’ learning outcomes: A meta-analysis. Review of Educational Research , 85 (4), 475–511. https://doi.org/10.3102/0034654314564881 .

Varma, K., & Linn, M. C. (2012). Using interactive technology to support students’ understanding of the greenhouse effect and global warming. Journal of Science Education and Technology , 21 (4), 453–464. https://doi.org/10.1007/s10956-011-9337-9 .

Wang, H. S., Chen, S., & Yen, M. H. (2021). Effects of metacognitive scaffolding on students’ performance and confidence judgments in simulation-based inquiry. Physical Review Physics Education Research , 17 (2). https://doi.org/10.1103/physrevphyseducres.17.020108 .

Wen, C. T., Chang, C. J., Chang, M. H., Chiang, F., Liu, S. H., Hwang, C. C., F. K., & Tsai, C. C. (2018). The learning analytics of model-based learning facilitated by a problem-solving simulation game. Instructional Science , 46 (6), 847–867. https://doi.org/10.1007/s11251-018-9461-5 .

Williams, P., Wray, J., Farrall, H., & Aspland, J. (2014). Fit for purpose: Traditional assessment is failing undergraduates with learning difficulties. Might eAssessment help? International Journal of Inclusive Education , 18 (6), 614–625. https://doi.org/10.1080/13603116.2013.802029 .

Windschitl, M., & Andre, T. (1998). Using computer simulations to enhance conceptual change: The roles of constructivist instruction and student epistemological beliefs. Journal of Research in Science Teaching , 35 (2), 145–160.

Wu, H. L., Weng, H. L., & She, H. C. (2016). Effects of scaffolds and scientific reasoning ability on web-based Scientific Inquiry. International Journal of Contemporary Educational Research , 3 (1), 12–24. https://doi.org/10.1103/PhysRevPhysEducRes.17.020108 .

Wu, J., Guo, R., Wang, Z., & Zeng, R. (2021). Integrating spherical video-based virtual reality into elementary school students’ scientific inquiry instruction: Effects on their problem-solving performance. Interactive Learning Environments , 29 (3), 496–509. https://doi.org/10.1080/10494820.2019.1587469 .

Zhu, M., Liu, O. L., & Lee, H. S. (2020). The effect of automated feedback on revision behavior and learning gains in formative assessment of scientific argument writing. Computers & Education , 143 (2), 1–15. https://doi.org/10.1016/j.compedu.2019.103668 .

Download references

This research was funded by the National Natural Science Foundation of China (number：62377005) and Guangdong Province Philosophy and Social Sciences Planning Project（number：GD24CJY04）.

Author information

Authors and affiliations.

Center for Educational Science and Technology, Beijing Normal University, Zhuhai, China

Yafeng Zheng & Yang Yang

Department of Computer and Artificial Intelligence, Shangqiu University Applied Science and Technology College, Kaifeng, China

Shebing Sun

School of International Education, Shandong University, Jinan, China

You can also search for this author in PubMed Google Scholar

Contributions

Yafeng Zheng: Conceptualization, Methodology, Writing - Original Draft, Funding acquisition. Shebing Sun: Formal analysis, Investigation, Writing - Original Draft. Yang Yang: Methodology, Writing - Review & Editing. Chang Xu: Conceptualization, Methodology, Writing - Review & Editing, Project administration.

Corresponding author

Correspondence to Chang Xu .

Ethics declarations

Conflict of interest.

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Zheng, Y., Sun, S., Yang, Y. et al. Exploring the effect of directive and reflective feedback on elementary school students’ scientific conceptual understanding, epistemological beliefs, and inquiry performance in online inquiry activities. Res Sci Educ (2024). https://doi.org/10.1007/s11165-024-10171-8

Download citation

Accepted : 03 May 2024

Published : 23 May 2024

DOI : https://doi.org/10.1007/s11165-024-10171-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Online Scientific Inquiry
Student Achievement
Directive Feedback
Reflective Feedback
Epistemological Beliefs
Find a journal
Publish with us
Track your research

Get science-backed answers as you write with Paperpal's Research feature

How to Write a Conclusion for Research Papers (with Examples)

The conclusion of a research paper is a crucial section that plays a significant role in the overall impact and effectiveness of your research paper. However, this is also the section that typically receives less attention compared to the introduction and the body of the paper. The conclusion serves to provide a concise summary of the key findings, their significance, their implications, and a sense of closure to the study. Discussing how can the findings be applied in real-world scenarios or inform policy, practice, or decision-making is especially valuable to practitioners and policymakers. The research paper conclusion also provides researchers with clear insights and valuable information for their own work, which they can then build on and contribute to the advancement of knowledge in the field.

The research paper conclusion should explain the significance of your findings within the broader context of your field. It restates how your results contribute to the existing body of knowledge and whether they confirm or challenge existing theories or hypotheses. Also, by identifying unanswered questions or areas requiring further investigation, your awareness of the broader research landscape can be demonstrated.

Remember to tailor the research paper conclusion to the specific needs and interests of your intended audience, which may include researchers, practitioners, policymakers, or a combination of these.

What is a conclusion in a research paper, summarizing conclusion, editorial conclusion, externalizing conclusion, importance of a good research paper conclusion, how to write a conclusion for your research paper, research paper conclusion examples.

How to write a research paper conclusion with Paperpal?

Frequently Asked Questions

A conclusion in a research paper is the final section where you summarize and wrap up your research, presenting the key findings and insights derived from your study. The research paper conclusion is not the place to introduce new information or data that was not discussed in the main body of the paper. When working on how to conclude a research paper, remember to stick to summarizing and interpreting existing content. The research paper conclusion serves the following purposes: 1

Warn readers of the possible consequences of not attending to the problem.
Recommend specific course(s) of action.
Restate key ideas to drive home the ultimate point of your research paper.
Provide a “take-home” message that you want the readers to remember about your study.

Types of conclusions for research papers

In research papers, the conclusion provides closure to the reader. The type of research paper conclusion you choose depends on the nature of your study, your goals, and your target audience. I provide you with three common types of conclusions:

A summarizing conclusion is the most common type of conclusion in research papers. It involves summarizing the main points, reiterating the research question, and restating the significance of the findings. This common type of research paper conclusion is used across different disciplines.

An editorial conclusion is less common but can be used in research papers that are focused on proposing or advocating for a particular viewpoint or policy. It involves presenting a strong editorial or opinion based on the research findings and offering recommendations or calls to action.

An externalizing conclusion is a type of conclusion that extends the research beyond the scope of the paper by suggesting potential future research directions or discussing the broader implications of the findings. This type of conclusion is often used in more theoretical or exploratory research papers.

Align your conclusion’s tone with the rest of your research paper. Start Writing with Paperpal Now!

The conclusion in a research paper serves several important purposes:

Offers Implications and Recommendations : Your research paper conclusion is an excellent place to discuss the broader implications of your research and suggest potential areas for further study. It’s also an opportunity to offer practical recommendations based on your findings.
Provides Closure : A good research paper conclusion provides a sense of closure to your paper. It should leave the reader with a feeling that they have reached the end of a well-structured and thought-provoking research project.
Leaves a Lasting Impression : Writing a well-crafted research paper conclusion leaves a lasting impression on your readers. It’s your final opportunity to leave them with a new idea, a call to action, or a memorable quote.

Writing a strong conclusion for your research paper is essential to leave a lasting impression on your readers. Here’s a step-by-step process to help you create and know what to put in the conclusion of a research paper: 2

Research Statement : Begin your research paper conclusion by restating your research statement. This reminds the reader of the main point you’ve been trying to prove throughout your paper. Keep it concise and clear.
Key Points : Summarize the main arguments and key points you’ve made in your paper. Avoid introducing new information in the research paper conclusion. Instead, provide a concise overview of what you’ve discussed in the body of your paper.
Address the Research Questions : If your research paper is based on specific research questions or hypotheses, briefly address whether you’ve answered them or achieved your research goals. Discuss the significance of your findings in this context.
Significance : Highlight the importance of your research and its relevance in the broader context. Explain why your findings matter and how they contribute to the existing knowledge in your field.
Implications : Explore the practical or theoretical implications of your research. How might your findings impact future research, policy, or real-world applications? Consider the “so what?” question.
Future Research : Offer suggestions for future research in your area. What questions or aspects remain unanswered or warrant further investigation? This shows that your work opens the door for future exploration.
Closing Thought : Conclude your research paper conclusion with a thought-provoking or memorable statement. This can leave a lasting impression on your readers and wrap up your paper effectively. Avoid introducing new information or arguments here.
Proofread and Revise : Carefully proofread your conclusion for grammar, spelling, and clarity. Ensure that your ideas flow smoothly and that your conclusion is coherent and well-structured.

Write your research paper conclusion 2x faster with Paperpal. Try it now!

Remember that a well-crafted research paper conclusion is a reflection of the strength of your research and your ability to communicate its significance effectively. It should leave a lasting impression on your readers and tie together all the threads of your paper. Now you know how to start the conclusion of a research paper and what elements to include to make it impactful, let’s look at a research paper conclusion sample.

How to write a research paper conclusion with Paperpal?

A research paper conclusion is not just a summary of your study, but a synthesis of the key findings that ties the research together and places it in a broader context. A research paper conclusion should be concise, typically around one paragraph in length. However, some complex topics may require a longer conclusion to ensure the reader is left with a clear understanding of the study’s significance. Paperpal, an AI writing assistant trusted by over 800,000 academics globally, can help you write a well-structured conclusion for your research paper.

Sign Up or Log In: Create a new Paperpal account or login with your details.
Navigate to Features : Once logged in, head over to the features’ side navigation pane. Click on Templates and you’ll find a suite of generative AI features to help you write better, faster.
Generate an outline: Under Templates, select ‘Outlines’. Choose ‘Research article’ as your document type.
Select your section: Since you’re focusing on the conclusion, select this section when prompted.
Choose your field of study: Identifying your field of study allows Paperpal to provide more targeted suggestions, ensuring the relevance of your conclusion to your specific area of research.
Provide a brief description of your study: Enter details about your research topic and findings. This information helps Paperpal generate a tailored outline that aligns with your paper’s content.
Generate the conclusion outline: After entering all necessary details, click on ‘generate’. Paperpal will then create a structured outline for your conclusion, to help you start writing and build upon the outline.
Write your conclusion: Use the generated outline to build your conclusion. The outline serves as a guide, ensuring you cover all critical aspects of a strong conclusion, from summarizing key findings to highlighting the research’s implications.
Refine and enhance: Paperpal’s ‘Make Academic’ feature can be particularly useful in the final stages. Select any paragraph of your conclusion and use this feature to elevate the academic tone, ensuring your writing is aligned to the academic journal standards.

By following these steps, Paperpal not only simplifies the process of writing a research paper conclusion but also ensures it is impactful, concise, and aligned with academic standards. Sign up with Paperpal today and write your research paper conclusion 2x faster .

The research paper conclusion is a crucial part of your paper as it provides the final opportunity to leave a strong impression on your readers. In the research paper conclusion, summarize the main points of your research paper by restating your research statement, highlighting the most important findings, addressing the research questions or objectives, explaining the broader context of the study, discussing the significance of your findings, providing recommendations if applicable, and emphasizing the takeaway message. The main purpose of the conclusion is to remind the reader of the main point or argument of your paper and to provide a clear and concise summary of the key findings and their implications. All these elements should feature on your list of what to put in the conclusion of a research paper to create a strong final statement for your work.

A strong conclusion is a critical component of a research paper, as it provides an opportunity to wrap up your arguments, reiterate your main points, and leave a lasting impression on your readers. Here are the key elements of a strong research paper conclusion: 1. Conciseness : A research paper conclusion should be concise and to the point. It should not introduce new information or ideas that were not discussed in the body of the paper. 2. Summarization : The research paper conclusion should be comprehensive enough to give the reader a clear understanding of the research’s main contributions. 3 . Relevance : Ensure that the information included in the research paper conclusion is directly relevant to the research paper’s main topic and objectives; avoid unnecessary details. 4 . Connection to the Introduction : A well-structured research paper conclusion often revisits the key points made in the introduction and shows how the research has addressed the initial questions or objectives. 5. Emphasis : Highlight the significance and implications of your research. Why is your study important? What are the broader implications or applications of your findings? 6 . Call to Action : Include a call to action or a recommendation for future research or action based on your findings.

The length of a research paper conclusion can vary depending on several factors, including the overall length of the paper, the complexity of the research, and the specific journal requirements. While there is no strict rule for the length of a conclusion, but it’s generally advisable to keep it relatively short. A typical research paper conclusion might be around 5-10% of the paper’s total length. For example, if your paper is 10 pages long, the conclusion might be roughly half a page to one page in length.

In general, you do not need to include citations in the research paper conclusion. Citations are typically reserved for the body of the paper to support your arguments and provide evidence for your claims. However, there may be some exceptions to this rule: 1. If you are drawing a direct quote or paraphrasing a specific source in your research paper conclusion, you should include a citation to give proper credit to the original author. 2. If your conclusion refers to or discusses specific research, data, or sources that are crucial to the overall argument, citations can be included to reinforce your conclusion’s validity.

The conclusion of a research paper serves several important purposes: 1. Summarize the Key Points 2. Reinforce the Main Argument 3. Provide Closure 4. Offer Insights or Implications 5. Engage the Reader. 6. Reflect on Limitations

Remember that the primary purpose of the research paper conclusion is to leave a lasting impression on the reader, reinforcing the key points and providing closure to your research. It’s often the last part of the paper that the reader will see, so it should be strong and well-crafted.

Makar, G., Foltz, C., Lendner, M., & Vaccaro, A. R. (2018). How to write effective discussion and conclusion sections. Clinical spine surgery, 31(8), 345-346.
Bunton, D. (2005). The structure of PhD conclusion chapters. Journal of English for academic purposes , 4 (3), 207-224.

Paperpal is a comprehensive AI writing toolkit that helps students and researchers achieve 2x the writing in half the time. It leverages 21+ years of STM experience and insights from millions of research articles to provide in-depth academic writing, language editing, and submission readiness support to help you write better, faster.

Get accurate academic translations, rewriting support, grammar checks, vocabulary suggestions, and generative AI assistance that delivers human precision at machine speed. Try for free or upgrade to Paperpal Prime starting at US$19 a month to access premium features, including consistency, plagiarism, and 30+ submission readiness checks to help you succeed.

Experience the future of academic writing – Sign up to Paperpal and start writing for free!

7 Ways to Improve Your Academic Writing Process

Paraphrasing in Academic Writing: Answering Top Author Queries

Preflight For Editorial Desk: The Perfect Hybrid (AI + Human) Assistance Against Compromised Manuscripts

You may also like, how to write a high-quality conference paper, academic editing: how to self-edit academic text with..., measuring academic success: definition & strategies for excellence, phd qualifying exam: tips for success , ai in education: it’s time to change the..., is it ethical to use ai-generated abstracts without..., what are journal guidelines on using generative ai..., quillbot review: features, pricing, and free alternatives, what is an academic paper types and elements , should you use ai tools like chatgpt for....

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings
Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Committee on Barriers and Opportunities in Completing 2-Year and 4-Year STEM Degrees; Board on Science Education; Division of Behavioral and Social Sciences and Education; Board on Higher Education and Workforce; Policy and Global Affairs; National Academy of Engineering; National Academies of Sciences, Engineering, and Medicine; Malcom S, Feder M, editors. Barriers and Opportunities for 2-Year and 4-Year STEM Degrees: Systemic Change to Support Students' Diverse Pathways. Washington (DC): National Academies Press (US); 2016 May 18.

Cover of Barriers and Opportunities for 2-Year and 4-Year STEM Degrees

Barriers and Opportunities for 2-Year and 4-Year STEM Degrees: Systemic Change to Support Students' Diverse Pathways.

Hardcopy Version at National Academies Press

7 Conclusions and Recommendations

There are many examples of unchanged policies and programs:

a “weed-out” culture in many STEM departments rather than a supportive environment;
graduation rates that are tracked on a 2-, 4-, or 6-year time clock, uninformed by data on median time to degree for different fields or the need to account for remediation time or the reality of part-time study;
recognition and rewards to institutions for the quantity of degrees awarded rather than the quality, relevance, and levels of learning that are expected of and provided to students; and
completion rates that are calculated on the basis of enrollment by first-time, full-time students and so discount part-time students and transfer students.

Several facts are worth noting. Institutions that take on the challenge of providing a high-quality STEM education to students from disadvantaged backgrounds often do so with fewer resources than elite institutions. Underrepresented minority students and first-generation students are more likely to enroll at a 2-year institution than a 4-year institution ( Van Noy and Zeidenberg, 2014 ). Historically black colleges and universities award about 20 percent of all of the STEM bachelor's degrees earned by black students in fields other than psychology and social sciences, and about one-third of black students who have earned a Ph.D. in these STEM fields attained a bachelor's degree in STEM from historically black colleges and universities (National Science Foundation, 2013).

Two overarching findings undergird our conclusions and recommendations:

The “STEM pipeline” metaphor focuses on the students who enter at one end of the education system and those who emerge with STEM degrees. The metaphor does not reflect the diverse ways that students now move across and within higher education institutions, the diversity of paths that lead students to STEM degrees, or the expanding range of careers for those with STEM degrees. The “STEM pathways” metaphor is a more comprehensive and inclusive way of examining how students progress through STEM degrees and the much broader kinds of supports that higher education needs to provide to enable these students to successfully complete a credential.

Undergraduate STEM reform efforts have been piecemeal and not institutional in nature, and those that do not attend to today's students, their challenges or to the policy environments in which the institutions operate are likely to be short-lived and largely ineffective.

In the following three sections, we present our conclusions and recommendations related to today's students, about the role of institutions in serving those students, and about the need for systemic and sustainable change. Our conclusions and recommendations are embedded in these sections. In addition, our recommendations are presented by stakeholder group in Box 7-1 .

Recommendations by Actors.

TODAY'S STEM STUDENTS

CONCLUSION 1 There is an opportunity to expand and diversify the nation's science, technology, engineering, and mathematics (STEM) workforce and STEM-skilled workers in all fields if there is a commitment to appropriately support students through degree completion and provide more opportunities to engage in high-quality STEM learning and experiences.

Interest in STEM degrees among all undergraduate degree seekers at 2-year and 4-year institutions is at an all-time high, including students from traditionally underrepresented groups. Interest in STEM degrees is not only reflected in what degrees students indicate they are most interested in earning when they first begin their undergraduate studies, but also in the fact that one-third of students who begin with an undeclared major select a STEM discipline as a major ( Eagan et al., 2014 ).

Students who enter college declaring that they are interested in pursuing STEM degrees but then decide to enroll in non-STEM majors most frequently do so after STEM introductory courses (or prerequisite introductory science and mathematics courses). These students turn away from STEM in response to the teaching methods and atmosphere they encountered in STEM classes ( President's Council of Advisors on Science and Technology, 2012 ; Seymour and Hewitt, 1997 ). Furthermore, many students who switch majors after their experiences in introductory STEM courses pass those courses. It seems that they abandon their goal of earning a STEM degree due to the way that STEM is taught and the difficulty in attaining support. That support, such as tutoring, mentoring, authentic STEM experiences, or other supports, improves retention in STEM majors ( Estrada, 2014 ). In other words, students are dissuaded from studying STEM rather than being drawn into studying a different discipline. While some of the switching may be the result of considered choices based on opportunities to explore attractive alternatives, lack of a supportive environment in STEM likely contributes to those decisions.

CONCLUSION 2 Science, technology, engineering, and mathematics (STEM) aspirants increasingly navigate the undergraduate education system in new and complex ways. It takes students longer for completion of degrees, there are many patterns of student mobility within and across institutions, and the accommodation and management of student enrollment patterns can affect how quickly and even whether a student earns a STEM degree.

CONCLUSION 3 National, state, and institutional undergraduate data systems often are not structured to gather information needed to understand how well the undergraduate education system and institutions of higher education are serving students.

Most large-scale data systems that include information on undergraduate students were built to track students in a pipeline model. Some systems focus primarily on gathering data on full-time or first-time students, while others do not account well for the swirling of students among institutions. These systems often rely on graduation rates as the sole metric of success for students and institutions: they do not systematically collect information on students' goals, reasons for transferring or leaving institutions, progress toward a credential, nor do they provide access to evidence-based teaching practices or student support systems.

RECOMMENDATION 1 Data collection systems should be adjusted to collect information to help departments and institutions better understand the nature of the student populations they serve and the pathways these students take to complete science, technology, engineering, and mathematics (STEM) degrees.

Colleges and universities need to more consistently leverage the information collected across their campuses (e.g., offices of institutional research, STEM departments, and student aid offices) to better understand who their students are, their movement among majors and institutions, the barriers they encounter in working toward their degrees, and the services or supports they need.
States and federal agencies should consider how the information they require institutions to collect might enable better tracking of students through pathways they take to earn a STEM degree within and especially across institutions. In addition, they should consider expanding measures of success, which increasingly inform funding formulas, beyond graduation rates.

RECOMMENDATION 2 Federal agencies, foundations, and other entities that fund research in undergraduate science, technology, engineering, and mathematics (STEM) education should prioritize research to assess whether enrollment mobility in STEM is a response to financial, institutional, individual, or other factors, both individually and collectively, and to improve understanding of how student progress in STEM, in comparison with other disciplines, is affected by enrollment mobility.

Many students move across institutions and into and out of STEM programs; the incidence is higher among community college students. It is often not clear what drives their decisions. One-half of community college STEM students enter into STEM after their first year of enrollment, and little is known about what factors are involved in their decisions and the ultimate implications for student outcomes. While late decisions can force students to take more than the required number of credits for a major because many STEM programs are highly structured with various requirements, early decisions may not be possible or even desirable if students are unsure about their career paths and need time to discover their interests. These decisions may be influenced by institutional policies (e.g., on early deadlines to declare program entry), discipline-based professional societies, and accrediting bodies. Research is needed on:

what kinds of exploration students undertake as they decide to major (or not) in a STEM field and how they make their decisions,
why students enter STEM programs at different times,
the factors that attract them to STEM majors,
how institutional structures might facilitate or delay their entry into STEM, and
to what extent the identified problems may be associated with changing student demographics.
INSTITUTIONAL SUPPORT FOR TODAY'S STEM STUDENTS

CONCLUSION 4 Better alignment of science, technology, engineering, and mathematics (STEM) programs, instructional practices, and student supports is needed in institutions to meet the needs of the populations they serve. Programming and policies that address the climate of STEM departments and classrooms, the availability of instructional supports and authentic STEM experiences, and the implementation of effective teaching practices together can help students overcome key barriers to earning a STEM degree, including time to degree and the price of a STEM degree.

Discipline-Based Education Research ( National Research Council, 2012 ) identifies the evidence-based practices that improve student learning and persistence in STEM programs. The study illustrates the importance of active instructional practices that engage students in the learning process and increase their interaction with peers, faculty, and teaching assistants. The report also points to the slow adoption of these practices. Research has also shown increased effects of evidence-based teaching practices when paired with co-curricular supports.

The need for and nature of student supports likely will differ by type of institution and student background. It would be useful for institutional leaders to collect the kind of data about students' current interests and needs to better determine how they can offer a range of interventions that are most appropriate to the current and changing needs of their students.

RECOMMENDATION 3 Federal agencies, foundations, and other entities that support research in undergraduate science, technology, engineering, and mathematics education should support studies with multiple methodologies and approaches to better understand the effectiveness of various co-curricular programs.

Future research on co-curricular programs should reflect the complexity and “messiness” of undergraduate education, and it should illuminate how the co-curricular support fits into the broader culture of institutions. There is a need for more studies that track students over time to assess both the short-term and long-term effects of program elements across academic pathways. Such studies should include data from similar cohorts of students who do not participate in the program as a comparison or control group. When possible and appropriate, participants should be randomly assigned to co-curricular program groups.

RECOMMENDATION 4 Institutions, states, and federal policy makers should better align educational policies with the range of education goals of students enrolled in 2-year and 4-year institutions. Policies should account for the fact that many students take more than 6 years to graduate, and should reward 2- year and 4-year institutions for their contributions to the educational success of students they serve, which includes not only those who graduate.

The states and the federal government should revise undergraduate accountability policies so that systems of assessment, evaluation, and accountability give credit to and do not penalize (i.e., in-state funding formulas) institutions for supporting students' progress toward their desired educational outcome. It is important that policies take into account the various ways that students are currently using different institutions in pursuit of a degree, certification, or technical skills.
The states and the federal government should extend financial aid eligibility to graduation for students making satisfactory progress toward a degree or certificate, so that students do not “time out” of financial aid eligibility.
Colleges and universities should shift their institutional policies toward a model in which all students who are admitted to a degree program are expected to complete that program and are provided the instruction, resources, and support they need to do so, rather than a model in which it is assumed that a large fraction of students will be unsuccessful and will leave science, technology, engineering, and mathematics programs. This model can save money because the time to degree is shortened and the number of drops, failures, withdrawals, and repeating of courses is reduced.

Systems of accountability for undergraduate education need to better align to the pathways that students actually are taking to earn STEM degrees. To do so, more thought needs to go into how each institution can track students' progression toward a degree or other outcome-—including gaining skills to upgrade current employment and earning a certificate while working toward an associate's degree—recognizing the long time to degree completion among many STEM students.

The culture of many STEM courses and departments is undergirded by the belief that “natural” ability, gender, or ethnicity is a significant determinant of a student's success in STEM . Related to this view is the tendency for introductory mathematics and science courses to be used as “gatekeeper” or “weeder” courses, which may discourage students from pursuing STEM degrees, through highly competitive classrooms and a lack of pedagogy that promotes active participation and emphasizes mastery and improvement. These courses often seek to select out and distinguish those with some perceived ability in STEM. The classroom and departmental culture needs to value diversity and be based on the understanding that all students aspiring to earn a STEM degree have the potential to succeed in STEM and provide all students the opportunity to make an informed decision about whether they want to continue pursuing STEM credentials.

RECOMMENDATION 5 Institutions of higher education, disciplinary societies, foundations, and federal agencies that fund undergraduate education should focus their efforts in a coordinated manner on critical issues to support science, technology, engineering, and mathematics (STEM) strategies, programs, and policies that can improve STEM instruction.

Colleges and universities should adjust faculty reward systems to better promote high-quality instruction and provide support for faculty to integrate effective teaching strategies into their practice. They should encourage educators to learn about and implement effective teaching methods by supporting participation in workshops, professional meetings, campus-based faculty development programs, and other related opportunities. Instructional quality is a key aspect of a student's undergraduate experience that could be addressed by providing incentives for more faculty members to align their classroom practices with evidence-based pedagogy.
Disciplinary and professional membership organizations should become more active in developing tools to support evidence-based teaching practices, and providing professional development in using these active pedagogies for new and potential faculty members and instructors.
The National Center for Education Statistics of the U.S. Department of Education should collect systematic data on tenured, tenure-track, and nontenure-track faculty and staff, as it previously did through the National Study of Postsecondary Faculty. Such data will make it possible to understand who is teaching STEM courses and whether they have participated in professional development programs to implement evidence-based instructional strategies. The Department of Education should support research on what supports are needed to allow all educators, including tenured, tenure-track faculty, full-time nontenured teaching faculty, adjunct faculty, and lecturers, to successfully implement such strategies.
Federal agencies, foundations, and other entities should invest in implementation research to better understand how to increase adoption of evidence-based instructional strategies.

RECOMMENDATION 6 Accrediting agencies, states, and institutions should take steps to support increased alignment of policies that can improve the transfer process for students.

Regional accrediting bodies should review student outcomes by participating colleges and require periodic updates of articulation agreements in response to those student outcomes.
States should encourage tracking transfer credits and using other metrics to measure the success of students who transfer.
Colleges and universities should work with other institutions in their regions to develop articulation agreements and student services that contribute to structured and supportive pathways for students seeking to transfer credits.

The pathways that students are taking to earn undergraduate STEM degrees have become increasingly complex, with greater numbers of students earning credits at more than one institution. Thus, issues of transfer and articulation are now relevant to an increasing proportion of STEM students, as well as students in other majors. The range of different regional, state, and institutional transfer and articulation policies that students encounter can be dizzying, and they can extend a student's time to completion and increase the cost of college, as well as being stressful to navigate.

Regional accrediting agencies, states, and institutions can all take steps to remove the barriers that students currently face when transferring credits among institutions. Removing these barriers may require creative and collaborative solutions, but they have the potential not only to improve students' educational experience, but also to make higher education institutions more efficient and effective.

RECOMMENDATION 7 State and federal agencies and accrediting bodies together should explore the efficacy and tradeoffs of different articulation agreements and transfer policies.

SYSTEMIC AND SUSTAINABLE CHANGE IN STEM EDUCATION

CONCLUSION 5 There is no single approach that will improve the educational outcomes of all science, technology, engineering, and mathematics (STEM) aspirants. The nature of U.S. undergraduate STEM education will require a series of interconnected and evidence-based approaches to create systemic organizational change for student success.

CONCLUSION 6 Improving undergraduate science, technology, engineering, and mathematics education for all students will require a more systemic approach to change that includes use of evidence to support institutional decisions, learning communities and faculty development networks, and partnerships across the education system.

Students need a higher education system that is less fragmented—or at least has clearer road markers—so that the diverse and complex pathways they take toward a degree do not create unnecessary barriers. Partnerships with elementary and secondary schools may be able to lead to better preparation for college, especially in mathematics. Partnerships with employers can lead to better articulation of the skills and knowledge that are relevant for their workforces, as well as opportunities for internships and work-related experiences that may improve students' understanding of and commitment to STEM education.

At the institutional level, program faculty and administrators need to recognize that successful improvements usually include strong leadership, including support for faculty to undertake the changes needed; awareness of how to overcome the barriers to adaptation and implementation of curricula that have been demonstrated to be effective; faculty who implement instructional practices developed through discipline-based education research; and data to monitor students' progress and to hold departments accountable for losses and recognize and reward them for student success.

RECOMMENDATION 8 Institutions should consider how expanded and improved co-curricular supports for science, technology, engineering, and mathematics (STEM) students can be informed by and integrated into work on more systemic reforms in undergraduate STEM education to more equitably serve their student populations.

To improve degree attainment rates, the quality of programs, and better serve their diverse student populations, institutions can consider a wide range of policies and programs: initiating or increasing opportunities for undergraduate student participation in research and other authentic STEM experiences; connecting students to experiences related to careers in their field of interest; expanding the use of educational technologies that have been effective in addressing the remediation needs of students; building student learning communities; and providing access to college and career guidance to help students understand the various and most efficient pathways to the degrees and careers they want. Students seem to benefit most from such supports when they are paired with evidence-based instructional strategies and when three or more co-curricular supports are bundled together ( Estrada, 2014 ). Such efforts will be more sustainable and effective if they are integrated into more systemic reform efforts.

RECOMMENDATION 9 Disciplinary departments, institutions, university associations, disciplinary societies, federal agencies, and accrediting bodies should work together to support systemic and long-lasting changes to undergraduate science, technology, engineering, and mathematics education.

STEM departments and entire academic units should support learning communities and networks that can help change faculty belief systems and practices and develop sustainable changes.
Colleges and universities should offer instructor training and mentoring to graduate students and postdoctoral scholars. Participating in such efforts as The Center for the Integration of Research, Teaching, and Learning (funded by the National Science Foundation; see Chapter 3 ) can educate graduate students about the value of treating their teaching as a form of scholarship and to value use of evidence-based approaches to teaching.
University associations and organizations should continue to facilitate undergraduate STEM educational reforms in their member institutions, particularly by examining reward structures and barriers to change and providing resources for data collection on student success, as well as by providing resources for interventions, support programs, and ways to share effective practices.
Disciplinary societies should s upport the development of continuing and intensive national and regional faculty development programs, awards, and recognition to encourage use of evidence-based instructional practices.
Federal agencies that support undergraduate STEM education should consider giving greater priority to supporting large-scale transformation strategies that are conceptualized to include and extend beyond instructional reform, and they should support both implementation research and research on barriers to reform that can support success for all students. They should increase the percentage of undergraduate STEM reform efforts and projects that focus on multiple levels—department, institution, discipline, government, and business and industry.
Following the policies adopted by some disciplinary accrediting bodies (e.g., the Accreditation Board for Engineering and Technology), regional and professional accrediting bodies should consider incorporating evidence-based instructional practices and faculty professional development efforts into their criteria and guidelines.

Looking from the top down, leadership is needed at every level to support change. Institutional leaders need to be committed to providing the supportive infrastructure that can make grassroots pedagogical and administrative changes possible (including active classrooms, technology, co-curricular supports, data systems, and teaching-learning centers). Loss of state support has negatively affected the operational model of many public institutions, forcing increased costs to be passed through to students, which disproportionately affects those who can least afford to attend, extending time to degree and may affect students' choices of major (e.g., when there is differential tuition for programs such as engineering). National accountability structures, though well intentioned, currently reward the most selective institutions while penalizing those with fewer resources, but the latter are the ones who often enroll and succeed in enrolling STEM students from disadvantaged and less selective backgrounds. The admonishment to “first, do no harm” should lead to a national discussion of how to recognize and honor the work of such institutions. At the same time, highly resourced institutions can be challenged to better support their STEM students through programs of active retention rather than “weeding out.”

Chang M, Sharkness J, Hurtado S, Newman C. What matters in college for retaining aspiring scientists and engineers from underrepresented racial groups. Journal of Research in Science Teaching. 2014; 51 (5):555–580.
Eagan K, Hurtado S, Figueroa T, Hughes B. Examining STEM Pathways among Students Who Begin College at Four-Year Institutions. National Academy of Sciences; Washington, DC: 2014. [April 2015]. (Commissioned paper prepared for the Committee on Barriers and Opportunities in Completing 2- and 4-Year STEM Degrees). http://sites .nationalacademies .org/cs/groups /dbassesite/documents /webpage/dbasse_088834.pdf .
Espinosa LL. Pathways and pipelines: Women of color in undergraduate STEM majors and the college experiences that contribute to persistence. Harvard Educational Review. 2011; 81 (2):209–241.
Estrada M. Ingredients for Improving the Culture of STEM Degree Attainment with Co-curricular Supports for Underrepresented Minority Students. 2014. [April 2015]. (Paper prepared for the Committee on Barriers and Opportunities in Completing 2- and 4-Year STEM Degrees). http://sites .nationalacademies .org/cs/groups /dbassesite/documents /webpage/dbasse_088832.pdf .
National Center for Education Statistics. Digest of Education Statistics 2013. Washington, DC: U.S. Department of Education; 2013.
National Research Council. Discipline-Based Education Research: Understanding and Improving Learning in Undergraduate Science and Engineering. Committee on the Status, Contributions, and Future Directions of Discipline-Based Education Research. Washington DC: The National Academies Press; 2012. (Board on Science Education, Division of Behavioral and Social Sciences and Education).
National Science Board. Science and Engineering Indicators 2014. Arlington VA: National Science Foundation; 2014. (NSB #14-01).
National Science Foundation and National Center for Science and Engineering Statistics. Women, Minorities, and Persons with Disabilities in Science and Engineering: 2013. Arlington, VA: National Science Foundation; 2013.
President's Council of Advisors on Science and Technology. Report to the President. 2012. [April 2015]. (Engage to Excel: Producing One Million Additional College Graduates with Degrees in Science, Technology, Engineering and Mathematics.). http://www .whitehouse .gov/sites/default/files /microsites/ostp /pcast-engage-to-excel-final_feb.pdf .
Salzman H, Van Noy M. Crossing the Boundaries: STEM Students in Four-Year and Community Colleges. 2014. [April 2015]. (Paper prepared for the Committee on Barriers and Opportunities in Completing 2- and 4-Year STEM Degrees). http://sites .nationalacademies .org/cs/groups /dbassesite/documents /webpage/dbasse_089924.pdf .
Seymour E, Hewitt N. Talking about Leaving: Why Undergraduates Leave the Sciences. Boulder, CO: Westview Press; 1997.
Van Noy M, Zeidenberg M. Hidden STEM Knowledge Producers: Community Colleges' Multiple Contributions to STEM Education and Workforce Development. 2014. [April 2015]. (Paper prepared for the Committee on Barriers and Opportunities in Completing 2- and 4-Year STEM Degrees). http://sites .nationalacademies .org/cs/groups /dbassesite/documents /webpage/dbasse_088831.pdf .
Cite this Page Committee on Barriers and Opportunities in Completing 2-Year and 4-Year STEM Degrees; Board on Science Education; Division of Behavioral and Social Sciences and Education; Board on Higher Education and Workforce; Policy and Global Affairs; National Academy of Engineering; National Academies of Sciences, Engineering, and Medicine; Malcom S, Feder M, editors. Barriers and Opportunities for 2-Year and 4-Year STEM Degrees: Systemic Change to Support Students' Diverse Pathways. Washington (DC): National Academies Press (US); 2016 May 18. 7, Conclusions and Recommendations.
PDF version of this title (1.2M)

In this Page

Recent activity.

Conclusions and Recommendations - Barriers and Opportunities for 2-Year and 4-Ye... Conclusions and Recommendations - Barriers and Opportunities for 2-Year and 4-Year STEM Degrees

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

Open access
Published: 23 May 2024

Providing insights into health data science education through artificial intelligence

Narjes Rohani 1 ,
Kobi Gal 2 , 5 ,
Michael Gallagher 3 &
Areti Manataki 4

BMC Medical Education volume 24 , Article number: 564 ( 2024 ) Cite this article

Metrics details

Health Data Science (HDS) is a novel interdisciplinary field that integrates biological, clinical, and computational sciences with the aim of analysing clinical and biological data through the utilisation of computational methods. Training healthcare specialists who are knowledgeable in both health and data sciences is highly required, important, and challenging. Therefore, it is essential to analyse students’ learning experiences through artificial intelligence techniques in order to provide both teachers and learners with insights about effective learning strategies and to improve existing HDS course designs.

We applied artificial intelligence methods to uncover learning tactics and strategies employed by students in an HDS massive open online course with over 3,000 students enrolled. We also used statistical tests to explore students’ engagement with different resources (such as reading materials and lecture videos) and their level of engagement with various HDS topics.

We found that students in HDS employed four learning tactics, such as actively connecting new information to their prior knowledge, taking assessments and practising programming to evaluate their understanding, collaborating with their classmates, and repeating information to memorise. Based on the employed tactics, we also found three types of learning strategies, including low engagement (Surface learners), moderate engagement (Strategic learners), and high engagement (Deep learners), which are in line with well-known educational theories. The results indicate that successful students allocate more time to practical topics, such as projects and discussions, make connections among concepts, and employ peer learning.

Conclusions

We applied artificial intelligence techniques to provide new insights into HDS education. Based on the findings, we provide pedagogical suggestions not only for course designers but also for teachers and learners that have the potential to improve the learning experience of HDS students.

Peer Review reports

In recent decades, data science and Artificial Intelligence (AI) techniques have shown promising applications in the field of medicine, leading to the emergence of an interdisciplinary field named Health Data Science (HDS) [ 1 , 2 ]. The availability of massive amounts of clinical and patient data has provided the opportunity for utilising computational tools to aid healthcare professionals in decision-making [ 3 ]. The application of data science in medicine has the potential to discover breakthrough findings that could improve global health through the early detection of diseases and personalised treatment recommendations [ 1 , 4 , 5 , 6 ].

Unfortunately, despite the high demand for data-literate healthcare specialists, according to the National Academy of Medicine, training students in this field can be challenging [ 7 ]. This difficulty is exacerbated by a variety of factors, such as the complexity of teaching both medical and computational concepts to students with diverse backgrounds, and the uncertainty over students’ learning preferences [ 2 , 8 , 9 ]. Consequently, both instructors and learners find it challenging to teach and learn HDS courses [ 8 , 9 , 10 , 11 ]. Conducting research to analyse HDS courses through AI techniques is thus necessary to understand how students regulate their learning, and identify areas for improvement. This information can be leveraged by instructors to facilitate better learning outcomes for HDS students and to design courses that are aligned with the HDS students’ needs and preferences [ 12 , 13 ].

Only a few studies [ 14 , 15 , 16 ] have been conducted to explore the learning preferences of students in HDS-related courses, specifically in the fields of bioinformatics [ 17 ] and precision medicine [ 15 ]. All of these studies relied on self-reported data, which can be biased and may not accurately reflect the true behaviours of students [ 18 , 19 ]. For example, Micheel et al. [ 15 ] conducted a survey study to discover the learning preferences of precision medicine healthcare professionals. Their findings showed that 80% of participants had multiple learning preferences. The largest group (39% of the participants) preferred a combination of watching, listening, and reading, whereas 19% of participants preferred a combination of watching and reading. The authors compared an intervention group that was exposed to a personalised course based on their learning preferences, with a control group that received standard training. The intervention group achieved significantly higher scores in both past-test and follow-up tests, suggesting that providing a customised course based on students’ preferences can significantly improve their learning outcomes. They also showed that the learning preferences of HDS students differ from those of medical students.

In another study, Holtzclaw et al. [ 16 ] explored four dimensions of learning preferences, including sensing/intuiting, visual/verbal, and active/reflective. The survey results indicated that the majority of bioinformatics students had visual (82%) and sequential (75%) learning preferences. The authors also conducted further analysis using pre- and post-course surveys, which confirmed their conclusions. According to this study, teaching genetic concepts through the use of visualisation techniques, such as diagrams and plots, was more effective for bioinformatics learners.

Although self-reported data can provide insight into students’ preferences and opinions, such information may be limited and biased due to several reasons [ 20 , 21 ]. First, the amount of data that is collected is not large enough to support rigorous statistical analysis. Second, the information reported by students may be influenced by their self-perceptions, which may not always align with their actual behaviours. Third, self-reported data may be subject to accidental or deliberate misreporting. We posit that applying AI techniques on clickstream data collected from students’ actual experience in a course can provide a more representative dataset for analysis [ 18 , 22 ].

Research findings in various disciplines suggest that the analysis of clickstream data using AI is a reliable approach to discovering students’ learning behaviours, such as their learning tactics and strategies [ 23 , 24 , 25 ]. For example, Jovanovic et al. [ 25 ] analysed the clickstream data from an engineering course and discovered four learning tactics and five learning strategies by using sequence mining and Agglomerative Hierarchical Clustering (AHC). Maldonado-Mahauad et al. [ 26 ] also applied a process mining technique to analyse clickstream data from three Massive Open Online Courses (MOOCs) in education, management and engineering. They discovered four learning patterns from students’ interactions with the course content and identified three groups of learners: “Comprehensive” learners who followed all course structure steps, “Targeting” students who focused on a specific set of activities that helped them pass the assessments, and “Sampling” learners with less goal-oriented strategy and low engagement.

Subsequently, Matcha et al. [ 24 ]. analysed the clickstream data from biology, Python programming and computer engineering courses using process mining and the Expectation Maximisation algorithm, which resulted in the discovery of various learning tactics in each course. Then, they applied AHC to the frequency of using the identified learning tactics by each student, which yielded three groups of students, namely low, moderate, and high engagement. Similarly, Crosslin et al. [ 27 ] used the same method to identify five learning tactics and four learning strategies employed in an online college-level history course.

Although AI methods have been employed to analyse the students’ learning experiences in a few disciplines [ 24 , 25 , 26 , 27 ], there is no research that analyses HDS students’ learning strategies using the power of AI algorithms. Given the fact that there is no data-driven insight into the learning behaviours of HDS students, multiple papers have emphasised the importance of conducting studies to analyse HDS education and learners’ experiences [ 28 , 29 , 30 , 31 ].

This paper directly addresses the above shortcoming by using AI techniques to analyse students’ interactions in an HDS MOOC with over 3,000 learners. We used statistical methods to explore students’ engagement with different types of educational resources (video lectures, reading materials, and so on) across three performance groups (students with low, moderate, and high performance). We also explored students’ engagement with different HDS topics covered in the course (e.g., medical image analysis, Python programming, and network biology) to identify topics that were most interesting or difficult for the students.

We identify a hierarchy of student activities in the course, ranging from low-level activities (e.g., watching videos, answering a question in a forum), mid-level learning tactics (e.g., collaborating with other students), and high-level strategies (e.g., deep learners).

We used statistical and AI methods to investigate the following research questions.

RQ1 - What type of educational resources in the HDS MOOC did the students engage with?

RQ2 - What health data science topics in the HDS MOOC did the students engage with?

RQ3 - What learning tactics and strategies did the students employ in the HDS MOOC?

RQ4 - Is there any association between students’ learning tactics and strategies and their performance?

With respect to RQ1, we found that overall, there were no large differences in engagement between readings and lecture videos, but students who achieved higher final grades engaged more than other students in all types of resources, especially in quizzes, labs, and projects. Regarding RQ2, among the taught topics, students were more engaged with Python programming and Sequence Processing.

With respect to RQ3 and RQ4, we identified the following four prevalent learning tactics employed by the HDS students: Elaboration – actively connecting new information to existing knowledge, Problem-solving – solving assessments and programming questions for better understanding, Peer learning - collaborating with peers to share knowledge, and Rehearsal – repeating information for better retention. Based on the frequency of using the identified learning tactics, we discovered three types of strategies employed by students, which are directly aligned with educational theory: low engagement (Surface learners), moderate engagement (Strategic learners), and high engagement (Deep learners). We found that the elaboration tactic had the highest correlation with overall student performance, and deep learners who had high final grades used this tactic more. Based on our findings, we provide pedagogical recommendations for course designers, teachers, and learners in HDS that can potentially improve HDS education.

In this section, we will provide a general description of the HDS MOOC, the student population, and the data that was available for analysis. Then, we will detail the methodology used to address each of the research questions.

Course and participants

The study is based on the Data Science in Stratified Healthcare and Precision Medicine (DSM) MOOC offered by the University of Edinburgh on Coursera [ 32 ]. We focused on the period between April 2018 and April 2022, and we analysed the clickstream data of 3,527 learners who engaged with at least one learning activity (see Additional file 1 for considered learning activities). The course completion rate for these students is 38%.

Demographic information of students shows that 37% were male, 28% were female, and 35% did not report their gender. Regarding their educational background, 15% held a master’s degree, 12% had a bachelor’s degree, 7% held a doctorate, and the remaining had a lower degree or did not report their level of educational attainment. There is a good location spread, with 34% of students based in America, 30% in Asia, 22% in Africa, 11% in Europe, and 2% in Oceania. This study used anonymised data and received institutional ethical approval.

DSM is an intermediate-level MOOC with a total of 43 videos, 13 reading materials, five quizzes, six discussion forums, one programming assignment and one peer-reviewed project assignment. The course covers the following five topics/weeks:

Course Introduction and Introduction to Programming.

DNA Sequence Processing and Medical Image Analysis.

Biological Network Modelling, Probabilistic Modelling, and Machine Learning.

Natural Language Processing and Process Modelling in Medicine.

Graph Data, and Ethical and Legal Aspects.

Each topic includes case studies, which are optional interview videos with specialists discussing real-world HDS projects and their research areas. The course assessment includes a quiz for each topic, as well as a programming assignment for the third topic and a peer-reviewed project on the last topic of the course. Final student grades are calculated (out of 100) using a weighted average of all quiz and assignment scores, with each quiz worth 10%, the programming assignment worth 20%, and the peer-reviewed assignment worth 30%.

Data analysis

Two-sided t-tests as well as Cohen’s D effect size were used to investigate any differences regarding engagement levels between different types of educational resources (RQ1), as well as between different course topics (RQ2). Each two-sided t-test was conducted by comparing the mean engagement scores of two groups, such as different types of educational resources (e.g., reading engagement and discussion engagement) or course topics (e.g., programming and network modelling). The null hypothesis (H0) for each test stated that there is no significant difference in engagement levels between the compared groups, while the alternative hypothesis (H1) stated that there is a significant difference in engagement levels between the groups compared. If the p -value obtained from the t-test was lower than 0.05, we rejected the null hypothesis, indicating a statistically significant difference in engagement levels. Due to the dataset’s large sample size, relying only on the p-value may not adequately represent meaningful differences. Therefore, we also calculated Cohen’s D effect size to quantify the standardised difference in engagement scores between the compared groups. By interpreting Cohen’s D alongside the results of the t-tests, our study aimed to assess not only the statistical significance but also the magnitude of the observed differences in engagement levels.

The schema of the methodology employed to find learning tactics and strategies employed by students (RQ3).

For RQ3, several AI techniques were used to analyse the clickstream data of the DSM course to uncover the learning tactics employed by the students. Following the approach by Matcha et al. [ 24 ], the clickstream data for each student from the beginning to the end of the course was divided into different learning sessions (see Fig. 1 .a and Fig. 1 .b). A learning session represents a consecutive series of learning actions performed by a student within one login into the learning platform. After pre-processing the learning sessions (this included considering two consecutive sessions with a time gap less than 30 min as one session), 44,505 learning sessions were identified. Process mining and clustering methods were employed to detect the learning tactics (see Fig. 1 .c). In particular, the probability of switching between different learning actions was estimated with the use of First-Order Markov Models (FOMMs) as implemented in the pMinerR package [ 33 ]. The number of possible learning tactics (no. tactics = 4) was determined based on a hierarchical clustering dendrogram. To identify the learning tactics, the Expectation-Maximisation (EM) algorithm was applied to the calculated transition probability matrix. For both First-Order Markov Models and Expectation-Maximisation, we utilised the default values for hyperparameters. The implementation of the methodology is available at: https://github.com/nrohani/HDS-EDM/tree/main .

Students often use several learning tactics while interacting with a course. Therefore, a learning strategy is defined as the goal-driven usage of a collection of learning tactics with the aim of acquiring knowledge or learning a new skill [ 24 , 27 , 34 , 35 ]. Based on previous work [ 24 ], the frequency of using each learning tactic by each student was calculated as a measure to group students into categories of learning strategies. The learning strategies were identified through agglomerative hierarchical clustering with the Ward algorithm (see Fig. 1 .d). The potential number of clusters (no. groups = 3) was determined based on the height of the dendrogram.

To answer RQ4, we explored both the association of a single learning tactic and the collection of learning tactics (learning strategy) with students’ final grades because it is useful to know whether any single tactic was more correlated to a higher final grade.

The Pearson correlation was used to check for any correlation between different learning tactics and students’ grades in different assessments. For each student, we computed the frequency of employing each learning tactic. Consequently, every student had a corresponding pair of values: their performance (correlation was calculated for each assessment score separately, as well as for the final grade) and the frequency of using each learning tactic. Subsequently, we determined the mean frequency of usage for each learning tactic across all students. Next, we computed the Pearson correlation coefficient between the mean frequencies of usage and mean performance to quantify the strength and direction of the linear relationship between each learning tactic and students’ performance. To assess the statistical significance of the calculated correlation, t-test for correlation coefficient was carried out, with the null hypothesis (H0) stating that there is no significant correlation between using the learning tactic and performance. H0 was rejected for p -values lower than 0.05, indicating that there is a statistically significant relationship between student usage of the learning tactic and their performance.

The Kruskal-Wallis test, which is a non-parametric test to compare the distributions of multiple groups, was used to test the association between students’ learning strategies and performance. We ranked the final grades within each group of learning strategies. Kruskal-Wallis test compared the distributions of final grades across the different groups of learning strategies. The test determined whether there are statistically significant differences in final grades achieved by each group of learning strategies or not (confidence interval set to 0.05).

In this study, similarly to previous work [ 24 , 27 ], we considered the number of clicks made by a student in the learning platform as a metric to assess their level of engagement. Also, following the course instructor’s recommendation, the students were categorised into three performance levels. LP: Low-Performance (final grade below 50, representing 62% of students), MP: Moderate-Performance (final grade between 50 and 80, representing 21% of students), and HP: High-Performance (final grade of 80 or higher, representing 16% of students). These performance categories were used in the next investigations to analyse the level of engagement of students in each performance group.

Engagement with different types of educational resources

Table 1 presents the relative number of visits to each type of educational resource. To provide a fair comparison between students’ engagement with each type of educational resource, we only measured the number of visits to the parent pages of each type of educational resource (on the Coursera platform there is a parent page for each resource before students go through the children/link pages of those resources).

The results in Table 1 show that, as expected, the relative number of visits to assessments (peer-reviewed project and quiz) is considerably higher than those to other learning materials (lectures, readings, labs, and discussions), demonstrating that students spend more time on assessments, particularly on the peer-reviewed project (Relative no. visits = 5.48). This could be because the peer-reviewed project needs more effort and accounts for 30% of the final grade. Therefore, it is not surprising that students visited this resource more than others.

Additionally, based on Table 1 , there is a small difference (t = 3.29e-74, p = 3.29e-74, d = 0.24) in the relative number of visits to video lectures (mean = 1.1) and reading materials (mean = 0.9). Similarly, the relative number of visits to programming labs is a bit higher than those to lectures and reading materials (p l = 9.89e-09, t l = 5.7466, p r = 7.64e-44, t r = 14.0826, d l =0.10, d r = 0.28). It can be inferred that students found programming labs either a bit more interesting or challenging, leading to more visits. However, the D effect size of the difference between lab and lecture is less than 0.2; therefore, the difference is not big.

It is worth noting that after measuring the average number of visits for children (linked) and parent (main) webpages related to each educational resource, students had relatively high engagement within each resource material. As an example, even though discussion forums have fewer visits, students exhibited higher engagement (mean no. visits = 11.29) and movement between posts within the forums compared to their visit to parent pages of discussion forums. However, to have a fair comparison between different types of educational resources, only the number of visits to the parent pages of each resource was measured (refer to Table 1 ).

We explored the engagement with different types of educational resources for the HP, MP, and LP groups (Fig. 2 ). For the HP group, engagement in the project (Relative no. visits = 18.16) was higher (d = 0.63, t = 28.085, p = 8.56e-110) than in the quiz (Relative no. visits = 3.37). Additionally, their engagement in the lab (Relative no. visits = 2.14) was slightly (+ 1.5 clicks) higher than in the lectures, readings, and discussions. Conversely, for the LP group, the quiz (Relative no. visits = 1.14) had the highest relative number of visits.

It can be inferred that, overall, there were no large differences in engagement between readings and lectures among HDS students, but students who achieved higher final grades (both HP and MP) visited more time all types of resources, especially in project. The discussion forums were also used more by HP students than by LP students (LP’s relative no. visits = 0.20, HP’s relative no. visits = 1.22, t = 10.470, p = 2.52e-23, d = 0.28).

Relative no. visits of parent pages of each type of educational resource. The figure shows log of the relative number of visits for each educational resources by each group of students. LP: low performance students, MP: moderate performance students, HP: high performance students.

Engagement with different HDS topics

The analysis regarding engagement with different HDS topics was based on two measures: (1) relative average video lecture watching time, and (2) click-based interaction with videos.

Figure 3 shows the average watching time, standardised by dividing by the length of each video lecture, for each topic. The bar plot demonstrates that students dedicated more time to “Introduction to Programming” and “Sequence Processing” compared to the other topics. Moreover, the figure shows that the engagement of students with the initial topics were more than the final topics.

Upon performance group-based analysis of video lecture watching time, we also found that HP students showed higher engagement in extra-curricular activities by spending more time watching the case studies (avg. watching time for HP = 0.62, avg. watching time for LP = 0.33). These case studies are optional interview videos in which HDS researchers and practitioners discuss their research projects. This finding is supported by previous research [ 24 , 36 , 37 ] that high achievers not only focus on the required syllabus but also aim to gain a deep understanding of the topics, often going beyond the syllabus.

The analysis of the students’ video interaction data reveals that students played, paused, and went forwards and backwards (seek) in the Introduction to Programming topic more than in other topics (as shown in Fig. 4 ). Our interpretation is that students potentially found this topic to be challenging, and therefore they rewatched certain parts to improve their understanding.

Apart from the Introduction to Programming topic, students had high click-based engagement with the Sequence Processing, Image Analysis, Network Modelling, and Machine Learning topics. It is also worth mentioning that although the case study videos were not mandatory, they achieved relatively high engagement according to the course instructor’s point of view. Based on the unexpected engagement of students with the case studies, one can infer that HDS students were interested in practical knowledge and real-world examples [ 24 , 36 , 37 ].

Learning tactics

Based on existing literature [ 24 , 25 , 27 ], a learning tactic is defined as a series of actions that a student carries out to fulfil a specific task in their learning procedure. After analysis of the students’ learning sessions, we discovered that the DSM students employed four learning tactics: Elaboration; Programming and Problem-solving; Peer learning; and Rehearsal. These learning tactics are in line with the Motivated Strategies for Learning Questionnaire (MSLQ) learning theory, and we named the data-driven tactics based on this learning theory [ 38 ].

The relative average watching time of each video lecture for students. The relative video watch was calculated by dividing the average watched time by the total length of each video lecture.

Elaboration is the longest (median 25 actions per session) and the most frequently used (45% of all learning sessions) learning tactic by DSM students (see Table 2 ). The dominant learning actions in this tactic are “video play” and “pause” (Fig. 5 .a). We can infer that this tactic primarily focuses on learning theoretical concepts, rather than programming practice and assessment participation. Students might have paused video lectures to reflect on their acquired information by taking notes, thinking about the new knowledge, or connecting concepts to their prior knowledge. This is in line with the MSLQ theory, which explains the elaboration tactic as a cognitive process in which students actively reflect on and connect new information to their prior knowledge, which can enhance their learning outcomes [ 38 , 39 ]. Our results also confirm that this tactic has a positive correlation with student performance. The positive Pearson correlation between the number of times students used the elaboration tactic and their final performance ( r = 0.38, p = 1.43e-113) provides evidence of the effectiveness of this learning tactic.

Average frequency of using each video action per student during watching each topic. “end” indicates watching a video until the end. “pause” means the student paused the video. “seek” means going forwards or backwards in a video. “play” indicates replaying a video after a pause. “start” means starting a video from the beginning. The topics in the x-axis are listed based on the order of teaching in the course (with the exception of Case Studies, which are spread across the course duration).

Programming and Problem-solving is the second most frequently used learning tactic (24% of all sessions). The dominant learning actions in this tactic are related to labs and quizzes (Fig. 5 .b). It can be concluded that DSM students employed this tactic for applying their acquired knowledge in solving programming labs as well as the programming assignment. It seems that they participated in quizzes to assess their knowledge and solve problems.

The positive Pearson correlation ( r = 0.35, p = 1.30e-92) between the frequency of using this tactic and their final performance suggests that this tactic is effective for achieving a high final grade. In conclusion, the students who practice programming more often tend to be more successful, which is consistent with [ 14 ]. This is because programming skills are an essential aspect of health data science and programming practice helps students develop critical thinking and problem-solving, which are essential skills for analysing health data. Therefore, students who invest time in practicing programming tend to have a better understanding of the material and perform better in the course assessments.

Peer learning is the third most frequently used learning tactic (19% of all sessions), in which DSM students engaged with discussion forums to ask questions, read others’ discussions, reply to peers, and solve the peer-reviewed project. The dominant learning actions in this tactic are the peer-reviewed project and discussion (Fig. 5 .c). The correlation analysis also shows a positive correlation between the number of times students used peer learning and their final performance ( r = 0.55, p = 3.04e-260), which is stronger than for the other learning tactics. This is not surprising, as 30% of the final grade is related to the peer-reviewed project, and students who engage more in peer learning are expected to achieve a higher grade. However, the correlation analysis between the number of times students used the peer learning tactic and their average grade in quizzes also shows a positive correlation ( r = 0.46, p = 5.29e-172). This supports the conclusion that the peer learning tactic is one of the most effective learning tactics, and this is in line with existing research that has shown that peer learning can lead to enhanced motivation, increased engagement, and improved learning outcomes [ 14 , 38 , 39 ].

Rehearsal is the least used learning tactic by DSM students (12% of sessions), and it is focused on acquiring theoretical knowledge. Although both Rehearsal and Elaboration learning tactics involve learning theoretical knowledge and mostly watching video lectures, the dominant learning actions in Rehearsal are video seek and video revisit actions (Fig. 5 .d). This suggests that students may have employed this learning tactic by reviewing certain parts of the video lectures instead of deeply understanding concepts through reflection and note-taking. In other words, Rehearsal is a simple tactic for memorising and superficially looking at learning materials. Based on prior research, the Rehearsal tactic may result in temporary retention of information rather than long-term retention [ 38 ]. As a result, some studies have indicated that the impact of this tactic may be restricted to low-level learning outcomes [ 39 ]. Although our Pearson correlation analysis shows a positive correlation between the number of times the Rehearsal tactic was used by a student and their final performance, this correlation is weak ( r = 0.19, p = 8.82e-29), and much weaker compared to the other learning tactics.

Frequency plot of each learning tactic, showing how many times each learning action was used in that tactic

Learning strategies

Three groups of learners, known as learning strategies [ 24 ], have been identified based on the frequency of using the learning tactics discussed in Sect. 3.3. The identified learning strategies can be mapped to well-recognised learning approaches introduced by [ 37 , 40 , 41 ]. Therefore, we used these learning theories to name the detected learning strategies and describe them according to the available educational information.

The results show that the majority of students (73% of all learners) are low engagement/surface learners who only used two learning tactics, Elaboration and Problem-solving, during their interaction with the course. They achieved a low final grade (m = 31, std = 27) and had low levels of engagement compared to the other two groups of learners (Fig. 6 ).

The second group of students (19% of all learners), high engagement/deep learners, used all learning tactics except Rehearsal with higher frequency, as shown in Fig. 6 . This group has the highest frequency of using the Elaboration, Problem-solving, and Peer learning tactics. This group also achieved the highest final grade (m = 68, std = 25), whereas surface learners achieved the lowest grade and were overall not successful in passing the course.

The third learning strategy group (8% of all learners) are moderate engagement/strategic learners who employed all learning tactics with moderate frequency, except for Rehearsal, which was used with relatively higher frequency. It can be inferred that these students strategised their learning by regulating their time in such a way as to only revisit and seek important parts of video lectures in order to achieve an acceptable grade. Although moderate engagement learners used all four discovered tactics, their final performance (m = 62, std = 25) is lower than deep learners. Deep learners did not use the rehearsal learning tactic, but it appears that the elaboration tactic, along with the other two tactics, was enough for them to achieve higher grades than moderate-engagement students.

The Kruskal-Wallis test shows a significant association between the discovered learning strategies and student final performance ( p = 2.51e-169, statistic = 776.43).

Average frequency of using each learning tactic by low, moderate, and high engagement learners (5b). as well as averaged final grade for each strategy group (5a)

This study employed artificial intelligence methods to provide insights into health data science education by analysing an MOOC with over 3,000 enrolled learners. The findings reveal that there is not a strong difference in the frequency of visits to reading materials, video lectures, and labs, although students tended to visit labs and lectures slightly more than reading materials. Also, based on the results, students who actively engaged with practical resources, such as labs, discussions, and projects, achieved higher final grades.

Furthermore, the results indicate that the students engaged more with the Sequence Processing and Python Programming topics. However, students moved forwards and backwards more in the programming topic videos compared to other topics. One inference is that students might find this topic challenging.

To analyse the students’ learning strategies, four learning tactics were identified that are in line with educational learning theories [ 38 , 41 ]. The most frequently used tactic is Elaboration , which involves learning by pausing and replaying video lectures, possibly to contemplate the taught concepts or take notes. Based on the MSLQ learning theory [ 38 ], this tactic assists learners in retaining information in their long-term memory by establishing connections between the items that need to be memorised. This tactic includes pausing and replaying a video lecture, potentially in order to rephrase or condense information into a summary, make comparisons and take notes in an active manner. This tactic supports a learner in combining and linking new information to their existing knowledge.

The second most used tactic is Programming and Problem-Solving , where students engaged with the programming labs and solved the programming assignment. Given the interdisciplinary nature of health data science, students need to develop their knowledge in programming and improve their problem-solving skills [ 42 ]. Therefore, this tactic can be effective for students to apply their theoretically acquired knowledge to solve problems by using programming.

The third tactic is Peer Learning , which involves communicating with peers in the discussion forums and solving the peer-reviewed assignment. This tactic was found to have a positive correlation with students’ final performance, which was stronger than for the other learning tactics. Existing literature confirms that peer learning is associated with high performance, especially in online courses where students do not have the opportunity to discuss the learning materials face to face [ 38 , 39 ]. It could be even more useful in the health data science field because students have diverse backgrounds; therefore, they can share their ideas and perspectives towards multi-disciplinary topics to develop in-depth knowledge and reflect on different approaches.

The final tactic is Rehearsal , which involves learning mostly by going forwards and backwards in video lectures instead of watching them from beginning to end. According to the MSLQ learning theory, it is a basic tactic for learning, which involves repeating information, again and again, to memorise it instead of deeply thinking about it. This tactic is effective for simple tasks and for calling information stored in the working memory, but not for acquiring new information that will be stored in the long-term memory. This tactic is believed to impact attention and the process of encoding information, but it does not seem to help students develop relationships between the information or integrate the information with their prior knowledge [ 38 , 39 ]. Based on a recent systematic review [ 39 ], one study [ 43 ] showed that this tactic has a weak positive impact on student performance; while two studies [ 44 , 45 ] did not find any significant association between rehearsal and performance. In this study, we found a weak correlation between rehearsal and student final grade (the weakest correlation compared to the other learning tactics). Interestingly, the Rehearsal tactic was not used much by deep learners, who achieved a higher final grade than surface and strategic learners.

Based on the frequency of using the learning tactics, three learning strategy groups were identified: low engagement/surface, high/engagement/deep, and moderate engagement/strategic. The learning strategies detected are highly accordant with the well-recognised learning approaches introduced by Biggs [ 41 ], Marton and Säljö [ 37 ], and Entwistle [ 40 ]. These scholars have described three learning approaches named deep, strategic, and surface learning, which are not the intrinsic characteristics of students [ 41 ], rather they are selected by students based on the task type and cognitive conditions. Also, students’ motivations and intuitions, the learning environment, the way the course is delivered, and the learning contents are the key factors that influence the choice of a learning approach by students [ 24 , 40 ].

The high engagement/deep learners’ group is characterised by a high level of engagement, a high frequency of employing various tactics, and a high number of quizzes and project submissions, which is consistent with a deep learning approach, by which students engage with high frequency with the course materials, they are highly engrossed in the ideas and actively try to relate them to previous knowledge [ 38 ]. Previous studies have shown that adopting the deep learning approach results in better academic performance [ 24 , 46 ]. Furthermore, the students with deep learning strategy obtained the highest marks in course assessments compared to other students, which indicates their in-depth knowledge. Also, the high use of the Elaboration, Problem-Solving, and Peer Learning tactics by these students reveals that they tend to focus on course materials for a long time (these tactics include long sessions), relate learning materials to their prior knowledge, focus on programming labs to solve problems and learn from peers and solve a project, which are all aligned with the characteristics of the deep learning approach.

The surface learning approach is adopted by students whose intention is to not fail and who want to achieve a passing mark rather than gain a deep understanding of the materials or obtain high marks. Therefore, these students mainly memorise the required information that is necessary for the exams, do not focus on abstract ideas, and mostly rely on details [ 37 ]. This approach has similar characteristics to the low engagement strategy in our study because the students using this strategy only used the Elaboration and Problem-Solving learning tactics with low engagement, resulting in low performance. They also did not use the Peer Learning tactic, which had the highest impact on student performance, because the peer-reviewed assignment corresponds to 30% of the final grade. In the DSM course, the passing score is 50 out of 100, and 50% of the final grade is related to quizzes. A deeper level of knowledge is required for the project compared to the quizzes. DSM students employing a surface approach tend to concentrate primarily on quizzes (by employing the problem-solving tactic) in order to achieve a passing score without investing significant effort in the project.

The strategic or achieving learning approach is described in educational theory as a combination of the surface and the deep approaches [ 46 ]. The main motivation of students adopting this approach is to get high scores and manage their efforts to make the most of the assessments done [ 47 ]. Therefore, they try to find the demands of assessments, manage their time, study in an organised manner, and routinely make sure that they use proper materials [ 47 ]. This learning approach is similar to the moderate engagement strategy in our study. The students with this strategy had moderate efforts, moderate frequency of using different tactics, and moderate performance in comparison to the two other strategies. They also mostly used the Rehearsal tactic, which shows that students moved forwards and backwards in video lectures instead of watching them from the start until the end. This is consistent with the characteristics of strategic learners who prefer to apply timely efficient tactics to manage their learning. Therefore, they used the Rehearsal tactic more than Elaboration because the Elaboration tactic is attributed to more effort, such as pausing and replaying videos instead of only seeking videos. This is also supported by the finding that the number of learning actions per learning session was higher in the Elaboration tactic.

It is worth pointing out that students use different learning strategies in different courses [ 48 ]. The learning tactics and strategies in health data science courses may differ from those in traditional biology or data science courses due to their interdisciplinary nature. Students in health data science must engage with both domain-specific biomedicine knowledge and data science concepts, requiring distinct strategies to facilitate their learning process [ 9 , 49 ]. The learning tactics and strategies identified in this study for the health data science course are unique, though they do share some similarities with the tactics and strategies reported in previous studies on biology and computer programming courses [ 24 ].

Recommendations for course design and education improvement

The identified insights about health data science students can help to design better courses and programmes in this field. Most educational design models [ 50 , 51 , 52 ] need information about students to design effective pedagogical frameworks (e.g., pedagogical strategy and tactics) and educational settings (e.g., learning tasks and organisational forms). For example, learning tactics and strategies could be defined in the form of pattern languages based on [ 50 ] for designing better educational frameworks. In other words, a key implication of our study is to provide health data science educational designers with insights about HDS students and their learning behaviours that can potentially assist them in designing better educational courses and frameworks. Our recommendations based on this study are as follows:

In the DSM course, there is a peer-reviewed project in the last week that is responsible for 30% of the final grade. Since many students were not successful in submitting the final assignment, our recommendation is to invite students to work on the assignment throughout an HDS course rather than only in the last week. This can be particularly helpful for LP students, as it can encourage them to remain engaged during all weeks [ 53 ].

We showed that students in DSM engaged with a diverse range of learning resources (lab, reading, video, quiz, and project). Previous research has shown that utilising diverse learning resources, such as reading materials, interactive video lectures, games, labs, and so on, can enhance students’ learning experiences [ 54 ]. As an example, some students may prefer to look at reading materials instead of videos, or vice versa. Therefore, the available resources should be diverse, as students are diverse in HDS courses. Additionally, previous research shows that integrating interactive resources, such as gamification tools, may increase student engagement and lead to improving their learning outcomes [ 55 , 56 ].

Our findings demonstrate that student engagement with topics decreased over the course, as evidenced by higher engagement with starting topics compared to ending topics. In the DSM course, as in many MOOCs [ 57 ], students have access to all topics/weeks upon enrolling on the course, which might overwhelm students given the large volume of learning materials. This can decrease their motivation, especially if they browse materials and assessments in the final topics and find them challenging. To address this issue, a potential solution could be to provide access to course material sequentially in such a way that a student can only have access to the subsequent topics upon the successful completion of previous topics.

Our results show that students had higher interaction with video lectures in the introductory Python programming topic compared to the other course topics, in particular higher video seek, pause, and play action. There are two possible explanations here. On one hand, students proficient in programming might have found the initial topic relatively straightforward and thus, did not engage with the entire video lecture from beginning to end. On the other hand, students with no programming background might have found the topic challenging and therefore rewatched certain parts of the videos. Given this mismatch, one might wonder how to best design a health data science course that works for diverse student backgrounds, including both computational and non-computational backgrounds. Our recommendation is to still provide introductory programming topics, but make them compulsory for students with no programming experience (so as to get up to speed with programming concepts) and optional for students with advanced programming skills (so that they are not disengaged). Once this is established as a baseline, subsequent programming-related tasks in the course should be designed at a balanced level of programming difficulty [ 58 , 59 ].

Based on the findings, peer learning in HDS can help students to achieve higher performance. Therefore, grouping students in such a way that each group contains students with different backgrounds and asking them to work on a project may help them not only better learn both computational and medical aspects of the course, but also help them to learn how to collaborate in an interdisciplinary community, which is essential for a career in health data science [ 49 , 60 ].

Recommendations for teachers and learners

The results of this study have implications not only for educational design, but also for learners and instructors. Learners sometimes are not aware of the most effective learning strategies, and informing them can possibly improve their future learning experiences [ 61 , 62 , 63 ]. However, course design is not the whole story, and teachers’ presentation approach also plays an important role in improving students’ learning outcomes. Therefore, we also provide some recommendations for teachers that may help them teach HDS more effectively.

Recommendations for learners

Applying multiple learning tactics when interacting with a course was found to be more effective than only using one or two learning tactics. Our findings, similar to previous research [ 48 ]. For example, in order to achieve a good grade in programming and enhance one’s programming skills, simply watching video lectures about programming is not enough. Students who practised coding and used the discussion forums to ask questions and solve the peer-reviewed project were more successful.

The results indicate that successful students not only relied on required knowledge for assessments but also went beyond the syllabus [ 37 ] and even engaged with optional sections of the DSM course (e.g. case studies). Therefore, we recommend to students to not only follow the essential parts of a health data science course, but also study additional resources to get a comprehensive knowledge of each topic.

Our findings demonstrate that students who paused and replayed video lectures in order to relate the taught concepts to their prior knowledge, take notes, or think deeply about the topics were more likely to achieve high performance in DSM. Our recommendation to health data science students is, therefore, to use the Elaboration tactic along with other effective learning tactics (Peer Learning and Problem-Solving). Using the Rehearsal learning tactic without deep comprehension is not always effective.

Recommendations for teachers

Previous studies [ 23 , 64 ] have shown that personalised feedback can help students to improve their learning. We recommend that instructors consider students’ learning tactics, strategies, and preferences when they are providing feedback to them.

Our results also show that although there is a relatively low number of posts in the DSM discussion forums, many students visited the discussion forums to read other students’ questions and answers. Given our finding that students who engaged in discussions more were more successful, teachers should encourage students to participate in the discussion forums. Students might be introverted or feel uncomfortable posting on discussion forums; therefore, teachers should motivate them through the use of appropriate techniques. As an example, a study showed that the active presence of teachers in discussions, through asking questions and following up with additional questions, can enhance students’ engagement [ 65 ]. Therefore, specifically for HDS courses, we recommend that teachers post a question in the discussion forums and ask students to share their opinions. Also, posting about cross-disciplinary research findings related to each topic might encourage students because it can show the application of each topic [ 66 ].

Limitations and directions for future work

The first limitation of this work is around generalisability. Given that in this study we analysed one health data science course, further research is needed to validate the generalisability of our findings. Also, given that the DSM course is a self-paced MOOC, our findings might not apply to other online courses or face-to-face classes. This is particularly important when considering the fact that students who enrol on MOOCs have different motivations [ 67 ] and it is possible that some of them did not focus on assessments because it is not part of their mandatory study programme. This limitation can impact findings related to student performance. As future work, we invite researchers to analyse the learning strategies employed by health data science students in other online or face-to-face courses. Furthermore, it is important to acknowledge the impact of user-friendly [ 68 ] and inclusive environment design [ 69 ] on students’ learning experiences in online courses. This study did not consider these factors, which may have influenced students’ learning strategies and preferences.

Another limitation of this study is to do with lack of access to temporal data (time spent to study each resource) for readings, discussions, and labs in the DSM course. Therefore, the student engagement with different topics (RQ2) was only explored based on the video lectures’ temporal data.

Our findings are limited to students’ clickstream data about the course on the Coursera platform. Since there are well-recognised survey tools, such as MSLQ [ 38 ] and self-regulation learning [ 70 ], for identifying students’ learning preferences and strategies that can uncover students’ perceptions about their learning regardless of the learning environment, it is worth collecting self-reported data and combining it with data-driven information as has been done for non-HDS courses [ 71 ], so as to strengthen results. We regard this as a fruitful avenue for future research.

Given that little is known about the learning behaviours and experiences of health data science students, conducting research to provide insight into health data science education is necessary. To address this important research gap, we employed artificial intelligence methods to analyse a health data science MOOC in order to understand students’ learning tactics, strategies, and engagement with learning materials and topics. We also provided suggestions supported by our findings for teachers, learners, and course designers in order to improve health data science education. The key findings of this study are the following:

Students who engaged more with practical resources, such as projects, labs, and discussions achieved higher final grades.

Among the topics taught, it seems that students were more engaged with Python Programming and Sequence Processing topics.

The Elaboration tactic (connecting new information to students’ prior knowledge) was used more, and this tactic was effective in achieving high performance.

The Peer Learning tactic had the highest correlation with the final grade.

The Rehearsal tactic (memorising information by repeating) had the lowest correlation with the final grade, and deep learners, who are the most successful students, did not use this learning tactic.

Deep learners utilised a range of different learning tactics throughout the course and engaged with all educational resources that enabled them in achieving higher final grades.

Data availability

The datasets generated and analysed during the current study are not publicly available due to ethical and legal restrictions. Data are however available from the Coursera platform upon reasonable request. The implementation of the methodology is available at: https://github.com/nrohani/HDS-EDM/tree/main .

Abbreviations

Agglomerative Hierarchical Clustering

Artificial Intelligence

Data Science in Stratified Healthcare and Precision Medicine

Health Data Science

High Performance

Low Performance

Massive Open Online Courses

Moderate Performance

Motivated Strategies for Learning Questionnaire

Research Question

Goldsmith J et al. The emergence and future of public health data science. Public Health Rev, 2021: p. 4.

Subrahmanya SVG, et al. The role of data science in healthcare advancements: applications, benefits, and future prospects. Ir J Med Sci (1971-). 2022;191(4):1473–83.

Article Google Scholar

Muni Kumar N, Manjula R. Role of big data analytics in rural health care-A step towards svasth bharath. Int J Comput Sci Inform Technol. 2014;5(6):7172–8.

Google Scholar

Burghard C. Big data and analytics key to accountable care success. IDC Health Insights. 2012;1:1–9.

Habehh H, Gohel S. Machine learning in healthcare. Curr Genom. 2021;22(4):291.

Saleem TJ, Chishti MA. Exploring the applications of machine learning in healthcare. Int J Sens Wirel Commun Control. 2020;10(4):458–72.

Matheny ME, Whicher D, Israni ST. Artificial intelligence in health care: a report from the National Academy of Medicine. JAMA. 2020;323(6):509–10.

Lillehaug S-I, Lajoie SP. AI in medical education—another grand challenge for medical informatics. Artif Intell Med. 1998;12(3):197–225.

Schneider MV, et al. Bioinformatics training: a review of challenges, actions and support requirements. Brief Bioinform. 2010;11(6):544–51.

Hardin J, et al. Data science in statistics curricula: preparing students to think with data. Am Stat. 2015;69(4):343–53.

Mainous III. Academic family physicians’ perception of genetic testing and integration into practice. Fam Med. 2013;45(4):257–62.

Kurilovas E. Advanced machine learning approaches to personalise learning: learning analytics and decision making. Behav Inform Technol. 2019;38(4):410–21.

Maseleno A, et al. Demystifying learning analytics in personalised learning. International Journal of Engineering and Technology (UAE); 2018.

Weisman D. Incorporating a collaborative web-based virtual laboratory in an undergraduate bioinformatics course. Biochem Mol Biol Educ. 2010;38(1):4–9.

Micheel CM, et al. Internet-based assessment of oncology health care professional learning style and optimization of materials for web-based learning: controlled trial with concealed allocation. J Med Internet Res. 2017;19(7):e265.

Holtzclaw JD, et al. Incorporating a new bioinformatics component into genetics at a historically Black college: outcomes and lessons. CBE—Life Sci Educ. 2006;5(1):52–64.

Sun K-t, Lin Y-c, Yu C-j. A study on learning effect among different learning styles in a web-based lab of science for elementary school students. Comput Educ. 2008;50(4):1411–22.

Gasevic D, et al. Detecting learning strategies with analytics: links with self-reported measures and academic performance. J Learn Analytics. 2017;4(2):113–28.

Winne PH, Jamieson-Noel D. Exploring students’ calibration of self reports about study tactics and achievement. Contemp Educ Psychol. 2002;27(4):551–72.

McDonald JD. Measuring personality constructs: the advantages and disadvantages of self-reports, informant reports and behavioural assessments. Enquire. 2008;1(1):1–19.

Rolstad S, Adler J, Rydén A. Response burden and questionnaire length: is shorter better? A review and meta-analysis. Value Health. 2011;14(8):1101–8.

Quick J et al. What college students say, and what they do: aligning self-regulated learning theory with behavioral logs . in Proceedings of the tenth international conference on learning analytics & knowledge . 2020.

Matcha W et al. Analytics of learning strategies: Associations with academic performance and feedback . in Proceedings of the 9th International Conference on Learning Analytics & Knowledge . 2019.

Matcha W, et al. Analytics of learning strategies: role of Course Design and Delivery Modality. J Learn Analytics. 2020;7(2):45–71.

Jovanović J, et al. Learning analytics to unveil learning strategies in a flipped classroom. Internet High Educ. 2017;33(4):74–85.

Maldonado-Mahauad J, et al. Mining theory-based patterns from big data: identifying self-regulated learning strategies in massive Open Online courses. Comput Hum Behav. 2018;80:179–96.

Crosslin M, et al. Understanding student learning pathways in traditional online history courses: utilizing process mining analysis on clickstream data. Journal of Research in Innovative Teaching & Learning; 2021.

Kolachalama VB, Garg PS. Machine learning and medical education. NPJ Digit Med. 2018;1(1):54.

Işık EB, et al. Grand challenges in bioinformatics education and training. Nat Biotechnol. 2023;41(8):1171–4.

Magana AJ, et al. A survey of scholarly literature describing the field of bioinformatics education and bioinformatics educational research. CBE—Life Sci Educ. 2014;13(4):607–23.

Brittain JM, Norris A. Delivery of health informatics education and training. Health Libr Rev. 2000;17(3):117–28.

Manataki A, Wong F. Data Science in Stratified Healthcare and Precision Medicine . https://www.coursera.org/learn/datascimed .

Gatta R et al. pMineR: an innovative R library for performing process mining in medicine . in Artificial Intelligence in Medicine: 16th Conference on Artificial Intelligence in Medicine, AIME 2017, Vienna, Austria, June 21–24 , 2017, Proceedings 16. 2017. Springer.

Derby SJ. Putting learning strategies to work. Educational Leadersh. 1989;46(4):4–10.

Hadwin AF, et al. Examining trace data to explore self-regulated learning. Metacognition Learn. 2007;2:107–24.

Floyd KS, Harrington S, Santiago J. The effect of engagement and perceived course value on deep and surface learning strategies. Informing Sci. 2009;12:181.

Marton F, Säljö R. On qualitative differences in learning: I—Outcome and process. Br J Educ Psychol. 1976;46(1):4–11.

Pintrich PR. A manual for the use of the Motivated Strategies for Learning Questionnaire (MSLQ) 1991.

Broadbent J, Poon WL. Self-regulated learning strategies & academic achievement in online higher education learning environments: a systematic review. Internet High Educ. 2015;27:1–13.

Entwistle NJ. Approaches to learning and perceptions of the learning environment: introduction to the special issue. High Educ, 1991: p. 201–4.

Biggs JB. Student approaches to learning and studying. Research Monograph. ERIC; 1987.

Sapci AH, Sapci HA. Teaching hands-on informatics skills to future health informaticians: a competency framework proposal and analysis of health care informatics curricula. JMIR Med Inf. 2020;8(1):e15748.

Puzziferro M. Online technologies self-efficacy and self-regulated learning as predictors of final grade and satisfaction in college-level online courses. Amer Jrnl Distance Educ. 2008;22(2):72–89.

Klingsieck KB, et al. Procrastination in a distance university setting. Distance Educ. 2012;33(3):295–310.

Wang S-L, Wu P-Y. The role of feedback and self-efficacy on web-based learning: the social cognitive perspective. Comput Educ. 2008;51(4):1589–98.

Chonkar SP, et al. The predominant learning approaches of medical students. BMC Med Educ. 2018;18:1–8.

Diseth Å, Martinsen Ø. Approaches to learning, cognitive style, and motives as predictors of academic achievement. Educational Psychol. 2003;23(2):195–207.

Simsek A, Balaban J. Learning strategies of successful and unsuccessful university students. Contemp Educational Technol. 2010;1(1):36–45.

Welch L, et al. Bioinformatics curriculum guidelines: toward a definition of core competencies. PLoS Comput Biol. 2014;10(3):e1003496.

Goodyear P. Educational design and networked learning: patterns, pattern languages and design practice. Australasian J Educational Technol, 2005. 21(1).

Young CP, Perović. N. ABC LD–A new toolkit for rapid learning design . in European Distance Education Network (EDEN) Conference . 2020.

Goodyear P, Carvalho L, Yeoman P. Activity-centred analysis and design (ACAD): core purposes, distinctive qualities and current developments. Education Tech Research Dev. 2021;69:445–64.

Rohani N et al. Early prediction of student performance in a health data science MOOC . in Proceedings of the 16th International Conference on Educational Data Mining . 2023. International Educational Data Mining Society.

Lee Y-H, Hsiao C, Ho C-H. The effects of various multimedia instructional materials on students’ learning responses and outcomes: a comparative experimental study. Comput Hum Behav. 2014;40:119–32.

Chen C-M, Sun Y-C. Assessing the effects of different multimedia materials on emotions and learning performance for visual and verbal style learners. Volume 59. Computers & Education; 2012. pp. 1273–85. 4.

Lopez CE, Tucker CS. Adaptive gamification and its impact on performance . in International conference on human-computer interaction . 2021. Springer.

Eriksson T, Adawi T, Stöhr C. Time is the bottleneck: a qualitative study exploring why learners drop out of MOOCs. J Comput High Educ. 2017;29(1):133–46.

Otero P, Hersh W, Ganesh AJ. Big data: are biomedical and health informatics training programs ready? Contribution of the IMIA working group for health and medical informatics education. Yearb Med Inform. 2014;9(1):177.

Gross LJ. Points of view: the interface of mathematics and biology: interdisciplinarity and the undergraduate biology curriculum: finding a balance. Cell Biol Educ. 2004;3(2):85–7.

Ranganathan S. Towards a career in bioinformatics. 2009, BioMed Cent. p. 1–4.

Azevedo R, Aleven V. International handbook of metacognition and learning technologies. Volume 26. Springer; 2013.

McCabe J. Metacognitive awareness of learning strategies in undergraduates. Mem Cognit. 2011;39:462–76.

Matcha W, Gašević D, Pardo A. A systematic review of empirical studies on learning analytics dashboards: a self-regulated learning perspective. IEEE Trans Learn Technol. 2019;13(2):226–45.

Lim L-A et al. Impact of learning analytics feedback on self-regulated learning: Triangulating behavioural logs with students’ recall . in LAK21: 11th international learning analytics and knowledge conference . 2021.

Nandi D, Hamilton M, Harland J. Evaluating the quality of interaction in asynchronous discussion forums in fully online courses. Distance Educ. 2012;33(1):5–30.

Martin F, Bolliger DU. Engagement matters: student perceptions on the importance of engagement strategies in the online learning environment. Online Learn. 2018;22(1):205–22.

Xiong Y, et al. Examining the relations among student motivation, engagement, and retention in a MOOC: a structural equation modeling approach. Global Educ Rev. 2015;2(3):23–33.

Ebner M, Holzinger A, Maurer H. Web 2.0 technology: Future interfaces for technology enhanced learning? in Universal Access in Human-Computer Interaction. Applications and Services: 4th International Conference on Universal Access in Human-Computer Interaction, UAHCI 2007 Held as Part of HCI International 2007 Beijing, China, July 22–27 , 2007 Proceedings, Part III 4. 2007. Springer.

Debevc M, Stjepanovič Z, Holzinger A. Development and evaluation of an e-learning course for deaf and hard of hearing based on the advanced adapted Pedagogical Index method. Interact Learn Environ. 2014;22(1):35–50.

Puustinen M, Pulkkinen L. Models of self-regulated learning: a review. Scandinavian J Educational Res. 2001;45(3):269–86.

Han F, Ellis RA, Pardo A. The descriptive features and quantitative aspects of students’ observed online learning: how are they related to self-reported perceptions and learning outcomes? IEEE Trans Learn Technol. 2022;15(1):32–41.

Download references

Acknowledgements

We would like to thank the Precision Medicine programme of the University of Edinburgh, as well as the Medical Research Council, for their support of this project aimed at enhancing health data science education. Additionally, we would like to express our appreciation to the Coursera platform and the students who participated in the course, whose contribution was invaluable to this research.

This work was supported by the Medical Research Council [grant number MR/N013166/1].

Author information

Authors and affiliations.

Usher Institute, University of Edinburgh, Edinburgh, UK

Narjes Rohani

School of Informatics, University of Edinburgh, Edinburgh, UK

Moray House School of Education and Sport, University of Edinburgh, Edinburgh, UK

Michael Gallagher

School of Computer Science, University of St Andrews, St Andrews, UK

Areti Manataki

Dept. of Software and Information Systems Engineering, Ben-Gurion University, Beersheba, Israel

You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to conceptualisation and design of the study. The implementation of the method as well as analysis of the results have been done by NR and supervised by AM, KG, and MG. The first draft of the paper was written by NR and improved by AM, KG, and MG. All authors read and accepted the final version of the paper.

Corresponding author

Correspondence to Areti Manataki .

Ethics declarations

Ethics approval and consent to participate.

This study does not involve experiments with human participants. Therefore, informed consent to participate is deemed unnecessary by the Informatics Research Ethics Committee at the University of Edinburgh. We have used secondary data that includes learner (i.e., human) data. This data is anonymised and it was collected by the Coursera platform and shared with us in the University of Edinburgh for research purposes. We have received ethics approval for this research (which involves the use of this secondary data) by the Informatics Research Ethics Committee at the University of Edinburgh [application number: #88883]. Coursera collects this data in accordance with its Terms of Use and its Privacy Notice. According to Coursera’s Privacy Notice, by using the Coursera website, learners agree to the use of their data for research purposes. Coursera has shared this data with us following its Research and Data Sharing Policies, which protect learners’ right to privacy.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Rohani, N., Gal, K., Gallagher, M. et al. Providing insights into health data science education through artificial intelligence. BMC Med Educ 24 , 564 (2024). https://doi.org/10.1186/s12909-024-05555-3

Download citation

Received : 14 February 2024

Accepted : 14 May 2024

Published : 23 May 2024

DOI : https://doi.org/10.1186/s12909-024-05555-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Health data science
Artificial intelligence
Learning analytics
Learning strategy
Learning tactic
Learning engagement
Health informatics
Medical education
Educational data mining

BMC Medical Education

ISSN: 1472-6920

Submission enquiries: [email protected]
General enquiries: [email protected]

SCIENCE & ENGINEERING INDICATORS

Research and development: u.s. trends and international comparisons.

Report PDF (808 KB)
Report - All Formats .ZIP (5.2 MB)
Supplemental Materials - All Formats .ZIP (513 KB)
MORE DOWNLOADS OPTIONS
Share on X/Twitter
Share on Facebook
Share on LinkedIn
Send as Email

U.S. GERD grew at a faster rate than GDP over 2010–21 on a compound annual growth rate basis. And while the United States remains the top R&D performer globally, other countries show continued growth in GERD and R&D intensity (R&D-to-GDP ratio). In 2021, the U.S. R&D intensity was 3.5%, based on internationally comparable OECD statistics. Other economies with R&D intensities above 3.0% include Israel and South Korea (both with intensities above 4.0%). Eight economies had intensities between 3.0% and 4.0%, including Taiwan, the United States, Japan, and Germany. Countries with intensities above 2.0% included the United Kingdom and China.

For the United States, the business sector continued to be the leading performer and funder of R&D. Manufacturing industries accounted for the largest proportion of R&D for companies with 10 or more employees, whereas the professional, scientific, and R&D services industry accounted for the largest proportion of R&D by microbusinesses. And U.S.-located companies continue to invest in software, AI, biotechnology, and nanotechnology R&D.

Consistent federal government support for R&D is a key feature of the U.S. R&D enterprise. The CHIPS and Science Act of 2022 appropriated $52.7 billion to revitalize the U.S. semiconductor industry along the supply chain, including $13.7 billion supporting R&D, workforce development, and related programs. More broadly, federal R&D funding constitutes the second-largest overall funding source and the largest source for U.S. basic research performance. The higher education sector was the largest performer of basic research and the largest recipient of federal R&D funding; in 2022, however, total R&D performance by the higher education sector did not increase after adjusting for inflation.

Barriers and Opportunities for 2-Year and 4-Year STEM Degrees: Systemic Change to Support Students' Diverse Pathways (2016)

TODAY’S STEM STUDENTS

INSTITUTIONAL SUPPORT FOR TODAY’S STEM STUDENTS

SYSTEMIC AND SUSTAINABLE CHANGE IN STEM EDUCATION

Welcome to OpenBook!

Get Email Updates

Science and technology education

Access options

Related Articles

Recruitment of Global Talent at the Institute of Zoology, Chinese Academy of Sciences (IOZ, CAS)

Full Professorship (W3) in “Organic Environmental Geochemistry (f/m/d)

Postdoc: deep learning for super-resolution microscopy

Postdoc: development of a novel MINFLUX microscope

Postdoctoral scholarship in Structural biology of neurodegeneration

Quick links

Expanding our views of science education to address sustainable development, empowerment, and social transformation

Sustainable development, environmental education and global challenges

Transforming our vision of education

Availability of data and materials

Abbreviations

Author information

Contributions

Corresponding author

Ethics declarations

Additional information

Rights and permissions

About this article

Share this article

My OpenLearn Profile

About this free course

introduction

INNOVATION IN TECHNOLOGY

THE INTERSECTION OF SCIENCE AND TECHNOLOGY

Images & Videos

Share This Book

Teaching the science of learning

Significance

Spaced practice

Interleaving

Retrieval practice

Elaboration

Concrete examples

Dual coding

Acknowledgements

Availability of data and materials

Contributions

Corresponding author

Ethics declarations

Publisher’s Note

Rights and permissions

About this article

Share this article

Why Is It Important for Students to Understand How Scientific Decisions are Made? ‘If You Don’t Understand How Scientists Decide What Makes One Claim More Believable, Then It’s Actually Very Hard to Understand Science,’ Says STEM Education Department Head William Sandoval

How Do Kids Typically Think About Science?

How Can Teachers Help Students Understand How Scientific Decisions Are Made?

More From College of Education News

Catherine Hartman Named Assistant Professor of Community College Leadership

Leading the Way

An Unforgettable Adventure: Meet Reginald Simon, MSA ‘24

Along with Stanford news and stories, show me:

AI in the classroom

Immersive environments

Gamification

Data-gathering and analysis

Exploring the effect of directive and reflective feedback on elementary school students’ scientific conceptual understanding, epistemological beliefs, and inquiry performance in online inquiry activities

Cite this article

Access this article

Author information

Contributions

Corresponding author

Ethics declarations

Additional information

Electronic Supplementary Material

Supplementary Material 1

About this article

Share this article

How to Write a Conclusion for Research Papers (with Examples)

Table of Contents

Frequently Asked Questions

Types of conclusions for research papers