affective components of critical thinking

Using affective learning to foster engagement and critical thinking

It takes time, patience and training, but a teaching approach that recognises the role that emotions play in learning can result in a more positive, effective and impactful student experience

Jyoti Devi Mahadeo

.css-76pyzs{margin-right:0.25rem;} ,, rabindra nepal.

• More on this topic

You may also like

Popular resources

.css-1txxx8u{overflow:hidden;max-height:81px;text-indent:0px;} Emotions and learning: what role do emotions play in how and why students learn?

A diy guide to starting your own journal, universities, ai and the common good, artificial intelligence and academic integrity: striking a balance, create an onboarding programme for neurodivergent students.

Students rarely mention how well designed their assignments were, or how effective the curriculum was, when asked to describe their ideal classes and teachers.  Rather, they offer an engaging account of classes they enjoyed or the teachers who were exemplars or constructively pushed them to do better . In sum, they talk about the human-to-human connection, a concept commonly referred to as “affective learning”.

So, what is affective learning?

Affective learning is the process of acquiring knowledge , skills and attitudes through emotional engagement. It recognises that emotions play a pivotal role in shaping cognitive processes, memory retention and decision-making. In the context of higher education, affective learning involves creating an environment that fosters positive emotions , such as curiosity and enthusiasm, to enhance learning outcomes. Integrating affective elements into instructional methods , through simple things such as learning students’ name s , will facilitate deeper understanding and meaningful retention of information. By acknowledging the interconnectedness of emotions and cognition, educators can tailor their approaches to promote a holistic and impactful learning experience, thereby making their teaching more effective. 

• Collection: Teaching critical thinking
• Teaching from the heart in 13 steps
• How learning through play pedagogy can develop 21st century business leaders

Traditionally, higher education has focused on cognitive aspects of learning, leaving affective learning aside. The importance of affect, emotions and motivations in the learning process cannot be overstated, as these elements are central to the internalisation of knowledge over time. A positive emotional state enhances motivation, making students more eager to participate and learn. Conversely, negative emotions, such as anxiety or fear , can reduce attention and information processing and so hinder the learning process.

Integrating affective learning into teaching

Here are additional practical examples of affective learning.

Affective learning can be integrated into teaching in several ways. Some of these will be discussed in depth below, and will allow educators to access the emotional states while creating an environment that supports students’ emotional well-being , which in turn is likely to result in higher levels of internalisation and positive outcomes , such as engagement and inclusivity and critical thinking .

Engagement and inclusivity

Academic engagement is often seen as an indicator of affective learning. The latter contributes to a positive and inclusive environment that motivates students to participate in collaborative projects and extracurricular activities and by making each student feel valued. Increased engagement leads to a more profound learning experience. Learning is not solely about acquiring knowledge; it’s about developing resilience and grit to overcome challenges. Moreover, affective learning acknowledges the significance of facing failures and setbacks as part of the growth process.

Teachers can offer reassurance to their students throughout the class, and communicate that making errors is not only permissible but an integral phase of the educational journey. A nurturing and emotionally secure learning environment, in which students feel comfortable sharing their emotions and apprehensions openly, is essential. Pedagogical activities, both within and outside the classroom, that support this include collaborative projects and team-building exercises. These activities not only enhance students’ interpersonal abilities and cooperative skills but also foster empathy through discussions about diverse viewpoints and experiences. Educators and administrators should ensure that courses remain accessible to students hailing from diverse academic backgrounds.

Fostering critical thinking

Affective learning plays a crucial role in nurturing critical-thinking abilities. Helping students express their emotions and opinions allows them to develop a deeper understanding of complex topics. Teachers can incorporate opportunities within formative and summative assessments for students to cultivate active listening and communication skills – for example, through in-class presentations and peer-to-peer Q&A sessions. Offering constructive feedback to students can help them to hone self-awareness and manage their emotions. 

This guidance can encompass practices such as mindfulness, stress-management and time-management techniques. Additionally, students can address their emotional intelligence and its pertinence to their field of study, as well as its potential impact on their academic and professional trajectories, through writing reflective essays for their course modules.

Challenges in implementing affective learning

While affective learning offers numerous benefits, its implementation comes with challenges:

• Time constraints : Under pressure to cover extensive syllabi, educators may find it difficult to find time to address emotions and build meaningful connections with their students. Simple formative tasks can offer a solution. Teachers could, for example, use reflective exercises and papers as formative or summative assignments, which will not require more teaching time to complete. Affective learning should be introduced gradually, and it is vital to acknowledge the diverse needs and backgrounds of students. Gathering regular formative feedback from students can help teachers to assess the effectiveness of their initiatives and make improvements. To help teachers learn students’ names , they could ask their administration team to prepare name tags for students to wear in class. In short, teachers have to exercise patience.
• Faculty training : Affective learning requires educators to develop a deep understanding of emotional intelligence and interpersonal skills. Faculty training programmes are essential to equip teachers with the tools to create an emotionally supportive environment where learners feel trusted and can communicate openly without fear of judgement. A potential solution is to introduce affective learning into the institution’s fellowship and induction programmes. 
• Assessment methods : Traditional assessment methods may not effectively measure the impact of affective learning on students’ academic performance. Developing new evaluation strategies that include emotional intelligence and personal growth is crucial to accurately gauge the effectiveness of this approach. Affective learning can be assessed through reflective assignments, such as our own experience with learning by teaching , journaling or group discussions . These can help students explore and process their emotions in a way that is related to the course material as well as to their learning experiences. Importantly, it is crucial to revise the assessment standards for gauging emotional learning to ensure their alignment with the capacity to assess emotional learning. This could involve instruments such the Griffith University affective learning scale (GUALS)  as well as models centred around meaning and student responses.

Using emotions to support learning

While establishing affective learning in higher education curricula, teachers have to acknowledge that there is no one “right” way for teachers to connect with students as they are all different. Affective learning recognises the profound impact of emotions on the learning process. By nurturing positive emotional experiences, educators can improve student engagement, critical thinking and overall academic performance. While challenges exist in integrating this approach, the benefits it offers in terms of enhanced long-term learning outcomes as well as positive mental health make it a valuable pedagogical tool for empowering minds and cultivating success in higher education. Effectively embracing affective learning will allow educators to create a transformative learning experience that prepares students for academic success as well as lifelong personal and professional growth.  

Jyoti Devi (Brinda) Mahadeo is assistant professor in the Faculty of Management, Law and Social Sciences at the University of Bradford, UK.

Rabindra Nepal is associate professor in the Faculty of Business and Law at the University of Wollongong, Australia.

If you would like advice and insight from academics and university staff delivered direct to your inbox each week,  sign up for the Campus newsletter .

Emotions and learning: what role do emotions play in how and why students learn?

Global perspectives: navigating challenges in higher education across borders, how to help young women see themselves as coders, contextual learning: linking learning to the real world, authentic assessment in higher education and the role of digital creative technologies, how hard can it be testing ai detection tools.

Register for free

and unlock a host of features on the THE site

• Table of Contents
• Random Entry
• Chronological
• Editorial Information
• About the SEP
• Editorial Board
• How to Cite the SEP
• Special Characters
• Advanced Tools
• Support the SEP
• PDFs for SEP Friends
• Make a Donation
• SEPIA for Libraries
• Entry Contents

Bibliography

Academic tools.

• Friends PDF Preview
• Author and Citation Info
• Back to Top

Critical Thinking

Critical thinking is a widely accepted educational goal. Its definition is contested, but the competing definitions can be understood as differing conceptions of the same basic concept: careful thinking directed to a goal. Conceptions differ with respect to the scope of such thinking, the type of goal, the criteria and norms for thinking carefully, and the thinking components on which they focus. Its adoption as an educational goal has been recommended on the basis of respect for students’ autonomy and preparing students for success in life and for democratic citizenship. “Critical thinkers” have the dispositions and abilities that lead them to think critically when appropriate. The abilities can be identified directly; the dispositions indirectly, by considering what factors contribute to or impede exercise of the abilities. Standardized tests have been developed to assess the degree to which a person possesses such dispositions and abilities. Educational intervention has been shown experimentally to improve them, particularly when it includes dialogue, anchored instruction, and mentoring. Controversies have arisen over the generalizability of critical thinking across domains, over alleged bias in critical thinking theories and instruction, and over the relationship of critical thinking to other types of thinking.

2.1 Dewey’s Three Main Examples

2.2 dewey’s other examples, 2.3 further examples, 2.4 non-examples, 3. the definition of critical thinking, 4. its value, 5. the process of thinking critically, 6. components of the process, 7. contributory dispositions and abilities, 8.1 initiating dispositions, 8.2 internal dispositions, 9. critical thinking abilities, 10. required knowledge, 11. educational methods, 12.1 the generalizability of critical thinking, 12.2 bias in critical thinking theory and pedagogy, 12.3 relationship of critical thinking to other types of thinking, other internet resources, related entries.

Use of the term ‘critical thinking’ to describe an educational goal goes back to the American philosopher John Dewey (1910), who more commonly called it ‘reflective thinking’. He defined it as

active, persistent and careful consideration of any belief or supposed form of knowledge in the light of the grounds that support it, and the further conclusions to which it tends. (Dewey 1910: 6; 1933: 9)

and identified a habit of such consideration with a scientific attitude of mind. His lengthy quotations of Francis Bacon, John Locke, and John Stuart Mill indicate that he was not the first person to propose development of a scientific attitude of mind as an educational goal.

In the 1930s, many of the schools that participated in the Eight-Year Study of the Progressive Education Association (Aikin 1942) adopted critical thinking as an educational goal, for whose achievement the study’s Evaluation Staff developed tests (Smith, Tyler, & Evaluation Staff 1942). Glaser (1941) showed experimentally that it was possible to improve the critical thinking of high school students. Bloom’s influential taxonomy of cognitive educational objectives (Bloom et al. 1956) incorporated critical thinking abilities. Ennis (1962) proposed 12 aspects of critical thinking as a basis for research on the teaching and evaluation of critical thinking ability.

Since 1980, an annual international conference in California on critical thinking and educational reform has attracted tens of thousands of educators from all levels of education and from many parts of the world. Also since 1980, the state university system in California has required all undergraduate students to take a critical thinking course. Since 1983, the Association for Informal Logic and Critical Thinking has sponsored sessions in conjunction with the divisional meetings of the American Philosophical Association (APA). In 1987, the APA’s Committee on Pre-College Philosophy commissioned a consensus statement on critical thinking for purposes of educational assessment and instruction (Facione 1990a). Researchers have developed standardized tests of critical thinking abilities and dispositions; for details, see the Supplement on Assessment . Educational jurisdictions around the world now include critical thinking in guidelines for curriculum and assessment.

For details on this history, see the Supplement on History .

2. Examples and Non-Examples

Before considering the definition of critical thinking, it will be helpful to have in mind some examples of critical thinking, as well as some examples of kinds of thinking that would apparently not count as critical thinking.

Dewey (1910: 68–71; 1933: 91–94) takes as paradigms of reflective thinking three class papers of students in which they describe their thinking. The examples range from the everyday to the scientific.

Transit : “The other day, when I was down town on 16th Street, a clock caught my eye. I saw that the hands pointed to 12:20. This suggested that I had an engagement at 124th Street, at one o’clock. I reasoned that as it had taken me an hour to come down on a surface car, I should probably be twenty minutes late if I returned the same way. I might save twenty minutes by a subway express. But was there a station near? If not, I might lose more than twenty minutes in looking for one. Then I thought of the elevated, and I saw there was such a line within two blocks. But where was the station? If it were several blocks above or below the street I was on, I should lose time instead of gaining it. My mind went back to the subway express as quicker than the elevated; furthermore, I remembered that it went nearer than the elevated to the part of 124th Street I wished to reach, so that time would be saved at the end of the journey. I concluded in favor of the subway, and reached my destination by one o’clock.” (Dewey 1910: 68–69; 1933: 91–92)

Ferryboat : “Projecting nearly horizontally from the upper deck of the ferryboat on which I daily cross the river is a long white pole, having a gilded ball at its tip. It suggested a flagpole when I first saw it; its color, shape, and gilded ball agreed with this idea, and these reasons seemed to justify me in this belief. But soon difficulties presented themselves. The pole was nearly horizontal, an unusual position for a flagpole; in the next place, there was no pulley, ring, or cord by which to attach a flag; finally, there were elsewhere on the boat two vertical staffs from which flags were occasionally flown. It seemed probable that the pole was not there for flag-flying.

“I then tried to imagine all possible purposes of the pole, and to consider for which of these it was best suited: (a) Possibly it was an ornament. But as all the ferryboats and even the tugboats carried poles, this hypothesis was rejected. (b) Possibly it was the terminal of a wireless telegraph. But the same considerations made this improbable. Besides, the more natural place for such a terminal would be the highest part of the boat, on top of the pilot house. (c) Its purpose might be to point out the direction in which the boat is moving.

“In support of this conclusion, I discovered that the pole was lower than the pilot house, so that the steersman could easily see it. Moreover, the tip was enough higher than the base, so that, from the pilot’s position, it must appear to project far out in front of the boat. Moreover, the pilot being near the front of the boat, he would need some such guide as to its direction. Tugboats would also need poles for such a purpose. This hypothesis was so much more probable than the others that I accepted it. I formed the conclusion that the pole was set up for the purpose of showing the pilot the direction in which the boat pointed, to enable him to steer correctly.” (Dewey 1910: 69–70; 1933: 92–93)

Bubbles : “In washing tumblers in hot soapsuds and placing them mouth downward on a plate, bubbles appeared on the outside of the mouth of the tumblers and then went inside. Why? The presence of bubbles suggests air, which I note must come from inside the tumbler. I see that the soapy water on the plate prevents escape of the air save as it may be caught in bubbles. But why should air leave the tumbler? There was no substance entering to force it out. It must have expanded. It expands by increase of heat, or by decrease of pressure, or both. Could the air have become heated after the tumbler was taken from the hot suds? Clearly not the air that was already entangled in the water. If heated air was the cause, cold air must have entered in transferring the tumblers from the suds to the plate. I test to see if this supposition is true by taking several more tumblers out. Some I shake so as to make sure of entrapping cold air in them. Some I take out holding mouth downward in order to prevent cold air from entering. Bubbles appear on the outside of every one of the former and on none of the latter. I must be right in my inference. Air from the outside must have been expanded by the heat of the tumbler, which explains the appearance of the bubbles on the outside. But why do they then go inside? Cold contracts. The tumbler cooled and also the air inside it. Tension was removed, and hence bubbles appeared inside. To be sure of this, I test by placing a cup of ice on the tumbler while the bubbles are still forming outside. They soon reverse” (Dewey 1910: 70–71; 1933: 93–94).

Dewey (1910, 1933) sprinkles his book with other examples of critical thinking. We will refer to the following.

Weather : A man on a walk notices that it has suddenly become cool, thinks that it is probably going to rain, looks up and sees a dark cloud obscuring the sun, and quickens his steps (1910: 6–10; 1933: 9–13).

Disorder : A man finds his rooms on his return to them in disorder with his belongings thrown about, thinks at first of burglary as an explanation, then thinks of mischievous children as being an alternative explanation, then looks to see whether valuables are missing, and discovers that they are (1910: 82–83; 1933: 166–168).

Typhoid : A physician diagnosing a patient whose conspicuous symptoms suggest typhoid avoids drawing a conclusion until more data are gathered by questioning the patient and by making tests (1910: 85–86; 1933: 170).

Blur : A moving blur catches our eye in the distance, we ask ourselves whether it is a cloud of whirling dust or a tree moving its branches or a man signaling to us, we think of other traits that should be found on each of those possibilities, and we look and see if those traits are found (1910: 102, 108; 1933: 121, 133).

Suction pump : In thinking about the suction pump, the scientist first notes that it will draw water only to a maximum height of 33 feet at sea level and to a lesser maximum height at higher elevations, selects for attention the differing atmospheric pressure at these elevations, sets up experiments in which the air is removed from a vessel containing water (when suction no longer works) and in which the weight of air at various levels is calculated, compares the results of reasoning about the height to which a given weight of air will allow a suction pump to raise water with the observed maximum height at different elevations, and finally assimilates the suction pump to such apparently different phenomena as the siphon and the rising of a balloon (1910: 150–153; 1933: 195–198).

Diamond : A passenger in a car driving in a diamond lane reserved for vehicles with at least one passenger notices that the diamond marks on the pavement are far apart in some places and close together in others. Why? The driver suggests that the reason may be that the diamond marks are not needed where there is a solid double line separating the diamond lane from the adjoining lane, but are needed when there is a dotted single line permitting crossing into the diamond lane. Further observation confirms that the diamonds are close together when a dotted line separates the diamond lane from its neighbour, but otherwise far apart.

Rash : A woman suddenly develops a very itchy red rash on her throat and upper chest. She recently noticed a mark on the back of her right hand, but was not sure whether the mark was a rash or a scrape. She lies down in bed and thinks about what might be causing the rash and what to do about it. About two weeks before, she began taking blood pressure medication that contained a sulfa drug, and the pharmacist had warned her, in view of a previous allergic reaction to a medication containing a sulfa drug, to be on the alert for an allergic reaction; however, she had been taking the medication for two weeks with no such effect. The day before, she began using a new cream on her neck and upper chest; against the new cream as the cause was mark on the back of her hand, which had not been exposed to the cream. She began taking probiotics about a month before. She also recently started new eye drops, but she supposed that manufacturers of eye drops would be careful not to include allergy-causing components in the medication. The rash might be a heat rash, since she recently was sweating profusely from her upper body. Since she is about to go away on a short vacation, where she would not have access to her usual physician, she decides to keep taking the probiotics and using the new eye drops but to discontinue the blood pressure medication and to switch back to the old cream for her neck and upper chest. She forms a plan to consult her regular physician on her return about the blood pressure medication.

Candidate : Although Dewey included no examples of thinking directed at appraising the arguments of others, such thinking has come to be considered a kind of critical thinking. We find an example of such thinking in the performance task on the Collegiate Learning Assessment (CLA+), which its sponsoring organization describes as

a performance-based assessment that provides a measure of an institution’s contribution to the development of critical-thinking and written communication skills of its students. (Council for Aid to Education 2017)

A sample task posted on its website requires the test-taker to write a report for public distribution evaluating a fictional candidate’s policy proposals and their supporting arguments, using supplied background documents, with a recommendation on whether to endorse the candidate.

Immediate acceptance of an idea that suggests itself as a solution to a problem (e.g., a possible explanation of an event or phenomenon, an action that seems likely to produce a desired result) is “uncritical thinking, the minimum of reflection” (Dewey 1910: 13). On-going suspension of judgment in the light of doubt about a possible solution is not critical thinking (Dewey 1910: 108). Critique driven by a dogmatically held political or religious ideology is not critical thinking; thus Paulo Freire (1968 [1970]) is using the term (e.g., at 1970: 71, 81, 100, 146) in a more politically freighted sense that includes not only reflection but also revolutionary action against oppression. Derivation of a conclusion from given data using an algorithm is not critical thinking.

What is critical thinking? There are many definitions. Ennis (2016) lists 14 philosophically oriented scholarly definitions and three dictionary definitions. Following Rawls (1971), who distinguished his conception of justice from a utilitarian conception but regarded them as rival conceptions of the same concept, Ennis maintains that the 17 definitions are different conceptions of the same concept. Rawls articulated the shared concept of justice as

a characteristic set of principles for assigning basic rights and duties and for determining… the proper distribution of the benefits and burdens of social cooperation. (Rawls 1971: 5)

Bailin et al. (1999b) claim that, if one considers what sorts of thinking an educator would take not to be critical thinking and what sorts to be critical thinking, one can conclude that educators typically understand critical thinking to have at least three features.

• It is done for the purpose of making up one’s mind about what to believe or do.
• The person engaging in the thinking is trying to fulfill standards of adequacy and accuracy appropriate to the thinking.
• The thinking fulfills the relevant standards to some threshold level.

One could sum up the core concept that involves these three features by saying that critical thinking is careful goal-directed thinking. This core concept seems to apply to all the examples of critical thinking described in the previous section. As for the non-examples, their exclusion depends on construing careful thinking as excluding jumping immediately to conclusions, suspending judgment no matter how strong the evidence, reasoning from an unquestioned ideological or religious perspective, and routinely using an algorithm to answer a question.

If the core of critical thinking is careful goal-directed thinking, conceptions of it can vary according to its presumed scope, its presumed goal, one’s criteria and threshold for being careful, and the thinking component on which one focuses. As to its scope, some conceptions (e.g., Dewey 1910, 1933) restrict it to constructive thinking on the basis of one’s own observations and experiments, others (e.g., Ennis 1962; Fisher & Scriven 1997; Johnson 1992) to appraisal of the products of such thinking. Ennis (1991) and Bailin et al. (1999b) take it to cover both construction and appraisal. As to its goal, some conceptions restrict it to forming a judgment (Dewey 1910, 1933; Lipman 1987; Facione 1990a). Others allow for actions as well as beliefs as the end point of a process of critical thinking (Ennis 1991; Bailin et al. 1999b). As to the criteria and threshold for being careful, definitions vary in the term used to indicate that critical thinking satisfies certain norms: “intellectually disciplined” (Scriven & Paul 1987), “reasonable” (Ennis 1991), “skillful” (Lipman 1987), “skilled” (Fisher & Scriven 1997), “careful” (Bailin & Battersby 2009). Some definitions specify these norms, referring variously to “consideration of any belief or supposed form of knowledge in the light of the grounds that support it and the further conclusions to which it tends” (Dewey 1910, 1933); “the methods of logical inquiry and reasoning” (Glaser 1941); “conceptualizing, applying, analyzing, synthesizing, and/or evaluating information gathered from, or generated by, observation, experience, reflection, reasoning, or communication” (Scriven & Paul 1987); the requirement that “it is sensitive to context, relies on criteria, and is self-correcting” (Lipman 1987); “evidential, conceptual, methodological, criteriological, or contextual considerations” (Facione 1990a); and “plus-minus considerations of the product in terms of appropriate standards (or criteria)” (Johnson 1992). Stanovich and Stanovich (2010) propose to ground the concept of critical thinking in the concept of rationality, which they understand as combining epistemic rationality (fitting one’s beliefs to the world) and instrumental rationality (optimizing goal fulfillment); a critical thinker, in their view, is someone with “a propensity to override suboptimal responses from the autonomous mind” (2010: 227). These variant specifications of norms for critical thinking are not necessarily incompatible with one another, and in any case presuppose the core notion of thinking carefully. As to the thinking component singled out, some definitions focus on suspension of judgment during the thinking (Dewey 1910; McPeck 1981), others on inquiry while judgment is suspended (Bailin & Battersby 2009, 2021), others on the resulting judgment (Facione 1990a), and still others on responsiveness to reasons (Siegel 1988). Kuhn (2019) takes critical thinking to be more a dialogic practice of advancing and responding to arguments than an individual ability.

In educational contexts, a definition of critical thinking is a “programmatic definition” (Scheffler 1960: 19). It expresses a practical program for achieving an educational goal. For this purpose, a one-sentence formulaic definition is much less useful than articulation of a critical thinking process, with criteria and standards for the kinds of thinking that the process may involve. The real educational goal is recognition, adoption and implementation by students of those criteria and standards. That adoption and implementation in turn consists in acquiring the knowledge, abilities and dispositions of a critical thinker.

Conceptions of critical thinking generally do not include moral integrity as part of the concept. Dewey, for example, took critical thinking to be the ultimate intellectual goal of education, but distinguished it from the development of social cooperation among school children, which he took to be the central moral goal. Ennis (1996, 2011) added to his previous list of critical thinking dispositions a group of dispositions to care about the dignity and worth of every person, which he described as a “correlative” (1996) disposition without which critical thinking would be less valuable and perhaps harmful. An educational program that aimed at developing critical thinking but not the correlative disposition to care about the dignity and worth of every person, he asserted, “would be deficient and perhaps dangerous” (Ennis 1996: 172).

Dewey thought that education for reflective thinking would be of value to both the individual and society; recognition in educational practice of the kinship to the scientific attitude of children’s native curiosity, fertile imagination and love of experimental inquiry “would make for individual happiness and the reduction of social waste” (Dewey 1910: iii). Schools participating in the Eight-Year Study took development of the habit of reflective thinking and skill in solving problems as a means to leading young people to understand, appreciate and live the democratic way of life characteristic of the United States (Aikin 1942: 17–18, 81). Harvey Siegel (1988: 55–61) has offered four considerations in support of adopting critical thinking as an educational ideal. (1) Respect for persons requires that schools and teachers honour students’ demands for reasons and explanations, deal with students honestly, and recognize the need to confront students’ independent judgment; these requirements concern the manner in which teachers treat students. (2) Education has the task of preparing children to be successful adults, a task that requires development of their self-sufficiency. (3) Education should initiate children into the rational traditions in such fields as history, science and mathematics. (4) Education should prepare children to become democratic citizens, which requires reasoned procedures and critical talents and attitudes. To supplement these considerations, Siegel (1988: 62–90) responds to two objections: the ideology objection that adoption of any educational ideal requires a prior ideological commitment and the indoctrination objection that cultivation of critical thinking cannot escape being a form of indoctrination.

Despite the diversity of our 11 examples, one can recognize a common pattern. Dewey analyzed it as consisting of five phases:

• suggestions , in which the mind leaps forward to a possible solution;
• an intellectualization of the difficulty or perplexity into a problem to be solved, a question for which the answer must be sought;
• the use of one suggestion after another as a leading idea, or hypothesis , to initiate and guide observation and other operations in collection of factual material;
• the mental elaboration of the idea or supposition as an idea or supposition ( reasoning , in the sense on which reasoning is a part, not the whole, of inference); and
• testing the hypothesis by overt or imaginative action. (Dewey 1933: 106–107; italics in original)

The process of reflective thinking consisting of these phases would be preceded by a perplexed, troubled or confused situation and followed by a cleared-up, unified, resolved situation (Dewey 1933: 106). The term ‘phases’ replaced the term ‘steps’ (Dewey 1910: 72), thus removing the earlier suggestion of an invariant sequence. Variants of the above analysis appeared in (Dewey 1916: 177) and (Dewey 1938: 101–119).

The variant formulations indicate the difficulty of giving a single logical analysis of such a varied process. The process of critical thinking may have a spiral pattern, with the problem being redefined in the light of obstacles to solving it as originally formulated. For example, the person in Transit might have concluded that getting to the appointment at the scheduled time was impossible and have reformulated the problem as that of rescheduling the appointment for a mutually convenient time. Further, defining a problem does not always follow after or lead immediately to an idea of a suggested solution. Nor should it do so, as Dewey himself recognized in describing the physician in Typhoid as avoiding any strong preference for this or that conclusion before getting further information (Dewey 1910: 85; 1933: 170). People with a hypothesis in mind, even one to which they have a very weak commitment, have a so-called “confirmation bias” (Nickerson 1998): they are likely to pay attention to evidence that confirms the hypothesis and to ignore evidence that counts against it or for some competing hypothesis. Detectives, intelligence agencies, and investigators of airplane accidents are well advised to gather relevant evidence systematically and to postpone even tentative adoption of an explanatory hypothesis until the collected evidence rules out with the appropriate degree of certainty all but one explanation. Dewey’s analysis of the critical thinking process can be faulted as well for requiring acceptance or rejection of a possible solution to a defined problem, with no allowance for deciding in the light of the available evidence to suspend judgment. Further, given the great variety of kinds of problems for which reflection is appropriate, there is likely to be variation in its component events. Perhaps the best way to conceptualize the critical thinking process is as a checklist whose component events can occur in a variety of orders, selectively, and more than once. These component events might include (1) noticing a difficulty, (2) defining the problem, (3) dividing the problem into manageable sub-problems, (4) formulating a variety of possible solutions to the problem or sub-problem, (5) determining what evidence is relevant to deciding among possible solutions to the problem or sub-problem, (6) devising a plan of systematic observation or experiment that will uncover the relevant evidence, (7) carrying out the plan of systematic observation or experimentation, (8) noting the results of the systematic observation or experiment, (9) gathering relevant testimony and information from others, (10) judging the credibility of testimony and information gathered from others, (11) drawing conclusions from gathered evidence and accepted testimony, and (12) accepting a solution that the evidence adequately supports (cf. Hitchcock 2017: 485).

Checklist conceptions of the process of critical thinking are open to the objection that they are too mechanical and procedural to fit the multi-dimensional and emotionally charged issues for which critical thinking is urgently needed (Paul 1984). For such issues, a more dialectical process is advocated, in which competing relevant world views are identified, their implications explored, and some sort of creative synthesis attempted.

If one considers the critical thinking process illustrated by the 11 examples, one can identify distinct kinds of mental acts and mental states that form part of it. To distinguish, label and briefly characterize these components is a useful preliminary to identifying abilities, skills, dispositions, attitudes, habits and the like that contribute causally to thinking critically. Identifying such abilities and habits is in turn a useful preliminary to setting educational goals. Setting the goals is in its turn a useful preliminary to designing strategies for helping learners to achieve the goals and to designing ways of measuring the extent to which learners have done so. Such measures provide both feedback to learners on their achievement and a basis for experimental research on the effectiveness of various strategies for educating people to think critically. Let us begin, then, by distinguishing the kinds of mental acts and mental events that can occur in a critical thinking process.

• Observing : One notices something in one’s immediate environment (sudden cooling of temperature in Weather , bubbles forming outside a glass and then going inside in Bubbles , a moving blur in the distance in Blur , a rash in Rash ). Or one notes the results of an experiment or systematic observation (valuables missing in Disorder , no suction without air pressure in Suction pump )
• Feeling : One feels puzzled or uncertain about something (how to get to an appointment on time in Transit , why the diamonds vary in spacing in Diamond ). One wants to resolve this perplexity. One feels satisfaction once one has worked out an answer (to take the subway express in Transit , diamonds closer when needed as a warning in Diamond ).
• Wondering : One formulates a question to be addressed (why bubbles form outside a tumbler taken from hot water in Bubbles , how suction pumps work in Suction pump , what caused the rash in Rash ).
• Imagining : One thinks of possible answers (bus or subway or elevated in Transit , flagpole or ornament or wireless communication aid or direction indicator in Ferryboat , allergic reaction or heat rash in Rash ).
• Inferring : One works out what would be the case if a possible answer were assumed (valuables missing if there has been a burglary in Disorder , earlier start to the rash if it is an allergic reaction to a sulfa drug in Rash ). Or one draws a conclusion once sufficient relevant evidence is gathered (take the subway in Transit , burglary in Disorder , discontinue blood pressure medication and new cream in Rash ).
• Knowledge : One uses stored knowledge of the subject-matter to generate possible answers or to infer what would be expected on the assumption of a particular answer (knowledge of a city’s public transit system in Transit , of the requirements for a flagpole in Ferryboat , of Boyle’s law in Bubbles , of allergic reactions in Rash ).
• Experimenting : One designs and carries out an experiment or a systematic observation to find out whether the results deduced from a possible answer will occur (looking at the location of the flagpole in relation to the pilot’s position in Ferryboat , putting an ice cube on top of a tumbler taken from hot water in Bubbles , measuring the height to which a suction pump will draw water at different elevations in Suction pump , noticing the spacing of diamonds when movement to or from a diamond lane is allowed in Diamond ).
• Consulting : One finds a source of information, gets the information from the source, and makes a judgment on whether to accept it. None of our 11 examples include searching for sources of information. In this respect they are unrepresentative, since most people nowadays have almost instant access to information relevant to answering any question, including many of those illustrated by the examples. However, Candidate includes the activities of extracting information from sources and evaluating its credibility.
• Identifying and analyzing arguments : One notices an argument and works out its structure and content as a preliminary to evaluating its strength. This activity is central to Candidate . It is an important part of a critical thinking process in which one surveys arguments for various positions on an issue.
• Judging : One makes a judgment on the basis of accumulated evidence and reasoning, such as the judgment in Ferryboat that the purpose of the pole is to provide direction to the pilot.
• Deciding : One makes a decision on what to do or on what policy to adopt, as in the decision in Transit to take the subway.

By definition, a person who does something voluntarily is both willing and able to do that thing at that time. Both the willingness and the ability contribute causally to the person’s action, in the sense that the voluntary action would not occur if either (or both) of these were lacking. For example, suppose that one is standing with one’s arms at one’s sides and one voluntarily lifts one’s right arm to an extended horizontal position. One would not do so if one were unable to lift one’s arm, if for example one’s right side was paralyzed as the result of a stroke. Nor would one do so if one were unwilling to lift one’s arm, if for example one were participating in a street demonstration at which a white supremacist was urging the crowd to lift their right arm in a Nazi salute and one were unwilling to express support in this way for the racist Nazi ideology. The same analysis applies to a voluntary mental process of thinking critically. It requires both willingness and ability to think critically, including willingness and ability to perform each of the mental acts that compose the process and to coordinate those acts in a sequence that is directed at resolving the initiating perplexity.

Consider willingness first. We can identify causal contributors to willingness to think critically by considering factors that would cause a person who was able to think critically about an issue nevertheless not to do so (Hamby 2014). For each factor, the opposite condition thus contributes causally to willingness to think critically on a particular occasion. For example, people who habitually jump to conclusions without considering alternatives will not think critically about issues that arise, even if they have the required abilities. The contrary condition of willingness to suspend judgment is thus a causal contributor to thinking critically.

Now consider ability. In contrast to the ability to move one’s arm, which can be completely absent because a stroke has left the arm paralyzed, the ability to think critically is a developed ability, whose absence is not a complete absence of ability to think but absence of ability to think well. We can identify the ability to think well directly, in terms of the norms and standards for good thinking. In general, to be able do well the thinking activities that can be components of a critical thinking process, one needs to know the concepts and principles that characterize their good performance, to recognize in particular cases that the concepts and principles apply, and to apply them. The knowledge, recognition and application may be procedural rather than declarative. It may be domain-specific rather than widely applicable, and in either case may need subject-matter knowledge, sometimes of a deep kind.

Reflections of the sort illustrated by the previous two paragraphs have led scholars to identify the knowledge, abilities and dispositions of a “critical thinker”, i.e., someone who thinks critically whenever it is appropriate to do so. We turn now to these three types of causal contributors to thinking critically. We start with dispositions, since arguably these are the most powerful contributors to being a critical thinker, can be fostered at an early stage of a child’s development, and are susceptible to general improvement (Glaser 1941: 175)

8. Critical Thinking Dispositions

Educational researchers use the term ‘dispositions’ broadly for the habits of mind and attitudes that contribute causally to being a critical thinker. Some writers (e.g., Paul & Elder 2006; Hamby 2014; Bailin & Battersby 2016a) propose to use the term ‘virtues’ for this dimension of a critical thinker. The virtues in question, although they are virtues of character, concern the person’s ways of thinking rather than the person’s ways of behaving towards others. They are not moral virtues but intellectual virtues, of the sort articulated by Zagzebski (1996) and discussed by Turri, Alfano, and Greco (2017).

On a realistic conception, thinking dispositions or intellectual virtues are real properties of thinkers. They are general tendencies, propensities, or inclinations to think in particular ways in particular circumstances, and can be genuinely explanatory (Siegel 1999). Sceptics argue that there is no evidence for a specific mental basis for the habits of mind that contribute to thinking critically, and that it is pedagogically misleading to posit such a basis (Bailin et al. 1999a). Whatever their status, critical thinking dispositions need motivation for their initial formation in a child—motivation that may be external or internal. As children develop, the force of habit will gradually become important in sustaining the disposition (Nieto & Valenzuela 2012). Mere force of habit, however, is unlikely to sustain critical thinking dispositions. Critical thinkers must value and enjoy using their knowledge and abilities to think things through for themselves. They must be committed to, and lovers of, inquiry.

A person may have a critical thinking disposition with respect to only some kinds of issues. For example, one could be open-minded about scientific issues but not about religious issues. Similarly, one could be confident in one’s ability to reason about the theological implications of the existence of evil in the world but not in one’s ability to reason about the best design for a guided ballistic missile.

Facione (1990a: 25) divides “affective dispositions” of critical thinking into approaches to life and living in general and approaches to specific issues, questions or problems. Adapting this distinction, one can usefully divide critical thinking dispositions into initiating dispositions (those that contribute causally to starting to think critically about an issue) and internal dispositions (those that contribute causally to doing a good job of thinking critically once one has started). The two categories are not mutually exclusive. For example, open-mindedness, in the sense of willingness to consider alternative points of view to one’s own, is both an initiating and an internal disposition.

Using the strategy of considering factors that would block people with the ability to think critically from doing so, we can identify as initiating dispositions for thinking critically attentiveness, a habit of inquiry, self-confidence, courage, open-mindedness, willingness to suspend judgment, trust in reason, wanting evidence for one’s beliefs, and seeking the truth. We consider briefly what each of these dispositions amounts to, in each case citing sources that acknowledge them.

• Attentiveness : One will not think critically if one fails to recognize an issue that needs to be thought through. For example, the pedestrian in Weather would not have looked up if he had not noticed that the air was suddenly cooler. To be a critical thinker, then, one needs to be habitually attentive to one’s surroundings, noticing not only what one senses but also sources of perplexity in messages received and in one’s own beliefs and attitudes (Facione 1990a: 25; Facione, Facione, & Giancarlo 2001).
• Habit of inquiry : Inquiry is effortful, and one needs an internal push to engage in it. For example, the student in Bubbles could easily have stopped at idle wondering about the cause of the bubbles rather than reasoning to a hypothesis, then designing and executing an experiment to test it. Thus willingness to think critically needs mental energy and initiative. What can supply that energy? Love of inquiry, or perhaps just a habit of inquiry. Hamby (2015) has argued that willingness to inquire is the central critical thinking virtue, one that encompasses all the others. It is recognized as a critical thinking disposition by Dewey (1910: 29; 1933: 35), Glaser (1941: 5), Ennis (1987: 12; 1991: 8), Facione (1990a: 25), Bailin et al. (1999b: 294), Halpern (1998: 452), and Facione, Facione, & Giancarlo (2001).
• Self-confidence : Lack of confidence in one’s abilities can block critical thinking. For example, if the woman in Rash lacked confidence in her ability to figure things out for herself, she might just have assumed that the rash on her chest was the allergic reaction to her medication against which the pharmacist had warned her. Thus willingness to think critically requires confidence in one’s ability to inquire (Facione 1990a: 25; Facione, Facione, & Giancarlo 2001).
• Courage : Fear of thinking for oneself can stop one from doing it. Thus willingness to think critically requires intellectual courage (Paul & Elder 2006: 16).
• Open-mindedness : A dogmatic attitude will impede thinking critically. For example, a person who adheres rigidly to a “pro-choice” position on the issue of the legal status of induced abortion is likely to be unwilling to consider seriously the issue of when in its development an unborn child acquires a moral right to life. Thus willingness to think critically requires open-mindedness, in the sense of a willingness to examine questions to which one already accepts an answer but which further evidence or reasoning might cause one to answer differently (Dewey 1933; Facione 1990a; Ennis 1991; Bailin et al. 1999b; Halpern 1998, Facione, Facione, & Giancarlo 2001). Paul (1981) emphasizes open-mindedness about alternative world-views, and recommends a dialectical approach to integrating such views as central to what he calls “strong sense” critical thinking. In three studies, Haran, Ritov, & Mellers (2013) found that actively open-minded thinking, including “the tendency to weigh new evidence against a favored belief, to spend sufficient time on a problem before giving up, and to consider carefully the opinions of others in forming one’s own”, led study participants to acquire information and thus to make accurate estimations.
• Willingness to suspend judgment : Premature closure on an initial solution will block critical thinking. Thus willingness to think critically requires a willingness to suspend judgment while alternatives are explored (Facione 1990a; Ennis 1991; Halpern 1998).
• Trust in reason : Since distrust in the processes of reasoned inquiry will dissuade one from engaging in it, trust in them is an initiating critical thinking disposition (Facione 1990a, 25; Bailin et al. 1999b: 294; Facione, Facione, & Giancarlo 2001; Paul & Elder 2006). In reaction to an allegedly exclusive emphasis on reason in critical thinking theory and pedagogy, Thayer-Bacon (2000) argues that intuition, imagination, and emotion have important roles to play in an adequate conception of critical thinking that she calls “constructive thinking”. From her point of view, critical thinking requires trust not only in reason but also in intuition, imagination, and emotion.
• Seeking the truth : If one does not care about the truth but is content to stick with one’s initial bias on an issue, then one will not think critically about it. Seeking the truth is thus an initiating critical thinking disposition (Bailin et al. 1999b: 294; Facione, Facione, & Giancarlo 2001). A disposition to seek the truth is implicit in more specific critical thinking dispositions, such as trying to be well-informed, considering seriously points of view other than one’s own, looking for alternatives, suspending judgment when the evidence is insufficient, and adopting a position when the evidence supporting it is sufficient.

Some of the initiating dispositions, such as open-mindedness and willingness to suspend judgment, are also internal critical thinking dispositions, in the sense of mental habits or attitudes that contribute causally to doing a good job of critical thinking once one starts the process. But there are many other internal critical thinking dispositions. Some of them are parasitic on one’s conception of good thinking. For example, it is constitutive of good thinking about an issue to formulate the issue clearly and to maintain focus on it. For this purpose, one needs not only the corresponding ability but also the corresponding disposition. Ennis (1991: 8) describes it as the disposition “to determine and maintain focus on the conclusion or question”, Facione (1990a: 25) as “clarity in stating the question or concern”. Other internal dispositions are motivators to continue or adjust the critical thinking process, such as willingness to persist in a complex task and willingness to abandon nonproductive strategies in an attempt to self-correct (Halpern 1998: 452). For a list of identified internal critical thinking dispositions, see the Supplement on Internal Critical Thinking Dispositions .

Some theorists postulate skills, i.e., acquired abilities, as operative in critical thinking. It is not obvious, however, that a good mental act is the exercise of a generic acquired skill. Inferring an expected time of arrival, as in Transit , has some generic components but also uses non-generic subject-matter knowledge. Bailin et al. (1999a) argue against viewing critical thinking skills as generic and discrete, on the ground that skilled performance at a critical thinking task cannot be separated from knowledge of concepts and from domain-specific principles of good thinking. Talk of skills, they concede, is unproblematic if it means merely that a person with critical thinking skills is capable of intelligent performance.

Despite such scepticism, theorists of critical thinking have listed as general contributors to critical thinking what they variously call abilities (Glaser 1941; Ennis 1962, 1991), skills (Facione 1990a; Halpern 1998) or competencies (Fisher & Scriven 1997). Amalgamating these lists would produce a confusing and chaotic cornucopia of more than 50 possible educational objectives, with only partial overlap among them. It makes sense instead to try to understand the reasons for the multiplicity and diversity, and to make a selection according to one’s own reasons for singling out abilities to be developed in a critical thinking curriculum. Two reasons for diversity among lists of critical thinking abilities are the underlying conception of critical thinking and the envisaged educational level. Appraisal-only conceptions, for example, involve a different suite of abilities than constructive-only conceptions. Some lists, such as those in (Glaser 1941), are put forward as educational objectives for secondary school students, whereas others are proposed as objectives for college students (e.g., Facione 1990a).

The abilities described in the remaining paragraphs of this section emerge from reflection on the general abilities needed to do well the thinking activities identified in section 6 as components of the critical thinking process described in section 5 . The derivation of each collection of abilities is accompanied by citation of sources that list such abilities and of standardized tests that claim to test them.

Observational abilities : Careful and accurate observation sometimes requires specialist expertise and practice, as in the case of observing birds and observing accident scenes. However, there are general abilities of noticing what one’s senses are picking up from one’s environment and of being able to articulate clearly and accurately to oneself and others what one has observed. It helps in exercising them to be able to recognize and take into account factors that make one’s observation less trustworthy, such as prior framing of the situation, inadequate time, deficient senses, poor observation conditions, and the like. It helps as well to be skilled at taking steps to make one’s observation more trustworthy, such as moving closer to get a better look, measuring something three times and taking the average, and checking what one thinks one is observing with someone else who is in a good position to observe it. It also helps to be skilled at recognizing respects in which one’s report of one’s observation involves inference rather than direct observation, so that one can then consider whether the inference is justified. These abilities come into play as well when one thinks about whether and with what degree of confidence to accept an observation report, for example in the study of history or in a criminal investigation or in assessing news reports. Observational abilities show up in some lists of critical thinking abilities (Ennis 1962: 90; Facione 1990a: 16; Ennis 1991: 9). There are items testing a person’s ability to judge the credibility of observation reports in the Cornell Critical Thinking Tests, Levels X and Z (Ennis & Millman 1971; Ennis, Millman, & Tomko 1985, 2005). Norris and King (1983, 1985, 1990a, 1990b) is a test of ability to appraise observation reports.

Emotional abilities : The emotions that drive a critical thinking process are perplexity or puzzlement, a wish to resolve it, and satisfaction at achieving the desired resolution. Children experience these emotions at an early age, without being trained to do so. Education that takes critical thinking as a goal needs only to channel these emotions and to make sure not to stifle them. Collaborative critical thinking benefits from ability to recognize one’s own and others’ emotional commitments and reactions.

Questioning abilities : A critical thinking process needs transformation of an inchoate sense of perplexity into a clear question. Formulating a question well requires not building in questionable assumptions, not prejudging the issue, and using language that in context is unambiguous and precise enough (Ennis 1962: 97; 1991: 9).

Imaginative abilities : Thinking directed at finding the correct causal explanation of a general phenomenon or particular event requires an ability to imagine possible explanations. Thinking about what policy or plan of action to adopt requires generation of options and consideration of possible consequences of each option. Domain knowledge is required for such creative activity, but a general ability to imagine alternatives is helpful and can be nurtured so as to become easier, quicker, more extensive, and deeper (Dewey 1910: 34–39; 1933: 40–47). Facione (1990a) and Halpern (1998) include the ability to imagine alternatives as a critical thinking ability.

Inferential abilities : The ability to draw conclusions from given information, and to recognize with what degree of certainty one’s own or others’ conclusions follow, is universally recognized as a general critical thinking ability. All 11 examples in section 2 of this article include inferences, some from hypotheses or options (as in Transit , Ferryboat and Disorder ), others from something observed (as in Weather and Rash ). None of these inferences is formally valid. Rather, they are licensed by general, sometimes qualified substantive rules of inference (Toulmin 1958) that rest on domain knowledge—that a bus trip takes about the same time in each direction, that the terminal of a wireless telegraph would be located on the highest possible place, that sudden cooling is often followed by rain, that an allergic reaction to a sulfa drug generally shows up soon after one starts taking it. It is a matter of controversy to what extent the specialized ability to deduce conclusions from premisses using formal rules of inference is needed for critical thinking. Dewey (1933) locates logical forms in setting out the products of reflection rather than in the process of reflection. Ennis (1981a), on the other hand, maintains that a liberally-educated person should have the following abilities: to translate natural-language statements into statements using the standard logical operators, to use appropriately the language of necessary and sufficient conditions, to deal with argument forms and arguments containing symbols, to determine whether in virtue of an argument’s form its conclusion follows necessarily from its premisses, to reason with logically complex propositions, and to apply the rules and procedures of deductive logic. Inferential abilities are recognized as critical thinking abilities by Glaser (1941: 6), Facione (1990a: 9), Ennis (1991: 9), Fisher & Scriven (1997: 99, 111), and Halpern (1998: 452). Items testing inferential abilities constitute two of the five subtests of the Watson Glaser Critical Thinking Appraisal (Watson & Glaser 1980a, 1980b, 1994), two of the four sections in the Cornell Critical Thinking Test Level X (Ennis & Millman 1971; Ennis, Millman, & Tomko 1985, 2005), three of the seven sections in the Cornell Critical Thinking Test Level Z (Ennis & Millman 1971; Ennis, Millman, & Tomko 1985, 2005), 11 of the 34 items on Forms A and B of the California Critical Thinking Skills Test (Facione 1990b, 1992), and a high but variable proportion of the 25 selected-response questions in the Collegiate Learning Assessment (Council for Aid to Education 2017).

Experimenting abilities : Knowing how to design and execute an experiment is important not just in scientific research but also in everyday life, as in Rash . Dewey devoted a whole chapter of his How We Think (1910: 145–156; 1933: 190–202) to the superiority of experimentation over observation in advancing knowledge. Experimenting abilities come into play at one remove in appraising reports of scientific studies. Skill in designing and executing experiments includes the acknowledged abilities to appraise evidence (Glaser 1941: 6), to carry out experiments and to apply appropriate statistical inference techniques (Facione 1990a: 9), to judge inductions to an explanatory hypothesis (Ennis 1991: 9), and to recognize the need for an adequately large sample size (Halpern 1998). The Cornell Critical Thinking Test Level Z (Ennis & Millman 1971; Ennis, Millman, & Tomko 1985, 2005) includes four items (out of 52) on experimental design. The Collegiate Learning Assessment (Council for Aid to Education 2017) makes room for appraisal of study design in both its performance task and its selected-response questions.

Consulting abilities : Skill at consulting sources of information comes into play when one seeks information to help resolve a problem, as in Candidate . Ability to find and appraise information includes ability to gather and marshal pertinent information (Glaser 1941: 6), to judge whether a statement made by an alleged authority is acceptable (Ennis 1962: 84), to plan a search for desired information (Facione 1990a: 9), and to judge the credibility of a source (Ennis 1991: 9). Ability to judge the credibility of statements is tested by 24 items (out of 76) in the Cornell Critical Thinking Test Level X (Ennis & Millman 1971; Ennis, Millman, & Tomko 1985, 2005) and by four items (out of 52) in the Cornell Critical Thinking Test Level Z (Ennis & Millman 1971; Ennis, Millman, & Tomko 1985, 2005). The College Learning Assessment’s performance task requires evaluation of whether information in documents is credible or unreliable (Council for Aid to Education 2017).

Argument analysis abilities : The ability to identify and analyze arguments contributes to the process of surveying arguments on an issue in order to form one’s own reasoned judgment, as in Candidate . The ability to detect and analyze arguments is recognized as a critical thinking skill by Facione (1990a: 7–8), Ennis (1991: 9) and Halpern (1998). Five items (out of 34) on the California Critical Thinking Skills Test (Facione 1990b, 1992) test skill at argument analysis. The College Learning Assessment (Council for Aid to Education 2017) incorporates argument analysis in its selected-response tests of critical reading and evaluation and of critiquing an argument.

Judging skills and deciding skills : Skill at judging and deciding is skill at recognizing what judgment or decision the available evidence and argument supports, and with what degree of confidence. It is thus a component of the inferential skills already discussed.

Lists and tests of critical thinking abilities often include two more abilities: identifying assumptions and constructing and evaluating definitions.

In addition to dispositions and abilities, critical thinking needs knowledge: of critical thinking concepts, of critical thinking principles, and of the subject-matter of the thinking.

We can derive a short list of concepts whose understanding contributes to critical thinking from the critical thinking abilities described in the preceding section. Observational abilities require an understanding of the difference between observation and inference. Questioning abilities require an understanding of the concepts of ambiguity and vagueness. Inferential abilities require an understanding of the difference between conclusive and defeasible inference (traditionally, between deduction and induction), as well as of the difference between necessary and sufficient conditions. Experimenting abilities require an understanding of the concepts of hypothesis, null hypothesis, assumption and prediction, as well as of the concept of statistical significance and of its difference from importance. They also require an understanding of the difference between an experiment and an observational study, and in particular of the difference between a randomized controlled trial, a prospective correlational study and a retrospective (case-control) study. Argument analysis abilities require an understanding of the concepts of argument, premiss, assumption, conclusion and counter-consideration. Additional critical thinking concepts are proposed by Bailin et al. (1999b: 293), Fisher & Scriven (1997: 105–106), Black (2012), and Blair (2021).

According to Glaser (1941: 25), ability to think critically requires knowledge of the methods of logical inquiry and reasoning. If we review the list of abilities in the preceding section, however, we can see that some of them can be acquired and exercised merely through practice, possibly guided in an educational setting, followed by feedback. Searching intelligently for a causal explanation of some phenomenon or event requires that one consider a full range of possible causal contributors, but it seems more important that one implements this principle in one’s practice than that one is able to articulate it. What is important is “operational knowledge” of the standards and principles of good thinking (Bailin et al. 1999b: 291–293). But the development of such critical thinking abilities as designing an experiment or constructing an operational definition can benefit from learning their underlying theory. Further, explicit knowledge of quirks of human thinking seems useful as a cautionary guide. Human memory is not just fallible about details, as people learn from their own experiences of misremembering, but is so malleable that a detailed, clear and vivid recollection of an event can be a total fabrication (Loftus 2017). People seek or interpret evidence in ways that are partial to their existing beliefs and expectations, often unconscious of their “confirmation bias” (Nickerson 1998). Not only are people subject to this and other cognitive biases (Kahneman 2011), of which they are typically unaware, but it may be counter-productive for one to make oneself aware of them and try consciously to counteract them or to counteract social biases such as racial or sexual stereotypes (Kenyon & Beaulac 2014). It is helpful to be aware of these facts and of the superior effectiveness of blocking the operation of biases—for example, by making an immediate record of one’s observations, refraining from forming a preliminary explanatory hypothesis, blind refereeing, double-blind randomized trials, and blind grading of students’ work. It is also helpful to be aware of the prevalence of “noise” (unwanted unsystematic variability of judgments), of how to detect noise (through a noise audit), and of how to reduce noise: make accuracy the goal, think statistically, break a process of arriving at a judgment into independent tasks, resist premature intuitions, in a group get independent judgments first, favour comparative judgments and scales (Kahneman, Sibony, & Sunstein 2021). It is helpful as well to be aware of the concept of “bounded rationality” in decision-making and of the related distinction between “satisficing” and optimizing (Simon 1956; Gigerenzer 2001).

Critical thinking about an issue requires substantive knowledge of the domain to which the issue belongs. Critical thinking abilities are not a magic elixir that can be applied to any issue whatever by somebody who has no knowledge of the facts relevant to exploring that issue. For example, the student in Bubbles needed to know that gases do not penetrate solid objects like a glass, that air expands when heated, that the volume of an enclosed gas varies directly with its temperature and inversely with its pressure, and that hot objects will spontaneously cool down to the ambient temperature of their surroundings unless kept hot by insulation or a source of heat. Critical thinkers thus need a rich fund of subject-matter knowledge relevant to the variety of situations they encounter. This fact is recognized in the inclusion among critical thinking dispositions of a concern to become and remain generally well informed.

Experimental educational interventions, with control groups, have shown that education can improve critical thinking skills and dispositions, as measured by standardized tests. For information about these tests, see the Supplement on Assessment .

What educational methods are most effective at developing the dispositions, abilities and knowledge of a critical thinker? In a comprehensive meta-analysis of experimental and quasi-experimental studies of strategies for teaching students to think critically, Abrami et al. (2015) found that dialogue, anchored instruction, and mentoring each increased the effectiveness of the educational intervention, and that they were most effective when combined. They also found that in these studies a combination of separate instruction in critical thinking with subject-matter instruction in which students are encouraged to think critically was more effective than either by itself. However, the difference was not statistically significant; that is, it might have arisen by chance.

Most of these studies lack the longitudinal follow-up required to determine whether the observed differential improvements in critical thinking abilities or dispositions continue over time, for example until high school or college graduation. For details on studies of methods of developing critical thinking skills and dispositions, see the Supplement on Educational Methods .

12. Controversies

Scholars have denied the generalizability of critical thinking abilities across subject domains, have alleged bias in critical thinking theory and pedagogy, and have investigated the relationship of critical thinking to other kinds of thinking.

McPeck (1981) attacked the thinking skills movement of the 1970s, including the critical thinking movement. He argued that there are no general thinking skills, since thinking is always thinking about some subject-matter. It is futile, he claimed, for schools and colleges to teach thinking as if it were a separate subject. Rather, teachers should lead their pupils to become autonomous thinkers by teaching school subjects in a way that brings out their cognitive structure and that encourages and rewards discussion and argument. As some of his critics (e.g., Paul 1985; Siegel 1985) pointed out, McPeck’s central argument needs elaboration, since it has obvious counter-examples in writing and speaking, for which (up to a certain level of complexity) there are teachable general abilities even though they are always about some subject-matter. To make his argument convincing, McPeck needs to explain how thinking differs from writing and speaking in a way that does not permit useful abstraction of its components from the subject-matters with which it deals. He has not done so. Nevertheless, his position that the dispositions and abilities of a critical thinker are best developed in the context of subject-matter instruction is shared by many theorists of critical thinking, including Dewey (1910, 1933), Glaser (1941), Passmore (1980), Weinstein (1990), Bailin et al. (1999b), and Willingham (2019).

McPeck’s challenge prompted reflection on the extent to which critical thinking is subject-specific. McPeck argued for a strong subject-specificity thesis, according to which it is a conceptual truth that all critical thinking abilities are specific to a subject. (He did not however extend his subject-specificity thesis to critical thinking dispositions. In particular, he took the disposition to suspend judgment in situations of cognitive dissonance to be a general disposition.) Conceptual subject-specificity is subject to obvious counter-examples, such as the general ability to recognize confusion of necessary and sufficient conditions. A more modest thesis, also endorsed by McPeck, is epistemological subject-specificity, according to which the norms of good thinking vary from one field to another. Epistemological subject-specificity clearly holds to a certain extent; for example, the principles in accordance with which one solves a differential equation are quite different from the principles in accordance with which one determines whether a painting is a genuine Picasso. But the thesis suffers, as Ennis (1989) points out, from vagueness of the concept of a field or subject and from the obvious existence of inter-field principles, however broadly the concept of a field is construed. For example, the principles of hypothetico-deductive reasoning hold for all the varied fields in which such reasoning occurs. A third kind of subject-specificity is empirical subject-specificity, according to which as a matter of empirically observable fact a person with the abilities and dispositions of a critical thinker in one area of investigation will not necessarily have them in another area of investigation.

The thesis of empirical subject-specificity raises the general problem of transfer. If critical thinking abilities and dispositions have to be developed independently in each school subject, how are they of any use in dealing with the problems of everyday life and the political and social issues of contemporary society, most of which do not fit into the framework of a traditional school subject? Proponents of empirical subject-specificity tend to argue that transfer is more likely to occur if there is critical thinking instruction in a variety of domains, with explicit attention to dispositions and abilities that cut across domains. But evidence for this claim is scanty. There is a need for well-designed empirical studies that investigate the conditions that make transfer more likely.

It is common ground in debates about the generality or subject-specificity of critical thinking dispositions and abilities that critical thinking about any topic requires background knowledge about the topic. For example, the most sophisticated understanding of the principles of hypothetico-deductive reasoning is of no help unless accompanied by some knowledge of what might be plausible explanations of some phenomenon under investigation.

Critics have objected to bias in the theory, pedagogy and practice of critical thinking. Commentators (e.g., Alston 1995; Ennis 1998) have noted that anyone who takes a position has a bias in the neutral sense of being inclined in one direction rather than others. The critics, however, are objecting to bias in the pejorative sense of an unjustified favoring of certain ways of knowing over others, frequently alleging that the unjustly favoured ways are those of a dominant sex or culture (Bailin 1995). These ways favour:

• reinforcement of egocentric and sociocentric biases over dialectical engagement with opposing world-views (Paul 1981, 1984; Warren 1998)
• distancing from the object of inquiry over closeness to it (Martin 1992; Thayer-Bacon 1992)
• indifference to the situation of others over care for them (Martin 1992)
• orientation to thought over orientation to action (Martin 1992)
• being reasonable over caring to understand people’s ideas (Thayer-Bacon 1993)
• being neutral and objective over being embodied and situated (Thayer-Bacon 1995a)
• doubting over believing (Thayer-Bacon 1995b)
• reason over emotion, imagination and intuition (Thayer-Bacon 2000)
• solitary thinking over collaborative thinking (Thayer-Bacon 2000)
• written and spoken assignments over other forms of expression (Alston 2001)
• attention to written and spoken communications over attention to human problems (Alston 2001)
• winning debates in the public sphere over making and understanding meaning (Alston 2001)

A common thread in this smorgasbord of accusations is dissatisfaction with focusing on the logical analysis and evaluation of reasoning and arguments. While these authors acknowledge that such analysis and evaluation is part of critical thinking and should be part of its conceptualization and pedagogy, they insist that it is only a part. Paul (1981), for example, bemoans the tendency of atomistic teaching of methods of analyzing and evaluating arguments to turn students into more able sophists, adept at finding fault with positions and arguments with which they disagree but even more entrenched in the egocentric and sociocentric biases with which they began. Martin (1992) and Thayer-Bacon (1992) cite with approval the self-reported intimacy with their subject-matter of leading researchers in biology and medicine, an intimacy that conflicts with the distancing allegedly recommended in standard conceptions and pedagogy of critical thinking. Thayer-Bacon (2000) contrasts the embodied and socially embedded learning of her elementary school students in a Montessori school, who used their imagination, intuition and emotions as well as their reason, with conceptions of critical thinking as

thinking that is used to critique arguments, offer justifications, and make judgments about what are the good reasons, or the right answers. (Thayer-Bacon 2000: 127–128)

Alston (2001) reports that her students in a women’s studies class were able to see the flaws in the Cinderella myth that pervades much romantic fiction but in their own romantic relationships still acted as if all failures were the woman’s fault and still accepted the notions of love at first sight and living happily ever after. Students, she writes, should

be able to connect their intellectual critique to a more affective, somatic, and ethical account of making risky choices that have sexist, racist, classist, familial, sexual, or other consequences for themselves and those both near and far… critical thinking that reads arguments, texts, or practices merely on the surface without connections to feeling/desiring/doing or action lacks an ethical depth that should infuse the difference between mere cognitive activity and something we want to call critical thinking. (Alston 2001: 34)

Some critics portray such biases as unfair to women. Thayer-Bacon (1992), for example, has charged modern critical thinking theory with being sexist, on the ground that it separates the self from the object and causes one to lose touch with one’s inner voice, and thus stigmatizes women, who (she asserts) link self to object and listen to their inner voice. Her charge does not imply that women as a group are on average less able than men to analyze and evaluate arguments. Facione (1990c) found no difference by sex in performance on his California Critical Thinking Skills Test. Kuhn (1991: 280–281) found no difference by sex in either the disposition or the competence to engage in argumentative thinking.

The critics propose a variety of remedies for the biases that they allege. In general, they do not propose to eliminate or downplay critical thinking as an educational goal. Rather, they propose to conceptualize critical thinking differently and to change its pedagogy accordingly. Their pedagogical proposals arise logically from their objections. They can be summarized as follows:

• Focus on argument networks with dialectical exchanges reflecting contesting points of view rather than on atomic arguments, so as to develop “strong sense” critical thinking that transcends egocentric and sociocentric biases (Paul 1981, 1984).
• Foster closeness to the subject-matter and feeling connected to others in order to inform a humane democracy (Martin 1992).
• Develop “constructive thinking” as a social activity in a community of physically embodied and socially embedded inquirers with personal voices who value not only reason but also imagination, intuition and emotion (Thayer-Bacon 2000).
• In developing critical thinking in school subjects, treat as important neither skills nor dispositions but opening worlds of meaning (Alston 2001).
• Attend to the development of critical thinking dispositions as well as skills, and adopt the “critical pedagogy” practised and advocated by Freire (1968 [1970]) and hooks (1994) (Dalgleish, Girard, & Davies 2017).

A common thread in these proposals is treatment of critical thinking as a social, interactive, personally engaged activity like that of a quilting bee or a barn-raising (Thayer-Bacon 2000) rather than as an individual, solitary, distanced activity symbolized by Rodin’s The Thinker . One can get a vivid description of education with the former type of goal from the writings of bell hooks (1994, 2010). Critical thinking for her is open-minded dialectical exchange across opposing standpoints and from multiple perspectives, a conception similar to Paul’s “strong sense” critical thinking (Paul 1981). She abandons the structure of domination in the traditional classroom. In an introductory course on black women writers, for example, she assigns students to write an autobiographical paragraph about an early racial memory, then to read it aloud as the others listen, thus affirming the uniqueness and value of each voice and creating a communal awareness of the diversity of the group’s experiences (hooks 1994: 84). Her “engaged pedagogy” is thus similar to the “freedom under guidance” implemented in John Dewey’s Laboratory School of Chicago in the late 1890s and early 1900s. It incorporates the dialogue, anchored instruction, and mentoring that Abrami (2015) found to be most effective in improving critical thinking skills and dispositions.

What is the relationship of critical thinking to problem solving, decision-making, higher-order thinking, creative thinking, and other recognized types of thinking? One’s answer to this question obviously depends on how one defines the terms used in the question. If critical thinking is conceived broadly to cover any careful thinking about any topic for any purpose, then problem solving and decision making will be kinds of critical thinking, if they are done carefully. Historically, ‘critical thinking’ and ‘problem solving’ were two names for the same thing. If critical thinking is conceived more narrowly as consisting solely of appraisal of intellectual products, then it will be disjoint with problem solving and decision making, which are constructive.

Bloom’s taxonomy of educational objectives used the phrase “intellectual abilities and skills” for what had been labeled “critical thinking” by some, “reflective thinking” by Dewey and others, and “problem solving” by still others (Bloom et al. 1956: 38). Thus, the so-called “higher-order thinking skills” at the taxonomy’s top levels of analysis, synthesis and evaluation are just critical thinking skills, although they do not come with general criteria for their assessment (Ennis 1981b). The revised version of Bloom’s taxonomy (Anderson et al. 2001) likewise treats critical thinking as cutting across those types of cognitive process that involve more than remembering (Anderson et al. 2001: 269–270). For details, see the Supplement on History .

As to creative thinking, it overlaps with critical thinking (Bailin 1987, 1988). Thinking about the explanation of some phenomenon or event, as in Ferryboat , requires creative imagination in constructing plausible explanatory hypotheses. Likewise, thinking about a policy question, as in Candidate , requires creativity in coming up with options. Conversely, creativity in any field needs to be balanced by critical appraisal of the draft painting or novel or mathematical theory.

• Abrami, Philip C., Robert M. Bernard, Eugene Borokhovski, David I. Waddington, C. Anne Wade, and Tonje Person, 2015, “Strategies for Teaching Students to Think Critically: A Meta-analysis”, Review of Educational Research , 85(2): 275–314. doi:10.3102/0034654314551063
• Aikin, Wilford M., 1942, The Story of the Eight-year Study, with Conclusions and Recommendations , Volume I of Adventure in American Education , New York and London: Harper & Brothers. [ Aikin 1942 available online ]
• Alston, Kal, 1995, “Begging the Question: Is Critical Thinking Biased?”, Educational Theory , 45(2): 225–233. doi:10.1111/j.1741-5446.1995.00225.x
• –––, 2001, “Re/Thinking Critical Thinking: The Seductions of Everyday Life”, Studies in Philosophy and Education , 20(1): 27–40. doi:10.1023/A:1005247128053
• American Educational Research Association, 2014, Standards for Educational and Psychological Testing / American Educational Research Association, American Psychological Association, National Council on Measurement in Education , Washington, DC: American Educational Research Association.
• Anderson, Lorin W., David R. Krathwohl, Peter W. Airiasian, Kathleen A. Cruikshank, Richard E. Mayer, Paul R. Pintrich, James Raths, and Merlin C. Wittrock, 2001, A Taxonomy for Learning, Teaching and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives , New York: Longman, complete edition.
• Bailin, Sharon, 1987, “Critical and Creative Thinking”, Informal Logic , 9(1): 23–30. [ Bailin 1987 available online ]
• –––, 1988, Achieving Extraordinary Ends: An Essay on Creativity , Dordrecht: Kluwer. doi:10.1007/978-94-009-2780-3
• –––, 1995, “Is Critical Thinking Biased? Clarifications and Implications”, Educational Theory , 45(2): 191–197. doi:10.1111/j.1741-5446.1995.00191.x
• Bailin, Sharon and Mark Battersby, 2009, “Inquiry: A Dialectical Approach to Teaching Critical Thinking”, in Juho Ritola (ed.), Argument Cultures: Proceedings of OSSA 09 , CD-ROM (pp. 1–10), Windsor, ON: OSSA. [ Bailin & Battersby 2009 available online ]
• –––, 2016a, “Fostering the Virtues of Inquiry”, Topoi , 35(2): 367–374. doi:10.1007/s11245-015-9307-6
• –––, 2016b, Reason in the Balance: An Inquiry Approach to Critical Thinking , Indianapolis: Hackett, 2nd edition.
• –––, 2021, “Inquiry: Teaching for Reasoned Judgment”, in Daniel Fasko, Jr. and Frank Fair (eds.), Critical Thinking and Reasoning: Theory, Development, Instruction, and Assessment , Leiden: Brill, pp. 31–46. doi: 10.1163/9789004444591_003
• Bailin, Sharon, Roland Case, Jerrold R. Coombs, and Leroi B. Daniels, 1999a, “Common Misconceptions of Critical Thinking”, Journal of Curriculum Studies , 31(3): 269–283. doi:10.1080/002202799183124
• –––, 1999b, “Conceptualizing Critical Thinking”, Journal of Curriculum Studies , 31(3): 285–302. doi:10.1080/002202799183133
• Blair, J. Anthony, 2021, Studies in Critical Thinking , Windsor, ON: Windsor Studies in Argumentation, 2nd edition. [Available online at https://windsor.scholarsportal.info/omp/index.php/wsia/catalog/book/106]
• Berman, Alan M., Seth J. Schwartz, William M. Kurtines, and Steven L. Berman, 2001, “The Process of Exploration in Identity Formation: The Role of Style and Competence”, Journal of Adolescence , 24(4): 513–528. doi:10.1006/jado.2001.0386
• Black, Beth (ed.), 2012, An A to Z of Critical Thinking , London: Continuum International Publishing Group.
• Bloom, Benjamin Samuel, Max D. Engelhart, Edward J. Furst, Walter H. Hill, and David R. Krathwohl, 1956, Taxonomy of Educational Objectives. Handbook I: Cognitive Domain , New York: David McKay.
• Boardman, Frank, Nancy M. Cavender, and Howard Kahane, 2018, Logic and Contemporary Rhetoric: The Use of Reason in Everyday Life , Boston: Cengage, 13th edition.
• Browne, M. Neil and Stuart M. Keeley, 2018, Asking the Right Questions: A Guide to Critical Thinking , Hoboken, NJ: Pearson, 12th edition.
• Center for Assessment & Improvement of Learning, 2017, Critical Thinking Assessment Test , Cookeville, TN: Tennessee Technological University.
• Cleghorn, Paul. 2021. “Critical Thinking in the Elementary School: Practical Guidance for Building a Culture of Thinking”, in Daniel Fasko, Jr. and Frank Fair (eds.), Critical Thinking and Reasoning: Theory, Development, Instruction, and Assessmen t, Leiden: Brill, pp. 150–167. doi: 10.1163/9789004444591_010
• Cohen, Jacob, 1988, Statistical Power Analysis for the Behavioral Sciences , Hillsdale, NJ: Lawrence Erlbaum Associates, 2nd edition.
• College Board, 1983, Academic Preparation for College. What Students Need to Know and Be Able to Do , New York: College Entrance Examination Board, ERIC document ED232517.
• Commission on the Relation of School and College of the Progressive Education Association, 1943, Thirty Schools Tell Their Story , Volume V of Adventure in American Education , New York and London: Harper & Brothers.
• Council for Aid to Education, 2017, CLA+ Student Guide . Available at http://cae.org/images/uploads/pdf/CLA_Student_Guide_Institution.pdf ; last accessed 2022 07 16.
• Dalgleish, Adam, Patrick Girard, and Maree Davies, 2017, “Critical Thinking, Bias and Feminist Philosophy: Building a Better Framework through Collaboration”, Informal Logic , 37(4): 351–369. [ Dalgleish et al. available online ]
• Dewey, John, 1910, How We Think , Boston: D.C. Heath. [ Dewey 1910 available online ]
• –––, 1916, Democracy and Education: An Introduction to the Philosophy of Education , New York: Macmillan.
• –––, 1933, How We Think: A Restatement of the Relation of Reflective Thinking to the Educative Process , Lexington, MA: D.C. Heath.
• –––, 1936, “The Theory of the Chicago Experiment”, Appendix II of Mayhew & Edwards 1936: 463–477.
• –––, 1938, Logic: The Theory of Inquiry , New York: Henry Holt and Company.
• Dominguez, Caroline (coord.), 2018a, A European Collection of the Critical Thinking Skills and Dispositions Needed in Different Professional Fields for the 21st Century , Vila Real, Portugal: UTAD. Available at http://bit.ly/CRITHINKEDUO1 ; last accessed 2022 07 16.
• ––– (coord.), 2018b, A European Review on Critical Thinking Educational Practices in Higher Education Institutions , Vila Real: UTAD. Available at http://bit.ly/CRITHINKEDUO2 ; last accessed 2022 07 16.
• ––– (coord.), 2018c, The CRITHINKEDU European Course on Critical Thinking Education for University Teachers: From Conception to Delivery , Vila Real: UTAD. Available at http:/bit.ly/CRITHINKEDU03; last accessed 2022 07 16.
• Dominguez Caroline and Rita Payan-Carreira (eds.), 2019, Promoting Critical Thinking in European Higher Education Institutions: Towards an Educational Protocol , Vila Real: UTAD. Available at http:/bit.ly/CRITHINKEDU04; last accessed 2022 07 16.
• Ennis, Robert H., 1958, “An Appraisal of the Watson-Glaser Critical Thinking Appraisal”, The Journal of Educational Research , 52(4): 155–158. doi:10.1080/00220671.1958.10882558
• –––, 1962, “A Concept of Critical Thinking: A Proposed Basis for Research on the Teaching and Evaluation of Critical Thinking Ability”, Harvard Educational Review , 32(1): 81–111.
• –––, 1981a, “A Conception of Deductive Logical Competence”, Teaching Philosophy , 4(3/4): 337–385. doi:10.5840/teachphil198143/429
• –––, 1981b, “Eight Fallacies in Bloom’s Taxonomy”, in C. J. B. Macmillan (ed.), Philosophy of Education 1980: Proceedings of the Thirty-seventh Annual Meeting of the Philosophy of Education Society , Bloomington, IL: Philosophy of Education Society, pp. 269–273.
• –––, 1984, “Problems in Testing Informal Logic, Critical Thinking, Reasoning Ability”, Informal Logic , 6(1): 3–9. [ Ennis 1984 available online ]
• –––, 1987, “A Taxonomy of Critical Thinking Dispositions and Abilities”, in Joan Boykoff Baron and Robert J. Sternberg (eds.), Teaching Thinking Skills: Theory and Practice , New York: W. H. Freeman, pp. 9–26.
• –––, 1989, “Critical Thinking and Subject Specificity: Clarification and Needed Research”, Educational Researcher , 18(3): 4–10. doi:10.3102/0013189X018003004
• –––, 1991, “Critical Thinking: A Streamlined Conception”, Teaching Philosophy , 14(1): 5–24. doi:10.5840/teachphil19911412
• –––, 1996, “Critical Thinking Dispositions: Their Nature and Assessability”, Informal Logic , 18(2–3): 165–182. [ Ennis 1996 available online ]
• –––, 1998, “Is Critical Thinking Culturally Biased?”, Teaching Philosophy , 21(1): 15–33. doi:10.5840/teachphil19982113
• –––, 2011, “Critical Thinking: Reflection and Perspective Part I”, Inquiry: Critical Thinking across the Disciplines , 26(1): 4–18. doi:10.5840/inquiryctnews20112613
• –––, 2013, “Critical Thinking across the Curriculum: The Wisdom CTAC Program”, Inquiry: Critical Thinking across the Disciplines , 28(2): 25–45. doi:10.5840/inquiryct20132828
• –––, 2016, “Definition: A Three-Dimensional Analysis with Bearing on Key Concepts”, in Patrick Bondy and Laura Benacquista (eds.), Argumentation, Objectivity, and Bias: Proceedings of the 11th International Conference of the Ontario Society for the Study of Argumentation (OSSA), 18–21 May 2016 , Windsor, ON: OSSA, pp. 1–19. Available at http://scholar.uwindsor.ca/ossaarchive/OSSA11/papersandcommentaries/105 ; last accessed 2022 07 16.
• –––, 2018, “Critical Thinking Across the Curriculum: A Vision”, Topoi , 37(1): 165–184. doi:10.1007/s11245-016-9401-4
• Ennis, Robert H., and Jason Millman, 1971, Manual for Cornell Critical Thinking Test, Level X, and Cornell Critical Thinking Test, Level Z , Urbana, IL: Critical Thinking Project, University of Illinois.
• Ennis, Robert H., Jason Millman, and Thomas Norbert Tomko, 1985, Cornell Critical Thinking Tests Level X & Level Z: Manual , Pacific Grove, CA: Midwest Publication, 3rd edition.
• –––, 2005, Cornell Critical Thinking Tests Level X & Level Z: Manual , Seaside, CA: Critical Thinking Company, 5th edition.
• Ennis, Robert H. and Eric Weir, 1985, The Ennis-Weir Critical Thinking Essay Test: Test, Manual, Criteria, Scoring Sheet: An Instrument for Teaching and Testing , Pacific Grove, CA: Midwest Publications.
• Facione, Peter A., 1990a, Critical Thinking: A Statement of Expert Consensus for Purposes of Educational Assessment and Instruction , Research Findings and Recommendations Prepared for the Committee on Pre-College Philosophy of the American Philosophical Association, ERIC Document ED315423.
• –––, 1990b, California Critical Thinking Skills Test, CCTST – Form A , Millbrae, CA: The California Academic Press.
• –––, 1990c, The California Critical Thinking Skills Test--College Level. Technical Report #3. Gender, Ethnicity, Major, CT Self-Esteem, and the CCTST , ERIC Document ED326584.
• –––, 1992, California Critical Thinking Skills Test: CCTST – Form B, Millbrae, CA: The California Academic Press.
• –––, 2000, “The Disposition Toward Critical Thinking: Its Character, Measurement, and Relationship to Critical Thinking Skill”, Informal Logic , 20(1): 61–84. [ Facione 2000 available online ]
• Facione, Peter A. and Noreen C. Facione, 1992, CCTDI: A Disposition Inventory , Millbrae, CA: The California Academic Press.
• Facione, Peter A., Noreen C. Facione, and Carol Ann F. Giancarlo, 2001, California Critical Thinking Disposition Inventory: CCTDI: Inventory Manual , Millbrae, CA: The California Academic Press.
• Facione, Peter A., Carol A. Sánchez, and Noreen C. Facione, 1994, Are College Students Disposed to Think? , Millbrae, CA: The California Academic Press. ERIC Document ED368311.
• Fisher, Alec, and Michael Scriven, 1997, Critical Thinking: Its Definition and Assessment , Norwich: Centre for Research in Critical Thinking, University of East Anglia.
• Freire, Paulo, 1968 [1970], Pedagogia do Oprimido . Translated as Pedagogy of the Oppressed , Myra Bergman Ramos (trans.), New York: Continuum, 1970.
• Gigerenzer, Gerd, 2001, “The Adaptive Toolbox”, in Gerd Gigerenzer and Reinhard Selten (eds.), Bounded Rationality: The Adaptive Toolbox , Cambridge, MA: MIT Press, pp. 37–50.
• Glaser, Edward Maynard, 1941, An Experiment in the Development of Critical Thinking , New York: Bureau of Publications, Teachers College, Columbia University.
• Groarke, Leo A. and Christopher W. Tindale, 2012, Good Reasoning Matters! A Constructive Approach to Critical Thinking , Don Mills, ON: Oxford University Press, 5th edition.
• Halpern, Diane F., 1998, “Teaching Critical Thinking for Transfer Across Domains: Disposition, Skills, Structure Training, and Metacognitive Monitoring”, American Psychologist , 53(4): 449–455. doi:10.1037/0003-066X.53.4.449
• –––, 2016, Manual: Halpern Critical Thinking Assessment , Mödling, Austria: Schuhfried. Available at https://pdfcoffee.com/hcta-test-manual-pdf-free.html; last accessed 2022 07 16.
• Hamby, Benjamin, 2014, The Virtues of Critical Thinkers , Doctoral dissertation, Philosophy, McMaster University. [ Hamby 2014 available online ]
• –––, 2015, “Willingness to Inquire: The Cardinal Critical Thinking Virtue”, in Martin Davies and Ronald Barnett (eds.), The Palgrave Handbook of Critical Thinking in Higher Education , New York: Palgrave Macmillan, pp. 77–87.
• Haran, Uriel, Ilana Ritov, and Barbara A. Mellers, 2013, “The Role of Actively Open-minded Thinking in Information Acquisition, Accuracy, and Calibration”, Judgment and Decision Making , 8(3): 188–201.
• Hatcher, Donald and Kevin Possin, 2021, “Commentary: Thinking Critically about Critical Thinking Assessment”, in Daniel Fasko, Jr. and Frank Fair (eds.), Critical Thinking and Reasoning: Theory, Development, Instruction, and Assessment , Leiden: Brill, pp. 298–322. doi: 10.1163/9789004444591_017
• Haynes, Ada, Elizabeth Lisic, Kevin Harris, Katie Leming, Kyle Shanks, and Barry Stein, 2015, “Using the Critical Thinking Assessment Test (CAT) as a Model for Designing Within-Course Assessments: Changing How Faculty Assess Student Learning”, Inquiry: Critical Thinking Across the Disciplines , 30(3): 38–48. doi:10.5840/inquiryct201530316
• Haynes, Ada and Barry Stein, 2021, “Observations from a Long-Term Effort to Assess and Improve Critical Thinking”, in Daniel Fasko, Jr. and Frank Fair (eds.), Critical Thinking and Reasoning: Theory, Development, Instruction, and Assessment , Leiden: Brill, pp. 231–254. doi: 10.1163/9789004444591_014
• Hiner, Amanda L. 2021. “Equipping Students for Success in College and Beyond: Placing Critical Thinking Instruction at the Heart of a General Education Program”, in Daniel Fasko, Jr. and Frank Fair (eds.), Critical Thinking and Reasoning: Theory, Development, Instruction, and Assessment , Leiden: Brill, pp. 188–208. doi: 10.1163/9789004444591_012
• Hitchcock, David, 2017, “Critical Thinking as an Educational Ideal”, in his On Reasoning and Argument: Essays in Informal Logic and on Critical Thinking , Dordrecht: Springer, pp. 477–497. doi:10.1007/978-3-319-53562-3_30
• –––, 2021, “Seven Philosophical Implications of Critical Thinking: Themes, Variations, Implications”, in Daniel Fasko, Jr. and Frank Fair (eds.), Critical Thinking and Reasoning: Theory, Development, Instruction, and Assessment , Leiden: Brill, pp. 9–30. doi: 10.1163/9789004444591_002
• hooks, bell, 1994, Teaching to Transgress: Education as the Practice of Freedom , New York and London: Routledge.
• –––, 2010, Teaching Critical Thinking: Practical Wisdom , New York and London: Routledge.
• Johnson, Ralph H., 1992, “The Problem of Defining Critical Thinking”, in Stephen P, Norris (ed.), The Generalizability of Critical Thinking , New York: Teachers College Press, pp. 38–53.
• Kahane, Howard, 1971, Logic and Contemporary Rhetoric: The Use of Reason in Everyday Life , Belmont, CA: Wadsworth.
• Kahneman, Daniel, 2011, Thinking, Fast and Slow , New York: Farrar, Straus and Giroux.
• Kahneman, Daniel, Olivier Sibony, & Cass R. Sunstein, 2021, Noise: A Flaw in Human Judgment , New York: Little, Brown Spark.
• Kenyon, Tim, and Guillaume Beaulac, 2014, “Critical Thinking Education and Debasing”, Informal Logic , 34(4): 341–363. [ Kenyon & Beaulac 2014 available online ]
• Krathwohl, David R., Benjamin S. Bloom, and Bertram B. Masia, 1964, Taxonomy of Educational Objectives, Handbook II: Affective Domain , New York: David McKay.
• Kuhn, Deanna, 1991, The Skills of Argument , New York: Cambridge University Press. doi:10.1017/CBO9780511571350
• –––, 2019, “Critical Thinking as Discourse”, Human Development, 62 (3): 146–164. doi:10.1159/000500171
• Lipman, Matthew, 1987, “Critical Thinking–What Can It Be?”, Analytic Teaching , 8(1): 5–12. [ Lipman 1987 available online ]
• –––, 2003, Thinking in Education , Cambridge: Cambridge University Press, 2nd edition.
• Loftus, Elizabeth F., 2017, “Eavesdropping on Memory”, Annual Review of Psychology , 68: 1–18. doi:10.1146/annurev-psych-010416-044138
• Makaiau, Amber Strong, 2021, “The Good Thinker’s Tool Kit: How to Engage Critical Thinking and Reasoning in Secondary Education”, in Daniel Fasko, Jr. and Frank Fair (eds.), Critical Thinking and Reasoning: Theory, Development, Instruction, and Assessment , Leiden: Brill, pp. 168–187. doi: 10.1163/9789004444591_011
• Martin, Jane Roland, 1992, “Critical Thinking for a Humane World”, in Stephen P. Norris (ed.), The Generalizability of Critical Thinking , New York: Teachers College Press, pp. 163–180.
• Mayhew, Katherine Camp, and Anna Camp Edwards, 1936, The Dewey School: The Laboratory School of the University of Chicago, 1896–1903 , New York: Appleton-Century. [ Mayhew & Edwards 1936 available online ]
• McPeck, John E., 1981, Critical Thinking and Education , New York: St. Martin’s Press.
• Moore, Brooke Noel and Richard Parker, 2020, Critical Thinking , New York: McGraw-Hill, 13th edition.
• Nickerson, Raymond S., 1998, “Confirmation Bias: A Ubiquitous Phenomenon in Many Guises”, Review of General Psychology , 2(2): 175–220. doi:10.1037/1089-2680.2.2.175
• Nieto, Ana Maria, and Jorge Valenzuela, 2012, “A Study of the Internal Structure of Critical Thinking Dispositions”, Inquiry: Critical Thinking across the Disciplines , 27(1): 31–38. doi:10.5840/inquiryct20122713
• Norris, Stephen P., 1985, “Controlling for Background Beliefs When Developing Multiple-choice Critical Thinking Tests”, Educational Measurement: Issues and Practice , 7(3): 5–11. doi:10.1111/j.1745-3992.1988.tb00437.x
• Norris, Stephen P. and Robert H. Ennis, 1989, Evaluating Critical Thinking (The Practitioners’ Guide to Teaching Thinking Series), Pacific Grove, CA: Midwest Publications.
• Norris, Stephen P. and Ruth Elizabeth King, 1983, Test on Appraising Observations , St. John’s, NL: Institute for Educational Research and Development, Memorial University of Newfoundland.
• –––, 1984, The Design of a Critical Thinking Test on Appraising Observations , St. John’s, NL: Institute for Educational Research and Development, Memorial University of Newfoundland. ERIC Document ED260083.
• –––, 1985, Test on Appraising Observations: Manual , St. John’s, NL: Institute for Educational Research and Development, Memorial University of Newfoundland.
• –––, 1990a, Test on Appraising Observations , St. John’s, NL: Institute for Educational Research and Development, Memorial University of Newfoundland, 2nd edition.
• –––, 1990b, Test on Appraising Observations: Manual , St. John’s, NL: Institute for Educational Research and Development, Memorial University of Newfoundland, 2nd edition.
• OCR [Oxford, Cambridge and RSA Examinations], 2011, AS/A Level GCE: Critical Thinking – H052, H452 , Cambridge: OCR. Past papers available at https://pastpapers.co/ocr/?dir=A-Level/Critical-Thinking-H052-H452; last accessed 2022 07 16.
• Ontario Ministry of Education, 2013, The Ontario Curriculum Grades 9 to 12: Social Sciences and Humanities . Available at http://www.edu.gov.on.ca/eng/curriculum/secondary/ssciences9to122013.pdf ; last accessed 2022 07 16.
• Passmore, John Arthur, 1980, The Philosophy of Teaching , London: Duckworth.
• Paul, Richard W., 1981, “Teaching Critical Thinking in the ‘Strong’ Sense: A Focus on Self-Deception, World Views, and a Dialectical Mode of Analysis”, Informal Logic , 4(2): 2–7. [ Paul 1981 available online ]
• –––, 1984, “Critical Thinking: Fundamental to Education for a Free Society”, Educational Leadership , 42(1): 4–14.
• –––, 1985, “McPeck’s Mistakes”, Informal Logic , 7(1): 35–43. [ Paul 1985 available online ]
• Paul, Richard W. and Linda Elder, 2006, The Miniature Guide to Critical Thinking: Concepts and Tools , Dillon Beach, CA: Foundation for Critical Thinking, 4th edition.
• Payette, Patricia, and Edna Ross, 2016, “Making a Campus-Wide Commitment to Critical Thinking: Insights and Promising Practices Utilizing the Paul-Elder Approach at the University of Louisville”, Inquiry: Critical Thinking Across the Disciplines , 31(1): 98–110. doi:10.5840/inquiryct20163118
• Possin, Kevin, 2008, “A Field Guide to Critical-Thinking Assessment”, Teaching Philosophy , 31(3): 201–228. doi:10.5840/teachphil200831324
• –––, 2013a, “Some Problems with the Halpern Critical Thinking Assessment (HCTA) Test”, Inquiry: Critical Thinking across the Disciplines , 28(3): 4–12. doi:10.5840/inquiryct201328313
• –––, 2013b, “A Serious Flaw in the Collegiate Learning Assessment (CLA) Test”, Informal Logic , 33(3): 390–405. [ Possin 2013b available online ]
• –––, 2013c, “A Fatal Flaw in the Collegiate Learning Assessment Test”, Assessment Update , 25 (1): 8–12.
• –––, 2014, “Critique of the Watson-Glaser Critical Thinking Appraisal Test: The More You Know, the Lower Your Score”, Informal Logic , 34(4): 393–416. [ Possin 2014 available online ]
• –––, 2020, “CAT Scan: A Critical Review of the Critical-Thinking Assessment Test”, Informal Logic , 40 (3): 489–508. [Available online at https://informallogic.ca/index.php/informal_logic/article/view/6243]
• Rawls, John, 1971, A Theory of Justice , Cambridge, MA: Harvard University Press.
• Rear, David, 2019, “One Size Fits All? The Limitations of Standardised Assessment in Critical Thinking”, Assessment & Evaluation in Higher Education , 44(5): 664–675. doi: 10.1080/02602938.2018.1526255
• Rousseau, Jean-Jacques, 1762, Émile , Amsterdam: Jean Néaulme.
• Scheffler, Israel, 1960, The Language of Education , Springfield, IL: Charles C. Thomas.
• Scriven, Michael, and Richard W. Paul, 1987, Defining Critical Thinking , Draft statement written for the National Council for Excellence in Critical Thinking Instruction. Available at http://www.criticalthinking.org/pages/defining-critical-thinking/766 ; last accessed 2022 07 16.
• Sheffield, Clarence Burton Jr., 2018, “Promoting Critical Thinking in Higher Education: My Experiences as the Inaugural Eugene H. Fram Chair in Applied Critical Thinking at Rochester Institute of Technology”, Topoi , 37(1): 155–163. doi:10.1007/s11245-016-9392-1
• Siegel, Harvey, 1985, “McPeck, Informal Logic and the Nature of Critical Thinking”, in David Nyberg (ed.), Philosophy of Education 1985: Proceedings of the Forty-First Annual Meeting of the Philosophy of Education Society , Normal, IL: Philosophy of Education Society, pp. 61–72.
• –––, 1988, Educating Reason: Rationality, Critical Thinking, and Education , New York: Routledge.
• –––, 1999, “What (Good) Are Thinking Dispositions?”, Educational Theory , 49(2): 207–221. doi:10.1111/j.1741-5446.1999.00207.x
• Simon, Herbert A., 1956, “Rational Choice and the Structure of the Environment”, Psychological Review , 63(2): 129–138. doi: 10.1037/h0042769
• Simpson, Elizabeth, 1966–67, “The Classification of Educational Objectives: Psychomotor Domain”, Illinois Teacher of Home Economics , 10(4): 110–144, ERIC document ED0103613. [ Simpson 1966–67 available online ]
• Skolverket, 2018, Curriculum for the Compulsory School, Preschool Class and School-age Educare , Stockholm: Skolverket, revised 2018. Available at https://www.skolverket.se/download/18.31c292d516e7445866a218f/1576654682907/pdf3984.pdf; last accessed 2022 07 15.
• Smith, B. Othanel, 1953, “The Improvement of Critical Thinking”, Progressive Education , 30(5): 129–134.
• Smith, Eugene Randolph, Ralph Winfred Tyler, and the Evaluation Staff, 1942, Appraising and Recording Student Progress , Volume III of Adventure in American Education , New York and London: Harper & Brothers.
• Splitter, Laurance J., 1987, “Educational Reform through Philosophy for Children”, Thinking: The Journal of Philosophy for Children , 7(2): 32–39. doi:10.5840/thinking1987729
• Stanovich Keith E., and Paula J. Stanovich, 2010, “A Framework for Critical Thinking, Rational Thinking, and Intelligence”, in David D. Preiss and Robert J. Sternberg (eds), Innovations in Educational Psychology: Perspectives on Learning, Teaching and Human Development , New York: Springer Publishing, pp 195–237.
• Stanovich Keith E., Richard F. West, and Maggie E. Toplak, 2011, “Intelligence and Rationality”, in Robert J. Sternberg and Scott Barry Kaufman (eds.), Cambridge Handbook of Intelligence , Cambridge: Cambridge University Press, 3rd edition, pp. 784–826. doi:10.1017/CBO9780511977244.040
• Tankersley, Karen, 2005, Literacy Strategies for Grades 4–12: Reinforcing the Threads of Reading , Alexandria, VA: Association for Supervision and Curriculum Development.
• Thayer-Bacon, Barbara J., 1992, “Is Modern Critical Thinking Theory Sexist?”, Inquiry: Critical Thinking Across the Disciplines , 10(1): 3–7. doi:10.5840/inquiryctnews199210123
• –––, 1993, “Caring and Its Relationship to Critical Thinking”, Educational Theory , 43(3): 323–340. doi:10.1111/j.1741-5446.1993.00323.x
• –––, 1995a, “Constructive Thinking: Personal Voice”, Journal of Thought , 30(1): 55–70.
• –––, 1995b, “Doubting and Believing: Both are Important for Critical Thinking”, Inquiry: Critical Thinking across the Disciplines , 15(2): 59–66. doi:10.5840/inquiryctnews199515226
• –––, 2000, Transforming Critical Thinking: Thinking Constructively , New York: Teachers College Press.
• Toulmin, Stephen Edelston, 1958, The Uses of Argument , Cambridge: Cambridge University Press.
• Turri, John, Mark Alfano, and John Greco, 2017, “Virtue Epistemology”, in Edward N. Zalta (ed.), The Stanford Encyclopedia of Philosophy (Winter 2017 Edition). URL = < https://plato.stanford.edu/archives/win2017/entries/epistemology-virtue/ >
• Vincent-Lancrin, Stéphan, Carlos González-Sancho, Mathias Bouckaert, Federico de Luca, Meritxell Fernández-Barrerra, Gwénaël Jacotin, Joaquin Urgel, and Quentin Vidal, 2019, Fostering Students’ Creativity and Critical Thinking: What It Means in School. Educational Research and Innovation , Paris: OECD Publishing.
• Warren, Karen J. 1988. “Critical Thinking and Feminism”, Informal Logic , 10(1): 31–44. [ Warren 1988 available online ]
• Watson, Goodwin, and Edward M. Glaser, 1980a, Watson-Glaser Critical Thinking Appraisal, Form A , San Antonio, TX: Psychological Corporation.
• –––, 1980b, Watson-Glaser Critical Thinking Appraisal: Forms A and B; Manual , San Antonio, TX: Psychological Corporation,
• –––, 1994, Watson-Glaser Critical Thinking Appraisal, Form B , San Antonio, TX: Psychological Corporation.
• Weinstein, Mark, 1990, “Towards a Research Agenda for Informal Logic and Critical Thinking”, Informal Logic , 12(3): 121–143. [ Weinstein 1990 available online ]
• –––, 2013, Logic, Truth and Inquiry , London: College Publications.
• Willingham, Daniel T., 2019, “How to Teach Critical Thinking”, Education: Future Frontiers , 1: 1–17. [Available online at https://prod65.education.nsw.gov.au/content/dam/main-education/teaching-and-learning/education-for-a-changing-world/media/documents/How-to-teach-critical-thinking-Willingham.pdf.]
• Zagzebski, Linda Trinkaus, 1996, Virtues of the Mind: An Inquiry into the Nature of Virtue and the Ethical Foundations of Knowledge , Cambridge: Cambridge University Press. doi:10.1017/CBO9781139174763

How to cite this entry . Preview the PDF version of this entry at the Friends of the SEP Society . Look up topics and thinkers related to this entry at the Internet Philosophy Ontology Project (InPhO). Enhanced bibliography for this entry at PhilPapers , with links to its database.

• Association for Informal Logic and Critical Thinking (AILACT)
• Critical Thinking Across the European Higher Education Curricula (CRITHINKEDU)
• Critical Thinking Definition, Instruction, and Assessment: A Rigorous Approach
• Critical Thinking Research (RAIL)
• Foundation for Critical Thinking
• Insight Assessment
• Partnership for 21st Century Learning (P21)
• The Critical Thinking Consortium
• The Nature of Critical Thinking: An Outline of Critical Thinking Dispositions and Abilities , by Robert H. Ennis

abilities | bias, implicit | children, philosophy for | civic education | decision-making capacity | Dewey, John | dispositions | education, philosophy of | epistemology: virtue | logic: informal

Copyright © 2022 by David Hitchcock < hitchckd @ mcmaster . ca >

• Accessibility

Support SEP

Mirror sites.

View this site from another server:

• Info about mirror sites

The Stanford Encyclopedia of Philosophy is copyright © 2024 by The Metaphysics Research Lab , Department of Philosophy, Stanford University

Library of Congress Catalog Data: ISSN 1095-5054

Thinking About Kahneman’s Contribution to Critical Thinking

A nobel laureate on the importance of 'thinking slow.'.

Updated April 11, 2024 | Reviewed by Lybi Ma

• Kahneman won a Nobel Memorial Prize in Economics for his work.
• He found that people are often irrational about economics.

During my Ph.D. studies, I recall focusing on reconceptualising what we know of as critical thinking to include reflective judgment (not jumping to conclusions and taking your time in your decision-making to consider the nature limits, and certainty of knowing) on par with the commonly accepted skills and dispositions components. The importance of reflective judgment wasn’t a particularly novel idea – a good deal of research on reflective judgment and similar processes akin to critical thinking had already been conducted (see King and Kitchener, 1994; Kuhn, 1999; 2000; Stanovich, 1999). However, reflective judgment – as opposed to intuitive judgment – didn’t seem to have ‘the presence’ in the discussion of critical thinking that it does today.

The same month I submitted my Ph.D. back in 2011, a book was released that massively helped to accomplish what I had been working to help facilitate – changing the terrain of thought surrounding critical thinking: Thinking, Fast, and Slow . Its author, Daniel Kahneman, passed away a couple of weeks ago at age 90. Psychology students will likely recognise the name associated with Amos Tversky and their classic work together in the 1970s on the availability, representativeness, and anchoring and adjustment heuristics (for example, Tversky and Kahneman, 1974). Indeed, such heuristics, alongside the affect heuristic (Kahneman and Frederick, 2002; Slovic and colleagues, 2002) play a large role in how we think about thinking and barriers to critical thought. In 2002, Kahneman won a Nobel Memorial Prize in Economics for his work on prospect theory concerning loss aversion and people’s often irrational approach to economics. Indeed, Kahneman’s resume is full of awards and achievements.

However, the accomplishment I will remember him best for is the publication of Thinking, Fast, and Slow and its contribution to the field of critical thinking. Funny enough, I don’t recall the term, critical thinking being used very often in the book, if at all – and I read it two or three times. No, critical thinking was not the focus of his book; rather system 1 (fast) and 2 (slow) thinking (see also Stanovich, 1999) – intuitive and reflective judgment. Not only did this book put into the spotlight many of the mechanics of reflective judgment for fellow academics and researchers of cognitive psychology, it also did so l for non-academic audiences – becoming a New York Times bestseller. Moreover, it won the Los Angeles Times Book Award for Current Interest, and the National Academy of Sciences Communication Award for Best Book (both in 2011). Good thinking was cool again in popular culture.

In the critical thinking literature, reflective judgment – regardless of what you want to call it (for example, system 2 thinking, epistemological understanding, ‘taking your time’) – is becoming more accepted as a core component of critical thinking. The field of critical thinking research and psychology more broadly, owes Kahneman a debt of gratitude for his contributions in helping shine a light on the importance of ‘thinking slow’. Thank you .

Kahneman, D. (2011). Thinking, fast and slow . 2UK: Penguin.

Kahneman, D., & Frederick, S. (2002). Representativeness revisited: Attribute substitution in intuitive judgment. Heuristics and biases: The Psychology of Intuitive Judgment , 49 (49-81), 74.

King, P. M., & Kitchener, K. S. (1994). Developing Reflective Judgment: Understanding and Promoting Intellectual Growth and Critical Thinking in Adolescents and Adults. CA: Jossey-Bass.

King, P. M., & Kitchener, K. S. (2004). Reflective judgment: Theory and research on the development of epistemic assumptions through adulthood. Educational Psychologist, 39 (1), 5–15.

Kuhn, D. (1999). A developmental model of critical thinking. Educational Researcher , 28 (2), 16-46.

Kuhn, D. (2000). Metacognitive development. Current Directions in Psychological Science , 9 (5), 178-181.

Slovic, P., Finucane, M., Peters, E., & MacGregor, D. G. (2002). Rational actors or rational fools: Implications of the affect heuristic for behavioral economics. The Journal of Socio-economics , 31 (4), 329-342.

Stanovich, K.E. (1999) Who is rational? Studies of individual differences in reasoning. Mahwah, Erlbaum.

Tversky, A., & Kahneman, D. (1974). Judgment under Uncertainty: Heuristics and Biases: Biases in judgments reveal some heuristics of thinking under uncertainty. Science , 185 (4157), 1124-1131.

Christopher Dwyer, Ph.D., is a lecturer at the Technological University of the Shannon in Athlone, Ireland.

• Find a Therapist
• Find a Treatment Center
• Find a Psychiatrist
• Find a Support Group
• Find Teletherapy
• United States
• Brooklyn, NY
• Chicago, IL
• Houston, TX
• Los Angeles, CA
• New York, NY
• Portland, OR
• San Diego, CA
• San Francisco, CA
• Seattle, WA
• Washington, DC
• Asperger's
• Bipolar Disorder
• Chronic Pain
• Eating Disorders
• Passive Aggression
• Personality
• Goal Setting
• Positive Psychology
• Stopping Smoking
• Low Sexual Desire
• Relationships
• Child Development
• Therapy Center NEW
• Diagnosis Dictionary
• Types of Therapy

Understanding what emotional intelligence looks like and the steps needed to improve it could light a path to a more emotionally adept world.

• Coronavirus Disease 2019
• Affective Forecasting
• Neuroscience

A consensus statement on critical thinking in nursing

Affiliation.

• 1 Department of Nursing Education, Eastern Michigan University, Ypsilanti 48197, USA.
• PMID: 11103973
• DOI: 10.3928/0148-4834-20001101-06

The purpose of this study was to define critical thinking in nursing. A Delphi technique with 5 rounds of input was used to achieve this purpose. An international panel of expert nurses from nine countries: Brazil, Canada, England, Iceland, Japan, Korea, Netherlands, Thailand, and 23 states in the U.S. participated in this study between 1995 and 1998. A consensus definition (statement) of critical thinking in nursing was achieved. The panel also identified and defined 10 habits of the mind (affective components) and 7 skills (cognitive components) of critical thinking in nursing. The habits of the mind of critical thinking in nursing included: confidence, contextual perspective, creativity, flexibility, inquisitiveness, intellectual integrity, intuition, open-mindedness, perseverance, and reflection. Skills of critical thinking in nursing included: analyzing, applying standards, discriminating, information seeking, logical reasoning, predicting and transforming knowledge. These findings can be used by practitioners, educators and researchers to advance understanding of the essential role of critical thinking in nursing.

Disposition Towards Critical Thinking and Student Engagement in Higher Education

• Published: 10 June 2022
• Volume 48 , pages 239–256, ( 2023 )

Cite this article

• Paula Álvarez-Huerta 1 ,
• Alexander Muela 2 &
• Inaki Larrea 1  

1186 Accesses

5 Citations

3 Altmetric

Explore all metrics

Developing student critical thinking skills is a core purpose of higher education, and requires the cognitive and disposition components of critical thinking to be fostered. The present study aims to examine the relationship between disposition towards critical thinking and engagement in higher education students. Participants were 836 students from two universities in Spain. Results showed a direct and positive relationship between student critical thinking disposition and several aspects of student engagement, such as reflective learning and participation in high-impact practices. These results could inform general pedagogical practices within the higher education curriculum so as to foster critical thinking disposition among future graduates.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Is Empathy the Key to Effective Teaching? A Systematic Review of Its Association with Teacher-Student Interactions and Student Outcomes

Karen Aldrup, Bastian Carstensen & Uta Klusmann

Why learning environment matters? An analysis on how the learning environment influences the academic motivation, learning strategies and engagement of college students

Ryan Francis O. Cayubit

All better than being disengaged: Student engagement patterns and their relations to academic self-concept and achievement

Katharina Schnitzler, Doris Holzberger & Tina Seidel

Data Availability

The data that support the findings of this study are available from the corresponding author, upon reasonable request.

Abrami, P. C., Bernard, R. M., Borokhovski, E., Waddington, D. I., Wade, C. A., & Persson, T. (2015). Strategies for teaching students to think critically: A meta-analysis. Review of Educational Research, 85 (2), 275–314. https://doi.org/10.3102/0034654314551063

Article   Google Scholar  

Agudo Saiz, D., Salcines Talledo, I., & González Fernández, N. (2020). Pensamiento crítico en ESO y Bachillerato: perspectiva de docentes y estudiantes de un IES en una provincia del norte de España. Tendencias Pedagógicas, 37, 121–133. https://doi.org/10.15366/tp2021.37.010 .

Akpur, U. (2020). Critical, reflective, creative thinking and their reflections on academic achievement. Thinking Skills and Creativity , 37 . https://doi.org/10.1016/j.tsc.2020.100683

Alsina, Á., Ayllón, S., Colomer, J., Fernández-Peña, R., Fullana, J., Pallisera, M., Pérez-Burriel, M., & Serra, L. (2017). Improving and evaluating reflective narratives: A rubric for higher education students. Teaching and Teacher Education, 63 , 148–158. https://doi.org/10.1016/j.tate.2016.12.015

Alvarez-Huerta, P., Muela, A., & Larrea, I. (2021). Student engagement and creative confidence beliefs in higher education. Thinking Skills and Creativity . https://doi.org/10.1016/j.tsc.2021.100821

Álvarez-Huerta, P., Muela, A., & Larrea, I. (2022). Disposition toward critical thinking andcreative confidence beliefs in higher education students: The mediating role of openness to diversity and challenge. Thinking Skills and Creativity, 101003 . https://doi.org/10.1016/j.tsc.2022.101003

Anaya, G. (1999). College impact on student learning: Comparing the use of self-reported gains, standardized test scores, and college grades. Research in Higher Education, 40 (5), 499–526. https://doi.org/10.1023/A:1018744326915

Arend, B. (2009). Encouraging critical thinking in online threaded discussions. The Journal of Educators Online.  https://files.eric.ed.gov/fulltext/EJ904064.pdf

Arslan, R., Gulveren, H., & Aydin, E. (2014). A research on critical thinking tendencies and factors that affect critical thinking of higher education students. International Journal of Business and Management , 9 (5). https://doi.org/10.5539/ijbm.v9n5p43

Bakadorova, O., Lazarides, R., & Raufelder, D. (2020). Effects of social and individual school self-concepts on school engagement during adolescence. European Journal of Psychology of Education, 35 (1), 73–91. https://doi.org/10.1007/s10212-019-00423-x

Bezanilla-Albisua, M. J., Poblete-Ruiz, M., Fernández-Nogueira, D., Arranz-Turnes, S., & Campo-Carrasco, L. (2018). El Pensamiento Crítico desde la Perspectiva de los Docentes Universitarios. Estudios Pedagógicos, 44 (1), 89–113. https://doi.org/10.4067/s0718-07052018000100089

Bezanilla, M. J., Fernández-Nogueira, D., Poblete, M., & Galindo-Domínguez, H. (2019). Methodologies for teaching-learning critical thinking in higher education: The teacher’s view. Thinking Skills and Creativity, 33 , 100584. https://doi.org/10.1016/j.tsc.2019.100584

Bezanilla, M. J., Galindo-Domínguez, H., & Poblete, M. (2021). Importance of teaching critical thinking in higher education and existing difficulties according to Teacher’s views. Multidisciplinary Journal of Educational Research, 11 (1), 20–48. https://doi.org/10.4471/remie.2021.6159

Bourner, T. (2003). Assessing reflective learning. Education + Training, 45 (5), 267–272. https://doi.org/10.1108/00400910310484321

Bravo, M. J., Galiana, L., Rodrigo, M. F., Navarro-Perez, J. J., & Oliver, A. (2020). An adaptation of the critical thinking disposition scale in Spanish youth. Thinking Skills and Creativity ,  38 , 100748.  https://doi.org/10.1016/j.tsc.2020.100748

Burbach, M. E., & MatkinFritz, G. S. (2004). Teaching critical thinking in an introductory leadership course utilizing active learning strategies: A confirmatory study. College Student Journal, 38 , 482–494.

Google Scholar  

Campbell, C. M., & Cabrera, A. F. (2014). Are grades and deep learning related? Research in Higher Education, 55 (5), 494–507. https://doi.org/10.1007/s11162-013-9323-6

Carini, R. M., Kuh, G. D., & Kleint, S. P. (2006). Student engagement and student learning: Testing the Linkages. Research in Higher Education, 47 (1), 1–32.

Chan, C. (2019). Using digital storytelling to facilitate critical thinking disposition in youth civic engagement: A randomized control trial. Children and Youth Services Review , 107 . https://doi.org/10.1016/j.childyouth.2019.104522

Chan, N. M., Ho, I. T., & Ku, K. Y. L. (2011). Epistemic beliefs and critical thinking of Chinese students. Learning and Individual Differences, 21 (1), 67–77. https://doi.org/10.1016/j.lindif.2010.11.001

Chen, Q., Liu, D., Zhou, C., & Tang, S. (2020). Relationship between critical thinking disposition and research competence among clinical nurses: A cross-sectional study. Journal of Clinical Nursing, 29 (7–8), 1332–1340. https://doi.org/10.1111/jocn.15201

Cheng, M. H. M., & Wan, Z. H. (2017). Exploring the effects of classroom learning environment on critical thinking skills and disposition: A study of Hong Kong 12th graders in Liberal Studies. Thinking Skills and Creativity, 24 , 152–163. https://doi.org/10.1016/J.TSC.2017.03.001

Comer, R. D., Schweiger, T. A., & Shelton, P. (2019). Impact of students’ strengths, critical thinking skills and disposition on academic success in the first year of a PharmD program. American Journal of Pharmaceutical Education, 83 (1), 93–99. https://doi.org/10.5688/ajpe6499

Daradoumis, T., & Arguedas, M. (2020). Cultivating students’ reflective learning in metacognitive activities through an affective pedagogical agent. Educational Technology & Society , 23(2), 19–31. https://www.jstor.org/stable/26921131

Darby, N. M., & Rashid, A. M. (2017). Critical thinking disposition: The effects of infusion approach in engineering drawing. Journal of Education and Learning, 6 (3), 305. https://doi.org/10.5539/jel.v6n3p305

Dumitru, D., Bigu, D., Elen, J., Ahern, A., Mcnally, C., & O’sullivan, J. J. (2018). A European review on critical thinking educational practices in higher education institutions . http://crithinkedu.utad.pt/en/crithinkedu/Itemrecord/moreinformation http://hdl.handle.net/10197/9865

Dunning, D., Heath, C., & Suls, J. M. (2016). Flawed self-assessment implications for health, education, and the workplace. Psychological Science in the Public Interest , 5 (3), 69. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.1022.3431&rep=rep1&type=pdf

Facione, P. A., Carol, A. S., Facione, N. C., & Gainen, J. (1995). The disposition towards critical thinking. Journal of General Education, 44 (1), 1–25.

Facione, P., Facione, N. C., & Giancarlo, C. A. (2000). The disposition towards critical thinking: Its character, measurement, and relationship to critical thinking skills. Informal Logic,   20 (1), 61–84.  https://doi.org/10.22329/il.v20i1.2254

Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G * Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39 (2), 175–191. https://doi.org/10.3758/BF03193146

Fullana, J., Pallisera, M., Colomer, J., Fernández-Peña, R., & Pérez-Burriel, M. (2016). Reflective learning in higher education. A qualitative study on students’ perceptions reflective learning in higher education. Studies in Higher Education, 41 (6), 1008–1022. https://doi.org/10.1080/03075079.2014.950563

Gallego-Noche, B., Goenechea, C., Antolínez-Domínguez, I., & Valero-Franco, C. (2021). Towards inclusion in Spanish higher education: Understanding the relationship between identification and discrimination. Social Inclusion, 9 (3), 81–93. https://doi.org/10.17645/si.v9i3.4065

Garvey, J. C., Brckalorenz, A., Latopolski, K., & Hurtado, S. S. (2018). High-impact practices and student–faculty interactions for students across sexual orientations. Journal of College Student Development, 59 (2), 210–226. https://doi.org/10.1353/csd.2018.0018

Giancarlo, C. A., & Facione, P. A. (2001). A look across four years at the disposition towards critical thinking among undergraduate students. The Journal of General Education, 50 (1), 29–55. https://doi.org/10.1353/jge.2001.0004

Halpern, D. F. (2003). Thinking critically about creative thinking. In M. A. Runco (Ed.), Critical creative processes (pp. 189–207). Hampton Press.

Hayek, J. C., Carini, R. M., O’Day, P. T., & Kuh, G. D. (2002). Triumph or tragedy: Comparing student engagement levels of members of Greek-letter organizations and other students. Journal of College Student Development, 43 (5), 643–663.

Huber, C. R., & Kuncel, N. R. (2016). Does college teach critical thinking? A meta-analysis. Review of Educational Research, 86 (2), 431–468. https://doi.org/10.3102/0034654315605917

Hyytinen, H., Toom, A., & Postareff, L. (2018). Unraveling the complex relationship in critical thinking, approaches to learning and self-efficacy beliefs among first-year educational science students. Learning and Individual Differences, 67 , 132–142. https://doi.org/10.1016/j.lindif.2018.08.004

Indrašienė, V., Jegelevičienė, V., Merfeldaitė, O., Penkauskienė, D., Pivorienė, J., Railienė, A., Sadauskas, J., & Valavičienė, N. (2021). Linking critical thinking and knowledge management: A conceptual analysis. Sustainability, 13 (3), 1–17. https://doi.org/10.3390/su13031476

Janssen, E. M., Mainhard, T., Buisman, R. S. M., Verkoeijen, P. P. J. L., Heijltjes, A. E. G., van Peppen, L. M., & van Gog, T. (2019). Training higher education teachers’ critical thinking and attitudes towardss teaching it. Contemporary Educational Psychology, 58 , 310–322. https://doi.org/10.1016/j.cedpsych.2019.03.007

Jayadeva, S., Brooks, R., Gupta, A., Abrahams, J., Lažetič, P., & Lainio, A. (2020). Are Spanish students customers? Paradoxical perceptions of the impact of marketisation on higher education in Spain. Sociological Research Online, 26 (1), 185–204. https://doi.org/10.1177/1360780420968577

Kahu, E. R. (2013). Framing student engagement in higher education. Studies in Higher Education, 38 (5), 758–773. https://doi.org/10.1080/03075079.2011.598505

Kang, F. (2015). Contribution of emotional intelligence towardss graduate students’ critical thinking disposition. International Journal of Education and Literacy Studies, 3 (4), 6–17. https://doi.org/10.7575/aiac.ijels.v.3n.4p.6

Ketonen, E. E., Malmberg, L. E., Salmela-Aro, K., Muukkonen, H., Tuominen, H., & Lonka, K. (2019). The role of study engagement in university students’ daily experiences: A multilevel test of moderation. Learning and Individual Differences, 69 , 196–205. https://doi.org/10.1016/j.lindif.2018.11.001

Kilgo, C. A., Ezell Sheets, J. K., & Pascarella, E. T. (2015). The link between high-impact practices and student learning: Some longitudinal evidence. Higher Education, 69 , 509–525. https://doi.org/10.1007/s10734-014-9788-z

Kim, Y. K., & Sax, L. J. (2009). Student-faculty interaction in research universities: Differences by student gender, race, social class, and first-generation status. Research in Higher Education, 50 (5), 437–459. https://doi.org/10.1007/s11162-009-9127-x

Kuh, G. D. (2008). High-impact educational practices: What they are, who has access to them, and why they matter . Association of American Colleges and Universities.

Kuh, G. D. (2010). Assessing what really matters to student learning inside the national survey of student engagement. Change: The Magazine of Higher Learning, 33 (3), 10–17. https://doi.org/10.1080/00091380109601795

Liyanage, I., Walker, T., & Shokouhi, H. (2021). Are we thinking critically about critical thinking? Uncovering uncertainties in internationalised higher education. Thinking Skills and Creativity , 39 . https://doi.org/10.1016/j.tsc.2020.100762

Loes, C. N., & Pascarella, E. T. (2017). Collaborative learning and critical thinking: Testing the link. Journal of Higher Education, 88 (5), 726–753. https://doi.org/10.1080/00221546.2017.1291257

Loes, C., Pascarella, E. T., & Umbach, P. D. (2012). Effects of diversity experiences on critical thinking skills: Who benefits? The Journal of Higher Education, 83 (1), 1–25. https://doi.org/10.1353/jhe.2012.0001

Lun, V. M. C., Fischer, R., & Ward, C. (2010). Exploring cultural differences in critical thinking: Is it about my thinking style or the language I speak? Learning and Individual Differences, 20 (6), 604–616. https://doi.org/10.1016/j.lindif.2010.07.001

Magno, C. (2010). The role of metacognitive skills in developing critical thinking. Metacognition and Learning, 5 (2), 137–156. https://doi.org/10.1007/s11409-010-9054-4

Mayhew, M. J., Rockenbach, A. N., Bowman, N. A., Seifert, T. A., Wolniak, G. C., Pascarella, E. T., & Terenzini, P. T. (2016). How college affects students, Volume 3, 21st century evidence that higher education works . Jossey-Bass.

McCormick, A. C., Kinzie, J., & Gonyea, R. M. (2013). Student engagement: Bridging research and practice to improve the quality of undergraduate education. In M. Paulsen (Ed.), Higher Education: Handbook of Theory and Research: Vol. XXVIII . Springer. https://doi.org/10.1007/1-4020-2456-8

Miller, A. L., & Dumford, A. D. (2016). Creative cognitive processes in higher education. Journal of Creative Behavior, 50 (4), 282–293. https://doi.org/10.1002/jocb.77

Morris, P. D., & Clark, L. M. (2018). Using NSSE data to analyze levels of engagement of distance learners. Quarterly Review of Distance Education, 19 (2), 1–53.

Myers, B. E., & Dyer, J. E. (2006). The influence of student learning style on critical thinking skill. Journal of Agricultural Education, 47 (1), 43–54.

Noohi, E., Karimi-Noghondar, M., & Haghdoost, A. (2012). Survey of critical thinking and clinical decision making in nursing student of Kerman University. Iranian journal of nursing and midwifery research, 17 (6), 440–444.

Pascarella, E. T., Palmer, B., Moye, M., & Pierson, C. T. (2001). Do diversity experiences influence the development of critical thinking? Journal of College Student Development, 42 (3), 257–271.

Qiang, R., Han, Q., Guo, Y., Bai, J., & Karwowski, M. (2020). Critical thinking disposition and scientific creativity: The mediating role of creative self-efficacy. Journal of Creative Behavior, 54 (1), 90–99. https://doi.org/10.1002/jocb.347

R Core Team (2021). R: A language and environment for statistical computing. R Foundation for Statistical Computing. Retrieved May 11, 2022, from https://www.R-project.org/

Rodzalan, S. A., & Saat, M. M. (2015). The perception of critical thinking and problem solving skill among Malaysian undergraduate students. Procedia - Social and Behavioral Sciences, 172 , 725–732. https://doi.org/10.1016/j.sbspro.2015.01.425

Rosseel, Y. (2012). Lavaan: An R package for structural equation modeling and more. Version 0.5–12 (BETA). Journal of Statistical Software, 48 (2), 1–36. https://doi.org/10.18637/jss.v048.i02

Sahanowas, S., & Santoshi, H. (2020). Critical thinking disposition of undergraduate students in relation to emotional intelligence: Gender as a moderator. Heliyon , 6 (11). https://doi.org/10.1016/j.heliyon.2020.e05477

Shirazi, F., & Heidari, S. (2019). The relationship between critical thinking skills and learning styles and academic achievement of nursing students. The Journal of Nursing Research, 27 (4), e38. https://doi.org/10.1097/jnr.0000000000000307

Snyder, L. G., & Snyder, M. J. (2008). Teaching critical thinking and problem solving skills. The Delta Pi Epsilon Journal , 50 , 90–99. https://pdfs.semanticscholar.org/9d73/986223dbfd7c799516bc0cc8f48e1869fbc5.pdf

Sosu, E. M. (2013). The development and psychometric validation of a critical thinking disposition scale. Thinking Skills and Creativity, 9 , 107–119. https://doi.org/10.1016/j.tsc.2012.09.002

Turan, M. B., & Koç, K. (2018). The impact of self-directed learning readiness on critical thinking and self-efficacy among the students of the school of physical education and sports. International Journal of Higher Education, 7 (6), 98–105. https://doi.org/10.5430/ijhe.v7n6p98

Vieira, M. J., Vidal, J., & Barrio, S. (2007). Una herramienta de evaluación para comparar la experiencia académica de los estudiantes universitarios. Revista De Investigación Educativa, 25 (2), 327–350.

Yuan, R., Yang, M., & Stapleton, P. (2020). Enhancing undergraduates’ critical thinking through research engagement: A practitioner research approach. Thinking Skills and Creativity, 38 , 100737. https://doi.org/10.1016/j.tsc.2020.100737

Zilvinskis, J., Masseria, A. A., & Pike, G. R. (2017). Student engagement and student learning : Examining the convergent and discriminant validity of the revised national survey of student engagement. Research in Higher Education, 58 , 880–903. https://doi.org/10.1007/s11162-017-9450-6

Download references

This research was supported by a grant from Gipuzkoako Foru Aldundia to the Faculty of Humanities and Education Sciences of Mondragon Unibertsitatea.

Author information

Authors and affiliations.

Innovation and Intervention in Inclusive Education, HUHEZI, Mondragon Unibertsitatea, Dorleta, 20540, Eskoriatza, Spain

Paula Álvarez-Huerta & Inaki Larrea

Department of Clinical and Health Psychology and Research Methodology, University of the Basque Country UPV/EHU, Tolosa Hiribidea, 70, 20018, Donostia, Spain

Alexander Muela

You can also search for this author in PubMed   Google Scholar

Contributions

PAH conceived the research; AM analyzed the data; IL contributed to the interpretation of the results. PAH, AM and IL wrote the manuscript. All authors provided critical feedback and helped shape the research, analysis and manuscript.

Corresponding author

Correspondence to Paula Álvarez-Huerta .

Ethics declarations

Ethics approval.

The study protocol was reviewed and approved by the Research Ethics Committee of Mondragon University (Mondragon Unibertsitatea).

Consent for Publication

All authors agree with the content, and all gave explicit consent to submit this article for publication with this journal.

Conflicts of Interest/Competing Interests

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Álvarez-Huerta, P., Muela, A. & Larrea, I. Disposition Towards Critical Thinking and Student Engagement in Higher Education. Innov High Educ 48 , 239–256 (2023). https://doi.org/10.1007/s10755-022-09614-9

Download citation

Accepted : 10 May 2022

Published : 10 June 2022

Issue Date : April 2023

DOI : https://doi.org/10.1007/s10755-022-09614-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Critical thinking disposition
• Student engagement
• Reflective learning
• High-impact practices
• Find a journal
• Publish with us
• Track your research

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 10 August 2023

A conceptual framework of cognitive-affective theory of mind: towards a precision identification of mental disorders

• Peng Zhou 1   na1 ,
• Huimin Ma 2   na1 ,
• Bochao Zou 2 ,
• Xiaowen Zhang 3 ,
• Shuyan Zhao 3 ,
• Yuxin Lin 2 ,
• Yidong Wang 4 ,
• Lei Feng 5 , 6 &
• Gang Wang 5 , 6  

npj Mental Health Research volume  2 , Article number:  12 ( 2023 ) Cite this article

2512 Accesses

1 Altmetric

Metrics details

• Diagnostic markers
• Human behaviour
• Psychiatric disorders

To explore the minds of others, which is traditionally referred to as Theory of Mind (ToM), is perhaps the most fundamental ability of humans as social beings. Impairments in ToM could lead to difficulties or even deficits in social interaction. The present study focuses on two core components of ToM, the ability to infer others’ beliefs and the ability to infer others’ emotions, which we refer to as cognitive and affective ToM respectively. Charting both typical and atypical trajectories underlying the cognitive-affective ToM promises to shed light on the precision identification of mental disorders, such as depressive disorders (DD) and autism spectrum disorder (ASD). However, most prior studies failed to capture the underlying processes involved in the cognitive-affective ToM in a fine-grained manner. To address this problem, we propose an innovative conceptual framework, referred to as visual theory of mind (V-ToM), by constructing visual scenes with emotional and cognitive meanings and by depicting explicitly a four-stage process of how humans make inferences about the beliefs and emotions of others. Through recording individuals’ eye movements while looking at the visual scenes, our model enables us to accurately measure each stage involved in the computation of cognitive-affective ToM, thereby allowing us to infer about potential difficulties that might occur in each stage. Our model is based on a large sample size ( n  > 700) and a novel audio-visual paradigm using visual scenes containing cognitive-emotional meanings. Here we report the obtained differential features among healthy controls, DD and ASD individuals that overcome the subjectivity of conventional questionnaire-based assessment, and therefore could serve as valuable references for mental health applications based on AI-aided digital medicine.

Similar content being viewed by others

A systematic review and multivariate meta-analysis of the physical and mental health benefits of touch interventions

Julian Packheiser, Helena Hartmann, … Frédéric Michon

Microdosing with psilocybin mushrooms: a double-blind placebo-controlled study

Federico Cavanna, Stephanie Muller, … Enzo Tagliazucchi

The serotonin theory of depression: a systematic umbrella review of the evidence

Joanna Moncrieff, Ruth E. Cooper, … Mark A. Horowitz

Introduction

Humans are by nature social beings. To explore the minds of others is perhaps the most fundamental ability of humans as social beings, commonly referred to as Theory of Mind (ToM) 1 , 2 , which begins at birth and could extend into the whole lifetime. Impairments in ToM could lead to difficulties or even deficits in social interaction. The study focuses on two disorders, depressive disorders (DD) and autism spectrum disorder (ASD), in which social impairments are commonly identified. According to the Diagnostic and Statistical Manual of Mental Disorders, fifth edition (APA, 2022), individuals with DD persistently suffer from depressed mood, markedly diminished interest or pleasure in almost all activities, feelings of worthlessness, fatigue or loss of energy and diminished ability to concentrate; individuals with ASD exhibit two clusters of symptoms: persistent deficits in social communication and social interaction, and restricted, repetitive patterns of behavior, interests, or activities. Both disorders severely impair individuals’ development, including their academic and career development as well as their overall quality of life.

ToM is often described as the ability to attribute mental states to oneself and others. It has traditionally been investigated using false-belief tasks 1 , 3 , 4 , 5 . It is generally acknowledged that typically developing (TD) children by 4 years of age can pass this explicit false-belief task, indicating the acquisition of representational ToM 6 . Note that in the standard false-belief tasks, the participants are asked to provide explicit verbal responses, which might pose particular difficulties for younger TD children and children with ASD. To solve this problem, non-verbal tasks that employ on-line techniques like eye tracking have been used to examine children’s spontaneous/implicit ToM. In a typical implicit ToM task proposed by Southgate et al. 7 , children were presented with an event where a girl was watching an object being hidden in a box; the object was then displaced while the girl was looking away. Children’s eye movements were recorded and analyzed to see whether they spontaneously anticipated the girl’s subsequent behavior on the basis of her false belief of the location of the object. The results of non-verbal ToM tasks revealed that TD children younger than 4, or even 7-month infants, already exhibited spontaneous behavioral patterns that reflected their ability to reason about others’ false beliefs 7 , 8 , 9 , 10 . By contrast, individuals with ASD have been found to exhibit marked difficulties in both explicit and implicit false-belief tasks. In addition, it has also been reported that high-functioning individuals with ASD who could generally pass the explicit false-belief tasks still failed in the non-verbal tasks to exhibit behavioral patterns that reflect spontaneous false-belief attribution 11 , 12 , 13 . In addition, research has shown that individuals with DD exhibit a significant slowdown of ToM as compared with healthy controls in false-belief tasks. (see, e.g., ref. 14 )

In addition to the ability to reason about others’ beliefs and intentions, ToM also includes the capacity to reason about others’ emotions 15 . We refer to these two components as cognitive ToM and affective ToM respectively. The division of cognitive and affective ToM has been empirically supported by prior research. For example, Raimo and colleagues found that rather than a general decline in ToM, elder adults showed a selective decline in cognitive ToM as compared with affective ToM (e.g., ref. 16 ). Research has also shown that the two components of ToM involve brain circuitry in frontal and temporal lobes 17 . However, we wish to note that the affective ToM has been relatively understudied. Emotions are extracted from people’s appraisals of events and are often used as important indicators of their immediate reactions to events. Therefore, our sensitivity to others’ emotions, the affective ToM, plays a central role in our social life 18 , 19 , 20 . A meta-analysis suggests that people with DD exhibited significant deficits in affective ToM 14 , 21 . Previous studies also found that individuals with ASD exhibited deficits in affective ToM, which was suggested to be closely relevant to their social impairments (for a review, see ref. 22 ).

The studies reviewed so far suggest that a good model that attempts to represent ToM must take into consideration both the cognitive and affective components. However, most prior studies failed to capture the underlying processes involved in the cognitive-affective ToM in a fine-grained manner. To address this problem, we propose an innovative conceptual framework that incorporates both cognitive and affective ToM, referred to as visual theory of mind (V-ToM), by constructing visual scenes with emotional and cognitive meanings and by depicting explicitly a four-stage process of how humans make inferences about the beliefs and emotions of others (see Fig. 1 ). The Differences between traditional ToM and our V-ToM are as follows: First, V-ToM successfully operationalizes ToM into fine-grained framework, which is realized by constructing visual scenes with emotional and cognitive meanings and by depicting explicitly a four-stage process of how humans make inferences about the beliefs and emotions of others. Second, through recording individuals’ eye movements while looking at the visual scenes, our model enables us to explicitly and accurately measure each stage involved in the computation of cognitive-affective ToM, thereby allowing us to infer about potential difficulties that might occur in each stage. The obtained differential features among healthy controls, DD and ASD individuals overcome the subjectivity of conventional questionnaire-based assessment, and therefore could serve as valuable references for mental health applications based on AI-aided digital medicine.

First, V-ToM successfully operationalizes ToM into fine-grained framework, which is realized by constructing visual scenes with emotional and cognitive meanings and by depicting explicitly a four-stage process of how humans make inferences about the beliefs and emotions of others. Second, through recording individuals’ eye movements while looking at the visual scenes, our model enables us to explicitly and accurately measure each stage involved in the computation of cognitive-affective ToM, thereby allowing us to infer about potential difficulties that might occur in each stage. The obtained differential features among normal people, DD and ASD individuals overcome the subjectivity of conventional questionnaire-based assessment, and therefore could serve as valuable references for mental health applications based on AI-aided digital medicine.

More specifically, by dividing the human computational processes underlying their cognitive-affective ToM into four stages (see the middle panel of Fig. 1 ), our model allows us to explain how humans process social information in a more fine-grained manner. The details of each of the four stages are provided in the next section. Although the underlying processes of human cognition unfold in a dynamic and rapid manner in real time, our model makes a bold attempt to characterize these processes by dividing them into four sub-components, each with measurable features. Our model is based on traditional visual attention paradigms 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , but unlike the traditional paradigms, our paradigm allows us to implement artificial intelligence (AI) algorithms when analyzing eye movement data, thereby enabling us to make inferences about potential difficulties that might occur in each of the four stages. This has important implications for understanding atypical cognitive-affective ToM like DD and ASD. By identifying the potential sources of their difficulties, we will be able to better understand the nature of their impairments, which will then lead to more effective identification and intervention programs for individuals with DD and those with ASD.

In this paper, we propose an innovative conceptual framework, V-ToM, by constructing visual scenes with emotional and cognitive meanings and by depicting explicitly a four-stage process of how humans make inferences about the beliefs and emotions of others. Although our tasks are based on individuals with DD and those with ASD, our model has the potential to be extended to other domains of ToM by incorporating new available data. In the following sections, we first introduce our conceptual framework (i.e., the V-ToM model) and then report findings from two tasks that investigated affective ToM and cognitive ToM respectively.

Conceptual framework

Our model provides an account for how cognitive-affective processes can be mapped onto eye movement patterns in an established visual world, thereby allowing us to infer the underlying mental states by analyzing these eye movement patterns. As shown in Fig. 2 , we divide the human cognitive processes underlying their cognitive-affective process into four stages. The detailed representations of each stage are provided in the color-matched boxes in the right panel of Fig. 2 . The first stage occurs at the perceptual level that involves the processing of visual information in an established visual world. Following the feature integration theory 41 , basic features (e.g., color, orientation) are registered early and automatically at the pre-attentive stage. Thus, the first stage includes early visual processing, like the detection of basic features, the detection of border and contour, and the segmentation and segregation of figure from background (see the pink panel in Fig. 2 ). Visual representations that contain both conceptual and propositional properties of the visual world are then established.

The relevant cognitive processes are explicitly divided into four stages: S1. Visual processing, S2. Mental processing, S3. Assessment, and S4. Transfer and implementation. The detailed representations of each stage are provided in the color-matched boxes in the right panel.

In the second stage, the established visual representations are brought to mental processing, which is analogous to the feature binding stage (see the blue panel in Fig. 2 ). Only representations that are of propositional semantic features (in the form of a propositional statement, e.g., this is a car , and the windows are broken ) enter this stage, therefore, all the mental representations are propositional in nature.

In the third stage, the newly established mental representations are assessed according to their relevance to the prototype (i.e., the extant mental representations that are established on the basis of prior experience and knowledge and are stored in long-term memory). The representations contained in the prototype are also propositional in nature (see the yellow panel in Fig. 2 ). For instance, if the prototype contains two mental representations, cars have a neutral connotation and broken windows have a negative connotation , then based on the relevance to the prototype, the two newly established mental representations in the example are evaluated differently. The representation this is a car is rendered as “neutral,” and the representation the windows are broken as “negative.” More specifically, this assessment process leads to two final propositional representations: the fact that this is a car has a neutral connotation and the fact that the windows are broken has a negative connotation . These two final mental representations are the output of this stage. It should be noted that individual differences might occur at this stage, because the prototype against which the newly established representations are evaluated is built upon prior knowledge and experience that might vary across individuals.

In the fourth stage, the final mental representations (the output of the assessment stage) are registered in working memory and transferred to implement cognitive-affective processes. In particular, both DD and ASD are quite likely to have representational differences in mind from health participants when presented with visual scenes of emotional and cognitive meanings. The representational differences guide the eye movements in the visual world, yielding eye movement patterns that reflect the final mental representations due to the computational processing of cognitive-affective information. For instance, DD patients have difficulty in disengaging from negative information 42 , and ASD patients have a low matching degree of proposition and prototype 43 . Therefore, reliable eye movement indicators (e.g., transit time, first fixation duration, total fixation duration, fixation proportions, and saccade path length) combined with AI algorithms, e.g., Naive Bayesian, logistic regression, Support Vector Machine (SVM), and deep learning, can serve as a reliable and accurate measure of cognitive-affective ToM.

Our model is based on two tasks using the audio-visual paradigm with eye movement measurements (detailed descriptions of the paradigm are provided in Fig. 3 ). Study 1 was designed to investigate the difference in affective ToM between individuals with DD and healthy controls. Study 2 was designed to investigate the difference in cognitive ToM between typically developing children and children with ASD. Compared with traditional paradigms of ToM research, our paradigm, for the first time, innovatively combines the investigations of these two core components of ToM. In addition, our audio-visual paradigm is based on well-constructed visual scenes with emotional and cognitive meanings and large sample size ( n  > 700), and therefore the obtained indicators serve as a highly objective measure.

Study 1: ( A ) A black background was shown to the participants for 1000 ms, and then a random positive or negative image was presented as the background scene. ( B ) Participants’ initial eye gaze patterns when the background scene was shown to them were recorded by a Tobii EyeX Controller eye tracker (sampling rate 60 Hz). ( C ) A random positive or negative face was presented randomly on the left or right of the background scene as the foreground stimulus in between 500 and 1000 ms. ( D ) The response time in which the participants judged the emotional attribute of the face was measured by pressing the button, and at the same time their eye movements were recorded. Study 2 : ( A ) In the visual stimulus, a strawberry, the liked object by the boy character Kangkang, was placed on the left high box and a green pepper, the disliked object, was placed on the right low box. The social element that Kangkang is facing is either a man (+social) or a tree (−social). ( B ) Participants’ initial eye gaze patterns when the visual stimuli were first shown to them were recorded by an EyeLink 1000 plus eye tracker (sampling rate 500 Hz). ( C ) A spoken sentence saying, “Look, which object do you think Kangkang will reach for?” was presented 500 ms after the appearance of the visual stimulus. ( D ) Each participant’s eye movement was recorded from the onset of the verb for 1000 ms.

Study 1: difference in affective ToM between individuals with DD and healthy controls

Two hundred and twelve people with diagnosed DD, including 79 males and 133 females, participated in the study (mean age = 32 (years);4 (months), SD = 12.79). Their diagnoses were confirmed by certified psychiatrists in hospitals using DSM-5 (APA, 2013), and were further complemented by two screening instruments, the Self-Rating Depression Scale (SDS) and the SCL-90 depression sub-scale 44 , 45 . In addition, 492 healthy controls without diagnosed DD, including 273 males and 219 females, participated as the control group (mean age = 34;1, SD = 14.24). The study was under the approval of the Ethics Committee of Beijing Anding Hospital, 201722FS-2, and written informed consent was obtained from all the participants. The participants were also recruited following the below eligibility criteria: (1) No history of previously diagnosed schizophrenia, schizophrenic affective disorder, or mental disorders associated with other diseases; (2) No history of alcohol or substance dependence; (3) Not on psychotropic medications for their conditions ; (4) Not suffering from any serious physical diseases that are not suitable to be included in this study. The experiments in this study were performed in accordance with the guidelines in the Declaration of Helsinki.

The experiment was conducted to investigate whether the attentional bias during emotion understanding could be used for DD identification. Using the competing-priming effect paradigm, emotional images and emotional faces were presented to the participants as visual stimuli. A total of four sets of visual stimuli were constructed. As shown in Fig. 4 , the four sets varied in the combinations of the emotional status of the images and the faces: a positive image with a positive face, a positive image with a negative face, a negative image with a positive face, and a negative image with a negative face. 40 positive and 40 negative images were constructed as the background scenes using the ThuPIS database 46 . The emotional faces were selected from the Taiwanese Facial Expression Image Database 47 , including 8 positive and 8 negative faces serving as the foreground stimuli. Both are open access database. Across the 4 sets of visual stimuli, each set consisted of 20 trials made up of the image and the face with the corresponding emotional status. All trials in the lists were arranged in random order.

Four combinations of the emotional status of the images and the faces: ( A ) positive image with positive face, ( B ) positive image with negative face, ( C ) negative image with positive face, and ( D ) negative image with negative face.

The experimental paradigm requires the participants to search for the presence of emotional faces in the background of emotional images, then judge the attributes of emotional faces. The reaction time data and eye movement data were recorded during this process. Before the actual test, each participant received a pre-test in which they needed to perform 8 consecutive trials correctly. The actual test consisted of 80 trials. Each trial proceeded as follows. First, a black background was shown to the participants for 1000 ms. Then, a random positive or negative image was presented as the background scene to attract the participants’ attention to the background. Following this, a random positive or negative face was presented randomly on the left or right of the background scene as the foreground stimulus in between 500 and 1000 ms. The participants were expected to switch their visual attention from the background scene to the foreground stimulus. Finally, the participants were asked to judge the emotional attribute of the faces by pressing the left (positive) or the right button (negative). Their response time was recorded at the same time. Response time indicates the speed at which the participant got distracted from the background scene and transferred to the foreground stimuli while judging the emotional status of the face. During the whole process, the participants’ eye movements were recorded using a Tobii EyeX Controller eye tracker with a sampling rate of 60 Hz. Prior to the formal experiment, participants went through a 4-point calibration procedure while seated approximately 60 cm from the screen. If the calibration error was beyond the threshold of 1° visual angle, the calibration and validation process repeated. The experimental stimuli were presented on a monitor with a resolution of 1600 × 900 pixels (21.5 inches) and programmed using C# with the framework of Windows Presentation Foundation.

This experimental paradigm fully integrates the idea of attentional bias hypothesis, mapping the effects of clinical depression onto attentional biases for emotional stimuli. During the experiment, participants were attracted by emotional images and emotional faces, and formed corresponding fixation points, which reflected the characteristics of attention orientation. In addition, to complete the attribute discrimination task of emotional faces, participants also needed to direct their gaze away from the emotional image area to the emotional face, further reflecting the ability to disengage attention. Finally, participants transferred their attention from the emotional image area to the emotional face area and made judgments, indicating attention transfer in the process of the experiment. By this means, the three attention components (attention orienting, attention disengagement, attention transfer) were unified in one paradigm. We believe that the current paradigm can accurately assess affective ToM, not the attentional control in the early visual stage, because the stimuli we devised were composed of not only facial images but also background images containing affective semantics. Although the facial images can be processed in early visual stage 48 , no strong evidence shows that the affective semantics of images can be processed in such an early stage. In addition, extant studies suggest that selective attention impairments in DD are specific to feature-based selective attention while spatial selective attention remains intact 49 . Therefore, by analyzing the eye movement trajectory information during the experiment, we can detect the differences of affective ToM between participants with DD and healthy controls.

Study 2: difference in cognitive ToM between TD children and children with ASD

Three hundred and thirty-two Mandarin-speaking children with ASD, who were diagnosed using DSM-IV-TR (APA, 2000) and DSM-5 (APA, 2013) by hospitals, and reconfirmed by our research team using the Autism Diagnostic Observation Schedule (ADOS), participated in the study. All the participants with ASD met the autism cut-off of the ADOS, with a mean score of 10.5 and a standard deviation of 3.5. Six hundred and twelve age-matched TD Mandarin-speaking children participated as the control group. The participants’ verbal IQ scores were assessed using Wechsler Preschool and Primary Scale of IntelligenceTM-IV (CN). All the participants in the ASD and TD groups had verbal IQ scores above 85. The study was under the approval of the Ethics Committee of the School of Medicine, Tsinghua University, 20170018, and all the participants or their guardians provided written informed consent. The experiments in this study were performed in accordance with the guidelines in the Declaration of Helsinki.

Sixteen pairs of visual stimuli were constructed by the research team. In each pair, two visual images were constructed, both containing a boy called Kangkang, two boxes (a high one and a low one) and two objects (one that Kangkang favors, referred to as the liked object, and the other one that he dislikes, referred to as the disliked object). The only difference between the two images was that in one of the pictures Kangkang was facing a man while in the other Kangkang was facing a tree. An example of the visual stimuli is presented in Fig. 5A, B . Across the trials, the audio stimulus that accompanied the visual images was always the same Mandarin spoken sentence: “Kan, ni juede Kangkang hui qu gou na yige?”[=Look, which object do you think Kangkang will reach for?]

( A ) a visual stimulus with a man, and ( B ) a visual stimulus with a tree.

In both Fig. 5A, B , a strawberry, the liked object of Kangkang, was placed on the top of the left high box, while a green pepper, the disliked object, was on the top of the right low box. The fact that the liked object was out of the reach of Kangkang made the different social meaning between the man in Fig. 5A and the tree in Fig. 5B critical—the man, rather than the tree, is both capable and ready to help. The experiment was conducted to investigate whether preschool Mandarin-speaking children with ASD were capable of making use of the social meaning of the man to predict Kangkang’s behavior (social cognition), as compared to TD children.

Across the 16 pairs of visual stimuli, both the position of the two boxes (i.e., whether they are on the left side or right side) and the location of the two objects (i.e., whether they are on the high box or low box) were counterbalanced. The 16 pairs of test objects were split into two lists, each list including only one of the test pictures in each pair (e.g., either Fig. 5A or Fig. 5B ) accompanied by the same spoken sentence. Each list was made up of 8 trials with an old man (e.g., Fig. 5A ) and 8 trials with a tree (e.g., Fig. 5B ), and the number of trials in terms of which box was on which side and which object was on which box was equal: four trials with the liked object on the high box on the left, four trials with the liked object on the high box on the right, four trials with the liked object on the low box on the left, and four trials with the liked object on the low box on the right. All trials in the two lists were randomly arranged. Participants from experimental and control groups were assigned to one of the two experimental lists randomly. 166 5-year-old children with ASD, 150 TD 5-year-old and 156 TD 4-year-old children were presented with List 1, and 166 5-year-old children with ASD, 150 TD 5-year-old and 156 TD 4-year-old children were presented with List 2.

Each participant received two pre-tests and one actual test. The first pre-test was designed to create the contrast between the liked and disliked objects. Participants were presented with 16 pairs of liked and disliked objects that would be used in the test phase, and explicitly instructed by the experimenter about which were the liked objects of Kangkang, and which were the disliked objects. After the training period, they were tested using 12 pairs of objects to see whether they could discriminate between the liked and disliked ones. Only children who passed all the test trials moved onto the next phase. The second phase was to establish the awareness of height. Participants were told that the old man could reach the high box while Kangkang could only reach the low box. After that, the participants were shown 12 test pictures, half of which contained a liked object on the high box and another half of which contained a disliked object on the high box. They were asked to indicate by pointing to the object that Kangkang could or could not reach for. Only children who answered all 12 questions correctly proceeded to the actual test phase.

The final test phase was to investigate whether participants could use the social cues embedded in the visual scenes to make inferences about others’ minds and behavior. Sixteen visual images were displayed by a monitor together with a spoken sentence saying “Look, which object do you think Kangkang will reach for?”, which was presented 500 ms after the visual stimulus appeared. Participants were seated approximately 64 cm from a 21 inch, 4:3 color monitor with 1024 × 768 pixel resolution. They were not asked to give any explicit responses, but just viewed the visual scenes freely. An EyeLink 1000 plus eye tracker was used to record the participants’ eye movements from the onset of the verb “reach” for 1000 ms because we believe they could infer Kangkang’s behavior after hearing the verb. The eye tracker was run under the free-to-move head mode with the monocular sampling rate of 500 Hz and a spatial resolution of 0.01 and an average error of less than 0.5.

The critical manipulation here was the presence of an old man or a tree when the liked object was placed on the high box. Note that typically developing children as young as 24 months of age know that the presence of a man indicates that the man could help them reach an object (e.g., refs. 50 , 51 ). If participants could make social inferences, they would demonstrate different eye movement patterns. To be more specific, they should show more fixations to the liked objects on the high box when an old man rather than when a tree was in the visual scene, because they inferred that the man could reach for the high box and he was willing to offer help while the tree could not, and they also inferred that Kangkang believed that the old man could help him achieve his goal. Based on these inferences, they would further reason about Kangkang’s action: he was likely to reach for the liked object placed on the high box when the old man was present, but he probably would reach for the disliked object placed on the low box when a tree was present. Therefore, there should be a significant difference in the eye gaze patterns between these two critical settings, by which we can further understand whether children could make inferences based on social cues in an established visual world. Note that no difference in eye movements was expected to exist when the liked object was placed on the low box, since Kangkang could reach the object by himself.

Reporting summary

Further information on research design is available in the Nature Research Reporting Summary linked to this article.

The results are illustrated in Figs. 6 and 7 . Error bars indicate standard errors. Figure 6 displays the results of Study 1. Eye movements were recorded in Study 1 which reflect the detailed attention deployment information (e.g., orientation, release, transfer) about how the participant got distracted from the background scene to the foreground stimulus. We followed the standard procedure for pre-processing eye movement data. First, the participants whose eye fixation durations were less than 80 ms (the minimum time to form a fixation) on more than 10 trials were excluded from further analysis, because fixation durations less than 80 ms indicate meaningless eye movements due to technical issues or lack of cooperation from the participants ( N  = 68). Second, participants whose average error rate was more than 20% either in the entire experiment or in one condition were excluded ( N  = 37). In addition, the data of participants with values greater than 3 SD above the average were also excluded as outliers ( N  = 76). This procedure was implemented to ensure that only meaningful eye movements were included in the final analyses. The procedure led to the inclusion of 523 participants in the final analyses. According to the attentional bias hypothesis, participants with DD should exhibit more difficulties than healthy controls when disengaging attention from negative stimuli and then transferring their attention to positive stimuli. And this can be characterized as transit time which is defined as the time difference between the onset of the positive/negative face and the time when the first fixation point formed on the face. Therefore, we calculated the difference in transit time between any two of the four foreground and background combinations. This process resulted in six combinations: nn-pn, nn-pp, np-pp, np-pn, nn-pn, and pn-pp. For further clarification, consider np-pp as an example, where np and pp are two combinations, np indicates negative (n) background with positive (p) face and pp indicates positive (p) background with positive face (p). The difference of transit time was then calculated as the transit time from negative background to positive face minus the transit time from positive background to positive face (np-pp). Other conditions can be interpreted accordingly. Therefore, np-pp represents the transit time difference between negative background with positive face and positive background with positive face. A significant positive value of this condition would indicate the difficulty in disengaging attention from negative stimuli. To test the effect of participant group on transit time differences, one-way multivariate analysis of variance of participant groups was performed with effect size calculated with partial eta squared (using IBM SPSS Statistics 24), and the result shows significant difference between the two groups (Wilks’ Lambda = 3.498, p  < 0.01, \({\eta }_{p}^{2}\)  = 0.136). To further investigate this significant difference, Fig. 6 shows the transit time difference for the two groups in each condition. As shown in the figure, only np-pp ( F (1,522) = 7.87, p  < 0.01, \({\eta }_{p}^{2}\)  = 0.014) and np-pn ( F (1,522) = 6.35, p  < 0.05, \(\,{\eta }_{p}^{2}\)  = 0.011), but no other conditions (all p s > 0.11), show significantly larger transit time differences between the DD group and the healthy controls, indicating that it was more challenging for participants with DD as compared with healthy controls in disengaging attention from negative stimuli and reducing preference to positive stimuli. Note that both the DD group and the healthy controls displayed a positive value of transit time difference, suggesting that both groups took longer time to disengage attention from negative stimuli than from positive ones. However, the magnitude of this difference was significantly greater for the DD group than for the healthy controls, indicating a second-order difference. This second-order difference might imply a fundamental disparity between the DD group and the healthy controls when processing emotional stimuli. Let’s discuss the non-significant conditions in more detail. The nn-pp condition occurred when both the background and the facial stimuli were negative for nn, but positive for pp, in which there was no competition between the positive and negative stimuli for attention, and thus failing to demonstrate a greater difficulty in disengaging attention from negative stimuli. Similarly, the lack of disengagement also applies to the nn-pn, nn-np, and pn-pp conditions, where there was no disengagement of attention from the negative background to the positive facial stimuli. To explain the non-significant results of the nn-np and pn-pp conditions, we present the following speculations. The nn-np value for the DD group was negative, indicating that they were less likely to transit their attention towards the positive stimuli when the faces were positive, thus yielding longer attention shift time for the np condition than for the nn condition. Concerning the pn-pp condition, the difference value was greater in the healthy controls than in the DD group, presumably because compared with the participants with DD, the healthy controls were less likely to shift their attention towards the negative stimuli in the pn condition, leading to a larger difference in the pn-pp condition. Overall, these findings verified the attentional bias hypothesis of people with DD by showing that they exhibited significant difficulties in disengaging attention from negative stimuli and reducing attention to positive stimuli, and thereby serving as a sensitive measure of individuals’ affective ToM.

The x-axis contains 6 conditions of difference in transit time between different foreground and background combinations (e.g. np-pp, where np and pp are two combinations, np indicates negative background with positive face and pp indicates positive background with positive face. The difference in transit time was then calculated as the transit time from negative background to positive face minus the transit time from positive background to positive face. Other conditions can be interpreted accordingly.). (error bars indicate standard errors, asterisk (*) denotes p  < 0.05).

Fixation proportions of the ASD 5-year-olds, the TD 4-year-olds and the TD 5-year-olds on the high box with the liked item (the left panel), on the low box with the liked item (the middle panel) and on the agent (the right panel) in scenes where there was a man vs. in scenes where there was a tree. (error bars indicate standard errors, asterisk (*) denotes p  < 0.05).

Figure 7 provides the results of Study 2. Eye movement data of all trials was included for both ASD and TD children according to our inclusion criteria. There are four areas of interest in the visual image: the high box area, the low box area, the social element area (man or tree), and the area including Kangkang. Each participant’s proportion of fixations onto these four areas per trial as the dependent variable was computed and further analyzed using generalized linear mixed models (GLMMs). GLMMs were computed for each participant group that included condition (i.e., when an old man versus when a tree was in the visual scene) as the fixed effect, and trial and participant as two random terms. The statistical models were chosen, because the eye fixation patterns were non-parametric binomial data in nature, and this model could include both fixed effects and by-participant as well as by-trial random effects. The model fitting process was realized using the glmer function in the lme4 package (v1.1.19) (Bates et al. 2013) under the R (v3.5.2) software environment (R Development Core Team 2017).

As indicated in Fig. 7 , when the liked object was placed on the high box, 4-year-old and 5-year-old TD children showed more fixations on the high box when the man was present compared to when the tree was present. A significant difference between the two critical settings was observed for both the TD 4-year-old children ( b  = 1.21, z  = 3.45, p  < 0.01) and the TD 5-year-old children ( b  = 0.14, z  = 3.18, p  < 0.01). However, this effect was not found for the 5-year-old ASD group ( b  = 0.77, z  = 1.16, p  = 0.35). In addition, there was no difference in the fixation proportions when the man was in the visual scene vs. when the tree was in the visual scene under the condition in which the liked object was placed on the low box. The results clearly indicate a difference in social inference between the TD children and children with ASD, thus serving as a sensitive measure of children’s cognitive ToM.

Discussions

The findings of the two studies clearly show how individuals with DD and ASD differ from healthy controls in cognitive-affective processes, and the obtained eye movement indicators are reliable and effective measures of cognitive-affective ToM. In addition, the findings can also inform us about the applicability and generalizability of our model. We discuss how V-ToM accounts for typical and atypical cognitive-affective ToM in relation to the findings of Study1 and Study 2 respectively. In particular, we focus on how the identification of DD and ASD can be related to the four stages proposed in V-ToM.

In Study 1, when presented with a typical visual stimulus like Fig. 8 , in which a car was moving on the road and then an accident happened, the participants first visually processed the scene, by means of segmenting the entire car from the background, constructing the border of the car, determining the orientation of the car, and analyzing the color of the car. Representations of the visual scene were then established (S1. Visual processing).

( A ) a car accident was first presented, and ( B ) an emotional face stimulus was then shown in the image.

After the visual representations were established, mental processing commenced immediately. At this stage, the mental representations were stored in working memory in the form of propositional statements, such as this is a car, the car is dented, the windows are broken, the car tilted and flew off the ground, the car is red , and so on (S2. Mental processing).

Then, the participants compared the mental representations established in S2 with the cognitive prototype stored in long-term memory, and established relevant prototypical representations in the form of propositions as well. Relevant prototypical representations included cars have a neutral connotation, dented surface has a negative connotation, broken windows have a negative connotation, tilted and flew-off conditions have a negative connotation, red color has a neutral connotation . Based on the prototype, the assessment process leads to corresponding final propositional representations: the fact that this is a car has a neutral connotation , the fact that the car is dented has a negative connotation , the fact that the windows are broken has a negative connotation , the fact that the car tilted and flew off the ground has a negative connotation and the fact that the car is red has a neutral connotation . These relevant mental presentations are the output of this stage (S3. Assessment).

The final representations computed in S3 were then registered in working memory and guided the eye movements in the visual world. In the task, the participants were then presented with a positive or negative face image, and were asked to shift attention from the background image and make judgments on the attribute of the face. Individuals with DD exhibited different eye movement patterns than healthy controls, presumably due to the differences between the two groups in any of the four stages proposed in our model. For example, individuals with DD might process visual information differently than healthy controls, leading to different visual representations (S1); they might establish mental representations that differ from healthy controls in degrees of positivity or negativity (S2); they might have different emotionally relevant prototypes from those of healthy controls (S3), or they might have difficulty in disengaging from negative information (S4). Differences in any of the four stages would lead to different eye movement patterns by individuals with DD as compared with healthy controls. Therefore, the eye movement patterns obtained from Study 1, e.g., start time of transfer (from face appeared to attention shifted) and transfer speed (from background scene to foreground stimuli), can serve as reliable indicators of emotion understanding (S4. Transfer and implementation).

In Study 2, when presented with a typical visual stimulus like Fig. 5 , where the liked object was placed on the high box and an old man (Fig. 5A ) or a tree (Fig. 5B ) was present in the scene, the participants first visually processed the images, by means of segmenting and segregating the elements from the background, and detecting the basic features of the elements, including border, contour, color, etc. Representations of the visual image were then established (S1. Visual processing).

Mental processing commenced immediately after S1. At this stage, the mental representations are established and stored in the form of propositional semantic statements. For instance, the mental representations of Fig. 5A include, this is Kangkang , Kangkang is short , this is a man, the man is tall , this is a high box , this is a strawberry , the strawberry is the liked object of Kangkang , the strawberry is outside Kangkang’s reach , the man can reach the strawberry , this is a low box , this is a green pepper , the green pepper is the disliked object of Kangkang , Kangkang can reach the green pepper , the man can reach the green pepper , and so on (S2. Mental processing).

The mental representations established in S2 are assessed according to their relevance to the prototype. The prototype is also represented in the form of propositional statements and is stored in long-term memory. In the social setting of Fig. 5A , the two relevant prototypical representations are: men are highly social beings and are willing to help and boys believe that men are willing to help . Evaluation of the established representations against the prototypical representations leads to two new mental representations: the man is ready to help and is going to help and Kangkang believes that the man is ready to help and is going to help . By contrast, in the social setting of Fig. 5B , the two relevant prototypical representations are: trees are not social beings and are unable to help and boys believe that trees are unable to help . The assessment against the prototype leads to two new mental representations: the tree is unable to help and Kangkang believes that the tree is unable to help (S3. Assessment).

The two new mental representations computed in S3 are registered in working memory and are transferred to guide the eye movements in the visual world. In this task, the participants are going to predict Kangkang’s behavior based on the relevant mental representations and make the social inference that Kangkang could ask the man to help him reach the liked object placed on the high box when presented with Fig. 5A . In contrast, when shown Fig. 5B , the participants would infer that Kangkang could not ask the tree for help and could only reach for the disliked object placed on the low box. Thus, the participants should show more looks at the area of the high box in the social setting of Fig. 5A containing the man, as compared to the same area of the high box in Fig. 5B with the tree (S4. Transfer and implementation).

When presented with visual images like Fig. 5 , children with ASD exhibited different eye movement patterns than their TD peers, presumably due to the difference between the two groups in any of the four stages described in our model. For instance, children with ASD might exhibit certain kinds of preference in processing and integrating visual information, thus leading to different visual representations than those of TD children (S1); they might establish different mental representations from those of their TD peers (S2); they might have a different set of socially relevant prototypical representations as compared with TD children (S3); or they might have a low matching degree of proposition and prototype (S4). Differences in any of the four stages would lead to different eye movement patterns by children with ASD as compared with TD children. Thus, the obtained eye movement patterns in Study 2 can be used as reliable indicators of social cognition. An anonymous reviewer raised an interesting question: whether there is an emotional development factor that could explain the eye movement pattern in Study 2. More specifically, the reviewer asked whether it is possible that ASD children do not develop emotionally at the same rate as TD children. We agree that this question is worthy of further exploration, but we will leave that for future research that directly investigates the connection between emotional development and theory of mind/social cognition in both ASD and TD children. Nonetheless, this potential factor does not affect our findings attesting to the significant difference between ASD and TD children.

The present paper provides a fine-grained conceptual framework of cognitive-affective ToM by dividing the relevant underlying processes into four stages. Compared with traditional models, our model, for the first time, innovatively integrates the two core components of ToM, the cognitive and affective components, into the construction of ToM. In addition, by detailed analysis of the four cognitive stages, our model allows us to explain how humans process cognitive-affective information in a fine-grained manner. Each stage is measurable, thus enabling us to make inferences about potential difficulties that might occur in each of the four stages. More specifically, we propose a theory of V-ToM by devising a novel audio-visual paradigm and by constructing visual scenes with cognitive and emotional meanings. In addition, the audio-visual paradigm is based on large sample size ( n  > 700) and thus the obtained eye movement patterns are robust and reliable indicators of the underlying cognitive-affective processes. The findings also suggest that the obtained eye movement patterns can be used effectively to distinguish between typical and atypical populations in the two core domains of ToM, and thus can potentially serve as clinical markers for DD and ASD.

We also wish to point out two limitations of the current study. The first limitation comes with the limited age range of the DD and ASD participants. We recruited only child participants for the cognitive ToM study and only adult participants for the affective ToM study. Ideally both adults and children should be included for each study, so that potential confounds due to age difference could be controlled for. The second limitation concerns the limited disorders being investigated in the study. The current study focused on the DD and ASD populations without comparing them to other DSM disorders. Ideally, to provide stronger evidence for specificity, future research should investigate a broader array of individuals with DSM disorders using the same paradigm, and then compare those groups against one another both within the adult population and the child/adolescent population.

To conclude, future work is needed to address the “real world” implications of these deficits and to develop effective transdiagnostic interventions for those individuals that are adversely affected. Traditional diagnostic methods for DD and ASD are largely based on the subjective assessment of the clinicians. By contrast, our model features a more objective and efficient measurement of DD and ASD. By extracting participants’ eye movement indicators from multiple dimensions automatically, we are able to identify the potential sources of their difficulties, thereby leading to more precise and reliable identification and intervention programs for individuals with DD and those with ASD. Note that the computability of the proposed V-ToM model operationalized abstract concepts of ToM into measurable processes that are potentially to be interfaced with AI technology, leading to AI-aided precision identification of mental disorders based on digital medicine.

Data availability

The full set of data and materials that support the findings of the experiments are available from the Beijing TeeView Technology Co., Ltd. Restrictions apply to the availability of these data, which were used under license for the experiments. Data are available on the request from the corresponding author H.M. with the permission of Beijing TeeView Technology Co., Ltd.

Code availability

The data were analyzed using standard statistical packages as discussed in the text. Analysis code is available on the request from the corresponding author H.M. with the permission of Beijing TeeView Technology Co., Ltd.

Baron-Cohen, S., Leslie, A. M. & Frith, U. Does the autistic child have a “theory of mind”? Cognition 21 , 37–46 (1985).

Article   CAS   PubMed   Google Scholar  

Premack, D. & Woodruff, G. Does the chimpanzee have a theory of mind? Behav. Brain Sci. 1 , 515–526 (1978).

Article   Google Scholar  

Wellman, H. M., Cross, D. & Watson, J. Meta‐analysis of theory‐of‐mind development: the truth about false belief. Child Dev. 72 , 655–684 (2001).

Wellman, H. M. & Liu, D. Scaling of theory‐of‐mind tasks. Child Dev. 75 , 523–541 (2004).

Article   PubMed   Google Scholar  

Wimmer, H. & Perner, J. Beliefs about beliefs: representation and constraining function of wrong beliefs in young children’s understanding of deception. Cognition 13 , 103–128 (1983).

Wellman, H. M. in Blackwell Handbook of Childhood Cognitive Development 167–187 (Wiley, 2002).

Southgate, V., Senju, A. & Csibra, G. Action anticipation through attribution of false belief by 2-year-olds. Psychol. Sci. 18 , 587–592 (2007).

Onishi, K. H. & Baillargeon, R. Do 15-month-old infants understand false beliefs? Science 308 , 255–258 (2005).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Song, H., Onishi, K. H., Baillargeon, R. & Fisher, C. Can an agent’s false belief be corrected by an appropriate communication? Psychological reasoning in 18-month-old infants. Cognition 109 , 295–315 (2008).

Article   PubMed   PubMed Central   Google Scholar  

Surian, L., Caldi, S. & Sperber, D. Attribution of beliefs by 13-month-old infants. Psychol. Sci. 18 , 580–586 (2007).

Senju, A., Southgate, V., White, S. & Frith, U. Mindblind eyes: an absence of spontaneous theory of mind in Asperger syndrome. Science 325 , 883–885 (2009).

Ruffman, T., Garnham, W. & Rideout, P. Social understanding in autism: eye gaze as a measure of core insights. J. Child Psychol. Psychiatry Allied Discip. 42 , 1083–1094 (2001).

Article   CAS   Google Scholar  

Abell, F., Happe, F. & Frith, U. Do triangles play tricks? Attribution of mental states to animated shapes in normal and abnormal development. Cogn. Dev. 15 , 1–16 (2000).

Mattern, M. et al. Behavioral evidence for an impairment of affective theory of mind capabilities in chronic depression. Psychopathology 48 , 240–250 (2015).

Tager-Flusberg, H. & Sullivan, K. A componential view of theory of mind: evidence from Williams syndrome. Cognition 76 , 59–90 (2000).

Raimo, S. et al. Cognitive and affective theory of mind across adulthood. Brain Sci. 12 , 899 (2022).

Platek, S. M., Keenan, J. P., Gallup, G. G. Jr. & Mohamed, F. B. Where am I? The neurological correlates of self and other. Cogn. Brain Res. 19 , 114–122 (2004).

Ellsworth, P. C. & Scherer, K. R. Appraisal Processes in Emotion (Oxford University Press, 2003).

Saxe, R. & Houlihan, S. D. Formalizing emotion concepts within a Bayesian model of theory of mind. Curr. Opin. Psychol. 17 , 15–21 (2017).

Scherer, K. R., Schorr, A. & Johnstone, T. Appraisal Processes in Emotion: Theory, Methods, Research (Oxford University Press, 2001).

Bora, E. & Berk, M. Theory of mind in major depressive disorder: a meta-analysis. J. Affect. Disord. 191 , 49–55 (2016).

Stewart, M. E., Barnard, L., Pearson, J., Hasan, R. & O’Brien, G. Presentation of depression in autism and Asperger syndrome: a review. Autism 10 , 103–116 (2006).

Baillargeon, R., Scott, R. M. & He, Z. False-belief understanding in infants. Trends Cogn. Sci. 14 , 110–118 (2010).

Barnhill, G. P. & Myles, B. S. Attributional style and depression in adolescents with Asperger syndrome. J. Posit. Behav. Interv. 3 , 175–182 (2001).

Beck, A. T. Cognitive Therapy of Depression (Guilford Press, 1979).

Bylinskii, Z., Judd, T., Oliva, A., Torralba, A. & Durand, F. What do different evaluation metrics tell us about saliency models? IEEE Trans. Pattern Anal. Mach. Intell. 41 , 740–757 (2018).

Cornia, M., Baraldi, L., Serra, G. & Cucchiara, R. A deep multi-level network for saliency prediction. In 2016 23rd International Conference on Pattern Recognition (ICPR) 3488–3493 (IEEE, 2016).

Cornia, M., Baraldi, L., Serra, G. & Cucchiara, R. Predicting human eye fixations via an lstm-based saliency attentive model. IEEE Trans. Image Process. 27 , 5142–5154 (2018).

Jetley, S., Murray, N. & Vig, E. End-to-end saliency mapping via probability distribution prediction. In Proceedings of the IEEE conference on computer vision and pattern recognition 5753–5761 (IEEE, 2016).

Jiang, M. et al. Learning to predict sequences of human visual fixations. IEEE Trans Neural Netw Learn. Syst. 27 , 1241–1252 (2016).

Kovács, Á. M., Téglás, E. & Endress, A. D. The social sense: susceptibility to others’ beliefs in human infants and adults. Science 330 , 1830–1834 (2010).

Kruthiventi, S. S. S., Ayush, K. & Babu, R. V. Deepfix: a fully convolutional neural network for predicting human eye fixations. IEEE Trans. Image Process. 26 , 4446–4456 (2017).

Liu, N., Han, J., Liu, T. & Li, X. Learning to predict eye fixations via multiresolution convolutional neural networks. IEEE Trans. Neural Netw. Learn. Syst. 29 , 392–404 (2016).

MacLeod, C., Mathews, A. & Tata, P. Attentional bias in emotional disorders. J Abnorm. Psychol. 95 , 15 (1986).

Mogg, K. & Bradley, B. P. Orienting of attention to threatening facial expressions presented under conditions of restricted awareness. Cogn. Emot. 13 , 713–740 (1999).

Pan, J., Sayrol, E., Giro-i-Nieto, X., McGuinness, K. & O’Connor, N. E. Shallow and deep convolutional networks for saliency prediction. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 598–606 (IEEE, 2016).

Phelps, E. A. & LeDoux, J. E. Contributions of the amygdala to emotion processing: from animal models to human behavior. Neuron 48 , 175–187 (2005).

Shen, C., Huang, X. & Zhao, Q. Predicting eye fixations on webpage with an ensemble of early features and high-level representations from deep network. IEEE Trans. Multimedia 17 , 2084–2093 (2015).

Vig, E., Dorr, M., Martinetz, T. & Barth, E. Intrinsic dimensionality predicts the saliency of natural dynamic scenes. IEEE Trans. Pattern Anal. Mach. Intell. 34 , 1080–1091 (2011).

Zhou, P., Zhan, L. & Ma, H. Understanding others’ minds: social inference in preschool children with autism spectrum disorder. J. Autism Dev. Disord. 49 , 4523–4534 (2019).

Treisman, A. M. & Gelade, G. A feature-integration theory of attention. Cogn. Psychol. 12 , 97–136 (1980).

Suslow, T., Hußlack, A., Kersting, A. & Bodenschatz, C. M. Attentional biases to emotional information in clinical depression: a systematic and meta-analytic review of eye tracking findings. J. Affect. Disord. 274 , 632–642 (2020).

Molesworth, C. J., Bowler, D. M. & Hampton, J. A. The prototype effect in recognition memory: Intact in autism? J. Child Psychol. Psychiatry 46 , 661–672 (2005).

Zung, W. W. K. A self-rating depression scale. Arch. Gen. Psychiatry 12 , 63–70 (1965).

Derogatis, L. R. SCL-90: an outpatient psychiatric rating scale-preliminary report. Psychopharmacol. Bull. 9 , 13–28 (1973).

CAS   PubMed   Google Scholar  

Bao, S., Ma, H. & Li, W. ThuPIS: a new affective image system for psychological analysis. In 2014 IEEE International Symposium on Bioelectronics and Bioinformatics (IEEE ISBB 2014) 1–4 (IEEE, 2014).

Chen, L.-F. & Yen, Y.-S. Taiwanese Facial Expression Image Database (Brain Mapping Laboratory, Institute of Brain Science, National Yang-Ming University, 2007).

Wolfe, J. M. Guided Search 6.0: an updated model of visual search. Psychonom. Bull. Rev . 28 , 1060–1092 (2021).

Keller, A. S., Leikauf, J. E., Holt-Gosselin, B., Staveland, B. R. & Williams, L. M. Paying attention to attention in depression. Transl. Psychiatry 9 , 1–12 (2019).

Masangkay, Z. S. et al. The early development of inferences about the visual percepts of others. Child Dev. 45 , 357–366 (1974).

Moll, H. & Tomasello, M. Level 1 perspective‐taking at 24 months of age. Br. J. Dev. Psychol. 24 , 603–613 (2006).

Download references

Acknowledgements

The authors would like to thank the children, the parents, the students, and the teachers at the Enqi Autism Platform, at the Rehabilitation and Education Centre for Children with Autism, at the Taolifangyuan Kindergarten, and at the Students Counselling Center of Tsinghua University, Beijing, China for their assistance and support in running the study; a small portion of the data of the people with DD has been presented at the International Conference on Image and Graphics, Shanghai, 13–15 September 2017 ( https://doi.org/10.1007/978-3-319-71607-7_43 ), and at the IEEE International Conference on Image Processing, Taipei, 22–25 September 2019 ( https://doi.org/10.1109/ICIP.2019.8803181 ). A small portion of the data of the children with ASD has been published in the Journal of Autism and Developmental Disorders 49 , 4523–4534 (2019), https://doi.org/10.1007/s10803-019-04167-x .

P.Z. was supported by the National Social Science Foundation of China [16BYY076]. H.M. was supported by the National Natural Science Foundation of China [U20B2062, 62227801]. B.Z. was supported by the National Natural Science Foundation of China [62206015].

Author information

These authors contributed equally: Peng Zhou, Huimin Ma

Authors and Affiliations

School of International Studies, Zhejiang University, Hangzhou, China

School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing, China

Huimin Ma, Bochao Zou & Yuxin Lin

Department of Foreign Languages and Literatures, Tsinghua University, Beijing, China

Xiaowen Zhang & Shuyan Zhao

Department of Electronic Engineering, Tsinghua University, Beijing, China

Yidong Wang

National Clinical Research Center for Mental Disorders & Beijing Key Laboratory of Mental Disorders, Beijing Anding Hospital, Capital Medical University, Beijing, China

Lei Feng & Gang Wang

Advanced Innovation Center for Human Brain Protection, Capital Medical University, Beijing, China

You can also search for this author in PubMed   Google Scholar

Contributions

P.Z. and H.M. conceived, designed and supervised the experiments, both contributing equally to the manuscript; P.Z., Y.L., S.Z., L.F., and G.W. collected and analyzed the data; P.Z., H.M., B.Z., X.Z., and S.Z. wrote the manuscript; all authors discussed the results and commented on the manuscript.

Corresponding author

Correspondence to Huimin Ma .

Ethics declarations

Competing interests.

The authors declare no competing interests. P.Z. and H.M. are inventors on a confidential provisional patent application that covers the subject matter of this article.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Reporting summary, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Zhou, P., Ma, H., Zou, B. et al. A conceptual framework of cognitive-affective theory of mind: towards a precision identification of mental disorders. npj Mental Health Res 2 , 12 (2023). https://doi.org/10.1038/s44184-023-00031-0

Download citation

Received : 27 March 2023

Accepted : 11 July 2023

Published : 10 August 2023

DOI : https://doi.org/10.1038/s44184-023-00031-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List

What influences students’ abilities to critically evaluate scientific investigations?

Ashley b. heim.

1 Department of Ecology and Evolutionary Biology, Cornell University, Ithaca, NY, United States of America

2 Laboratory of Atomic and Solid State Physics, Cornell University, Ithaca, NY, United States of America

David Esparza

Michelle k. smith, n. g. holmes, associated data.

All raw data files are available from the Cornell Institute for Social and Economic Research (CISER) data and reproduction archive ( https://archive.ciser.cornell.edu/studies/2881 ).

Critical thinking is the process by which people make decisions about what to trust and what to do. Many undergraduate courses, such as those in biology and physics, include critical thinking as an important learning goal. Assessing critical thinking, however, is non-trivial, with mixed recommendations for how to assess critical thinking as part of instruction. Here we evaluate the efficacy of assessment questions to probe students’ critical thinking skills in the context of biology and physics. We use two research-based standardized critical thinking instruments known as the Biology Lab Inventory of Critical Thinking in Ecology (Eco-BLIC) and Physics Lab Inventory of Critical Thinking (PLIC). These instruments provide experimental scenarios and pose questions asking students to evaluate what to trust and what to do regarding the quality of experimental designs and data. Using more than 3000 student responses from over 20 institutions, we sought to understand what features of the assessment questions elicit student critical thinking. Specifically, we investigated (a) how students critically evaluate aspects of research studies in biology and physics when they are individually evaluating one study at a time versus comparing and contrasting two and (b) whether individual evaluation questions are needed to encourage students to engage in critical thinking when comparing and contrasting. We found that students are more critical when making comparisons between two studies than when evaluating each study individually. Also, compare-and-contrast questions are sufficient for eliciting critical thinking, with students providing similar answers regardless of if the individual evaluation questions are included. This research offers new insight on the types of assessment questions that elicit critical thinking at the introductory undergraduate level; specifically, we recommend instructors incorporate more compare-and-contrast questions related to experimental design in their courses and assessments.

Introduction

Critical thinking and its importance.

Critical thinking, defined here as “the ways in which one uses data and evidence to make decisions about what to trust and what to do” [ 1 ], is a foundational learning goal for almost any undergraduate course and can be integrated in many points in the undergraduate curriculum. Beyond the classroom, critical thinking skills are important so that students are able to effectively evaluate data presented to them in a society where information is so readily accessible [ 2 , 3 ]. Furthermore, critical thinking is consistently ranked as one of the most necessary outcomes of post-secondary education for career advancement by employers [ 4 ]. In the workplace, those with critical thinking skills are more competitive because employers assume they can make evidence-based decisions based on multiple perspectives, keep an open mind, and acknowledge personal limitations [ 5 , 6 ]. Despite the importance of critical thinking skills, there are mixed recommendations on how to elicit and assess critical thinking during and as a result of instruction. In response, here we evaluate the degree to which different critical thinking questions elicit students’ critical thinking skills.

Assessing critical thinking in STEM

Across STEM (i.e., science, technology, engineering, and mathematics) disciplines, several standardized assessments probe critical thinking skills. These assessments focus on aspects of critical thinking and ask students to evaluate experimental methods [ 7 – 11 ], form hypotheses and make predictions [ 12 , 13 ], evaluate data [ 2 , 12 – 14 ], or draw conclusions based on a scenario or figure [ 2 , 12 – 14 ]. Many of these assessments are open-response, so they can be difficult to score, and several are not freely available.

In addition, there is an ongoing debate regarding whether critical thinking is a domain-general or context-specific skill. That is, can someone transfer their critical thinking skills from one domain or context to another (domain-general) or do their critical thinking skills only apply in their domain or context of expertise (context-specific)? Research on the effectiveness of teaching critical thinking has found mixed results, primarily due to a lack of consensus definition of and assessment tools for critical thinking [ 15 , 16 ]. Some argue that critical thinking is domain-general—or what Ennis refers to as the “general approach”—because it is an overlapping skill that people use in various aspects of their lives [ 17 ]. In contrast, others argue that critical thinking must be elicited in a context-specific domain, as prior knowledge is needed to make informed decisions in one’s discipline [ 18 , 19 ]. Current assessments include domain-general components [ 2 , 7 , 8 , 14 , 20 , 21 ], asking students to evaluate, for instance, experiments on the effectiveness of dietary supplements in athletes [ 20 ] and context-specific components, such as to measure students’ abilities to think critically in domains such as neuroscience [ 9 ] and biology [ 10 ].

Others maintain the view that critical thinking is a context-specific skill for the purpose of undergraduate education, but argue that it should be content accessible [ 22 – 24 ], as “thought processes are intertwined with what is being thought about” [ 23 ]. From this viewpoint, the context of the assessment would need to be embedded in a relatively accessible context to assess critical thinking independent of students’ content knowledge. Thus, to effectively elicit critical thinking among students, instructors should use assessments that present students with accessible domain-specific information needed to think deeply about the questions being asked [ 24 , 25 ].

Within the context of STEM, current critical thinking assessments primarily ask students to evaluate a single experimental scenario (e.g., [ 10 , 20 ]), though compare-and-contrast questions about more than one scenario can be a powerful way to elicit critical thinking [ 26 , 27 ]. Generally included in the “Analysis” level of Bloom’s taxonomy [ 28 – 30 ], compare-and-contrast questions encourage students to recognize, distinguish between, and relate features between scenarios and discern relevant patterns or trends, rather than compile lists of important features [ 26 ]. For example, a compare-and-contrast assessment may ask students to compare the hypotheses and research methods used in two different experimental scenarios, instead of having them evaluate the research methods of a single experiment. Alternatively, students may inherently recall and use experimental scenarios based on their prior experiences and knowledge as they evaluate an individual scenario. In addition, evaluating a single experimental scenario individually may act as metacognitive scaffolding [ 31 , 32 ]—a process which “guides students by asking questions about the task or suggesting relevant domain-independent strategies [ 32 ]—to support students in their compare-and-contrast thinking.

Purpose and research questions

Our primary objective of this study was to better understand what features of assessment questions elicit student critical thinking using two existing instruments in STEM: the Biology Lab Inventory of Critical Thinking in Ecology (Eco-BLIC) and Physics Lab Inventory of Critical Thinking (PLIC). We focused on biology and physics since critical thinking assessments were already available for these disciplines. Specifically, we investigated (a) how students critically evaluate aspects of research studies in biology and physics when they are individually evaluating one study at a time or comparing and contrasting two studies and (b) whether individual evaluation questions are needed to encourage students to engage in critical thinking when comparing and contrasting.

Providing undergraduates with ample opportunities to practice critical thinking skills in the classroom is necessary for evidence-based critical thinking in their future careers and everyday life. While most critical thinking instruments in biology and physics contexts have undergone some form of validation to ensure they are accurately measuring the intended construct, to our knowledge none have explored how different question types influence students’ critical thinking. This research offers new insight on the types of questions that elicit critical thinking, which can further be applied by educators and researchers across disciplines to measure cognitive student outcomes and incorporate more effective critical thinking opportunities in the classroom.

Ethics statement

The procedures for this study were approved by the Institutional Review Board of Cornell University (Eco-BLIC: #1904008779; PLIC: #1608006532). Informed consent was obtained by all participating students via online consent forms at the beginning of the study, and students did not receive compensation for participating in this study unless their instructor offered credit for completing the assessment.

Participants and assessment distribution

We administered the Eco-BLIC to undergraduate students across 26 courses at 11 institutions (six doctoral-granting, three Master’s-granting, and two Baccalaureate-granting) in Fall 2020 and Spring 2021 and received 1612 usable responses. Additionally, we administered the PLIC to undergraduate students across 21 courses at 11 institutions (six doctoral-granting, one Master’s-granting, three four-year colleges, and one 2-year college) in Fall 2020 and Spring 2021 and received 1839 usable responses. We recruited participants via convenience sampling by emailing instructors of primarily introductory ecology-focused courses or introductory physics courses who expressed potential interest in implementing our instrument in their course(s). Both instruments were administered online via Qualtrics and students were allowed to complete the assessments outside of class. The demographic distribution of the response data is presented in Table 1 , all of which were self-reported by students. The values presented in this table represent all responses we received.

Instrument description

Question types.

Though the content and concepts featured in the Eco-BLIC and PLIC are distinct, both instruments share a similar structure and set of question types. The Eco-BLIC—which was developed using a structure similar to that of the PLIC [ 1 ]—includes two predator-prey scenarios based on relationships between (a) smallmouth bass and mayflies and (b) great-horned owls and house mice. Within each scenario, students are presented with a field-based study and a laboratory-based study focused on a common research question about feeding behaviors of smallmouth bass or house mice, respectively. The prompts for these two Eco-BLIC scenarios are available in S1 and S2 Appendices. The PLIC focuses on two research groups conducting different experiments to test the relationship between oscillation periods of masses hanging on springs [ 1 ]; the prompts for this scenario can be found in S3 Appendix . The descriptive prompts in both the Eco-BLIC and PLIC also include a figure presenting data collected by each research group, from which students are expected to draw conclusions. The research scenarios (e.g., field-based group and lab-based group on the Eco-BLIC) are written so that each group has both strengths and weaknesses in their experimental designs.

After reading the prompt for the first experimental group (Group 1) in each instrument, students are asked to identify possible claims from Group 1’s data (data evaluation questions). Students next evaluate the strengths and weaknesses of various study features for Group 1 (individual evaluation questions). Examples of these individual evaluation questions are in Table 2 . They then suggest next steps the group should pursue (next steps items). Students are then asked to read about the prompt describing the second experimental group’s study (Group 2) and again answer questions about the possible claims, strengths and weaknesses, and next steps of Group 2’s study (data evaluation questions, individual evaluation questions, and next steps items). Once students have independently evaluated Groups 1 and 2, they answer a series of questions to compare the study approaches of Group 1 versus Group 2 (group comparison items). In this study, we focus our analysis on the individual evaluation questions and group comparison items.

The Eco-BLIC examples are derived from the owl/mouse scenario.

Instrument versions

To determine whether the individual evaluation questions impacted the assessment of students’ critical thinking, students were randomly assigned to take one of two versions of the assessment via Qualtrics branch logic: 1) a version that included the individual evaluation and group comparison items or 2) a version with only the group comparison items, with the individual evaluation questions removed. We calculated the median time it took students to answer each of these versions for both the Eco-BLIC and PLIC.

Think-aloud interviews

We also conducted one-on-one think-aloud interviews with students to elicit feedback on the assessment questions (Eco-BLIC n = 21; PLIC n = 4). Students were recruited via convenience sampling at our home institution and were primarily majoring in biology or physics. All interviews were audio-recorded and screen captured via Zoom and lasted approximately 30–60 minutes. We asked participants to discuss their reasoning for answering each question as they progressed through the instrument. We did not analyze these interviews in detail, but rather used them to extract relevant examples of critical thinking that helped to explain our quantitative findings. Multiple think-aloud interviews were conducted with students using previous versions of the PLIC [ 1 ], though these data are not discussed here.

Data analyses

Our analyses focused on (1) investigating the alignment between students’ responses to the individual evaluation questions and the group comparison items and (2) comparing student responses between the two instrument versions. If individual evaluation and group comparison items elicit critical thinking in the same way, we would expect to see the same frequency of responses for each question type, as per Fig 1 . For example, if students evaluated one study feature of Group 1 as a strength and the same study feature for Group 2 as a strength, we would expect that students would respond that both groups were highly effective for this study feature on the group comparison item (i.e., data represented by the purple circle in the top right quadrant of Fig 1 ). Alternatively, if students evaluated one study feature of Group 1 as a strength and the same study feature for Group 2 as a weakness, we would expect that students would indicate that Group 1 was more effective than Group 2 on the group comparison item (i.e., data represented by the green circle in the lower right quadrant of Fig 1 ).

The x- and y-axes represent rankings on the individual evaluation questions for Groups 1 and 2 (or field and lab groups), respectively. The colors in the legend at the top of the figure denote responses to the group comparison items. In this idealized example, all pie charts are the same size to indicate that the student answers are equally proportioned across all answer combinations.

We ran descriptive statistics to summarize student responses to questions and examine distributions and frequencies of the data on the Eco-BLIC and PLIC. We also conducted chi-square goodness-of-fit tests to analyze differences in student responses between versions within the relevant questions from the same instrument. In all of these tests, we used a Bonferroni correction to lower the chances of receiving a false positive and account for multiple comparisons. We generated figures—primarily multi-pie chart graphs and heat maps—to visualize differences between individual evaluation and group comparison items and between versions of each instrument with and without individual evaluation questions, respectively. All aforementioned data analyses and figures were conducted or generated in the R statistical computing environment (v. 4.1.1) and Microsoft Excel.

We asked students to evaluate different experimental set-ups on the Eco-BLIC and PLIC two ways. Students first evaluated the strengths and weaknesses of study features for each scenario individually (individual evaluation questions, Table 2 ) and, subsequently, answered a series of questions to compare and contrast the study approaches of both research groups side-by-side (group comparison items, Table 2 ). Through analyzing the individual evaluation questions, we found that students generally ranked experimental features (i.e., those related to study set-up, data collection and summary methods, and analysis and outcomes) of the independent research groups as strengths ( Fig 2 ), evidenced by the mean scores greater than 2 on a scale from 1 (weakness) to 4 (strength).

Each box represents the interquartile range (IQR). Lines within each box represent the median. Circles represent outliers of mean scores for each question.

Individual evaluation versus compare-and-contrast evaluation

Our results indicate that when students consider Group 1 or Group 2 individually, they mark most study features as strengths (consistent with the means in Fig 2 ), shown by the large circles in the upper right quadrant across the three experimental scenarios ( Fig 3 ). However, the proportion of colors on each pie chart shows that students select a range of responses when comparing the two groups [e.g., Group 1 being more effective (green), Group 2 being more effective (blue), both groups being effective (purple), and neither group being effective (orange)]. We infer that students were more discerning (i.e., more selective) when they were asked to compare the two groups across the various study features ( Fig 3 ). In short, students think about the groups differently if they are rating either Group 1 or Group 2 in the individual evaluation questions versus directly comparing Group 1 to Group 2.

The x- and y-axes represent students’ rankings on the individual evaluation questions for Groups 1 and 2 on each assessment, respectively, where 1 indicates weakness and 4 indicates strength. The overall size of each pie chart represents the proportion of students who responded with each pair of ratings. The colors in the pie charts denote the proportion of students’ responses who chose each option on the group comparison items. (A) Eco-BLIC bass-mayfly scenario (B) Eco-BLIC owl-mouse scenario (C) PLIC oscillation periods of masses hanging on springs scenario.

These results are further supported by student responses from the think-aloud interviews. For example, one interview participant responding to the bass-mayfly scenario of the Eco-BLIC explained that accounting for bias/error in both the field and lab groups in this scenario was a strength (i.e., 4). This participant mentioned that Group 1, who performed the experiment in the field, “[had] outliers, so they must have done pretty well,” and that Group 2, who collected organisms in the field but studied them in lab, “did a good job of accounting for bias.” However, when asked to compare between the groups, this student argued that Group 2 was more effective at accounting for bias/error, noting that “they controlled for more variables.”

Another individual who was evaluating “repeated trials for each mass” in the PLIC expressed a similar pattern. In response to ranking this feature of Group 1 as a strength, they explained: “Given their uncertainties and how small they are, [the group] seems like they’ve covered their bases pretty well.” Similarly, they evaluated this feature of Group 2 as a strength as well, simply noting: “Same as the last [group], I think it’s a strength.” However, when asked to compare between Groups 1 and 2, this individual argued that Group 1 was more effective because they conducted more trials.

Individual evaluation questions to support compare and contrast thinking

Given that students were more discerning when they directly compared two groups for both biology and physics experimental scenarios, we next sought to determine if the individual evaluation questions for Group 1 or Group 2 were necessary to elicit or helpful to support student critical thinking about the investigations. To test this, students were randomly assigned to one of two versions of the instrument. Students in one version saw individual evaluation questions about Group 1 and Group 2 and then saw group comparison items for Group 1 versus Group 2. Students in the second version only saw the group comparison items. We found that students assigned to both versions responded similarly to the group comparison questions, indicating that the individual evaluation questions did not promote additional critical thinking. We visually represent these similarities across versions with and without the individual evaluation questions in Fig 4 as heat maps.

The x-axis denotes students’ responses on the group comparison items (i.e., whether they ranked Group 1 as more effective, Group 2 as more effective, both groups as highly effective, or neither group as effective/both groups were minimally effective). The y-axis lists each of the study features that students compared between the field and lab groups. White and lighter shades of red indicate a lower percentage of student responses, while brighter red indicates a higher percentage of student responses. (A) Eco-BLIC bass-mayfly scenario. (B) Eco-BLIC owl-mouse scenario. (C) PLIC oscillation periods of masses hanging on springs scenario.

We ran chi-square goodness-of-fit tests on the answers between student responses on both instrument versions and there were no significant differences on the Eco-BLIC bass-mayfly scenario ( Fig 4A ; based on an adjusted p -value of 0.006) or owl-mouse questions ( Fig 4B ; based on an adjusted p-value of 0.004). There were only three significant differences (out of 53 items) in how students responded to questions on both versions of the PLIC ( Fig 4C ; based on an adjusted p -value of 0.0005). The items that students responded to differently ( p <0.0005) across both versions were items where the two groups were identical in their design; namely, the equipment used (i.e., stopwatches), the variables measured (i.e., time and mass), and the number of bounces of the spring per trial (i.e., five bounces). We calculated Cramer’s C (Vc; [ 33 ]), a measure commonly applied to Chi-square goodness of fit models to understand the magnitude of significant results. We found that the effect sizes for these three items were small (Vc = 0.11, Vc = 0.10, Vc = 0.06, respectively).

The trend that students answer the Group 1 versus Group 2 comparison questions similarly, regardless of whether they responded to the individual evaluation questions, is further supported by student responses from the think-aloud interviews. For example, one participant who did not see the individual evaluation questions for the owl-mouse scenario of the Eco-BLIC independently explained that sampling mice from other fields was a strength for both the lab and field groups. They explained that for the lab group, “I think that [the mice] coming from multiple nearby fields is good…I was curious if [mouse] behavior was universal.” For the field group, they reasoned, “I also noticed it was just from a single nearby field…I thought that was good for control.” However, this individual ultimately reasoned that the field group was “more effective for sampling methods…it’s better to have them from a single field because you know they were exposed to similar environments.” Thus, even without individual evaluation questions available, students can still make individual evaluations when comparing and contrasting between groups.

We also determined that removing the individual evaluation questions decreased the duration of time students needed to complete the Eco-BLIC and PLIC. On the Eco-BLIC, the median time to completion for the version with individual evaluation and group comparison questions was approximately 30 minutes, while the version with only the group comparisons had a median time to completion of 18 minutes. On the PLIC, the median time to completion for the version with individual evaluation questions and group comparison questions was approximately 17 minutes, while the version with only the group comparisons had a median time to completion of 15 minutes.

To determine how to elicit critical thinking in a streamlined manner using introductory biology and physics material, we investigated (a) how students critically evaluate aspects of experimental investigations in biology and physics when they are individually evaluating one study at a time versus comparing and contrasting two and (b) whether individual evaluation questions are needed to encourage students to engage in critical thinking when comparing and contrasting.

Students are more discerning when making comparisons

We found that students were more discerning when comparing between the two groups in the Eco-BLIC and PLIC rather than when evaluating each group individually. While students tended to independently evaluate study features of each group as strengths ( Fig 2 ), there was greater variation in their responses to which group was more effective when directly comparing between the two groups ( Fig 3 ). Literature evaluating the role of contrasting cases provides plausible explanations for our results. In that work, contrasting between two cases supports students in identifying deep features of the cases, compared with evaluating one case after the other [ 34 – 37 ]. When presented with a single example, students may deem certain study features as unimportant or irrelevant, but comparing study features side-by-side allows students to recognize the distinct features of each case [ 38 ]. We infer, therefore, that students were better able to recognize the strengths and weaknesses of the two groups in each of the assessment scenarios when evaluating the groups side by side, rather than in isolation [ 39 , 40 ]. This result is somewhat surprising, however, as students could have used their knowledge of experimental designs as a contrasting case when evaluating each group. Future work, therefore, should evaluate whether experts use their vast knowledge base of experimental studies as discerning contrasts when evaluating each group individually. This work would help determine whether our results here suggest that students do not have a sufficient experiment-base to use as contrasts or if the students just do not use their experiment-base when evaluating the individual groups. Regardless, our study suggests that critical thinking assessments should ask students to compare and contrast experimental scenarios, rather than just evaluate individual cases.

Individual evaluation questions do not influence answers to compare and contrast questions

We found that individual evaluation questions were unnecessary for eliciting or supporting students’ critical thinking on the two assessments. Students responded to the group comparison items similarly whether or not they had received the individual evaluation questions. The exception to this pattern was that students responded differently to three group comparison items on the PLIC when individual evaluation questions were provided. These three questions constituted a small portion of the PLIC and showed a small effect size. Furthermore, removing the individual evaluation questions decreased the median time for students to complete the Eco-BLIC and PLIC. It is plausible that spending more time thinking about the experimental methods while responding to the individual evaluation questions would then prepare students to be better discerners on the group comparison questions. However, the overall trend is that individual evaluation questions do not have a strong impact on how students evaluate experimental scenarios, nor do they set students up to be better critical thinkers later. This finding aligns with prior research suggesting that students tend to disregard details when they evaluate a single case, rather than comparing and contrasting multiple cases [ 38 ], further supporting our findings about the effectiveness of the group comparison questions.

Practical implications

Individual evaluation questions were not effective for students to engage in critical thinking nor to prepare them for subsequent questions that elicit their critical thinking. Thus, researchers and instructors could make critical thinking assessments more effective and less time-consuming by encouraging comparisons between cases. Additionally, the study raises a question about whether instruction should incorporate more experimental case studies throughout their courses and assessments so that students have a richer experiment-base to use as contrasts when evaluating individual experimental scenarios. To help students discern information about experimental design, we suggest that instructors consider providing them with multiple experimental studies (i.e., cases) and asking them to compare and contrast between these studies.

Future directions and limitations

When designing critical thinking assessments, questions should ask students to make meaningful comparisons that require them to consider the important features of the scenarios. One challenge of relying on compare-and-contrast questions in the Eco-BLIC and PLIC to elicit students’ critical thinking is ensuring that students are comparing similar yet distinct study features across experimental scenarios, and that these comparisons are meaningful [ 38 ]. For example, though sample size is different between experimental scenarios in our instruments, it is a significant feature that has implications for other aspects of the research like statistical analyses and behaviors of the animals. Therefore, one limitation of our study could be that we exclusively focused on experimental method evaluation questions (i.e., what to trust), and we are unsure if the same principles hold for other dimensions of critical thinking (i.e., what to do). Future research should explore whether questions that are not in a compare-and-contrast format also effectively elicit critical thinking, and if so, to what degree.

As our question schema in the Eco-BLIC and PLIC were designed for introductory biology and physics content, it is unknown how effective this question schema would be for upper-division biology and physics undergraduates who we would expect to have more content knowledge and prior experiences for making comparisons in their respective disciplines [ 18 , 41 ]. For example, are compare-and-contrast questions still needed to elicit critical thinking among upper-division students, or would critical thinking in this population be more effectively assessed by incorporating more sophisticated data analyses in the research scenarios? Also, if students with more expert-like thinking have a richer set of experimental scenarios to inherently use as contrasts when comparing, we might expect their responses on the individual evaluation questions and group comparisons to better align. To further examine how accessible and context-specific the Eco-BLIC and PLIC are, novel scenarios could be developed that incorporate topics and concepts more commonly addressed in upper-division courses. Additionally, if instructors offer students more experience comparing and contrasting experimental scenarios in the classroom, would students be more discerning on the individual evaluation questions?

While a single consensus definition of critical thinking does not currently exist [ 15 ], continuing to explore critical thinking in other STEM disciplines beyond biology and physics may offer more insight into the context-specific nature of critical thinking [ 22 , 23 ]. Future studies should investigate critical thinking patterns in other STEM disciplines (e.g., mathematics, engineering, chemistry) through designing assessments that encourage students to evaluate aspects of at least two experimental studies. As undergraduates are often enrolled in multiple courses simultaneously and thus have domain-specific knowledge in STEM, would we observe similar patterns in critical thinking across additional STEM disciplines?

Lastly, we want to emphasize that we cannot infer every aspect of critical thinking from students’ responses on the Eco-BLIC and PLIC. However, we suggest that student responses on the think-aloud interviews provide additional qualitative insight into how and why students were making comparisons in each scenario and their overall critical thinking processes.

Conclusions

Overall, we found that comparing and contrasting two different experiments is an effective and efficient way to elicit context-specific critical thinking in introductory biology and physics undergraduates using the Eco-BLIC and the PLIC. Students are more discerning (i.e., critical) and engage more deeply with the scenarios when making comparisons between two groups. Further, students do not evaluate features of experimental studies differently when individual evaluation questions are provided or removed. These novel findings hold true across both introductory biology and physics, based on student responses on the Eco-BLIC and PLIC, respectively—though there is much more to explore regarding critical thinking processes of students across other STEM disciplines and in more advanced stages of their education. Undergraduate students in STEM need to be able to critically think for career advancement, and the Eco-BLIC and PLIC are two means of measuring students’ critical thinking in biology and physics experimental contexts via comparing and contrasting. This research offers new insight on the types of questions that elicit critical thinking, which can further be applied by educators and researchers across disciplines to teach and measure cognitive student outcomes. Specifically, we recommend instructors incorporate more compare-and-contrast questions related to experimental design in their courses to efficiently elicit undergraduates’ critical thinking.

Supporting information

S1 appendix, s2 appendix, s3 appendix, acknowledgments.

We thank the members of the Cornell Discipline-based Education Research group for their feedback on this article, as well as our advisory board (Jenny Knight, Meghan Duffy, Luanna Prevost, and James Hewlett) and the AAALab for their ideas and suggestions. We also greatly appreciate the instructors who shared the Eco-BLIC and PLIC in their classes and the students who participated in this study.

Funding Statement

This work was supported by the National Science Foundation under grants DUE-1909602 (MS & NH) and DUE-1611482 (NH). NSF: nsf.gov The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Data Availability