critical thinking skills inventory

Why Schools Need to Change Yes, We Can Define, Teach, and Assess Critical Thinking Skills

Jeff Heyck-Williams (He, His, Him) Director of the Two Rivers Learning Institute in Washington, DC

Today’s learners face an uncertain present and a rapidly changing future that demand far different skills and knowledge than were needed in the 20th century. We also know so much more about enabling deep, powerful learning than we ever did before. Our collective future depends on how well young people prepare for the challenges and opportunities of 21st-century life.

Critical thinking is a thing. We can define it; we can teach it; and we can assess it.

While the idea of teaching critical thinking has been bandied around in education circles since at least the time of John Dewey, it has taken greater prominence in the education debates with the advent of the term “21st century skills” and discussions of deeper learning. There is increasing agreement among education reformers that critical thinking is an essential ingredient for long-term success for all of our students.

However, there are still those in the education establishment and in the media who argue that critical thinking isn’t really a thing, or that these skills aren’t well defined and, even if they could be defined, they can’t be taught or assessed.

To those naysayers, I have to disagree. Critical thinking is a thing. We can define it; we can teach it; and we can assess it. In fact, as part of a multi-year Assessment for Learning Project , Two Rivers Public Charter School in Washington, D.C., has done just that.

Before I dive into what we have done, I want to acknowledge that some of the criticism has merit.

First, there are those that argue that critical thinking can only exist when students have a vast fund of knowledge. Meaning that a student cannot think critically if they don’t have something substantive about which to think. I agree. Students do need a robust foundation of core content knowledge to effectively think critically. Schools still have a responsibility for building students’ content knowledge.

However, I would argue that students don’t need to wait to think critically until after they have mastered some arbitrary amount of knowledge. They can start building critical thinking skills when they walk in the door. All students come to school with experience and knowledge which they can immediately think critically about. In fact, some of the thinking that they learn to do helps augment and solidify the discipline-specific academic knowledge that they are learning.

The second criticism is that critical thinking skills are always highly contextual. In this argument, the critics make the point that the types of thinking that students do in history is categorically different from the types of thinking students do in science or math. Thus, the idea of teaching broadly defined, content-neutral critical thinking skills is impossible. I agree that there are domain-specific thinking skills that students should learn in each discipline. However, I also believe that there are several generalizable skills that elementary school students can learn that have broad applicability to their academic and social lives. That is what we have done at Two Rivers.

Defining Critical Thinking Skills

We began this work by first defining what we mean by critical thinking. After a review of the literature and looking at the practice at other schools, we identified five constructs that encompass a set of broadly applicable skills: schema development and activation; effective reasoning; creativity and innovation; problem solving; and decision making.

We then created rubrics to provide a concrete vision of what each of these constructs look like in practice. Working with the Stanford Center for Assessment, Learning and Equity (SCALE) , we refined these rubrics to capture clear and discrete skills.

For example, we defined effective reasoning as the skill of creating an evidence-based claim: students need to construct a claim, identify relevant support, link their support to their claim, and identify possible questions or counter claims. Rubrics provide an explicit vision of the skill of effective reasoning for students and teachers. By breaking the rubrics down for different grade bands, we have been able not only to describe what reasoning is but also to delineate how the skills develop in students from preschool through 8th grade.

Before moving on, I want to freely acknowledge that in narrowly defining reasoning as the construction of evidence-based claims we have disregarded some elements of reasoning that students can and should learn. For example, the difference between constructing claims through deductive versus inductive means is not highlighted in our definition. However, by privileging a definition that has broad applicability across disciplines, we are able to gain traction in developing the roots of critical thinking. In this case, to formulate well-supported claims or arguments.

Teaching Critical Thinking Skills

The definitions of critical thinking constructs were only useful to us in as much as they translated into practical skills that teachers could teach and students could learn and use. Consequently, we have found that to teach a set of cognitive skills, we needed thinking routines that defined the regular application of these critical thinking and problem-solving skills across domains. Building on Harvard’s Project Zero Visible Thinking work, we have named routines aligned with each of our constructs.

For example, with the construct of effective reasoning, we aligned the Claim-Support-Question thinking routine to our rubric. Teachers then were able to teach students that whenever they were making an argument, the norm in the class was to use the routine in constructing their claim and support. The flexibility of the routine has allowed us to apply it from preschool through 8th grade and across disciplines from science to economics and from math to literacy.

Kathryn Mancino, a 5th grade teacher at Two Rivers, has deliberately taught three of our thinking routines to students using the anchor charts above. Her charts name the components of each routine and has a place for students to record when they’ve used it and what they have figured out about the routine. By using this structure with a chart that can be added to throughout the year, students see the routines as broadly applicable across disciplines and are able to refine their application over time.

Assessing Critical Thinking Skills

By defining specific constructs of critical thinking and building thinking routines that support their implementation in classrooms, we have operated under the assumption that students are developing skills that they will be able to transfer to other settings. However, we recognized both the importance and the challenge of gathering reliable data to confirm this.

With this in mind, we have developed a series of short performance tasks around novel discipline-neutral contexts in which students can apply the constructs of thinking. Through these tasks, we have been able to provide an opportunity for students to demonstrate their ability to transfer the types of thinking beyond the original classroom setting. Once again, we have worked with SCALE to define tasks where students easily access the content but where the cognitive lift requires them to demonstrate their thinking abilities.

These assessments demonstrate that it is possible to capture meaningful data on students’ critical thinking abilities. They are not intended to be high stakes accountability measures. Instead, they are designed to give students, teachers, and school leaders discrete formative data on hard to measure skills.

While it is clearly difficult, and we have not solved all of the challenges to scaling assessments of critical thinking, we can define, teach, and assess these skills . In fact, knowing how important they are for the economy of the future and our democracy, it is essential that we do.

Jeff Heyck-Williams (He, His, Him)

Director of the two rivers learning institute.

Jeff Heyck-Williams is the director of the Two Rivers Learning Institute and a founder of Two Rivers Public Charter School. He has led work around creating school-wide cultures of mathematics, developing assessments of critical thinking and problem-solving, and supporting project-based learning.

Read More About Why Schools Need to Change

Nurturing STEM Identity and Belonging: The Role of Equitable Program Implementation in Project Invent

Alexis Lopez (she/her)

May 9, 2024

Bring Your Vision for Student Success to Life with NGLC and Bravely

March 13, 2024

For Ethical AI, Listen to Teachers

Jason Wilmot

October 23, 2023

How it works

For Business

Join Mind Tools

Article • 8 min read

Critical Thinking

Developing the right mindset and skills.

By the Mind Tools Content Team

We make hundreds of decisions every day and, whether we realize it or not, we're all critical thinkers.

We use critical thinking each time we weigh up our options, prioritize our responsibilities, or think about the likely effects of our actions. It's a crucial skill that helps us to cut out misinformation and make wise decisions. The trouble is, we're not always very good at it!

In this article, we'll explore the key skills that you need to develop your critical thinking skills, and how to adopt a critical thinking mindset, so that you can make well-informed decisions.

What Is Critical Thinking?

Critical thinking is the discipline of rigorously and skillfully using information, experience, observation, and reasoning to guide your decisions, actions, and beliefs. You'll need to actively question every step of your thinking process to do it well.

Collecting, analyzing and evaluating information is an important skill in life, and a highly valued asset in the workplace. People who score highly in critical thinking assessments are also rated by their managers as having good problem-solving skills, creativity, strong decision-making skills, and good overall performance. [1]

Key Critical Thinking Skills

Critical thinkers possess a set of key characteristics which help them to question information and their own thinking. Focus on the following areas to develop your critical thinking skills:

Being willing and able to explore alternative approaches and experimental ideas is crucial. Can you think through "what if" scenarios, create plausible options, and test out your theories? If not, you'll tend to write off ideas and options too soon, so you may miss the best answer to your situation.

To nurture your curiosity, stay up to date with facts and trends. You'll overlook important information if you allow yourself to become "blinkered," so always be open to new information.

But don't stop there! Look for opposing views or evidence to challenge your information, and seek clarification when things are unclear. This will help you to reassess your beliefs and make a well-informed decision later. Read our article, Opening Closed Minds , for more ways to stay receptive.

Logical Thinking

You must be skilled at reasoning and extending logic to come up with plausible options or outcomes.

It's also important to emphasize logic over emotion. Emotion can be motivating but it can also lead you to take hasty and unwise action, so control your emotions and be cautious in your judgments. Know when a conclusion is "fact" and when it is not. "Could-be-true" conclusions are based on assumptions and must be tested further. Read our article, Logical Fallacies , for help with this.

Use creative problem solving to balance cold logic. By thinking outside of the box you can identify new possible outcomes by using pieces of information that you already have.

Self-Awareness

Many of the decisions we make in life are subtly informed by our values and beliefs. These influences are called cognitive biases and it can be difficult to identify them in ourselves because they're often subconscious.

Practicing self-awareness will allow you to reflect on the beliefs you have and the choices you make. You'll then be better equipped to challenge your own thinking and make improved, unbiased decisions.

One particularly useful tool for critical thinking is the Ladder of Inference . It allows you to test and validate your thinking process, rather than jumping to poorly supported conclusions.

Developing a Critical Thinking Mindset

Combine the above skills with the right mindset so that you can make better decisions and adopt more effective courses of action. You can develop your critical thinking mindset by following this process:

Gather Information

First, collect data, opinions and facts on the issue that you need to solve. Draw on what you already know, and turn to new sources of information to help inform your understanding. Consider what gaps there are in your knowledge and seek to fill them. And look for information that challenges your assumptions and beliefs.

Be sure to verify the authority and authenticity of your sources. Not everything you read is true! Use this checklist to ensure that your information is valid:

• Are your information sources trustworthy ? (For example, well-respected authors, trusted colleagues or peers, recognized industry publications, websites, blogs, etc.)
• Is the information you have gathered up to date ?
• Has the information received any direct criticism ?
• Does the information have any errors or inaccuracies ?
• Is there any evidence to support or corroborate the information you have gathered?
• Is the information you have gathered subjective or biased in any way? (For example, is it based on opinion, rather than fact? Is any of the information you have gathered designed to promote a particular service or organization?)

If any information appears to be irrelevant or invalid, don't include it in your decision making. But don't omit information just because you disagree with it, or your final decision will be flawed and bias.

Now observe the information you have gathered, and interpret it. What are the key findings and main takeaways? What does the evidence point to? Start to build one or two possible arguments based on what you have found.

You'll need to look for the details within the mass of information, so use your powers of observation to identify any patterns or similarities. You can then analyze and extend these trends to make sensible predictions about the future.

To help you to sift through the multiple ideas and theories, it can be useful to group and order items according to their characteristics. From here, you can compare and contrast the different items. And once you've determined how similar or different things are from one another, Paired Comparison Analysis can help you to analyze them.

The final step involves challenging the information and rationalizing its arguments.

Apply the laws of reason (induction, deduction, analogy) to judge an argument and determine its merits. To do this, it's essential that you can determine the significance and validity of an argument to put it in the correct perspective. Take a look at our article, Rational Thinking , for more information about how to do this.

Once you have considered all of the arguments and options rationally, you can finally make an informed decision.

Afterward, take time to reflect on what you have learned and what you found challenging. Step back from the detail of your decision or problem, and look at the bigger picture. Record what you've learned from your observations and experience.

Critical thinking involves rigorously and skilfully using information, experience, observation, and reasoning to guide your decisions, actions and beliefs. It's a useful skill in the workplace and in life.

You'll need to be curious and creative to explore alternative possibilities, but rational to apply logic, and self-aware to identify when your beliefs could affect your decisions or actions.

You can demonstrate a high level of critical thinking by validating your information, analyzing its meaning, and finally evaluating the argument.

Critical Thinking Infographic

See Critical Thinking represented in our infographic: An Elementary Guide to Critical Thinking .

You've accessed 1 of your 2 free resources.

Get unlimited access

Discover more content

Snyder's hope theory.

Cultivating Aspiration in Your Life

Mindfulness in the Workplace

Focusing the Mind by Staying Present

Add comment

Comments (1)

priyanka ghogare

Gain essential management and leadership skills

Busy schedule? No problem. Learn anytime, anywhere. 

Subscribe to unlimited access to meticulously researched, evidence-based resources.

Join today and save on an annual membership!

Sign-up to our newsletter

Subscribing to the Mind Tools newsletter will keep you up-to-date with our latest updates and newest resources.

Subscribe now

Business Skills

Personal Development

Leadership and Management

Member Extras

Most Popular

Latest Updates

Better Public Speaking

How to Build Confidence in Others

Mind Tools Store

About Mind Tools Content

Discover something new today

How to create psychological safety at work.

Speaking up without fear

How to Guides

Pain Points Podcast - Presentations Pt 1

How do you get better at presenting?

How Emotionally Intelligent Are You?

Boosting Your People Skills

Self-Assessment

What's Your Leadership Style?

Learn About the Strengths and Weaknesses of the Way You Like to Lead

Recommended for you

The science of a good night's sleep infographic.

Infographic Transcript

Infographic

Business Operations and Process Management

Strategy Tools

Customer Service

Business Ethics and Values

Handling Information and Data

Project Management

Knowledge Management

Self-Development and Goal Setting

Time Management

Presentation Skills

Learning Skills

Career Skills

Communication Skills

Negotiation, Persuasion and Influence

Working With Others

Difficult Conversations

Creativity Tools

Self-Management

Work-Life Balance

Stress Management and Wellbeing

Coaching and Mentoring

Change Management

Team Management

Managing Conflict

Delegation and Empowerment

Performance Management

Leadership Skills

Developing Your Team

Talent Management

Problem Solving

Decision Making

Member Podcast

• Top Courses
• Online Degrees
• Find your New Career
• Join for Free

What Are Critical Thinking Skills and Why Are They Important?

Learn what critical thinking skills are, why they’re important, and how to develop and apply them in your workplace and everyday life.

We often use critical thinking skills without even realizing it. When you make a decision, such as which cereal to eat for breakfast, you're using critical thinking to determine the best option for you that day.

Critical thinking is like a muscle that can be exercised and built over time. It is a skill that can help propel your career to new heights. You'll be able to solve workplace issues, use trial and error to troubleshoot ideas, and more.

We'll take you through what it is and some examples so you can begin your journey in mastering this skill.

What is critical thinking?

Critical thinking is the ability to interpret, evaluate, and analyze facts and information that are available, to form a judgment or decide if something is right or wrong.

More than just being curious about the world around you, critical thinkers make connections between logical ideas to see the bigger picture. Building your critical thinking skills means being able to advocate your ideas and opinions, present them in a logical fashion, and make decisions for improvement.

Build job-ready skills with a Coursera Plus subscription

• Get access to 7,000+ learning programs from world-class universities and companies, including Google, Yale, Salesforce, and more
• Try different courses and find your best fit at no additional cost
• Earn certificates for learning programs you complete
• A subscription price of $59/month, cancel anytime

Why is critical thinking important?

Critical thinking is useful in many areas of your life, including your career. It makes you a well-rounded individual, one who has looked at all of their options and possible solutions before making a choice.

According to the University of the People in California, having critical thinking skills is important because they are [ 1 ]:

Crucial for the economy

Essential for improving language and presentation skills

Very helpful in promoting creativity

Important for self-reflection

The basis of science and democracy 

Critical thinking skills are used every day in a myriad of ways and can be applied to situations such as a CEO approaching a group project or a nurse deciding in which order to treat their patients.

Examples of common critical thinking skills

Critical thinking skills differ from individual to individual and are utilized in various ways. Examples of common critical thinking skills include:

Identification of biases: Identifying biases means knowing there are certain people or things that may have an unfair prejudice or influence on the situation at hand. Pointing out these biases helps to remove them from contention when it comes to solving the problem and allows you to see things from a different perspective.

Research: Researching details and facts allows you to be prepared when presenting your information to people. You’ll know exactly what you’re talking about due to the time you’ve spent with the subject material, and you’ll be well-spoken and know what questions to ask to gain more knowledge. When researching, always use credible sources and factual information.

Open-mindedness: Being open-minded when having a conversation or participating in a group activity is crucial to success. Dismissing someone else’s ideas before you’ve heard them will inhibit you from progressing to a solution, and will often create animosity. If you truly want to solve a problem, you need to be willing to hear everyone’s opinions and ideas if you want them to hear yours.

Analysis: Analyzing your research will lead to you having a better understanding of the things you’ve heard and read. As a true critical thinker, you’ll want to seek out the truth and get to the source of issues. It’s important to avoid taking things at face value and always dig deeper.

Problem-solving: Problem-solving is perhaps the most important skill that critical thinkers can possess. The ability to solve issues and bounce back from conflict is what helps you succeed, be a leader, and effect change. One way to properly solve problems is to first recognize there’s a problem that needs solving. By determining the issue at hand, you can then analyze it and come up with several potential solutions.

How to develop critical thinking skills

You can develop critical thinking skills every day if you approach problems in a logical manner. Here are a few ways you can start your path to improvement:

1. Ask questions.

Be inquisitive about everything. Maintain a neutral perspective and develop a natural curiosity, so you can ask questions that develop your understanding of the situation or task at hand. The more details, facts, and information you have, the better informed you are to make decisions.

2. Practice active listening.

Utilize active listening techniques, which are founded in empathy, to really listen to what the other person is saying. Critical thinking, in part, is the cognitive process of reading the situation: the words coming out of their mouth, their body language, their reactions to your own words. Then, you might paraphrase to clarify what they're saying, so both of you agree you're on the same page.

3. Develop your logic and reasoning.

This is perhaps a more abstract task that requires practice and long-term development. However, think of a schoolteacher assessing the classroom to determine how to energize the lesson. There's options such as playing a game, watching a video, or challenging the students with a reward system. Using logic, you might decide that the reward system will take up too much time and is not an immediate fix. A video is not exactly relevant at this time. So, the teacher decides to play a simple word association game.

Scenarios like this happen every day, so next time, you can be more aware of what will work and what won't. Over time, developing your logic and reasoning will strengthen your critical thinking skills.

Learn tips and tricks on how to become a better critical thinker and problem solver through online courses from notable educational institutions on Coursera. Start with Introduction to Logic and Critical Thinking from Duke University or Mindware: Critical Thinking for the Information Age from the University of Michigan.

Article sources

University of the People, “ Why is Critical Thinking Important?: A Survival Guide , https://www.uopeople.edu/blog/why-is-critical-thinking-important/.” Accessed May 18, 2023.

Keep reading

Coursera staff.

Editorial Team

Coursera’s editorial team is comprised of highly experienced professional editors, writers, and fact...

This content has been made available for informational purposes only. Learners are advised to conduct additional research to ensure that courses and other credentials pursued meet their personal, professional, and financial goals.

• A Model for the National Assessment of Higher Order Thinking
• International Critical Thinking Essay Test
• Online Critical Thinking Basic Concepts Test
• Online Critical Thinking Basic Concepts Sample Test

Consequential Validity: Using Assessment to Drive Instruction

Translate this page from English...

*Machine translated pages not guaranteed for accuracy. Click Here for our professional translations.

Critical Thinking Testing and Assessment

The purpose of assessment in instruction is improvement. The purpose of assessing instruction for critical thinking is improving the teaching of discipline-based thinking (historical, biological, sociological, mathematical, etc.) It is to improve students’ abilities to think their way through content using disciplined skill in reasoning. The more particular we can be about what we want students to learn about critical thinking, the better we can devise instruction with that particular end in view.

The Foundation for Critical Thinking offers assessment instruments which share in the same general goal: to enable educators to gather evidence relevant to determining the extent to which instruction is teaching students to think critically (in the process of learning content). To this end, the Fellows of the Foundation recommend:

that academic institutions and units establish an oversight committee for critical thinking, and

that this oversight committee utilizes a combination of assessment instruments (the more the better) to generate incentives for faculty, by providing them with as much evidence as feasible of the actual state of instruction for critical thinking.

The following instruments are available to generate evidence relevant to critical thinking teaching and learning:

Course Evaluation Form : Provides evidence of whether, and to what extent, students perceive faculty as fostering critical thinking in instruction (course by course). Machine-scoreable.

Online Critical Thinking Basic Concepts Test : Provides evidence of whether, and to what extent, students understand the fundamental concepts embedded in critical thinking (and hence tests student readiness to think critically). Machine-scoreable.

Critical Thinking Reading and Writing Test : Provides evidence of whether, and to what extent, students can read closely and write substantively (and hence tests students' abilities to read and write critically). Short-answer.

International Critical Thinking Essay Test : Provides evidence of whether, and to what extent, students are able to analyze and assess excerpts from textbooks or professional writing. Short-answer.

Commission Study Protocol for Interviewing Faculty Regarding Critical Thinking : Provides evidence of whether, and to what extent, critical thinking is being taught at a college or university. Can be adapted for high school. Based on the California Commission Study . Short-answer.

Protocol for Interviewing Faculty Regarding Critical Thinking : Provides evidence of whether, and to what extent, critical thinking is being taught at a college or university. Can be adapted for high school. Short-answer.

Protocol for Interviewing Students Regarding Critical Thinking : Provides evidence of whether, and to what extent, students are learning to think critically at a college or university. Can be adapted for high school). Short-answer. 

Criteria for Critical Thinking Assignments : Can be used by faculty in designing classroom assignments, or by administrators in assessing the extent to which faculty are fostering critical thinking.

Rubrics for Assessing Student Reasoning Abilities : A useful tool in assessing the extent to which students are reasoning well through course content.  

All of the above assessment instruments can be used as part of pre- and post-assessment strategies to gauge development over various time periods.

Consequential Validity

All of the above assessment instruments, when used appropriately and graded accurately, should lead to a high degree of consequential validity. In other words, the use of the instruments should cause teachers to teach in such a way as to foster critical thinking in their various subjects. In this light, for students to perform well on the various instruments, teachers will need to design instruction so that students can perform well on them. Students cannot become skilled in critical thinking without learning (first) the concepts and principles that underlie critical thinking and (second) applying them in a variety of forms of thinking: historical thinking, sociological thinking, biological thinking, etc. Students cannot become skilled in analyzing and assessing reasoning without practicing it. However, when they have routine practice in paraphrasing, summariz­ing, analyzing, and assessing, they will develop skills of mind requisite to the art of thinking well within any subject or discipline, not to mention thinking well within the various domains of human life.

For full copies of this and many other critical thinking articles, books, videos, and more, join us at the Center for Critical Thinking Community Online - the world's leading online community dedicated to critical thinking!   Also featuring interactive learning activities, study groups, and even a social media component, this learning platform will change your conception of intellectual development.

• SUGGESTED TOPICS
• The Magazine
• Newsletters
• Managing Yourself
• Managing Teams
• Work-life Balance
• The Big Idea
• Data & Visuals
• Reading Lists
• Case Selections
• HBR Learning
• Topic Feeds
• Account Settings
• Email Preferences

A Short Guide to Building Your Team’s Critical Thinking Skills

• Matt Plummer

Critical thinking isn’t an innate skill. It can be learned.

Most employers lack an effective way to objectively assess critical thinking skills and most managers don’t know how to provide specific instruction to team members in need of becoming better thinkers. Instead, most managers employ a sink-or-swim approach, ultimately creating work-arounds to keep those who can’t figure out how to “swim” from making important decisions. But it doesn’t have to be this way. To demystify what critical thinking is and how it is developed, the author’s team turned to three research-backed models: The Halpern Critical Thinking Assessment, Pearson’s RED Critical Thinking Model, and Bloom’s Taxonomy. Using these models, they developed the Critical Thinking Roadmap, a framework that breaks critical thinking down into four measurable phases: the ability to execute, synthesize, recommend, and generate.

With critical thinking ranking among the most in-demand skills for job candidates , you would think that educational institutions would prepare candidates well to be exceptional thinkers, and employers would be adept at developing such skills in existing employees. Unfortunately, both are largely untrue.

• Matt Plummer (@mtplummer) is the founder of Zarvana, which offers online programs and coaching services to help working professionals become more productive by developing time-saving habits. Before starting Zarvana, Matt spent six years at Bain & Company spin-out, The Bridgespan Group, a strategy and management consulting firm for nonprofits, foundations, and philanthropists.  

Partner Center

• ADEA Connect

• Communities
• Career Opportunities
• New Thinking
• ADEA Governance
• House of Delegates
• Board of Directors
• Advisory Committees
• Sections and Special Interest Groups
• Governance Documents and Publications
• Dental Faculty Code of Conduct
• ADEAGies Foundation
• About ADEAGies Foundation
• ADEAGies Newsroom
• Gies Awards
• Press Center
• Strategic Directions
• 2023 Annual Report
• ADEA Membership
• Institutions
• Faculty and Staff
• Individuals
• Corporations
• ADEA Members
• Predoctoral Dental
• Allied Dental
• Nonfederal Advanced Dental
• U.S. Federal Dental
• Students, Residents and Fellows
• Corporate Members
• Member Directory
• Directory of Institutional Members (DIM)
• 5 Questions With
• ADEA Member to Member Recruitment
• Students, Residents, and Fellows
• Information For
• Deans & Program Directors
• Current Students & Residents
• Prospective Students
• Educational Meetings
• Upcoming Events
• 2025 Annual Session & Exhibition
• eLearn Webinars
• Past Events
• Professional Development
• eLearn Micro-credentials
• Leadership Institute
• Leadership Institute Alumni Association (LIAA)
• Faculty Development Programs
• ADEA Scholarships, Awards and Fellowships
• Academic Fellowship
• For Students
• For Dental Educators
• For Leadership Institute Fellows
• Teaching Resources
• ADEA weTeach®
• MedEdPORTAL

Critical Thinking Skills Toolbox

• Resources for Teaching
• Policy Topics
• Task Force Report
• Opioid Epidemic
• Financing Dental Education
• Holistic Review
• Sex-based Health Differences
• Access, Diversity and Inclusion
• ADEA Commission on Change and Innovation in Dental Education
• Tool Resources
• Campus Liaisons
• Policy Resources
• Policy Publications
• Holistic Review Workshops
• Leading Conversations Webinar Series
• Collaborations
• Summer Health Professions Education Program
• Minority Dental Faculty Development Program
• Federal Advocacy
• Dental School Legislators
• Policy Letters and Memos
• Legislative Process
• Federal Advocacy Toolkit
• State Information
• Opioid Abuse
• Tracking Map
• Loan Forgiveness Programs
• State Advocacy Toolkit
• Canadian Information
• Dental Schools
• Provincial Information
• ADEA Advocate
• Books and Guides
• About ADEA Publications
• 2023-24 Official Guide
• Dental School Explorer
• Dental Education Trends
• Ordering Publications
• ADEA Bookstore
• Newsletters
• About ADEA Newsletters
• Bulletin of Dental Education
• Charting Progress
• Subscribe to Newsletter
• Journal of Dental Education
• Subscriptions
• Submissions FAQs
• Data, Analysis and Research
• Educational Institutions
• Applicants, Enrollees and Graduates
• Dental School Seniors
• ADEA AADSAS® (Dental School)
• AADSAS Applicants
• Health Professions Advisors
• Admissions Officers
• ADEA CAAPID® (International Dentists)
• CAAPID Applicants
• Program Finder
• ADEA DHCAS® (Dental Hygiene Programs)
• DHCAS Applicants
• Program Directors
• ADEA PASS® (Advanced Dental Education Programs)
• PASS Applicants
• PASS Evaluators
• DentEd Jobs
• Information For:

• Introduction
• Overview of Critical Thinking Skills
• Teaching Observations
• Avenues for Research

CTS Tools for Faculty and Student Assessment

• Critical Thinking and Assessment
• Conclusions
• Bibliography
• Helpful Links
• Appendix A. Author's Impressions of Vignettes

A number of critical thinking skills inventories and measures have been developed:

     Watson-Glaser Critical Thinking Appraisal (WGCTA)      Cornell Critical Thinking Test      California Critical Thinking Disposition Inventory (CCTDI)      California Critical Thinking Skills Test (CCTST)      Health Science Reasoning Test (HSRT)      Professional Judgment Rating Form (PJRF)      Teaching for Thinking Student Course Evaluation Form      Holistic Critical Thinking Scoring Rubric      Peer Evaluation of Group Presentation Form

Excluding the Watson-Glaser Critical Thinking Appraisal and the Cornell Critical Thinking Test, Facione and Facione developed the critical thinking skills instruments listed above. However, it is important to point out that all of these measures are of questionable utility for dental educators because their content is general rather than dental education specific. (See Critical Thinking and Assessment .)

Table 7. Purposes of Critical Thinking Skills Instruments

  Reliability and Validity

Reliability means that individual scores from an instrument should be the same or nearly the same from one administration of the instrument to another. The instrument can be assumed to be free of bias and measurement error (68). Alpha coefficients are often used to report an estimate of internal consistency. Scores of .70 or higher indicate that the instrument has high reliability when the stakes are moderate. Scores of .80 and higher are appropriate when the stakes are high.

Validity means that individual scores from a particular instrument are meaningful, make sense, and allow researchers to draw conclusions from the sample to the population that is being studied (69) Researchers often refer to "content" or "face" validity. Content validity or face validity is the extent to which questions on an instrument are representative of the possible questions that a researcher could ask about that particular content or skills.

Watson-Glaser Critical Thinking Appraisal-FS (WGCTA-FS)

The WGCTA-FS is a 40-item inventory created to replace Forms A and B of the original test, which participants reported was too long.70 This inventory assesses test takers' skills in:

     (a) Inference: the extent to which the individual recognizes whether assumptions are clearly stated      (b) Recognition of assumptions: whether an individual recognizes whether assumptions are clearly stated      (c) Deduction: whether an individual decides if certain conclusions follow the information provided      (d) Interpretation: whether an individual considers evidence provided and determines whether generalizations from data are warranted      (e) Evaluation of arguments: whether an individual distinguishes strong and relevant arguments from weak and irrelevant arguments

Researchers investigated the reliability and validity of the WGCTA-FS for subjects in academic fields. Participants included 586 university students. Internal consistencies for the total WGCTA-FS among students majoring in psychology, educational psychology, and special education, including undergraduates and graduates, ranged from .74 to .92. The correlations between course grades and total WGCTA-FS scores for all groups ranged from .24 to .62 and were significant at the p < .05 of p < .01. In addition, internal consistency and test-retest reliability for the WGCTA-FS have been measured as .81. The WGCTA-FS was found to be a reliable and valid instrument for measuring critical thinking (71).

Cornell Critical Thinking Test (CCTT)

There are two forms of the CCTT, X and Z. Form X is for students in grades 4-14. Form Z is for advanced and gifted high school students, undergraduate and graduate students, and adults. Reliability estimates for Form Z range from .49 to .87 across the 42 groups who have been tested. Measures of validity were computed in standard conditions, roughly defined as conditions that do not adversely affect test performance. Correlations between Level Z and other measures of critical thinking are about .50.72 The CCTT is reportedly as predictive of graduate school grades as the Graduate Record Exam (GRE), a measure of aptitude, and the Miller Analogies Test, and tends to correlate between .2 and .4.73

California Critical Thinking Disposition Inventory (CCTDI)

Facione and Facione have reported significant relationships between the CCTDI and the CCTST. When faculty focus on critical thinking in planning curriculum development, modest cross-sectional and longitudinal gains have been demonstrated in students' CTS.74 The CCTDI consists of seven subscales and an overall score. The recommended cut-off score for each scale is 40, the suggested target score is 50, and the maximum score is 60. Scores below 40 on a specific scale are weak in that CT disposition, and scores above 50 on a scale are strong in that dispositional aspect. An overall score of 280 shows serious deficiency in disposition toward CT, while an overall score of 350 (while rare) shows across the board strength. The seven subscales are analyticity, self-confidence, inquisitiveness, maturity, open-mindedness, systematicity, and truth seeking (75).

In a study of instructional strategies and their influence on the development of critical thinking among undergraduate nursing students, Tiwari, Lai, and Yuen found that, compared with lecture students, PBL students showed significantly greater improvement in overall CCTDI (p = .0048), Truth seeking (p = .0008), Analyticity (p =.0368) and Critical Thinking Self-confidence (p =.0342) subscales from the first to the second time points; in overall CCTDI (p = .0083), Truth seeking (p= .0090), and Analyticity (p =.0354) subscales from the second to the third time points; and in Truth seeking (p = .0173) and Systematicity (p = .0440) subscales scores from the first to the fourth time points (76). California Critical Thinking Skills Test (CCTST)

Studies have shown the California Critical Thinking Skills Test captured gain scores in students' critical thinking over one quarter or one semester. Multiple health science programs have demonstrated significant gains in students' critical thinking using site-specific curriculum. Studies conducted to control for re-test bias showed no testing effect from pre- to post-test means using two independent groups of CT students. Since behavioral science measures can be impacted by social-desirability bias-the participant's desire to answer in ways that would please the researcher-researchers are urged to have participants take the Marlowe Crowne Social Desirability Scale simultaneously when measuring pre- and post-test changes in critical thinking skills. The CCTST is a 34-item instrument. This test has been correlated with the CCTDI with a sample of 1,557 nursing education students. Results show that, r = .201, and the relationship between the CCTST and the CCTDI is significant at p< .001. Significant relationships between CCTST and other measures including the GRE total, GRE-analytic, GRE-Verbal, GRE-Quantitative, the WGCTA, and the SAT Math and Verbal have also been reported. The two forms of the CCTST, A and B, are considered statistically significant. Depending on the testing, context KR-20 alphas range from .70 to .75. The newest version is CCTST Form 2000, and depending on the testing context, KR-20 alphas range from .78-.84.77

The Health Science Reasoning Test (HSRT)

Items within this inventory cover the domain of CT cognitive skills identified by a Delphi group of experts whose work resulted in the development of the CCTDI and CCTST. This test measures health science undergraduate and graduate students' CTS. Although test items are set in health sciences and clinical practice contexts, test takers are not required to have discipline-specific health sciences knowledge. For this reason, the test may have limited utility in dental education (78).

Preliminary estimates of internal consistency show that overall KR-20 coefficients range from .77 to .83.79 The instrument has moderate reliability on analysis and inference subscales, although the factor loadings appear adequate. The low K-20 coefficients may be result of small sample size, variance in item response, or both (see following table).

Table 8. Estimates of Internal Consistency and Factor Loading by Subscale for HSRT

Professional Judgment Rating Form (PJRF)

The scale consists of two sets of descriptors. The first set relates primarily to the attitudinal (habits of mind) dimension of CT. The second set relates primarily to CTS.

A single rater should know the student well enough to respond to at least 17 or the 20 descriptors with confidence. If not, the validity of the ratings may be questionable. If a single rater is used and ratings over time show some consistency, comparisons between ratings may be used to assess changes. If more than one rater is used, then inter-rater reliability must be established among the raters to yield meaningful results. While the PJRF can be used to assess the effectiveness of training programs for individuals or groups, the evaluation of participants' actual skills are best measured by an objective tool such as the California Critical Thinking Skills Test.

Teaching for Thinking Student Course Evaluation Form

Course evaluations typically ask for responses of "agree" or "disagree" to items focusing on teacher behavior. Typically the questions do not solicit information about student learning. Because contemporary thinking about curriculum is interested in student learning, this form was developed to address differences in pedagogy and subject matter, learning outcomes, student demographics, and course level characteristic of education today. This form also grew out of a "one size fits all" approach to teaching evaluations and a recognition of the limitations of this practice. It offers information about how a particular course enhances student knowledge, sensitivities, and dispositions. The form gives students an opportunity to provide feedback that can be used to improve instruction.

Holistic Critical Thinking Scoring Rubric

This assessment tool uses a four-point classification schema that lists particular opposing reasoning skills for select criteria. One advantage of a rubric is that it offers clearly delineated components and scales for evaluating outcomes. This rubric explains how students' CTS will be evaluated, and it provides a consistent framework for the professor as evaluator. Users can add or delete any of the statements to reflect their institution's effort to measure CT. Like most rubrics, this form is likely to have high face validity since the items tend to be relevant or descriptive of the target concept. This rubric can be used to rate student work or to assess learning outcomes. Experienced evaluators should engage in a process leading to consensus regarding what kinds of things should be classified and in what ways.80 If used improperly or by inexperienced evaluators, unreliable results may occur.

Peer Evaluation of Group Presentation Form

This form offers a common set of criteria to be used by peers and the instructor to evaluate student-led group presentations regarding concepts, analysis of arguments or positions, and conclusions.81 Users have an opportunity to rate the degree to which each component was demonstrated. Open-ended questions give users an opportunity to cite examples of how concepts, the analysis of arguments or positions, and conclusions were demonstrated.

Table 8. Proposed Universal Criteria for Evaluating Students' Critical Thinking Skills 

Aside from the use of the above-mentioned assessment tools, Dexter et al. recommended that all schools develop universal criteria for evaluating students' development of critical thinking skills (82).

Their rationale for the proposed criteria is that if faculty give feedback using these criteria, graduates will internalize these skills and use them to monitor their own thinking and practice (see Table 4).

• Application Information
• ADEA GoDental
• ADEA AADSAS
• ADEA CAAPID
• Events & Professional Development
• Scholarships, Awards & Fellowships
• Publications & Data
• Official Guide to Dental Schools
• Data, Analysis & Research
• Follow Us On:

• ADEA Privacy Policy
• Terms of Use
• Website Feedback
• Website Help

• Reference Manager
• Simple TEXT file

People also looked at

Original research article, performance assessment of critical thinking: conceptualization, design, and implementation.

• 1 Lynch School of Education and Human Development, Boston College, Chestnut Hill, MA, United States
• 2 Graduate School of Education, Stanford University, Stanford, CA, United States
• 3 Department of Business and Economics Education, Johannes Gutenberg University, Mainz, Germany

Enhancing students’ critical thinking (CT) skills is an essential goal of higher education. This article presents a systematic approach to conceptualizing and measuring CT. CT generally comprises the following mental processes: identifying, evaluating, and analyzing a problem; interpreting information; synthesizing evidence; and reporting a conclusion. We further posit that CT also involves dealing with dilemmas involving ambiguity or conflicts among principles and contradictory information. We argue that performance assessment provides the most realistic—and most credible—approach to measuring CT. From this conceptualization and construct definition, we describe one possible framework for building performance assessments of CT with attention to extended performance tasks within the assessment system. The framework is a product of an ongoing, collaborative effort, the International Performance Assessment of Learning (iPAL). The framework comprises four main aspects: (1) The storyline describes a carefully curated version of a complex, real-world situation. (2) The challenge frames the task to be accomplished (3). A portfolio of documents in a range of formats is drawn from multiple sources chosen to have specific characteristics. (4) The scoring rubric comprises a set of scales each linked to a facet of the construct. We discuss a number of use cases, as well as the challenges that arise with the use and valid interpretation of performance assessments. The final section presents elements of the iPAL research program that involve various refinements and extensions of the assessment framework, a number of empirical studies, along with linkages to current work in online reading and information processing.

Introduction

In their mission statements, most colleges declare that a principal goal is to develop students’ higher-order cognitive skills such as critical thinking (CT) and reasoning (e.g., Shavelson, 2010 ; Hyytinen et al., 2019 ). The importance of CT is echoed by business leaders ( Association of American Colleges and Universities [AACU], 2018 ), as well as by college faculty (for curricular analyses in Germany, see e.g., Zlatkin-Troitschanskaia et al., 2018 ). Indeed, in the 2019 administration of the Faculty Survey of Student Engagement (FSSE), 93% of faculty reported that they “very much” or “quite a bit” structure their courses to support student development with respect to thinking critically and analytically. In a listing of 21st century skills, CT was the most highly ranked among FSSE respondents ( Indiana University, 2019 ). Nevertheless, there is considerable evidence that many college students do not develop these skills to a satisfactory standard ( Arum and Roksa, 2011 ; Shavelson et al., 2019 ; Zlatkin-Troitschanskaia et al., 2019 ). This state of affairs represents a serious challenge to higher education – and to society at large.

In view of the importance of CT, as well as evidence of substantial variation in its development during college, its proper measurement is essential to tracking progress in skill development and to providing useful feedback to both teachers and learners. Feedback can help focus students’ attention on key skill areas in need of improvement, and provide insight to teachers on choices of pedagogical strategies and time allocation. Moreover, comparative studies at the program and institutional level can inform higher education leaders and policy makers.

The conceptualization and definition of CT presented here is closely related to models of information processing and online reasoning, the skills that are the focus of this special issue. These two skills are especially germane to the learning environments that college students experience today when much of their academic work is done online. Ideally, students should be capable of more than naïve Internet search, followed by copy-and-paste (e.g., McGrew et al., 2017 ); rather, for example, they should be able to critically evaluate both sources of evidence and the quality of the evidence itself in light of a given purpose ( Leu et al., 2020 ).

In this paper, we present a systematic approach to conceptualizing CT. From that conceptualization and construct definition, we present one possible framework for building performance assessments of CT with particular attention to extended performance tasks within the test environment. The penultimate section discusses some of the challenges that arise with the use and valid interpretation of performance assessment scores. We conclude the paper with a section on future perspectives in an emerging field of research – the iPAL program.

Conceptual Foundations, Definition and Measurement of Critical Thinking

In this section, we briefly review the concept of CT and its definition. In accordance with the principles of evidence-centered design (ECD; Mislevy et al., 2003 ), the conceptualization drives the measurement of the construct; that is, implementation of ECD directly links aspects of the assessment framework to specific facets of the construct. We then argue that performance assessments designed in accordance with such an assessment framework provide the most realistic—and most credible—approach to measuring CT. The section concludes with a sketch of an approach to CT measurement grounded in performance assessment .

Concept and Definition of Critical Thinking

Taxonomies of 21st century skills ( Pellegrino and Hilton, 2012 ) abound, and it is neither surprising that CT appears in most taxonomies of learning, nor that there are many different approaches to defining and operationalizing the construct of CT. There is, however, general agreement that CT is a multifaceted construct ( Liu et al., 2014 ). Liu et al. (2014) identified five key facets of CT: (i) evaluating evidence and the use of evidence; (ii) analyzing arguments; (iii) understanding implications and consequences; (iv) developing sound arguments; and (v) understanding causation and explanation.

There is empirical support for these facets from college faculty. A 2016–2017 survey conducted by the Higher Education Research Institute (HERI) at the University of California, Los Angeles found that a substantial majority of faculty respondents “frequently” encouraged students to: (i) evaluate the quality or reliability of the information they receive; (ii) recognize biases that affect their thinking; (iii) analyze multiple sources of information before coming to a conclusion; and (iv) support their opinions with a logical argument ( Stolzenberg et al., 2019 ).

There is general agreement that CT involves the following mental processes: identifying, evaluating, and analyzing a problem; interpreting information; synthesizing evidence; and reporting a conclusion (e.g., Erwin and Sebrell, 2003 ; Kosslyn and Nelson, 2017 ; Shavelson et al., 2018 ). We further suggest that CT includes dealing with dilemmas of ambiguity or conflict among principles and contradictory information ( Oser and Biedermann, 2020 ).

Importantly, Oser and Biedermann (2020) posit that CT can be manifested at three levels. The first level, Critical Analysis , is the most complex of the three levels. Critical Analysis requires both knowledge in a specific discipline (conceptual) and procedural analytical (deduction, inclusion, etc.) knowledge. The second level is Critical Reflection , which involves more generic skills “… necessary for every responsible member of a society” (p. 90). It is “a basic attitude that must be taken into consideration if (new) information is questioned to be true or false, reliable or not reliable, moral or immoral etc.” (p. 90). To engage in Critical Reflection, one needs not only apply analytic reasoning, but also adopt a reflective stance toward the political, social, and other consequences of choosing a course of action. It also involves analyzing the potential motives of various actors involved in the dilemma of interest. The third level, Critical Alertness , involves questioning one’s own or others’ thinking from a skeptical point of view.

Wheeler and Haertel (1993) categorized higher-order skills, such as CT, into two types: (i) when solving problems and making decisions in professional and everyday life, for instance, related to civic affairs and the environment; and (ii) in situations where various mental processes (e.g., comparing, evaluating, and justifying) are developed through formal instruction, usually in a discipline. Hence, in both settings, individuals must confront situations that typically involve a problematic event, contradictory information, and possibly conflicting principles. Indeed, there is an ongoing debate concerning whether CT should be evaluated using generic or discipline-based assessments ( Nagel et al., 2020 ). Whether CT skills are conceptualized as generic or discipline-specific has implications for how they are assessed and how they are incorporated into the classroom.

In the iPAL project, CT is characterized as a multifaceted construct that comprises conceptualizing, analyzing, drawing inferences or synthesizing information, evaluating claims, and applying the results of these reasoning processes to various purposes (e.g., solve a problem, decide on a course of action, find an answer to a given question or reach a conclusion) ( Shavelson et al., 2019 ). In the course of carrying out a CT task, an individual typically engages in activities such as specifying or clarifying a problem; deciding what information is relevant to the problem; evaluating the trustworthiness of information; avoiding judgmental errors based on “fast thinking”; avoiding biases and stereotypes; recognizing different perspectives and how they can reframe a situation; considering the consequences of alternative courses of actions; and communicating clearly and concisely decisions and actions. The order in which activities are carried out can vary among individuals and the processes can be non-linear and reciprocal.

In this article, we focus on generic CT skills. The importance of these skills derives not only from their utility in academic and professional settings, but also the many situations involving challenging moral and ethical issues – often framed in terms of conflicting principles and/or interests – to which individuals have to apply these skills ( Kegan, 1994 ; Tessier-Lavigne, 2020 ). Conflicts and dilemmas are ubiquitous in the contexts in which adults find themselves: work, family, civil society. Moreover, to remain viable in the global economic environment – one characterized by increased competition and advances in second generation artificial intelligence (AI) – today’s college students will need to continually develop and leverage their CT skills. Ideally, colleges offer a supportive environment in which students can develop and practice effective approaches to reasoning about and acting in learning, professional and everyday situations.

Measurement of Critical Thinking

Critical thinking is a multifaceted construct that poses many challenges to those who would develop relevant and valid assessments. For those interested in current approaches to the measurement of CT that are not the focus of this paper, consult Zlatkin-Troitschanskaia et al. (2018) .

In this paper, we have singled out performance assessment as it offers important advantages to measuring CT. Extant tests of CT typically employ response formats such as forced-choice or short-answer, and scenario-based tasks (for an overview, see Liu et al., 2014 ). They all suffer from moderate to severe construct underrepresentation; that is, they fail to capture important facets of the CT construct such as perspective taking and communication. High fidelity performance tasks are viewed as more authentic in that they provide a problem context and require responses that are more similar to what individuals confront in the real world than what is offered by traditional multiple-choice items ( Messick, 1994 ; Braun, 2019 ). This greater verisimilitude promises higher levels of construct representation and lower levels of construct-irrelevant variance. Such performance tasks have the capacity to measure facets of CT that are imperfectly assessed, if at all, using traditional assessments ( Lane and Stone, 2006 ; Braun, 2019 ; Shavelson et al., 2019 ). However, these assertions must be empirically validated, and the measures should be subjected to psychometric analyses. Evidence of the reliability, validity, and interpretative challenges of performance assessment (PA) are extensively detailed in Davey et al. (2015) .

We adopt the following definition of performance assessment:

A performance assessment (sometimes called a work sample when assessing job performance) … is an activity or set of activities that requires test takers, either individually or in groups, to generate products or performances in response to a complex, most often real-world task. These products and performances provide observable evidence bearing on test takers’ knowledge, skills, and abilities—their competencies—in completing the assessment ( Davey et al., 2015 , p. 10).

A performance assessment typically includes an extended performance task and short constructed-response and selected-response (i.e., multiple-choice) tasks (for examples, see Zlatkin-Troitschanskaia and Shavelson, 2019 ). In this paper, we refer to both individual performance- and constructed-response tasks as performance tasks (PT) (For an example, see Table 1 in section “iPAL Assessment Framework”).

Table 1. The iPAL assessment framework.

An Approach to Performance Assessment of Critical Thinking: The iPAL Program

The approach to CT presented here is the result of ongoing work undertaken by the International Performance Assessment of Learning collaborative (iPAL 1 ). iPAL is an international consortium of volunteers, primarily from academia, who have come together to address the dearth in higher education of research and practice in measuring CT with performance tasks ( Shavelson et al., 2018 ). In this section, we present iPAL’s assessment framework as the basis of measuring CT, with examples along the way.

iPAL Background

The iPAL assessment framework builds on the Council of Aid to Education’s Collegiate Learning Assessment (CLA). The CLA was designed to measure cross-disciplinary, generic competencies, such as CT, analytic reasoning, problem solving, and written communication ( Klein et al., 2007 ; Shavelson, 2010 ). Ideally, each PA contained an extended PT (e.g., examining a range of evidential materials related to the crash of an aircraft) and two short PT’s: one in which students either critique an argument or provide a solution in response to a real-world societal issue.

Motivated by considerations of adequate reliability, in 2012, the CLA was later modified to create the CLA+. The CLA+ includes two subtests: a PT and a 25-item Selected Response Question (SRQ) section. The PT presents a document or problem statement and an assignment based on that document which elicits an open-ended response. The CLA+ added the SRQ section (which is not linked substantively to the PT scenario) to increase the number of student responses to obtain more reliable estimates of performance at the student-level than could be achieved with a single PT ( Zahner, 2013 ; Davey et al., 2015 ).

iPAL Assessment Framework

Methodological foundations.

The iPAL framework evolved from the Collegiate Learning Assessment developed by Klein et al. (2007) . It was also informed by the results from the AHELO pilot study ( Organisation for Economic Co-operation and Development [OECD], 2012 , 2013 ), as well as the KoKoHs research program in Germany (for an overview see, Zlatkin-Troitschanskaia et al., 2017 , 2020 ). The ongoing refinement of the iPAL framework has been guided in part by the principles of Evidence Centered Design (ECD) ( Mislevy et al., 2003 ; Mislevy and Haertel, 2006 ; Haertel and Fujii, 2017 ).

In educational measurement, an assessment framework plays a critical intermediary role between the theoretical formulation of the construct and the development of the assessment instrument containing tasks (or items) intended to elicit evidence with respect to that construct ( Mislevy et al., 2003 ). Builders of the assessment framework draw on the construct theory and operationalize it in a way that provides explicit guidance to PT’s developers. Thus, the framework should reflect the relevant facets of the construct, where relevance is determined by substantive theory or an appropriate alternative such as behavioral samples from real-world situations of interest (criterion-sampling; McClelland, 1973 ), as well as the intended use(s) (for an example, see Shavelson et al., 2019 ). By following the requirements and guidelines embodied in the framework, instrument developers strengthen the claim of construct validity for the instrument ( Messick, 1994 ).

An assessment framework can be specified at different levels of granularity: an assessment battery (“omnibus” assessment, for an example see below), a single performance task, or a specific component of an assessment ( Shavelson, 2010 ; Davey et al., 2015 ). In the iPAL program, a performance assessment comprises one or more extended performance tasks and additional selected-response and short constructed-response items. The focus of the framework specified below is on a single PT intended to elicit evidence with respect to some facets of CT, such as the evaluation of the trustworthiness of the documents provided and the capacity to address conflicts of principles.

From the ECD perspective, an assessment is an instrument for generating information to support an evidentiary argument and, therefore, the intended inferences (claims) must guide each stage of the design process. The construct of interest is operationalized through the Student Model , which represents the target knowledge, skills, and abilities, as well as the relationships among them. The student model should also make explicit the assumptions regarding student competencies in foundational skills or content knowledge. The Task Model specifies the features of the problems or items posed to the respondent, with the goal of eliciting the evidence desired. The assessment framework also describes the collection of task models comprising the instrument, with considerations of construct validity, various psychometric characteristics (e.g., reliability) and practical constraints (e.g., testing time and cost). The student model provides grounds for evidence of validity, especially cognitive validity; namely, that the students are thinking critically in responding to the task(s).

In the present context, the target construct (CT) is the competence of individuals to think critically, which entails solving complex, real-world problems, and clearly communicating their conclusions or recommendations for action based on trustworthy, relevant and unbiased information. The situations, drawn from actual events, are challenging and may arise in many possible settings. In contrast to more reductionist approaches to assessment development, the iPAL approach and framework rests on the assumption that properly addressing these situational demands requires the application of a constellation of CT skills appropriate to the particular task presented (e.g., Shavelson, 2010 , 2013 ). For a PT, the assessment framework must also specify the rubric by which the responses will be evaluated. The rubric must be properly linked to the target construct so that the resulting score profile constitutes evidence that is both relevant and interpretable in terms of the student model (for an example, see Zlatkin-Troitschanskaia et al., 2019 ).

iPAL Task Framework

The iPAL ‘omnibus’ framework comprises four main aspects: A storyline , a challenge , a document library , and a scoring rubric . Table 1 displays these aspects, brief descriptions of each, and the corresponding examples drawn from an iPAL performance assessment (Version adapted from original in Hyytinen and Toom, 2019 ). Storylines are drawn from various domains; for example, the worlds of business, public policy, civics, medicine, and family. They often involve moral and/or ethical considerations. Deriving an appropriate storyline from a real-world situation requires careful consideration of which features are to be kept in toto , which adapted for purposes of the assessment, and which to be discarded. Framing the challenge demands care in wording so that there is minimal ambiguity in what is required of the respondent. The difficulty of the challenge depends, in large part, on the nature and extent of the information provided in the document library , the amount of scaffolding included, as well as the scope of the required response. The amount of information and the scope of the challenge should be commensurate with the amount of time available. As is evident from the table, the characteristics of the documents in the library are intended to elicit responses related to facets of CT. For example, with regard to bias, the information provided is intended to play to judgmental errors due to fast thinking and/or motivational reasoning. Ideally, the situation should accommodate multiple solutions of varying degrees of merit.

The dimensions of the scoring rubric are derived from the Task Model and Student Model ( Mislevy et al., 2003 ) and signal which features are to be extracted from the response and indicate how they are to be evaluated. There should be a direct link between the evaluation of the evidence and the claims that are made with respect to the key features of the task model and student model . More specifically, the task model specifies the various manipulations embodied in the PA and so informs scoring, while the student model specifies the capacities students employ in more or less effectively responding to the tasks. The score scales for each of the five facets of CT (see section “Concept and Definition of Critical Thinking”) can be specified using appropriate behavioral anchors (for examples, see Zlatkin-Troitschanskaia and Shavelson, 2019 ). Of particular importance is the evaluation of the response with respect to the last dimension of the scoring rubric; namely, the overall coherence and persuasiveness of the argument, building on the explicit or implicit characteristics related to the first five dimensions. The scoring process must be monitored carefully to ensure that (trained) raters are judging each response based on the same types of features and evaluation criteria ( Braun, 2019 ) as indicated by interrater agreement coefficients.

The scoring rubric of the iPAL omnibus framework can be modified for specific tasks ( Lane and Stone, 2006 ). This generic rubric helps ensure consistency across rubrics for different storylines. For example, Zlatkin-Troitschanskaia et al. (2019 , p. 473) used the following scoring scheme:

Based on our construct definition of CT and its four dimensions: (D1-Info) recognizing and evaluating information, (D2-Decision) recognizing and evaluating arguments and making decisions, (D3-Conseq) recognizing and evaluating the consequences of decisions, and (D4-Writing), we developed a corresponding analytic dimensional scoring … The students’ performance is evaluated along the four dimensions, which in turn are subdivided into a total of 23 indicators as (sub)categories of CT … For each dimension, we sought detailed evidence in students’ responses for the indicators and scored them on a six-point Likert-type scale. In order to reduce judgment distortions, an elaborate procedure of ‘behaviorally anchored rating scales’ (Smith and Kendall, 1963) was applied by assigning concrete behavioral expectations to certain scale points (Bernardin et al., 1976). To this end, we defined the scale levels by short descriptions of typical behavior and anchored them with concrete examples. … We trained four raters in 1 day using a specially developed training course to evaluate students’ performance along the 23 indicators clustered into four dimensions (for a description of the rater training, see Klotzer, 2018).

Shavelson et al. (2019) examined the interrater agreement of the scoring scheme developed by Zlatkin-Troitschanskaia et al. (2019) and “found that with 23 items and 2 raters the generalizability (“reliability”) coefficient for total scores to be 0.74 (with 4 raters, 0.84)” ( Shavelson et al., 2019 , p. 15). In the study by Zlatkin-Troitschanskaia et al. (2019 , p. 478) three score profiles were identified (low-, middle-, and high-performer) for students. Proper interpretation of such profiles requires care. For example, there may be multiple possible explanations for low scores such as poor CT skills, a lack of a disposition to engage with the challenge, or the two attributes jointly. These alternative explanations for student performance can potentially pose a threat to the evidentiary argument. In this case, auxiliary information may be available to aid in resolving the ambiguity. For example, student responses to selected- and short-constructed-response items in the PA can provide relevant information about the levels of the different skills possessed by the student. When sufficient data are available, the scores can be modeled statistically and/or qualitatively in such a way as to bring them to bear on the technical quality or interpretability of the claims of the assessment: reliability, validity, and utility evidence ( Davey et al., 2015 ; Zlatkin-Troitschanskaia et al., 2019 ). These kinds of concerns are less critical when PT’s are used in classroom settings. The instructor can draw on other sources of evidence, including direct discussion with the student.

Use of iPAL Performance Assessments in Educational Practice: Evidence From Preliminary Validation Studies

The assessment framework described here supports the development of a PT in a general setting. Many modifications are possible and, indeed, desirable. If the PT is to be more deeply embedded in a certain discipline (e.g., economics, law, or medicine), for example, then the framework must specify characteristics of the narrative and the complementary documents as to the breadth and depth of disciplinary knowledge that is represented.

At present, preliminary field trials employing the omnibus framework (i.e., a full set of documents) indicated that 60 min was generally an inadequate amount of time for students to engage with the full set of complementary documents and to craft a complete response to the challenge (for an example, see Shavelson et al., 2019 ). Accordingly, it would be helpful to develop modified frameworks for PT’s that require substantially less time. For an example, see a short performance assessment of civic online reasoning, requiring response times from 10 to 50 min ( Wineburg et al., 2016 ). Such assessment frameworks could be derived from the omnibus framework by focusing on a reduced number of facets of CT, and specifying the characteristics of the complementary documents to be included – or, perhaps, choices among sets of documents. In principle, one could build a ‘family’ of PT’s, each using the same (or nearly the same) storyline and a subset of the full collection of complementary documents.

Paul and Elder (2007) argue that the goal of CT assessments should be to provide faculty with important information about how well their instruction supports the development of students’ CT. In that spirit, the full family of PT’s could represent all facets of the construct while affording instructors and students more specific insights on strengths and weaknesses with respect to particular facets of CT. Moreover, the framework should be expanded to include the design of a set of short answer and/or multiple choice items to accompany the PT. Ideally, these additional items would be based on the same narrative as the PT to collect more nuanced information on students’ precursor skills such as reading comprehension, while enhancing the overall reliability of the assessment. Areas where students are under-prepared could be addressed before, or even in parallel with the development of the focal CT skills. The parallel approach follows the co-requisite model of developmental education. In other settings (e.g., for summative assessment), these complementary items would be administered after the PT to augment the evidence in relation to the various claims. The full PT taking 90 min or more could serve as a capstone assessment.

As we transition from simply delivering paper-based assessments by computer to taking full advantage of the affordances of a digital platform, we should learn from the hard-won lessons of the past so that we can make swifter progress with fewer missteps. In that regard, we must take validity as the touchstone – assessment design, development and deployment must all be tightly linked to the operational definition of the CT construct. Considerations of reliability and practicality come into play with various use cases that highlight different purposes for the assessment (for future perspectives, see next section).

The iPAL assessment framework represents a feasible compromise between commercial, standardized assessments of CT (e.g., Liu et al., 2014 ), on the one hand, and, on the other, freedom for individual faculty to develop assessment tasks according to idiosyncratic models. It imposes a degree of standardization on both task development and scoring, while still allowing some flexibility for faculty to tailor the assessment to meet their unique needs. In so doing, it addresses a key weakness of the AAC&U’s VALUE initiative 2 (retrieved 5/7/2020) that has achieved wide acceptance among United States colleges.

The VALUE initiative has produced generic scoring rubrics for 15 domains including CT, problem-solving and written communication. A rubric for a particular skill domain (e.g., critical thinking) has five to six dimensions with four ordered performance levels for each dimension (1 = lowest, 4 = highest). The performance levels are accompanied by language that is intended to clearly differentiate among levels. 3 Faculty are asked to submit student work products from a senior level course that is intended to yield evidence with respect to student learning outcomes in a particular domain and that, they believe, can elicit performances at the highest level. The collection of work products is then graded by faculty from other institutions who have been trained to apply the rubrics.

A principal difficulty is that there is neither a common framework to guide the design of the challenge, nor any control on task complexity and difficulty. Consequently, there is substantial heterogeneity in the quality and evidential value of the submitted responses. This also causes difficulties with task scoring and inter-rater reliability. Shavelson et al. (2009) discuss some of the problems arising with non-standardized collections of student work.

In this context, one advantage of the iPAL framework is that it can provide valuable guidance and an explicit structure for faculty in developing performance tasks for both instruction and formative assessment. When faculty design assessments, their focus is typically on content coverage rather than other potentially important characteristics, such as the degree of construct representation and the adequacy of their scoring procedures ( Braun, 2019 ).

Concluding Reflections

Challenges to interpretation and implementation.

Performance tasks such as those generated by iPAL are attractive instruments for assessing CT skills (e.g., Shavelson, 2010 ; Shavelson et al., 2019 ). The attraction mainly rests on the assumption that elaborated PT’s are more authentic (direct) and more completely capture facets of the target construct (i.e., possess greater construct representation) than the widely used selected-response tests. However, as Messick (1994) noted authenticity is a “promissory note” that must be redeemed with empirical research. In practice, there are trade-offs among authenticity, construct validity, and psychometric quality such as reliability ( Davey et al., 2015 ).

One reason for Messick (1994) caution is that authenticity does not guarantee construct validity. The latter must be established by drawing on multiple sources of evidence ( American Educational Research Association et al., 2014 ). Following the ECD principles in designing and developing the PT, as well as the associated scoring rubrics, constitutes an important type of evidence. Further, as Leighton (2019) argues, response process data (“cognitive validity”) is needed to validate claims regarding the cognitive complexity of PT’s. Relevant data can be obtained through cognitive laboratory studies involving methods such as think aloud protocols or eye-tracking. Although time-consuming and expensive, such studies can yield not only evidence of validity, but also valuable information to guide refinements of the PT.

Going forward, iPAL PT’s must be subjected to validation studies as recommended in the Standards for Psychological and Educational Testing by American Educational Research Association et al. (2014) . With a particular focus on the criterion “relationships to other variables,” a framework should include assumptions about the theoretically expected relationships among the indicators assessed by the PT, as well as the indicators’ relationships to external variables such as intelligence or prior (task-relevant) knowledge.

Complementing the necessity of evaluating construct validity, there is the need to consider potential sources of construct-irrelevant variance (CIV). One pertains to student motivation, which is typically greater when the stakes are higher. If students are not motivated, then their performance is likely to be impacted by factors unrelated to their (construct-relevant) ability ( Lane and Stone, 2006 ; Braun et al., 2011 ; Shavelson, 2013 ). Differential motivation across groups can also bias comparisons. Student motivation might be enhanced if the PT is administered in the context of a course with the promise of generating useful feedback on students’ skill profiles.

Construct-irrelevant variance can also occur when students are not equally prepared for the format of the PT or fully appreciate the response requirements. This source of CIV could be alleviated by providing students with practice PT’s. Finally, the use of novel forms of documentation, such as those from the Internet, can potentially introduce CIV due to differential familiarity with forms of representation or contents. Interestingly, this suggests that there may be a conflict between enhancing construct representation and reducing CIV.

Another potential source of CIV is related to response evaluation. Even with training, human raters can vary in accuracy and usage of the full score range. In addition, raters may attend to features of responses that are unrelated to the target construct, such as the length of the students’ responses or the frequency of grammatical errors ( Lane and Stone, 2006 ). Some of these sources of variance could be addressed in an online environment, where word processing software could alert students to potential grammatical and spelling errors before they submit their final work product.

Performance tasks generally take longer to administer and are more costly than traditional assessments, making it more difficult to reliably measure student performance ( Messick, 1994 ; Davey et al., 2015 ). Indeed, it is well known that more than one performance task is needed to obtain high reliability ( Shavelson, 2013 ). This is due to both student-task interactions and variability in scoring. Sources of student-task interactions are differential familiarity with the topic ( Hyytinen and Toom, 2019 ) and differential motivation to engage with the task. The level of reliability required, however, depends on the context of use. For use in formative assessment as part of an instructional program, reliability can be lower than use for summative purposes. In the former case, other types of evidence are generally available to support interpretation and guide pedagogical decisions. Further studies are needed to obtain estimates of reliability in typical instructional settings.

With sufficient data, more sophisticated psychometric analyses become possible. One challenge is that the assumption of unidimensionality required for many psychometric models might be untenable for performance tasks ( Davey et al., 2015 ). Davey et al. (2015) provide the example of a mathematics assessment that requires students to demonstrate not only their mathematics skills but also their written communication skills. Although the iPAL framework does not explicitly address students’ reading comprehension and organization skills, students will likely need to call on these abilities to accomplish the task. Moreover, as the operational definition of CT makes evident, the student must not only deploy several skills in responding to the challenge of the PT, but also carry out component tasks in sequence. The former requirement strongly indicates the need for a multi-dimensional IRT model, while the latter suggests that the usual assumption of local item independence may well be problematic ( Lane and Stone, 2006 ). At the same time, the analytic scoring rubric should facilitate the use of latent class analysis to partition data from large groups into meaningful categories ( Zlatkin-Troitschanskaia et al., 2019 ).

Future Perspectives

Although the iPAL consortium has made substantial progress in the assessment of CT, much remains to be done. Further refinement of existing PT’s and their adaptation to different languages and cultures must continue. To this point, there are a number of examples: The refugee crisis PT (cited in Table 1 ) was translated and adapted from Finnish to US English and then to Colombian Spanish. A PT concerning kidney transplants was translated and adapted from German to US English. Finally, two PT’s based on ‘legacy admissions’ to US colleges were translated and adapted to Colombian Spanish.

With respect to data collection, there is a need for sufficient data to support psychometric analysis of student responses, especially the relationships among the different components of the scoring rubric, as this would inform both task development and response evaluation ( Zlatkin-Troitschanskaia et al., 2019 ). In addition, more intensive study of response processes through cognitive laboratories and the like are needed to strengthen the evidential argument for construct validity ( Leighton, 2019 ). We are currently conducting empirical studies, collecting data on both iPAL PT’s and other measures of CT. These studies will provide evidence of convergent and discriminant validity.

At the same time, efforts should be directed at further development to support different ways CT PT’s might be used—i.e., use cases—especially those that call for formative use of PT’s. Incorporating formative assessment into courses can plausibly be expected to improve students’ competency acquisition ( Zlatkin-Troitschanskaia et al., 2017 ). With suitable choices of storylines, appropriate combinations of (modified) PT’s, supplemented by short-answer and multiple-choice items, could be interwoven into ordinary classroom activities. The supplementary items may be completely separate from the PT’s (as is the case with the CLA+), loosely coupled with the PT’s (as in drawing on the same storyline), or tightly linked to the PT’s (as in requiring elaboration of certain components of the response to the PT).

As an alternative to such integration, stand-alone modules could be embedded in courses to yield evidence of students’ generic CT skills. Core curriculum courses or general education courses offer ideal settings for embedding performance assessments. If these assessments were administered to a representative sample of students in each cohort over their years in college, the results would yield important information on the development of CT skills at a population level. For another example, these PA’s could be used to assess the competence profiles of students entering Bachelor’s or graduate-level programs as a basis for more targeted instructional support.

Thus, in considering different use cases for the assessment of CT, it is evident that several modifications of the iPAL omnibus assessment framework are needed. As noted earlier, assessments built according to this framework are demanding with respect to the extensive preliminary work required by a task and the time required to properly complete it. Thus, it would be helpful to have modified versions of the framework, focusing on one or two facets of the CT construct and calling for a smaller number of supplementary documents. The challenge to the student should be suitably reduced.

Some members of the iPAL collaborative have developed PT’s that are embedded in disciplines such as engineering, law and education ( Crump et al., 2019 ; for teacher education examples, see Jeschke et al., 2019 ). These are proving to be of great interest to various stakeholders and further development is likely. Consequently, it is essential that an appropriate assessment framework be established and implemented. It is both a conceptual and an empirical question as to whether a single framework can guide development in different domains.

Performance Assessment in Online Learning Environment

Over the last 15 years, increasing amounts of time in both college and work are spent using computers and other electronic devices. This has led to formulation of models for the new literacies that attempt to capture some key characteristics of these activities. A prominent example is a model proposed by Leu et al. (2020) . The model frames online reading as a process of problem-based inquiry that calls on five practices to occur during online research and comprehension:

1. Reading to identify important questions,

2. Reading to locate information,

3. Reading to critically evaluate information,

4. Reading to synthesize online information, and

5. Reading and writing to communicate online information.

The parallels with the iPAL definition of CT are evident and suggest there may be benefits to closer links between these two lines of research. For example, a report by Leu et al. (2014) describes empirical studies comparing assessments of online reading using either open-ended or multiple-choice response formats.

The iPAL consortium has begun to take advantage of the affordances of the online environment (for examples, see Schmidt et al. and Nagel et al. in this special issue). Most obviously, Supplementary Materials can now include archival photographs, audio recordings, or videos. Additional tasks might include the online search for relevant documents, though this would add considerably to the time demands. This online search could occur within a simulated Internet environment, as is the case for the IEA’s ePIRLS assessment ( Mullis et al., 2017 ).

The prospect of having access to a wealth of materials that can add to task authenticity is exciting. Yet it can also add ambiguity and information overload. Increased authenticity, then, should be weighed against validity concerns and the time required to absorb the content in these materials. Modifications of the design framework and extensive empirical testing will be required to decide on appropriate trade-offs. A related possibility is to employ some of these materials in short-answer (or even selected-response) items that supplement the main PT. Response formats could include highlighting text or using a drag-and-drop menu to construct a response. Students’ responses could be automatically scored, thereby containing costs. With automated scoring, feedback to students and faculty, including suggestions for next steps in strengthening CT skills, could also be provided without adding to faculty workload. Therefore, taking advantage of the online environment to incorporate new types of supplementary documents should be a high priority and, perhaps, to introduce new response formats as well. Finally, further investigation of the overlap between this formulation of CT and the characterization of online reading promulgated by Leu et al. (2020) is a promising direction to pursue.

Data Availability Statement

All datasets generated for this study are included in the article/supplementary material.

Author Contributions

HB wrote the article. RS, OZ-T, and KB were involved in the preparation and revision of the article and co-wrote the manuscript. All authors contributed to the article and approved the submitted version.

This study was funded in part by the Spencer Foundation (Grant No. #201700123).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We would like to thank all the researchers who have participated in the iPAL program.

• ^ https://www.ipal-rd.com/
• ^ https://www.aacu.org/value
• ^ When test results are reported by means of substantively defined categories, the scoring is termed “criterion-referenced”. This is, in contrast to results, reported as percentiles; such scoring is termed “norm-referenced”.

American Educational Research Association, American Psychological Association, and National Council on Measurement in Education (2014). Standards for Educational and Psychological Testing. Washington, D.C: American Educational Research Association.

Google Scholar

Arum, R., and Roksa, J. (2011). Academically Adrift: Limited Learning on College Campuses. Chicago, IL: University of Chicago Press.

Association of American Colleges and Universities (n.d.). VALUE: What is value?. Available online at:: https://www.aacu.org/value (accessed May 7, 2020).

Association of American Colleges and Universities [AACU] (2018). Fulfilling the American Dream: Liberal Education and the Future of Work. Available online at:: https://www.aacu.org/research/2018-future-of-work (accessed May 1, 2020).

Braun, H. (2019). Performance assessment and standardization in higher education: a problematic conjunction? Br. J. Educ. Psychol. 89, 429–440. doi: 10.1111/bjep.12274

PubMed Abstract | CrossRef Full Text | Google Scholar

Braun, H. I., Kirsch, I., and Yamoto, K. (2011). An experimental study of the effects of monetary incentives on performance on the 12th grade NAEP reading assessment. Teach. Coll. Rec. 113, 2309–2344.

Crump, N., Sepulveda, C., Fajardo, A., and Aguilera, A. (2019). Systematization of performance tests in critical thinking: an interdisciplinary construction experience. Rev. Estud. Educ. 2, 17–47.

Davey, T., Ferrara, S., Shavelson, R., Holland, P., Webb, N., and Wise, L. (2015). Psychometric Considerations for the Next Generation of Performance Assessment. Washington, DC: Center for K-12 Assessment & Performance Management, Educational Testing Service.

Erwin, T. D., and Sebrell, K. W. (2003). Assessment of critical thinking: ETS’s tasks in critical thinking. J. Gen. Educ. 52, 50–70. doi: 10.1353/jge.2003.0019

CrossRef Full Text | Google Scholar

Haertel, G. D., and Fujii, R. (2017). “Evidence-centered design and postsecondary assessment,” in Handbook on Measurement, Assessment, and Evaluation in Higher Education , 2nd Edn, eds C. Secolsky and D. B. Denison (Abingdon: Routledge), 313–339. doi: 10.4324/9781315709307-26

Hyytinen, H., and Toom, A. (2019). Developing a performance assessment task in the Finnish higher education context: conceptual and empirical insights. Br. J. Educ. Psychol. 89, 551–563. doi: 10.1111/bjep.12283

Hyytinen, H., Toom, A., and Shavelson, R. J. (2019). “Enhancing scientific thinking through the development of critical thinking in higher education,” in Redefining Scientific Thinking for Higher Education: Higher-Order Thinking, Evidence-Based Reasoning and Research Skills , eds M. Murtonen and K. Balloo (London: Palgrave MacMillan).

Indiana University (2019). FSSE 2019 Frequencies: FSSE 2019 Aggregate. Available online at:: http://fsse.indiana.edu/pdf/FSSE_IR_2019/summary_tables/FSSE19_Frequencies_(FSSE_2019).pdf (accessed May 1, 2020).

Jeschke, C., Kuhn, C., Lindmeier, A., Zlatkin-Troitschanskaia, O., Saas, H., and Heinze, A. (2019). Performance assessment to investigate the domain specificity of instructional skills among pre-service and in-service teachers of mathematics and economics. Br. J. Educ. Psychol. 89, 538–550. doi: 10.1111/bjep.12277

Kegan, R. (1994). In Over Our Heads: The Mental Demands of Modern Life. Cambridge, MA: Harvard University Press.

Klein, S., Benjamin, R., Shavelson, R., and Bolus, R. (2007). The collegiate learning assessment: facts and fantasies. Eval. Rev. 31, 415–439. doi: 10.1177/0193841x07303318

Kosslyn, S. M., and Nelson, B. (2017). Building the Intentional University: Minerva and the Future of Higher Education. Cambridge, MAL: The MIT Press.

Lane, S., and Stone, C. A. (2006). “Performance assessment,” in Educational Measurement , 4th Edn, ed. R. L. Brennan (Lanham, MA: Rowman & Littlefield Publishers), 387–432.

Leighton, J. P. (2019). The risk–return trade-off: performance assessments and cognitive validation of inferences. Br. J. Educ. Psychol. 89, 441–455. doi: 10.1111/bjep.12271

Leu, D. J., Kiili, C., Forzani, E., Zawilinski, L., McVerry, J. G., and O’Byrne, W. I. (2020). “The new literacies of online research and comprehension,” in The Concise Encyclopedia of Applied Linguistics , ed. C. A. Chapelle (Oxford: Wiley-Blackwell), 844–852.

Leu, D. J., Kulikowich, J. M., Kennedy, C., and Maykel, C. (2014). “The ORCA Project: designing technology-based assessments for online research,” in Paper Presented at the American Educational Research Annual Meeting , Philadelphia, PA.

Liu, O. L., Frankel, L., and Roohr, K. C. (2014). Assessing critical thinking in higher education: current state and directions for next-generation assessments. ETS Res. Rep. Ser. 1, 1–23. doi: 10.1002/ets2.12009

McClelland, D. C. (1973). Testing for competence rather than for “intelligence.”. Am. Psychol. 28, 1–14. doi: 10.1037/h0034092

McGrew, S., Ortega, T., Breakstone, J., and Wineburg, S. (2017). The challenge that’s bigger than fake news: civic reasoning in a social media environment. Am. Educ. 4, 4-9, 39.

Mejía, A., Mariño, J. P., and Molina, A. (2019). Incorporating perspective analysis into critical thinking performance assessments. Br. J. Educ. Psychol. 89, 456–467. doi: 10.1111/bjep.12297

Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educ. Res. 23, 13–23. doi: 10.3102/0013189x023002013

Mislevy, R. J., Almond, R. G., and Lukas, J. F. (2003). A brief introduction to evidence-centered design. ETS Res. Rep. Ser. 2003, i–29. doi: 10.1002/j.2333-8504.2003.tb01908.x

Mislevy, R. J., and Haertel, G. D. (2006). Implications of evidence-centered design for educational testing. Educ. Meas. Issues Pract. 25, 6–20. doi: 10.1111/j.1745-3992.2006.00075.x

Mullis, I. V. S., Martin, M. O., Foy, P., and Hooper, M. (2017). ePIRLS 2016 International Results in Online Informational Reading. Available online at:: http://timssandpirls.bc.edu/pirls2016/international-results/ (accessed May 1, 2020).

Nagel, M.-T., Zlatkin-Troitschanskaia, O., Schmidt, S., and Beck, K. (2020). “Performance assessment of generic and domain-specific skills in higher education economics,” in Student Learning in German Higher Education , eds O. Zlatkin-Troitschanskaia, H. A. Pant, M. Toepper, and C. Lautenbach (Berlin: Springer), 281–299. doi: 10.1007/978-3-658-27886-1_14

Organisation for Economic Co-operation and Development [OECD] (2012). AHELO: Feasibility Study Report , Vol. 1. Paris: OECD. Design and implementation.

Organisation for Economic Co-operation and Development [OECD] (2013). AHELO: Feasibility Study Report , Vol. 2. Paris: OECD. Data analysis and national experiences.

Oser, F. K., and Biedermann, H. (2020). “A three-level model for critical thinking: critical alertness, critical reflection, and critical analysis,” in Frontiers and Advances in Positive Learning in the Age of Information (PLATO) , ed. O. Zlatkin-Troitschanskaia (Cham: Springer), 89–106. doi: 10.1007/978-3-030-26578-6_7

Paul, R., and Elder, L. (2007). Consequential validity: using assessment to drive instruction. Found. Crit. Think. 29, 31–40.

Pellegrino, J. W., and Hilton, M. L. (eds) (2012). Education for life and work: Developing Transferable Knowledge and Skills in the 21st Century. Washington DC: National Academies Press.

Shavelson, R. (2010). Measuring College Learning Responsibly: Accountability in a New Era. Redwood City, CA: Stanford University Press.

Shavelson, R. J. (2013). On an approach to testing and modeling competence. Educ. Psychol. 48, 73–86. doi: 10.1080/00461520.2013.779483

Shavelson, R. J., Zlatkin-Troitschanskaia, O., Beck, K., Schmidt, S., and Marino, J. P. (2019). Assessment of university students’ critical thinking: next generation performance assessment. Int. J. Test. 19, 337–362. doi: 10.1080/15305058.2018.1543309

Shavelson, R. J., Zlatkin-Troitschanskaia, O., and Marino, J. P. (2018). “International performance assessment of learning in higher education (iPAL): research and development,” in Assessment of Learning Outcomes in Higher Education: Cross-National Comparisons and Perspectives , eds O. Zlatkin-Troitschanskaia, M. Toepper, H. A. Pant, C. Lautenbach, and C. Kuhn (Berlin: Springer), 193–214. doi: 10.1007/978-3-319-74338-7_10

Shavelson, R. J., Klein, S., and Benjamin, R. (2009). The limitations of portfolios. Inside Higher Educ. Available online at: https://www.insidehighered.com/views/2009/10/16/limitations-portfolios

Stolzenberg, E. B., Eagan, M. K., Zimmerman, H. B., Berdan Lozano, J., Cesar-Davis, N. M., Aragon, M. C., et al. (2019). Undergraduate Teaching Faculty: The HERI Faculty Survey 2016–2017. Los Angeles, CA: UCLA.

Tessier-Lavigne, M. (2020). Putting Ethics at the Heart of Innovation. Stanford, CA: Stanford Magazine.

Wheeler, P., and Haertel, G. D. (1993). Resource Handbook on Performance Assessment and Measurement: A Tool for Students, Practitioners, and Policymakers. Palm Coast, FL: Owl Press.

Wineburg, S., McGrew, S., Breakstone, J., and Ortega, T. (2016). Evaluating Information: The Cornerstone of Civic Online Reasoning. Executive Summary. Stanford, CA: Stanford History Education Group.

Zahner, D. (2013). Reliability and Validity–CLA+. Council for Aid to Education. Available online at:: https://pdfs.semanticscholar.org/91ae/8edfac44bce3bed37d8c9091da01d6db3776.pdf .

Zlatkin-Troitschanskaia, O., and Shavelson, R. J. (2019). Performance assessment of student learning in higher education [Special issue]. Br. J. Educ. Psychol. 89, i–iv, 413–563.

Zlatkin-Troitschanskaia, O., Pant, H. A., Lautenbach, C., Molerov, D., Toepper, M., and Brückner, S. (2017). Modeling and Measuring Competencies in Higher Education: Approaches to Challenges in Higher Education Policy and Practice. Berlin: Springer VS.

Zlatkin-Troitschanskaia, O., Pant, H. A., Toepper, M., and Lautenbach, C. (eds) (2020). Student Learning in German Higher Education: Innovative Measurement Approaches and Research Results. Wiesbaden: Springer.

Zlatkin-Troitschanskaia, O., Shavelson, R. J., and Pant, H. A. (2018). “Assessment of learning outcomes in higher education: international comparisons and perspectives,” in Handbook on Measurement, Assessment, and Evaluation in Higher Education , 2nd Edn, eds C. Secolsky and D. B. Denison (Abingdon: Routledge), 686–697.

Zlatkin-Troitschanskaia, O., Shavelson, R. J., Schmidt, S., and Beck, K. (2019). On the complementarity of holistic and analytic approaches to performance assessment scoring. Br. J. Educ. Psychol. 89, 468–484. doi: 10.1111/bjep.12286

Keywords : critical thinking, performance assessment, assessment framework, scoring rubric, evidence-centered design, 21st century skills, higher education

Citation: Braun HI, Shavelson RJ, Zlatkin-Troitschanskaia O and Borowiec K (2020) Performance Assessment of Critical Thinking: Conceptualization, Design, and Implementation. Front. Educ. 5:156. doi: 10.3389/feduc.2020.00156

Received: 30 May 2020; Accepted: 04 August 2020; Published: 08 September 2020.

Reviewed by:

Copyright © 2020 Braun, Shavelson, Zlatkin-Troitschanskaia and Borowiec. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Henry I. Braun, [email protected]

This article is part of the Research Topic

Assessing Information Processing and Online Reasoning as a Prerequisite for Learning in Higher Education

• Table of Contents
• Random Entry
• Chronological
• Editorial Information
• About the SEP
• Editorial Board
• How to Cite the SEP
• Special Characters
• Advanced Tools
• Support the SEP
• PDFs for SEP Friends
• Make a Donation
• SEPIA for Libraries
• Back to Entry
• Entry Contents
• Entry Bibliography
• Academic Tools
• Friends PDF Preview
• Author and Citation Info
• Back to Top

Supplement to Critical Thinking

How can one assess, for purposes of instruction or research, the degree to which a person possesses the dispositions, skills and knowledge of a critical thinker?

In psychometrics, assessment instruments are judged according to their validity and reliability.

Roughly speaking, an instrument is valid if it measures accurately what it purports to measure, given standard conditions. More precisely, the degree of validity is “the degree to which evidence and theory support the interpretations of test scores for proposed uses of tests” (American Educational Research Association 2014: 11). In other words, a test is not valid or invalid in itself. Rather, validity is a property of an interpretation of a given score on a given test for a specified use. Determining the degree of validity of such an interpretation requires collection and integration of the relevant evidence, which may be based on test content, test takers’ response processes, a test’s internal structure, relationship of test scores to other variables, and consequences of the interpretation (American Educational Research Association 2014: 13–21). Criterion-related evidence consists of correlations between scores on the test and performance on another test of the same construct; its weight depends on how well supported is the assumption that the other test can be used as a criterion. Content-related evidence is evidence that the test covers the full range of abilities that it claims to test. Construct-related evidence is evidence that a correct answer reflects good performance of the kind being measured and an incorrect answer reflects poor performance.

An instrument is reliable if it consistently produces the same result, whether across different forms of the same test (parallel-forms reliability), across different items (internal consistency), across different administrations to the same person (test-retest reliability), or across ratings of the same answer by different people (inter-rater reliability). Internal consistency should be expected only if the instrument purports to measure a single undifferentiated construct, and thus should not be expected of a test that measures a suite of critical thinking dispositions or critical thinking abilities, assuming that some people are better in some of the respects measured than in others (for example, very willing to inquire but rather closed-minded). Otherwise, reliability is a necessary but not a sufficient condition of validity; a standard example of a reliable instrument that is not valid is a bathroom scale that consistently under-reports a person’s weight.

Assessing dispositions is difficult if one uses a multiple-choice format with known adverse consequences of a low score. It is pretty easy to tell what answer to the question “How open-minded are you?” will get the highest score and to give that answer, even if one knows that the answer is incorrect. If an item probes less directly for a critical thinking disposition, for example by asking how often the test taker pays close attention to views with which the test taker disagrees, the answer may differ from reality because of self-deception or simple lack of awareness of one’s personal thinking style, and its interpretation is problematic, even if factor analysis enables one to identify a distinct factor measured by a group of questions that includes this one (Ennis 1996). Nevertheless, Facione, Sánchez, and Facione (1994) used this approach to develop the California Critical Thinking Dispositions Inventory (CCTDI). They began with 225 statements expressive of a disposition towards or away from critical thinking (using the long list of dispositions in Facione 1990a), validated the statements with talk-aloud and conversational strategies in focus groups to determine whether people in the target population understood the items in the way intended, administered a pilot version of the test with 150 items, and eliminated items that failed to discriminate among test takers or were inversely correlated with overall results or added little refinement to overall scores (Facione 2000). They used item analysis and factor analysis to group the measured dispositions into seven broad constructs: open-mindedness, analyticity, cognitive maturity, truth-seeking, systematicity, inquisitiveness, and self-confidence (Facione, Sánchez, and Facione 1994). The resulting test consists of 75 agree-disagree statements and takes 20 minutes to administer. A repeated disturbing finding is that North American students taking the test tend to score low on the truth-seeking sub-scale (on which a low score results from agreeing to such statements as the following: “To get people to agree with me I would give any reason that worked”. “Everyone always argues from their own self-interest, including me”. “If there are four reasons in favor and one against, I’ll go with the four”.) Development of the CCTDI made it possible to test whether good critical thinking abilities and good critical thinking dispositions go together, in which case it might be enough to teach one without the other. Facione (2000) reports that administration of the CCTDI and the California Critical Thinking Skills Test (CCTST) to almost 8,000 post-secondary students in the United States revealed a statistically significant but weak correlation between total scores on the two tests, and also between paired sub-scores from the two tests. The implication is that both abilities and dispositions need to be taught, that one cannot expect improvement in one to bring with it improvement in the other.

A more direct way of assessing critical thinking dispositions would be to see what people do when put in a situation where the dispositions would reveal themselves. Ennis (1996) reports promising initial work with guided open-ended opportunities to give evidence of dispositions, but no standardized test seems to have emerged from this work. There are however standardized aspect-specific tests of critical thinking dispositions. The Critical Problem Solving Scale (Berman et al. 2001: 518) takes as a measure of the disposition to suspend judgment the number of distinct good aspects attributed to an option judged to be the worst among those generated by the test taker. Stanovich, West and Toplak (2011: 800–810) list tests developed by cognitive psychologists of the following dispositions: resistance to miserly information processing, resistance to myside thinking, absence of irrelevant context effects in decision-making, actively open-minded thinking, valuing reason and truth, tendency to seek information, objective reasoning style, tendency to seek consistency, sense of self-efficacy, prudent discounting of the future, self-control skills, and emotional regulation.

It is easier to measure critical thinking skills or abilities than to measure dispositions. The following eight currently available standardized tests purport to measure them: the Watson-Glaser Critical Thinking Appraisal (Watson & Glaser 1980a, 1980b, 1994), the Cornell Critical Thinking Tests Level X and Level Z (Ennis & Millman 1971; Ennis, Millman, & Tomko 1985, 2005), the Ennis-Weir Critical Thinking Essay Test (Ennis & Weir 1985), the California Critical Thinking Skills Test (Facione 1990b, 1992), the Halpern Critical Thinking Assessment (Halpern 2016), the Critical Thinking Assessment Test (Center for Assessment & Improvement of Learning 2017), the Collegiate Learning Assessment (Council for Aid to Education 2017), the HEIghten Critical Thinking Assessment (https://territorium.com/heighten/), and a suite of critical thinking assessments for different groups and purposes offered by Insight Assessment (https://www.insightassessment.com/products). The Critical Thinking Assessment Test (CAT) is unique among them in being designed for use by college faculty to help them improve their development of students’ critical thinking skills (Haynes et al. 2015; Haynes & Stein 2021). Also, for some years the United Kingdom body OCR (Oxford Cambridge and RSA Examinations) awarded AS and A Level certificates in critical thinking on the basis of an examination (OCR 2011). Many of these standardized tests have received scholarly evaluations at the hands of, among others, Ennis (1958), McPeck (1981), Norris and Ennis (1989), Fisher and Scriven (1997), Possin (2008, 2013a, 2013b, 2013c, 2014, 2020) and Hatcher and Possin (2021). Their evaluations provide a useful set of criteria that such tests ideally should meet, as does the description by Ennis (1984) of problems in testing for competence in critical thinking: the soundness of multiple-choice items, the clarity and soundness of instructions to test takers, the information and mental processing used in selecting an answer to a multiple-choice item, the role of background beliefs and ideological commitments in selecting an answer to a multiple-choice item, the tenability of a test’s underlying conception of critical thinking and its component abilities, the set of abilities that the test manual claims are covered by the test, the extent to which the test actually covers these abilities, the appropriateness of the weighting given to various abilities in the scoring system, the accuracy and intellectual honesty of the test manual, the interest of the test to the target population of test takers, the scope for guessing, the scope for choosing a keyed answer by being test-wise, precautions against cheating in the administration of the test, clarity and soundness of materials for training essay graders, inter-rater reliability in grading essays, and clarity and soundness of advance guidance to test takers on what is required in an essay. Rear (2019) has challenged the use of standardized tests of critical thinking as a way to measure educational outcomes, on the grounds that  they (1) fail to take into account disputes about conceptions of critical thinking, (2) are not completely valid or reliable, and (3) fail to evaluate skills used in real academic tasks. He proposes instead assessments based on discipline-specific content.

There are also aspect-specific standardized tests of critical thinking abilities. Stanovich, West and Toplak (2011: 800–810) list tests of probabilistic reasoning, insights into qualitative decision theory, knowledge of scientific reasoning, knowledge of rules of logical consistency and validity, and economic thinking. They also list instruments that probe for irrational thinking, such as superstitious thinking, belief in the superiority of intuition, over-reliance on folk wisdom and folk psychology, belief in “special” expertise, financial misconceptions, overestimation of one’s introspective powers, dysfunctional beliefs, and a notion of self that encourages egocentric processing. They regard these tests along with the previously mentioned tests of critical thinking dispositions as the building blocks for a comprehensive test of rationality, whose development (they write) may be logistically difficult and would require millions of dollars.

A superb example of assessment of an aspect of critical thinking ability is the Test on Appraising Observations (Norris & King 1983, 1985, 1990a, 1990b), which was designed for classroom administration to senior high school students. The test focuses entirely on the ability to appraise observation statements and in particular on the ability to determine in a specified context which of two statements there is more reason to believe. According to the test manual (Norris & King 1985, 1990b), a person’s score on the multiple-choice version of the test, which is the number of items that are answered correctly, can justifiably be given either a criterion-referenced or a norm-referenced interpretation.

On a criterion-referenced interpretation, those who do well on the test have a firm grasp of the principles for appraising observation statements, and those who do poorly have a weak grasp of them. This interpretation can be justified by the content of the test and the way it was developed, which incorporated a method of controlling for background beliefs articulated and defended by Norris (1985). Norris and King synthesized from judicial practice, psychological research and common-sense psychology 31 principles for appraising observation statements, in the form of empirical generalizations about tendencies, such as the principle that observation statements tend to be more believable than inferences based on them (Norris & King 1984). They constructed items in which exactly one of the 31 principles determined which of two statements was more believable. Using a carefully constructed protocol, they interviewed about 100 students who responded to these items in order to determine the thinking that led them to choose the answers they did (Norris & King 1984). In several iterations of the test, they adjusted items so that selection of the correct answer generally reflected good thinking and selection of an incorrect answer reflected poor thinking. Thus they have good evidence that good performance on the test is due to good thinking about observation statements and that poor performance is due to poor thinking about observation statements. Collectively, the 50 items on the final version of the test require application of 29 of the 31 principles for appraising observation statements, with 13 principles tested by one item, 12 by two items, three by three items, and one by four items. Thus there is comprehensive coverage of the principles for appraising observation statements. Fisher and Scriven (1997: 135–136) judge the items to be well worked and sound, with one exception. The test is clearly written at a grade 6 reading level, meaning that poor performance cannot be attributed to difficulties in reading comprehension by the intended adolescent test takers. The stories that frame the items are realistic, and are engaging enough to stimulate test takers’ interest. Thus the most plausible explanation of a given score on the test is that it reflects roughly the degree to which the test taker can apply principles for appraising observations in real situations. In other words, there is good justification of the proposed interpretation that those who do well on the test have a firm grasp of the principles for appraising observation statements and those who do poorly have a weak grasp of them.

To get norms for performance on the test, Norris and King arranged for seven groups of high school students in different types of communities and with different levels of academic ability to take the test. The test manual includes percentiles, means, and standard deviations for each of these seven groups. These norms allow teachers to compare the performance of their class on the test to that of a similar group of students.

Copyright © 2022 by David Hitchcock < hitchckd @ mcmaster . ca >

• Accessibility

Support SEP

Mirror sites.

View this site from another server:

• Info about mirror sites

The Stanford Encyclopedia of Philosophy is copyright © 2024 by The Metaphysics Research Lab , Department of Philosophy, Stanford University

Library of Congress Catalog Data: ISSN 1095-5054

TSA (Thinking Skills Assessment)

• What is the TSA?

How do I register?

When do i take the test, practice materials.

• How do I get my results?

New arrangements for this test for 2025-entry will be communicated at the earliest opportunity. In the meantime, you are welcome to explore the test preparation and practice materials which you may find helpful. 

What is the TSA? 

The Thinking Skills Assessment (TSA) is divided into two parts: a 90-minute, multiple-choice Thinking Skills Assessment and a 30-minute writing task.  If applying for  Philosophy, Politics and Economics , you will be required to take both sections of the TSA.

However, you will be required to take Section 1 only if you are applying for:

• Economics and Management
• Experimental Psychology
• Human Sciences
• History and Economics
• Psychology, Philosophy and Linguistics .

New arrangements for this test for 2025-entry will be communicated at the earliest opportunity. 

New arrangements for 2025-entry will be communicated by the start of the new admissions cycle in Spring. 

Section 1 specimen and past papers

The first section of the TSA is made up of 50 multiple-choice questions. Below you can find a specimen paper, in addition to past papers going back to 2008. You can also download answer sheets for each paper and a score conversion chart. 

• TSA specimen test
• TSA specimen answer sheet
• TSA specimen answer key
• TSA specimen explained answers
• TSA Oxford 2022 Section 1
• TSA Oxford 2022 Section 1 answer key and score conversion  
• TSA Oxford 2021 Section 1
• TSA Oxford 2021 Section 1 answer key and score conversion
• TSA Oxford 2020 Section 1
• TSA Oxford 2020 Section 1 answer key and score conversion
• TSA Oxford 2019 Section 1
• TSA Oxford 2019 Section 1 answer key
• TSA Oxford 2019 Section 1 score conversion
• TSA Oxford 2018 Section 1
• TSA Oxford 2018 Section 1 answer key
• TSA Oxford 2018 Section 1 score conversion
• TSA Oxford 2017 Section 1
• TSA Oxford 2017 Section 1 answer key
• TSA Oxford 2017 Section 1 score conversion
• TSA Oxford 2016 Section 1
• TSA Oxford 2016 Section 1 answer key
• TSA Oxford 2016 Section 1 score conversion
• TSA Oxford 2015 Section 1
• TSA Oxford 2015 Section 1 answer key
• TSA Oxford 2015 Section 1 score conversion
• TSA Oxford 2014 Section 1
• TSA Oxford 2014 Section 1 answer key
• TSA Oxford 2014 Section 1 score conversion
• TSA Oxford 2013 Section 1
• TSA Oxford 2013 Section 1 answer key
• TSA Oxford 2013 Section 1 score conversion
• TSA Oxford 2012 Section 1
• TSA Oxford 2012 Section 1 answer key
• TSA Oxford 2012 Section 1 score conversion
• TSA Oxford 2011 Section 1
• TSA Oxford 2011 Section 1 answer key
• TSA Oxford 2011 Section 1 score conversion
• TSA Oxford 2010 Section 1
• TSA Oxford 2010 Section 1 answer key
• TSA Oxford 2010 Section 1 score conversion
• TSA Oxford 2009 Section 1
• TSA Oxford 2009 Section 1 answer key
• TSA Oxford 2009 Section 1 score conversion
• TSA Oxford 2008 Section 1
• TSA Oxford 2008 Section 1 answer key
• TSA Oxford 2008 Section 1 score conversion

Section 2 specimen and past papers

Section 2 of the TSA test is a writing task. You will have 30 minutes to write a single short essay. There will be a choice of four essay questions, on general subjects that do not require any specialised knowledge.

The writing task gives you an opportunity to show that you can communicate effectively in writing, organising your ideas and presenting them clearly and concisely. You should start by planning the essay carefully, deciding what are the main points that you want to make in the limited time available, and how to organise your answer to explain and convey them clearly. You will have plenty of space if you want to use it, but a concise and well-structured answer may be more effective than a longer essay.

It is important that your answer is relevant to the question, and addresses it directly. If the question requires you to make judgements and express your own opinions, try to provide coherent arguments to support your views, and consider the merits of possible counter-arguments. Your essay will be judged by the quality of the writing, and the way you use what you know.

The document below, written by an Oxford tutor, gives several example questions from past papers and discusses ways to approach them. 

• Guidance on TSA half-hour essay questions

Below you can find a specimen paper and past papers going back to 2008. 

• TSA Oxford specimen Section 2
• TSA Oxford Writing Task specimen answer sheet
• TSA Oxford 2022 Section 2
• TSA Oxford 2021 Section 2
• TSA Oxford 2020 Section 2
• TSA Oxford 2019 Section 2
• TSA Oxford 2018 Section 2
• TSA Oxford 2017 Section 2 
• TSA Oxford 2016 Section 2
• TSA Oxford 2015 Section 2
• TSA Oxford 2014 Section 2
• TSA Oxford 2013 Section 2
• TSA Oxford 2012 Section 2
• TSA Oxford 2011 Section 2
• TSA Oxford 2010 Section 2
• TSA Oxford 2009 Section 2
• TSA Oxford 2008 Section 2

Test question guide and explanation of results

• TSA Question Guide

Explanation of results :

Section 1  scores 1 mark per question. Scores are calculated on the TSA scale to one decimal place (running approximately 0–100). The scale is an estimate of the candidate’s ability, which makes scoring comparable by factoring in the question and overall test difficulty, using the  Rasch statistical technique . Marking of this section is automated. 

Section 2  is reviewed by the admissions tutor(s) of the college you apply to.

• Explanation of Results 2023
• Explanation of Results Section 1 2023  
• Explanation of Results 2022
• Explanation of Results Section 1 2022
• Explanation of Results 2021
• Explanation of Results Section 1 2021
• Explanation of Results 2020
• Explanation of Results Section 1 2020
• Explanation of Results 2019
• Explanation of Results Section 1 2019
• Explanation of results 2018
• Explanation of results Section 1 2018
• Explanation of results 2017
• Explanation of results Section 1 2017
• Explanation of results 2016
• Explanation of results 2015
• Explanation of results 2014
• Explanation of results 2013
• Explanation of results 2012
• Explanation of results 2011
• Explanation of results 2010
• Explanation of results 2009
• Explanation of results 2008
• Explanation of results 2007

Further reading

• John Butterworth and Geoff Thwaites, Thinking Skills (Cambridge University Press, 2013)
• Anne Thomson, Critical Reasoning: A Practical Introduction (Routledge, 2008)
• Nigel Warburton, Thinking from A to Z (Routledge, 2000)
• Alec  Fisher, Critical Thinking: An Introduction (Cambridge University Press, 2011)

How do I get my results?         

New arrangements for this test for 2025-entry will be communicated at the earliest opportunity. 

YOU MUST TAKE THE TSA IF YOU ARE APPLYING FOR:

Experimental Psychology Human Sciences Philosophy, Politics and Economics Psychology, Philosophy and Linguistics

YOU MUST TAKE THE TSA (SECTION 1) IF YOU ARE APPLYING FOR:

Economics and Management   History and Economics

Oxford only admissions tests:

BMSAT CAT HAT MAT MLAT PAT Philosophy test TSA

CAN'T FIND WHAT YOU'RE LOOKING FOR?

Try our extensive database of FAQs or submit your own question...

ANY QUESTIONS?

Follow us on social media

Follow us on social media to get the most up-to-date application information throughout the year, and to hear from our students.

Get 25% off all test packages.

Get 25% off all test packages!

Click below to get 25% off all test packages.

Critical Thinking Tests

• 228 questions

Critical thinking tests, sometimes known as critical reasoning tests, are often used by employers. They evaluate how a candidate makes logical deductions after scrutinising the evidence provided, while avoiding fallacies or non-factual opinions. Critical thinking tests can form part of an assessment day, or be used as a screening test before an interview.

What is a critical thinking test?

A critical thinking test assesses your ability to use a range of logical skills to evaluate given information and make a judgement. The test is presented in such a way that candidates are expected to quickly scrutinise the evidence presented and decide on the strength of the arguments.

Critical thinking tests show potential employers that you do not just accept data and can avoid subconscious bias and opinions – instead, you can find logical connections between ideas and find alternative interpretations.

This test is usually timed, so quick, clear, logical thinking will help candidates get the best marks. Critical thinking tests are designed to be challenging, and often used as part of the application process for upper-management-level roles.

What does critical thinking mean?

Critical thinking is the intellectual skill set that ensures you can process and consider information, challenge and analyse data, and then reach a conclusion that can be defended and justified.

In the most simple terms, critical reasoning skills will make sure that you are not simply accepting information at face value with little or no supporting evidence.

It also means that you are less likely to be swayed by ‘false news’ or opinions that cannot be backed with facts – which is important in high-level jobs that require logical thinking.

For more information about logical thinking, please see our article all about logical reasoning .

Which professions use critical thinking tests, and why?

Typically, critical thinking tests are taken as part of the application process for jobs that require advanced skills in judgement, analysis and decision making. The higher the position, the more likely that you will need to demonstrate reliable critical reasoning and good logic.

The legal sector is the main industry that uses critical thinking assessments – making decisions based on facts, without opinion and intuition, is vital in legal matters.

A candidate for a legal role needs to demonstrate their intellectual skills in problem-solving without pre-existing knowledge or subconscious bias – and the critical thinking test is a simple and effective way to screen candidates.

Another industry that uses critical thinking tests as part of the recruitment process is banking. In a similar way to the legal sector, those that work in banking are required to make decisions without allowing emotion, intuition or opinion to cloud coherent analysis and conclusions.

Critical thinking tests also sometimes comprise part of the recruitment assessment for graduate and management positions across numerous industries.

The format of the test: which skills are tested?

The test itself, no matter the publisher, is multiple choice.

As a rule, the questions present a paragraph of information for a scenario that may include numerical data. There will then be a statement and a number of possible answers.

The critical thinking test is timed, so decisions need to be made quickly and accurately; in most tests there is a little less than a minute for each question. Having experience of the test structure and what each question is looking for will make the experience smoother for you.

There are typically five separate sections in a critical thinking test, and each section may have multiple questions.

Inference questions assess your ability to judge whether a statement is true, false, or impossible to determine based on the given data and scenario. You usually have five possible answers: absolutely true, absolutely false, possibly true, possibly false, or not possible to determine.

Assumptions

In this section, you are being assessed on your ability to avoid taking things for granted. Each question gives a scenario including data, and you need to evaluate whether there are any assumptions present.

Here you are given a scenario and a number of deductions that may be applicable. You need to assess the given deductions to see which is the logical conclusion – does it follow?

Interpretation

In the interpretation stage, you need to read and analyse a paragraph of information, then interpret a set of possible conclusions, to see which one is correct. You are looking for the conclusion that follows beyond reasonable doubt.

Evaluation of Arguments

In this section, you are given a scenario and a set of arguments that can be for or against. You need to determine which are strong arguments and which are weak, in terms of the information that you have. This decision is made based on the way they address the scenario and how relevant they are to the content.

How best to prepare for a critical thinking test

The best way to prepare for any type of aptitude test is to practice, and critical thinking tests are no different.

Taking practice tests, as mentioned above, will give you confidence as it makes you better understand the structure, layout and timing of the real tests, so you can concentrate on the actual scenarios and questions.

Practice tests should be timed. This will help you get used to working through the scenarios and assessing the conclusions under time constraints – which is a good way to make sure that you perform quickly as well as accurately.

In some thinking skills assessments , a timer will be built in, but you might need to time yourself.

Consistent practice will also enable you to pinpoint any areas of the critical thinking test that require improvement. Our tests offer explanations for each answer, similar to the examples provided above.

Publishers of critical thinking tests

The watson glaser critical thinking test.

The Watson-Glaser Critical Thinking Appraisal (W-GCTA) is the most popular and widely used critical thinking test. This test has been in development for 85 years and is published by TalentLens .

The W-GCTA is seen as a successful tool for assessing cognitive abilities, allowing recruiting managers to predict job success, find good managers and identify future leaders. It is available in multiple languages including English, French and Spanish.

The test itself can be used as part of an assessment day or as a screening assessment before an interview. It consists of 40 questions on the 5 sections mentioned above, and is timed at 30 minutes. Click here for more information on Watson Glaser tests .

SHL critical reasoning test

SHL is a major aptitude test publisher, which offers critical thinking as part of its testing battery for pre-employment checks.

SHL tests cover all kinds of behavioural and aptitude tests, from logic to inference, verbal to numerical – and with a number of test batteries available online, they are one of the most popular choices for recruiters.

Cornell critical thinking test

The Cornell critical thinking test was made to test students and first developed in 1985. It is an American system that helps teachers, parents and administrators to confidently predict future performance for college admission, gifted and advanced placement programs, and even career success.

Prepare yourself for leading employers

5 Example critical thinking practice questions with answers

In this section, you need to deduce whether the inferred statement is true, false or impossible to deduce.

The UK Government has published data that shows 82% of people under the age of 30 are not homeowners. A charity that helps homeless people has published data that shows 48% of people that are considered homeless are under 30.

The lack of affordable housing on the sales market is the reason so many under-30s are homeless.

• Definitely True
• Probably True
• Impossible to Deduce
• Probably False
• Definitely False

The information given does not infer the conclusion given, so it is impossible to deduce if the inference is correct – there is just not enough information to judge the inference as correct.

The removal of the five-substitution rule in British football will benefit clubs with a smaller roster.

Clubs with more money would prefer the five-substitute rule to continue.

• Assumption Made

Assumption Not Made

This is an example of a fallacy that could cause confusion for a candidate – it encourages you to bring in any pre-existing knowledge of football clubs.

It would be easy to assume the assumption has been made when you consider that the more money a club has, the more players they should have on the roster. However, the statement does not make the assumption that the clubs with more money would prefer to continue with the five-substitute rule.

All boys love football. Football is a sport, therefore:

• All boys love all sports
• Girls do not love football
• Boys are more likely to choose to play football than any other sport

In this section we are looking for the conclusion that follows the logic of the statement. In this example, we cannot deduce that girls do not love football, because there is not enough information to support that.

In the same way the conclusion that all boys love all sports does not follow – we are not given enough information to make that assumption. So, the conclusion that follows is 3: boys are more likely to choose football than any other sport because all boys like football.

The British Museum has a range of artefacts on display, including the largest privately owned collection of WWII weaponry.

There is a larger privately owned collection of WWII weaponry in the USA.

• Conclusion Follows

Conclusion Does Not Follow

The fact that the collection is in the British Museum does not make a difference to the fact it is the largest private collection – so there cannot be a larger collection elsewhere.

The Department for Education should lower standards in examinations to make it fairer for less able students.

• Yes – top grades are too hard for lower-income students
• No – less fortunate students are not capable of higher standards
• Yes – making the standards lower will benefit all students
• No – private school students will suffer if grade standards are lower
• The strongest argument is the right answer, not the one that you might personally believe.

In this case, we need to assess which argument is most relevant to the statement. Both 1 and 4 refer to students in particular situations, which isn’t relevant to the statement. The same can be said about 2, so the strongest argument is 3, since it is relevant and addresses the statement given.

Sample Critical Thinking Tests question Test your knowledge!

What implication can be drawn from the information in the passage?

A company’s internal audit revealed that departments with access to advanced analytics tools reported higher levels of strategic decision-making. These departments also showed a higher rate of reaching their quarterly objectives.

• Strategic decision-making has no link to the achievement of quarterly objectives.
• Access to advanced analytics does not influence a department's ability to make strategic decisions.
• Advanced analytics tools are the sole reason for departments reaching their quarterly objectives.
• Departments without access to advanced analytics tools are unable to make strategic decisions.
• Advanced analytics tools may facilitate better strategic decision-making, which can lead to the achievement of objectives.

After reading the passage below, what conclusion is best supported by the information provided?

• Job satisfaction increases when employees start their day earlier.
• Starting early may lead to more efficient task completion and less job-related stress.
• Workers who start their day later are more efficient at completing tasks.
• There is a direct correlation between job satisfaction and starting work early.
• The study concludes that job-related stress is unaffected by the start time of the workday.

Based on the passage below, which of the following assumptions is implicit?

• Inter-departmental cooperation is the sole factor influencing project completion rates.
• The increase in project completion rates is due entirely to the specialized team-building module.
• Team-building exercises have no effect on inter-departmental cooperation.
• The specialized team-building module may contribute to improvements in inter-departmental cooperation.
• Departments that have not undergone the training will experience a decrease in project completion rates.

What is the flaw in the argument presented in the passage below?

• The assumption that a casual dress code is suitable for all company types.
• High-tech companies have a casual dress code to increase employee productivity specifically.
• The argument correctly suggests that a casual dress code will increase employee morale in every company.
• Morale and productivity cannot be affected by a company's dress code.
• A casual dress code is more important than other factors in determining a company's success.

Which statement is an inference that can be drawn from the passage below?

• Telecommuting employees are less productive than on-site workers.
• The reduction in operational costs is directly caused by the increase in telecommuting employees.
• Telecommuting may have contributed to the decrease in operational costs.
• Operational costs are unaffected by employee work locations.
• The number of telecommuting employees has no impact on operational costs.

Start your success journey

Access one of our Watson Glaser tests for FREE.

I’ve practiced hundreds of numerical questions and still have plenty more to try.

Ellen used Practice Aptitude Tests to prepare for her upcoming interview at HSBC.

Hire better talent

At Neuroworx we help companies build perfect teams

Critical Thinking Tests Tips

The most important factor in your success will be practice. If you have taken some practice tests, not only will you start to recognise the way questions are worded and become familiar with what each question is looking for, you will also be able to find out whether there are any parts that you need extra practice with.

It is important to find out which test you will be taking, as some generic critical thinking practice tests might not help if you are taking specific publisher tests (see the section below).

2 Fact vs fallacy

Practice questions can also help you recognise the difference between fact and fallacy in the test. A fallacy is simply an error or something misleading in the scenario paragraph that encourages you to choose an invalid argument. This might be a presumption or a misconception, but if it isn’t spotted it can make finding the right answer impossible.

3 Ignore what you already know

There is no need for pre-existing knowledge to be brought into the test, so no research is needed. In fact, it is important that you ignore any subconscious bias when you are considering the questions – you need logic and facts to get the correct answer, not intuition or instinct.

4 Read everything carefully

Read all the given information thoroughly. This might sound straightforward, but knowing that the test is timed can encourage candidates to skip content and risk misunderstanding the content or miss crucial details.

During the test itself, you will receive instructions that will help you to understand what is being asked of you on each section. There is likely to be an example question and answer, so ensure you take the time to read them fully.

5 Stay aware of the time you've taken

This test is usually timed, so don’t spend too long on a question. If you feel it is going to take too much time, leave it and come back to it at the end (if you have time). Critical thinking tests are complex by design, so they do have quite generous time limits.

For further advice, check out our full set of tips for critical thinking tests .

Prepare for your Watson Glaser Test

Immediate access. Cancel anytime.

• 30 Numerical reasoning tests
• 30 Verbal reasoning tests
• 30 Diagrammatic reasoning tests
• 30 Situational judgement tests
• 34 Publisher packages e.g. Watson Glaser
• 252 Employer packages e.g. HSBC
• 29 Extra packages e.g Mechanical
• Dashboard performance tracking
• Full solutions and explanations
• Tips, tricks, guides and resources
• Access to free tests
• Basic performance tracking
• Solutions & explanations
• Tips and resources

Critical Thinking Tests FAQs

What are the basics of critical thinking.

In essence, critical thinking is the intellectual process of considering information on its merits, and reaching an analysis or conclusion from that information that can be defended and rationalised with evidence.

How do you know if you have good critical thinking skills?

You are likely to be someone with good critical thinking skills if you can build winning arguments; pick holes in someone’s theory if it’s inconsistent with known facts; reflect on the biases inherent in your own experiences and assumptions; and look at problems using a systematic methodology.

Reviews of our Watson Glaser tests

What our customers say about our Watson Glaser tests

Jozef Bailey

United Kingdom

April 05, 2022

Doesn't cover all aspects of Watson-Glaser tests but useful

The WGCTA uses more categories to assess critical thinking, but this was useful for the inference section.

April 01, 2022

Just practicing for an interview

Good information and liked that it had a countdown clock, to give you that real feel in the test situation.

Jerico Kadhir

March 31, 2022

Aptitude test

It was OK, I didn't understand personally whether or not the "cannot say" option was acceptable or not in a lot of the questions, as it may have been a trick option.

Salvarina Viknesuari

March 15, 2022

I like the test because the platform is simple and engaging while the test itself is different than most of the Watson Glaser tests I've taken.

Alexis Sheridan

March 02, 2022

Some of the ratios were harder than I thought!

I like how clear the design and layout is - makes things very easy (even if the content itself is not!)

Cyril Lekgetho

February 17, 2022

Mental arithmetic

I enjoyed the fact that there were multiple questions pertaining to one passage of information, rather than multiple passages. However I would've appreciated a more varied question type.

Madupoju Manish

February 16, 2022

Analytics are the best questions

I like the test because of its time schedule. The way the questions are prepared makes it easy to crack the original test.

Chelsea Franklin

February 02, 2022

Interesting

I haven't done something like this for ages. Very good for the brain - although I certainly experienced some fog whilst doing it.

[email protected]

January 04, 2022

Population/exchange rates were the hardest

Great test as it felt a bit time pressured. Very different types of questions in terms of difficulty.

faezeh tavakoli

January 02, 2022

More attention to detail + be more time conscious

It was asking about daily stuff we all deal with, but as an assessment it's scrutinising how we approach these problems.

By using our website you agree with our Cookie Policy.

You are using an outdated browser.

• About Wabash
• Majors & More
• Admissions & Aid
• Student Life
• Give to Wabash
• Event Calendar
• Directories
• Wabash Employment
• Alumni & Friends
• Parents & Families

by Thomas F. Nelson Laird, Indiana University Center for Postsecondary Research

Summer 2005

Highlights of the CCTDI:

• Straightforward 75-question survey; relatively inexpensive to administer; takes about 20 minutes to complete; questionnaire can be completed by paper and pencil or online.
• Survey addresses the "dispositional" dimension of critical thinking—as opposed to the "skills" dimension, which is evaluated in the Critical Thinking Skills Test ( CCTST ). Survey assesses how students feel they approach these seven qualities: truth-seeking, open-mindedness, analytical tendencies, systematic tendencies, critical thinking self-confidence, inquisitiveness, and cognitive maturity.

Uses of the CCTDI:

• As a one-time test to gain understanding of how students view themselves as critical thinkers. Students’ strengths toward critical thinking are noted and areas for improvement identified.  
• As a pre- and post-test of a particular curricular or co-curricular experience in order to study how a student’s attitude toward critical thinking develops in relation to that experience.  
• Can be combined with demographic surveys to examine the relationship between student attitudes toward critical thinking and student characteristics (such as socioeconomic status or major).

Jill Cellars Rogers Center of Inquiry in the Liberal Arts at Wabash College

Introduction

The ideal critical thinker is habitually inquisitive, well-informed, trustful of reason, open-minded, flexible, fair-minded in evaluation, honest in facing personal biases, prudent in making judgments, willing to reconsider, clear about issues, orderly in complex matters, diligent in seeking relevant information, reasonable in the selection of criteria, focused in inquiry, and persistent in seeking results that are as precise as the subject and the circumstances of inquiry permit.                                           – American Philosophical Association, The Delphi Report [1]

Experts from several fields agree that a critical thinker must possess both a set of thinking skills and the habits of mind necessary to use those skills. The California Critical Thinking Dispositions Inventory ( CCTDI ) is a survey instrument designed to measure whether a person habitually exhibits the mindset of an ideal critical thinker. (A companion survey, the California Critical Thinking Skills Test, measures actual critical thinking skills.) The CCTDI, a 75-item questionnaire designed by Peter and Noreen Facione, is available through Insight Assessment (formerly the California Academic Press). The survey is designed for use with students in postsecondary settings (undergraduate, graduate, and professional) and with adults outside of educational environments. The CCTDI is used for student assessment as well as program evaluation, professional development, and training.

The following review provides a summary of several aspects of the CCTDI, including how it is used, how much it costs, what it measures, how it can be used for the purposes of student assessment, and why someone would use it when assessing liberal arts education.  

Administration and Cost

The CCTDI must be ordered from Insight Assessment. It is a tool that can be used with groups of any size (a class, a department, or an entire campus). It is available in paper form or as a web-based survey. Either version takes 20 minutes or less to complete. A "specimen kit" containing a manual, a copy of the instrument, and a copy of the fill-in answer form is available for purchase ($60). For those seriously considering using the CCTDI, it is advisable to obtain the specimen kit prior to ordering the surveys and answer sheets. The manual describes the survey’s history and properties as well as procedures for its administration.

Paper Version

The paper version of test/tool booklets (six-page documents that contain directions and the 75 statements about which students will rate their level of agreement/disagreement) and answer forms (scannable forms on which students fill in bubbles corresponding to their responses) must be ordered. At this writing, answer forms can be ordered in bundles of 25 ($150), 50 ($275), or 100 ($485). Because the booklets are separate from the answer forms, they can be used more than once. For this reason, answer forms can also be ordered on their own in packets of 25 ($110), 50 ($190), and 100 ($335).

Booklets and answer forms are shipped to the purchaser, who determines to whom and how the CCTDI will be administered (e.g., in class, by mail, at orientation). A student filling out the paper form will receive a test/tool booklet and an answer form. Answer forms are then collected from the students and shipped back to Insight Assessment, where they are scanned and scored using a system called CapScore. Insight Assessment then sends the investigator a data file and a report summarizing the survey results.

Online Version

To use the online version, an order needs to be placed with Insight Assessment. The cost is $6 per student. The software application is made available to the administrator by Insight Assessment and needs to be set up on a computer or group of computers. (For specifics on the application or for a demo version contact Insight Assessment.) The software application administers the CCTDI and compiles students’ responses. The data and reporting of the results are available instantly. The system can also be set up to give each student a critical thinking "dispositions profile" immediately upon completion of the instrument. A computer lab is an ideal setting for administering the online version.

About the CCTDI

In 1990, with sponsorship from the American Philosophical Association, a group of scholars from several disciplines developed a definition of critical thinking that had a skills dimension and a dispositional (i.e., affective and attitudinal) dimension. Building on the scholars' definition regarding the habits of mind of an ideal critical thinker, Peter and Noreen Facione developed and tested the CCTDI as a measure of the dispositional side of critical thinking. In its final form, the CCTDI has 75 items. Each respondent can choose from six responses, ranging from "strongly agree" to "strongly disagree." Since 1990, the CCTDI have been developed in several languages, including English, Spanish, and Chinese. The instrument uses seven sub-scales to capture different aspects of the disposition to think critically: truth-seeking, open-mindedness, critical thinking self-confidence, inquisitiveness, cognitive maturity, and the inclination to analyze and systematize. (For a brief definition of each, visit Insight Assessment.) Examples of survey items are listed below, under the corresponding sub-scale.

Example Items for Each CCTDI Sub-scale

The CCTDI total score is the sum of the scores for each of the seven sub-scales. The total score indicates whether a person is generally disposed to think critically—whether the individual habitually exhibits the characteristics of an ideal critical thinker. The total score ranges from 70 to 420. Students who score less than 210 are defined as negatively disposed toward critical thinking, students with scores between 210 and 280 are defined as ambivalently disposed, and students with scores above 280 are defined as positively disposed. [4,6]  The score range for each of the seven sub-scales is from 10–40, and students can be considered negatively (scores less than 30), ambivalently (scores between 30 and 40), or positively (scores greater than 40) disposed to each of the characteristics.

The CCTDI has been tested by its developers and by several independent researchers. [2, 4, 6, 7, 9] Among those who have examined the instrument, there is general agreement that the survey validly and reliably measures the disposition toward critical thinking and is therefore appropriate for inclusion in research and assessment. In contrast to some of the findings of the survey developers, several researchers have identified concerns about the appropriate number of sub-scales and some of the statistical properties of particular sub-scales. [2, 7, 9]  Further work is needed to review these concerns, though the issues raised do not appear to be serious, and using the seven sub-scales defined by the instrument’s authors still appears to be appropriate.

How the CCTDI Can Be Used in Assessment

The CCTDI can be used at a single point in time to gain an understanding of how students view themselves as critical thinkers. This information can be useful in determining whether individual students or groups of students have the dispositions deemed necessary for a class, at the end of a program, or for entry into a particular professional setting. For example, in the field of nursing, which has recommended CCTDI scores [3] , it may be useful to know a student's disposition toward critical thinking upon entry into a program or prior to his or her entry into a clinical setting. Scores can be used to identify strengths and areas for improvement. (A person's predisposition and motivation to think critically is interrelated with actual critical thinking ability; both work together to create a critical thinker. Therefore, institutions interested in assessing critical thinking characteristics of their students might choose to consider using both the CCTDI for attitude and the CCTST for ability.)

In addition, the instrument can be used to test how an experience or set of experiences influence students’ dispositions toward critical thinking. Positive changes in individual predisposition to critical thinking linked to curricular programs have been demonstrated. Students can be tested in their first year of college and again at the end of their senior year to determine how the entire collegiate experience affected their dispositions to think critically. [6] One can also test the effects of specific collegiate programs, courses, or experiences. For this purpose, it is important to measure carefully different student characteristics and differences in what students have done at college in addition to the experiences under study. In one of my own studies [8] , I looked at the effects of students’ experiences with diversity on several outcomes, including the disposition toward critical thinking, as measured by the CCTDI.

The CCTDI is useful because it is a relatively short survey that captures a meaningful concept (the disposition to think critically) with clear connections to valued educational outcomes. It is adaptable to different settings and can be administered to any size group. For these reasons, it can play a valuable role in the assessment of a liberal arts education. However, because it measures a single outcome, the CCTDI is most helpful when combined with information gathered from other instruments and methods. It is a valuable tool to keep in one’s assessment "toolkit."

References  

• American Philosophical Association. (1990). Critical thinking: A statement of expert consensus for purposes of educational assessment and instruction. The Delphi Report Executive Summary: Research findings and recommendations prepared for the committee on pre-college philosophy. (ERIC Document Reproduction Service No. ED315423)
• Bondy, K. N., Koenigseder, L. A., Ishee, J. H., & Williams, B. G. (2001). Psychometric properties of the California critical thinking tests. Journal of Nursing Measurement , 9, 309–328.
• Facione, N. C. & Facione, P. A. (1997). Critical thinking assessment in nursing education programs: An aggregate data analysis . Millbrae, CA: The California Academic Press.
• Facione, P. A., Facione, N. C., & Giancarlo, C. A. (1998). The California critical thinking disposition inventory test manual (Revised) . Millbrae, CA: The California Academic Press.
• Facione, P. A., Sánchez, C. A., Facione, N. C., & Gainen, J. (1995). The disposition toward critical thinking. The Journal of General Education, 44 , 1–25.
• Giancarlo, C. A. & Facione, P. A. (2001). A look across four years at the disposition toward critical thinking among undergraduate students. The Journal of General Education , 50 , 29–55.
• Kakai, H. (2003). Re-examining the factor structure of the California Critical Thinking Disposition Inventory. Perceptual & Motor Skills, 96 , 435–438.
• Nelson Laird, T. F. (2005). College students’ experiences with diversity and their effects on academic self-confidence, social agency, and disposition toward critical thinking. Research in Higher Education, 46 , 365–387.
• Walsh, C. M. & Hardy, R. C. (1997). Factor structure stability of the California Critical Thinking Disposition Inventory across sex and various students’ majors. Perceptual & Motor Skills , 85 , 1211–1228.  
• The Bachelor
• Audio Podcasts
• The Class of 2024 Commencement Speakers
• Pickett ’26 Receives “Realizing the Dream” Scholarship
• 2024 Day of Giving: Celebrate. Participate. Elevate.
• Hayden Kammer ’24 Earns Fulbright Scholarship
• WM: The Making of 'Something Rotten!'

News Archives

Wabash College Communications and Marketing

Video player.

What Is the Watson Glaser Test?

Who uses the watson glaser test and why, why is it so important to be a critical thinker, what is the watson glaser red model, how to pass a watson glaser test in 2024, how to prepare for a watson glaser critical appraisal in 2024, frequently asked questions, the watson glaser critical thinking appraisal.

Updated May 10, 2024

Modern employers have changed the way that they recruit new candidates. They are no longer looking for people who have the technical skills on paper that match the job description.

Instead, they are looking for candidates who can demonstrably prove that they have a wider range of transferrable skills.

One of those key skills is the ability to think critically .

Firms (particularly those in sectors such as law, finance, HR and marketing ) need to know that their employees can look beyond the surface of the information presented to them.

They want confidence that their staff members can understand, analyze and evaluate situations or work-related tasks. There is more on the importance of critical thinking later in this article.

This is where the Watson Glaser Critical Thinking test comes into play.

The Watson Glaser critical thinking test is a unique assessment that provides a detailed analysis of a participant’s ability to think critically.

The test lasts 30 minutes and applicants can expect to be tested on around 40 questions in five distinct areas :

Assumptions

Interpretation.

The questions are multiple-choice and may be phrased as true/false statements in a bid to see how well the participant has understood and interpreted the information provided.

Employers around the world use it during recruitment campaigns to help hiring managers effectively filter their prospective candidates .

The Watson Glaser test has been used for more than 85 years; employers trust the insights that the test can provide.

In today’s competitive jobs market where every candidate has brought the best of themselves, it can be increasingly difficult for employers to decide between applicants.

On paper, two candidates may appear identical, with a similar level of education, work experience, and even interests and skills.

But that does not necessarily mean both or either of them is right for the job.

There is much information available on creating an effective cover letter and resume, not to mention advice on making a good impression during an interview.

As a result, employers are increasingly turning to psychometric testing to look beyond the information that they have.

They want to find the right fit: someone who has the skills that they need now and in the future. And with recruitment costs rising each year, making the wrong hiring decision can be catastrophic.

This is where the Watson Glaser test can help.

It can provide hiring managers with the additional support and guidance they need to help them make an informed decision.

The Watson Glaser test is popular among firms working in professional services (such as law, banking and insurance) . It is used for recruitment for junior and senior positions and some of the world’s most recognized establishments are known for their use of the test.

The Bank of England, Deloitte, Hiscox, Linklaters and Hogan Lovells are just a few employers who enhance their recruitment processes through Watson Glaser testing.

Critical thinking is all about logic and rational thought. Finding out someone’s critical thinking skill level is about knowing whether they can assess whether they are being told the truth and how they can use inferences and assumptions to aid their decision-making.

If you are working in a high-pressure environment, having an instinctive ability to look beyond the information provided to the underlying patterns of cause-and-effect can be crucial to do your job well.

Although it is often thought of concerning law firms and finance teams, it is easy to see how critical thinking skills could be applied to a wide range of professions.

For example, HR professionals dealing with internal disputes may need to think critically. Or social workers and other health professionals may need to use critical thinking to assess whether someone is vulnerable and in need of help and support when that person does not or cannot say openly.

Practice Watson Glaser Test with TestHQ

Critical thinking is about questioning what you already know . It is about understanding how to find the facts and the truth about a situation or argument without being influenced by other people’s opinions .

It is also about looking at the bigger picture and seeing how decisions made now may have short-term benefits but long-term consequences.

For those working in senior managerial roles, this ability to think objectively can make a big difference to business success.

As part of the critical thinking assessment, the Watson Glaser Test focuses on the acronym, 'RED':

• R ecognize assumptions
• E valuate arguments
• D raw conclusions

Put simply, the RED model ensures you can understand how to move beyond subconscious bias in your thinking. It ensures that you can identify the truth and understand the differences between fact and opinion.

To recognize assumptions , you must understand yourself and others: what your thought patterns and past experiences have led you to conclude about the world.

Evaluating arguments requires you to genuinely consider the merits of all options in a situation, and not just choose the one you feel that you ‘ought’ to.

Finally, to draw an accurate and beneficial conclusion you must trust your decision-making and understanding of the situation.

Watson Glaser Practice Test Questions & Answers

As mentioned earlier, the Watson Glaser Test assesses five core elements. Here, they will be examined in more depth:

This part of the test is about your ability to draw conclusions based on facts . These facts may be directly provided or may be assumptions that you have previously made.

Within the assessment, you can expect to be provided with a selection of text. Along with the text will be a statement.

You may need to decide whether that statement is true, probably true, insufficient data (neither true nor false), probably false or false.

The test looks to see if your answer was based on a conclusion that could be inferred from the text provided or if it is based on an assumption you previously made.

Take a Watson Glaser Practice Test

Example Statement:

500 students recently attended a voluntary conference in New York. During the conference, two of the main topics discussed were issues relating to diversity and climate change. This is because these are the two issues that the students selected that are important to them.

Many people make decisions based on assumptions. But you need to be able to identify when assumptions are being made.

Within the Watson Glaser test , you will be provided with a written statement as well as an assumption.

You will be asked to declare whether that assumption was made in the text provided or not .

This is an important part of the test; it allows employers to understand if you have any expectations about whether things are true or not . For roles in law or finance, this is a vital skill.

We need to save money, so we’ll visit the local shops in the nearest town rather than the local supermarket

As a core part of critical thinking, 'deduction' is the ability to use logic and reasoning to come to an informed decision .

You will be presented with several facts, along with a variety of conclusions. You will be tasked with confirming whether those conclusions can be made from the information provided in that statement.

The answers are commonly in a ‘Yes, it follows/No, it does not follow’ form.

It is sometimes sunny on Wednesdays. All sunny days are fun. Therefore…

If you need to prepare for a number of different employment tests and want to outsmart the competition, choose a Premium Membership from TestHQ . You will get access to three PrepPacks of your choice, from a database that covers all the major test providers and employers and tailored profession packs.

Get a Premium Package Now

Critical thinking is also about interpreting the information correctly. It is about using the information provided to come to a valuable, informed decision .

Like the deduction questions, you will be provided with a written statement, which you must assume to be true.

You will also be provided with a suggested interpretation of that written statement. You must decide if that interpretation is correct based on the information provided, using a yes/no format.

A study of toddlers shows that their speech can change significantly between the ages of 10 months and three years old. At 1 year old, a child may learn their first word whereas at three years old they may know 200 words

Evaluation of Arguments

This final part requires you to identify whether an argument is strong or weak . You will be presented with a written statement and several arguments that can be used for or against it. You need to identify which is the strongest argument and which is the weakest based on the information provided.

Should all 18-year-olds go to college to study for a degree after they have graduated from high school?

There are no confirmed pass/fail scores for Watson Glaser tests; different sectors have different interpretations of what is a good score .

Law firms, for example, will require a pass mark of at least 75–80% because the ability to think critically is an essential aspect of working as a lawyer.

As a comparative test, you need to consider what the comparative ‘norm’ is for your chosen profession. Your score will be compared to other candidates taking the test and you need to score better than them.

It is important to try and score as highly as you possibly can. Your Watson Glaser test score can set you apart from other candidates; you need to impress the recruiters as much as possible.

Your best chance of achieving a high score is to practice as much as possible in advance.

Everyone will have their own preferred study methods, and what works for one person may not necessarily work for another.

However, there are some basic techniques everyone can use, which will enhance your study preparation ahead of the test:

Step 1 . Pay Attention to Online Practice Tests

There are numerous free online training aids available; these can be beneficial as a starting point to your preparation.

However, it should be noted that they are often not as detailed as the actual exam questions.

When researching for online test questions, make sure that any questions are specific to the Watson Glaser Test , not just critical thinking.

General critical thinking questions can help you improve your skills but will not familiarize you with this test. Therefore, make sure you practice any questions which follow the ‘rules’ and structure of a Watson Glaser Test .

Step 2 . Paid-for Preparation Packs Can Be Effective

If you are looking for something that mimics the complexity of a Watson Glaser test , you may wish to look at investing in a preparation pack.

There are plenty of options available from sites such as TestHQ . These are often far more comprehensive than free practice tests.

They may also include specific drills (which take you through each of the five stages of the test) as well as study guides, practice tests and suggestions of how to improve your score.

Psychologically, if you have purchased a preparation pack, you may be more inclined to increase your pre-test practice/study when compared to using free tools, due to having invested money.

Step 3 . Apply Critical Thinking to All Aspects of Your Daily Routine

The best way to improve your critical thinking score is to practice it every day.

It is not just about using your skills to pass an exam question; it is about being able to think critically in everyday scenarios.

Therefore, when you are reading the news or online articles, try to think whether you are being given facts or you are making deductions and assumptions from the information provided.

The more you practice your critical thinking in these scenarios, the more it will become second nature to you.

You could revert to the RED model: recognize the assumptions being made, by you and the author; evaluate the arguments and decide which, if any, are strong; and draw conclusions from the information provided and perhaps see if they differ from conclusions drawn using your external knowledge.

Prepare for Watson Glaser Test with TestHQ

Nine Top Tips for Ensuring Success in Your Watson Glaser Test

If you are getting ready to participate in a Watson Glaser test, you must be clear about what you are being asked to do.

Here are a few tips that can help you to improve your Watson Glaser test score.

1. Practice, Practice, Practice

Critical thinking is a skill that should become second nature to you. You should practice as much as possible, not just so that you can pass the test, but also to feel confident in using your skills in reality.

2. The Best Success Is Based on the Long-Term Study

To succeed in your Watson Glaser test , you need to spend time preparing.

Those who begin studying in the weeks and months beforehand will be far more successful than those who leave their study to the last minute.

3. Acquaint Yourself With the Test Format

The Watson Glaser test has a different type of question to other critical thinking tests.

Make sure that you are aware of what to expect from the test questions. The last thing you want is to be surprised on test day.

4. Read the Instructions Carefully

This is one of the simplest but most effective tips. Your critical thinking skills start with understanding what you are being asked to do. Take your time over the question.

Although you may only have 30 minutes to complete the test, it is still important that you do not rush through and submit the wrong answers. You do not get a higher score if you finish early, so use your time wisely.

5. Only Use the Information Provided in the Question

Remember, the purpose of the test is to see if you can come to a decision based on the provided written statement.

This means that you must ignore anything that you think you already know and focus only on the information given in the question.

6. Widen Your Non-Fictional Reading

Reading a variety of journals, newspapers and reports, and watching examples of debates and arguments will help you to improve your skills.

You will start to understand how the same basic facts can be presented in different ways and cause people to draw different conclusions.

From there, you can start to enhance your critical thinking skills to go beyond the perspective provided in any given situation.

7. Be Self-Aware

We all have our own biases and prejudices whether we know them or not. It is important to think about how your own opinions and life experiences may impact how you perceive and understand situations.

For example, someone who has grown up with a lot of money may have a different interpretation of what it is like to go without, compared to someone who has grown up in extreme poverty.

It is important to have this self-awareness as it is important for understanding other people; this is useful if you are working in sectors such as law.

8. Read the Explanations During Your Preparation

To make the most of practice tests, make sure you read the analysis explaining the answers, regardless of if you got the question right or wrong.

This is the crux of your study; it will explain the reasoning why a certain answer is correct, and this will help you understand how to choose the correct answers.

9. Practice Your Timings

You know that you will have five sections to complete in the test. You also know that you have 30 minutes to complete the test.

Therefore, make sure that your timings are in sync within your practice, so you can work your way through the test in its entirety.

Time yourself on how long each section takes you and put in extra work on your slowest.

What score do you need to pass the Watson Glaser test?

There is no standard benchmark score to pass the Watson Glaser test . Each business sector has its own perception of what constitutes a good score and every employer will set its own requirements.

It is wise to aim for a Watson Glaser test score of at least 75%. To score 75% or higher, you will need to correctly answer at least 30 of the 40 questions.

The employing organization will use your test results to compare your performance with other candidates within the selection pool. The higher you score in the Watson Glaser test , the better your chances of being hired.

Can you fail a Watson Glaser test?

It is not possible to fail a Watson Glaser test . However, your score may not be high enough to meet the benchmark set by the employing organization.

By aiming for a score of at least 75%, you stand a good chance of progressing to the next stage of the recruitment process.

Are Watson Glaser tests hard?

Many candidates find the Watson Glaser test hard. The test is designed to assess five different aspects of logical reasoning skills. Candidates must work under pressure, which adds another dimension of difficulty.

By practicing your critical thinking skills, you can improve your chances of achieving a high score on the Watson Glaser test .

How do I prepare for Watson Glaser?

To prepare for Watson Glaser , you will need to practice your critical thinking abilities. This can be achieved through a range of activities; for example, reading a variety of newspapers, journals and other literature.

Try applying the RED model to your reading – recognize the assumptions being made (both by you and the writer), evaluate the arguments and decide which of these (if any) are strong.

You should also practice drawing conclusions from the information available to you.

Online Watson Glaser practice assessments are a useful way to prepare for Watson Glaser. These practice tests will give you an idea of what to expect on the day, although the questions are not usually as detailed as those in the actual test.

You might also consider using a paid-for Watson Glaser preparation pack, such as the one available from TestHQ . Preparation packs provide a comprehensive test guide, including practice tests and recommendations on how to improve your test score.

How long does the Watson Glaser test take?

Candidates are allowed 30 minutes to complete the Watson Glaser test . The multiple-choice test questions are grouped into five distinct areas – assumptions, deduction, evaluation, inference and interpretation.

Which firms use the Watson Glaser test?

Companies all over the world use the Watson Glaser test as part of their recruitment campaigns.

It is a popular choice for professional service firms, including banking, law, and insurance. Firms using the Watson Glaser test include the Bank of England, Hiscox, Deloitte and Clifford Chance.

How many times can you take the Watson Glaser test?

Most employers will only allow you to take the Watson Glaser test once per application. However, you may take the Watson Glaser test more than once throughout your career.

What is the next step after passing the Watson Glaser test?

The next step after passing the Watson Glaser test will vary between employers. Some firms will ask you to attend a face-to-face interview after passing the Watson Glaser test, others will ask you to attend an assessment center. Speak to the hiring manager to find out the process for the firm you are applying for.

Start preparing in advance for the Watson Glaser test

The Watson Glaser test differs from other critical thinking tests. It has its own rules and formations, and the exam is incredibly competitive. If you are asked to participate in a Watson Glaser test it is because your prospective employer is looking for the ‘best of the best’. Your aim is not to simply pass the test; it is to achieve a higher score than anyone else taking that test .

Therefore, taking the time to prepare for the Watson Glaser test is vital for your chances of success. You need to be confident that you know what you are being asked to do, and that you can use your critical thinking skills to make informed decisions.

Your study is about more than helping you to pass a test; it is about providing you with the skills and capability to think critically about information in the ‘real world’ .

You might also be interested in these other Psychometric Success articles:

Or explore the Aptitude Tests / Test Types sections.

Critical thinking definition

Critical thinking, as described by Oxford Languages, is the objective analysis and evaluation of an issue in order to form a judgement.

Active and skillful approach, evaluation, assessment, synthesis, and/or evaluation of information obtained from, or made by, observation, knowledge, reflection, acumen or conversation, as a guide to belief and action, requires the critical thinking process, which is why it's often used in education and academics.

Some even may view it as a backbone of modern thought.

However, it's a skill, and skills must be trained and encouraged to be used at its full potential.

People turn up to various approaches in improving their critical thinking, like:

• Developing technical and problem-solving skills
• Engaging in more active listening
• Actively questioning their assumptions and beliefs
• Seeking out more diversity of thought
• Opening up their curiosity in an intellectual way etc.

Is critical thinking useful in writing?

Critical thinking can help in planning your paper and making it more concise, but it's not obvious at first. We carefully pinpointed some the questions you should ask yourself when boosting critical thinking in writing:

• What information should be included?
• Which information resources should the author look to?
• What degree of technical knowledge should the report assume its audience has?
• What is the most effective way to show information?
• How should the report be organized?
• How should it be designed?
• What tone and level of language difficulty should the document have?

Usage of critical thinking comes down not only to the outline of your paper, it also begs the question: How can we use critical thinking solving problems in our writing's topic?

Let's say, you have a Powerpoint on how critical thinking can reduce poverty in the United States. You'll primarily have to define critical thinking for the viewers, as well as use a lot of critical thinking questions and synonyms to get them to be familiar with your methods and start the thinking process behind it.

Are there any services that can help me use more critical thinking?

We understand that it's difficult to learn how to use critical thinking more effectively in just one article, but our service is here to help.

We are a team specializing in writing essays and other assignments for college students and all other types of customers who need a helping hand in its making. We cover a great range of topics, offer perfect quality work, always deliver on time and aim to leave our customers completely satisfied with what they ordered.

The ordering process is fully online, and it goes as follows:

• Select the topic and the deadline of your essay.
• Provide us with any details, requirements, statements that should be emphasized or particular parts of the essay writing process you struggle with.
• Leave the email address, where your completed order will be sent to.
• Select your prefered payment type, sit back and relax!

With lots of experience on the market, professionally degreed essay writers , online 24/7 customer support and incredibly low prices, you won't find a service offering a better deal than ours.

Secondary Menu

California critical thinking skills test (cctst), instructions.

Please click here to view detailed instructions for accessing and completing the California Critical Thinking Skills Test (CCTST)

If you have any questions about the study or the CCTST questionnaire, you are welcome to contact Dr. Jennifer Hill, Director of the Office of Assessment, at (919) 668-1617 or  [email protected] . 

Click here for instructions to begin the California Critical Thinking Skills Test (CCTST)

• Gen. Ed. Assessment
• Starting the DAP
• Handbook on Assessment at Duke
• Assessment Resources
• Presentations
• Duke's Quality Enhancement Plan (QEP)

California Critical Thinking Skills Test

Get data for Admissions Evaluating Critical Thinking Skills   Student Success Advising National Benchmarking Accreditation Educational Research

The  California Critical Thinking Skills Test (CCTST)  is an educational assessment that measures all the core reasoning skills needed for reflective decision-making. The CCTST provides valid and reliable data on critical thinking skills of individuals and of groups.  It is designed for use with undergraduate and graduate students. It is available in many languages and its OVERALL skills score can be benchmarked using one of many percentile comparisons. Clients most commonly use the  CCTST for admissions, advising and retention, studies of curriculum effectiveness, accreditation, and the documentation of student learning outcomes.

For assessment specs, administration, metrics reported, and more, scroll down. Contact us by using the “Request A Quote” button to ask a question. Or phone us at 650-697-5628 to speak with an assessment services client support specialist.

Seamless Testing. Results You Can Trust.

Higher education.

CCTST is calibrated for undergraduate and graduate level college students across the full spectrum of disciplines and fields of study.

Administration

Administered online with a secure, multi-lingual interface, it’s user-friendly and accessible anywhere.

Support Materials

User Manual includes all needed information about administering the assessment and interpreting the resulting individual and group scores.

Assessment Specs

55 minutes timed administration; 40 engaging, scenario-based questions

Deliverables

Group graphics with statistical summary of scores; Excel spreadsheet of responses to all custom demographic questions, and all scores for each person tested. Optional individual score reports for administrators and/or test takers.

Results Reported

Metrics include scores for 8 critical skills, plus an OVERALL rating. Population percentile scores are available for benchmarking.

All of the CCTST metrics are on a 100-point scale with a corresponding qualitative rating (Superior, Strong, Moderate, Weak, Not Manifested).

Available in English, Arabic, Chinese Simplified, Chinese Traditional, Dutch, French, German, Indonesian-Bahasa, Italian, Japanese, Korean, Norwegian, Portuguese, Spanish, Swedish, Thai, Turkish, and Vietnamese languages.

CCTST provides 8  cognitive skill scores to focus future development and training. Items are drawn from a scientifically developed and tested item pool.

• OVERALL Critical Thinking Skills – Sustained use of critical thinking to form reasoned judgments
• Analysis  – Accurate identification of the problem and decision-critical elements
• Interpretation  – Discovering and determining significance and contextual meaning
• Inference –  Drawing warranted and logical conclusions from reasons and evidence
• Evaluation  – Assessing credibility of claims and the strength of arguments
• Explanation  – Providing the evidence, reasons, assumptions, or rationale for judgments and decisions
• Induction  – Reasoned judgment in ambiguous, risky, and uncertain contexts
• Deduction  – Reasoned judgment in precisely defined, logically rigorous contexts
• Numeracy  – Sustained use of critical thinking skills in quantitative contexts (quantitative reasoning)

The  California Critical Thinking Skills Test (CCTST)  Report Package includes an individual test-taker report for each person assessed and group summary reports for each group and sub-group in the sample.

Reports are generated immediately after the conclusion of testing and are available for clients to download making real time assessment possible. Read more about how our  customer support specialists work with clients to select their reporting options on our Services tab or contact us for a consultation.

Group Analytics

• Clients can generate and download Excel spreadsheet files of all scores (OVERALL, Percentile ranking and all cognitive score metrics). At the option of the client, these also include the responses to custom demographic questions added by the client to the assessment profile, and percentile scores corresponding to the external comparison group selected by the client.
• Presentation-ready tables and graphic representations of the score distribution for OVERALL critical thinking skills and for the additional cognitive skill metrics.
• Customers who have added custom demographic questions can generate sub-group reports for these variables, or for specific testing sessions or time periods.

Optional Individual Test-Taker Reports

• An overall score of critical thinking skills (OVERALL Score). OVERALL is reported on a 100-point scale accompanied by a qualitative rating (Superior, Strong, Moderate, Weak, Not Manifested), and a comparison percentile score.
• Scores for each cognitive skill metric. These metrics are scored on a 100-point scale and are accompanied by a categorical interpretation of the strength of the score indicating areas of strength and areas for future development.
• The Individual Test Taker Report can be pushed to an email address of the client’s choosing (for example, to an admissions office email, institutional assessment email, dean’s office email, etc.).
• The client controls whether individual reports are made available to the test-taker.

Need to expedite your project?  We can have your first online testing assignment available for your students within 24 hours.  Request a Quote or get started by calling 650-697-5628 and speaking with one of our assessment specialists today.

Unlock your exclusive access to our resource library, training tools.

 Understand the depth of our metrics with hands-on tools designed to elucidate the reasoning behind our results. These tools empower you to interpret and apply our data in your professional journey.

Published Articles

Explore how our products are used in real-world scenarios, with comprehensive studies that showcase their impact and effectiveness.

Analytical Reports

Delve into critical thinking insights from our founders, offering fresh perspectives and groundbreaking approaches.

Your subscription includes access to our resource library and periodic emails that keep you informed and ahead in your field. Your privacy is important to us. We promise to keep your information safe and never spam you. You can unsubscribe at any time.

Critical Thinking test

By 123test team . Updated May 12, 2023

Critical Thinking test reviews

This Critical Thinking test measures your ability to think critically and draw logical conclusions based on written information. Critical Thinking tests are often used in job assessments in the legal sector to assess a candidate's  analytical critical  thinking skills. A well known example of a critical thinking test is the Watson-Glaser Critical Thinking Appraisal .

Need more practice?

Score higher on your critical thinking test.

The test comprises of the following five sections with a total of 10 questions:

• Analysing Arguments
• Assumptions
• Interpreting Information

Instructions Critical Thinking test

Each question presents one or more paragraphs of text and a question about the information in the text. It's your job to figure out which of the options is the correct answer.

Below is a statement that is followed by an argument. You should consider this argument to be true. It is then up to you to determine whether the argument is strong or weak. Do not let your personal opinion about the statement play a role in your evaluation of the argument.

Statement: It would be good if people would eat vegetarian more often. Argument: No, because dairy also requires animals to be kept that will have to be eaten again later.

Is this a strong or weak argument?

Strong argument Weak argument

Statement: Germany should no longer use the euro as its currency Argument: No, because that means that the 10 billion Deutschmark that the introduction of the euro has cost is money thrown away.

Overfishing is the phenomenon that too much fish is caught in a certain area, which leads to the disappearance of the fish species in that area. This trend can only be reversed by means of catch reduction measures. These must therefore be introduced and enforced.

Assumption: The disappearance of fish species in areas of the oceans is undesirable.

Is the assumption made from the text?

Assumption is made Assumption is not made

As a company, we strive for satisfied customers. That's why from now on we're going to keep track of how quickly our help desk employees pick up the phone. Our goal is for that phone to ring for a maximum of 20 seconds.

Assumption: The company has tools or ways to measure how quickly help desk employees pick up the phone.

• All reptiles lay eggs
• All reptiles are vertebrates
• All snakes are reptiles
• All vertebrates have brains
• Some reptiles hatch their eggs themselves
• Most reptiles have two lungs
• Many snakes only have one lung
• Cobras are poisonous snakes
• All reptiles are animals

Conclusion: Some snakes hatch their eggs themselves.

Does the conclusion follow the statements?

Conclusion follows Conclusion does not follow

(Continue with the statements from question 5.)

Conclusion: Some animals that lay eggs only have one lung.

In the famous 1971 Stanford experiment, 24 normal, healthy male students were randomly assigned as 'guards' (12) or 'prisoners' (12). The guards were given a uniform and instructed to keep order, but not to use force. The prisoners were given prison uniforms. Soon after the start of the experiment, the guards made up all kinds of sentences for the prisoners. Insurgents were shot down with a fire extinguisher and public undressing or solitary confinement was also a punishment. The aggression of the guards became stronger as the experiment progressed. At one point, the abuses took place at night, because the guards thought that the researchers were not watching. It turned out that some guards also had fun treating the prisoners very cruelly. For example, prisoners got a bag over their heads and were chained to their ankles. Originally, the experiment would last 14 days. However, after six days the experiment was stopped.

The students who took part in the research did not expect to react the way they did in such a situation.

To what extent is this conclusion true, based on the given text?

True Probably true More information required Probably false False

(Continue with the text from 'Stanford experiment' in question 7.)

The results of the experiment support the claim that every young man (or at least some young men) is capable of turning into a sadist fairly quickly.

• A flag is a tribute to the nation and should therefore not be hung outside at night. Hoisting the flag therefore happens at sunrise, bringing it down at sunset. Only when a country flag is illuminated by spotlights on both sides, it may remain hanging after sunset. There is a simple rule of thumb for the time of bringing down the flag. This is the moment when there is no longer any visible difference between the individual colors of the flag.
• A flag may not touch the ground.
• On the Dutch flag, unless entitled to do so, no decorations or other additions should be made. Also the use of a flag purely for decoration should be avoided. However, flag cloth may be used for decoration - for example in the form of drapes.
• The orange pennant is only used on birthdays of members of the Royal House and on King's Day. The orange pennant should be as long or slightly longer than the diagonal of the flag.

Conclusion: One can assume that no Dutch flag will fly at government buildings at night, unless it is illuminated by spotlights on both sides.

Does the conclusion follow, based on the given text?

(Continue with the text from 'Dutch flag protocol' in question 9.)

Conclusion: If the protocol is followed, the orange pennant will always be longer than the horizontal bands/stripes of the flag.

Please answer the questions below. Not all questions are required but it will help us improve this test.

My educational level is

-- please select -- primary school high school college university PhD other

Students’ critical thinking skills using HOTS-based assessment on pH calculation

• Split-Screen
• Article contents
• Figures & tables
• Supplementary Data
• Peer Review
• Open the PDF for in another window
• Reprints and Permissions
• Cite Icon Cite
• Search Site

Ika Farida Yuliana , Fina Faizah , Fatayah Fatayah , Rendy Priyasmika; Students’ critical thinking skills using HOTS-based assessment on pH calculation. AIP Conf. Proc. 24 May 2024; 3106 (1): 040013. https://doi.org/10.1063/5.0215096

Download citation file:

• Ris (Zotero)
• Reference Manager

One of the Indonesian government’s efforts to meet the 21st-century skills of students is to implement the curriculum of 2013. According to the Ministry of Education and Culture, the HOTS writing guidebook states that, in the context of 21 st -century learning, students must learn and master essential skills, including critical thinking, problem- solving, creative and innovative thinking, collaborating and communicating effectively, which are higher order thinking skills. Familiarising students to think critically makes students able to analyse a problem that occurs in society so that they can produce an idea or way to solve these problems. This research is qualitative research with the type of case study research and uses the analysis method of Miles and Huberman. The research subjects in this study were students last XI A and XI B MA Fathul Hidayah Pangean, with a total of 42 students. The results showed that the critical thinking skills of MA Fathul Hidayah Pangean students in classes XI A and XI B were 42 students classified as low, with an average percentage of 26,80%. Judging from the 5 aspects of critical thinking as a whole, it is relatively low. This is because students are not used to solving HOTS-based questions, so even though in mastering the pH calculation material, they can understand the material shown by excellent learning outcomes with a percentage of 82,7%, when faced with HOTS questions, student learning outcomes become 44,3% with medium criteria. This resulted in students’ critical thinking skills being still low.

Citing articles via

Publish with us - request a quote.

Sign up for alerts

• Online ISSN 1551-7616
• Print ISSN 0094-243X
• For Researchers
• For Librarians
• For Advertisers
• Our Publishing Partners  
• Physics Today
• Conference Proceedings
• Special Topics

pubs.aip.org

• Privacy Policy
• Terms of Use

Connect with AIP Publishing

This feature is available to subscribers only.

Sign In or Create an Account