qualitative research measurement tools

Understanding qualitative measurement: The what, why, and how

Last updated

30 January 2024

Reviewed by

Short on time? Get an AI generated summary of this article instead

You’ll need to collect data to determine the success of any project, from product launches to employee culture initiatives. How that data is collected is just as important as what it reveals.

There are many ways to gather and analyze data, from in-person interviews to emailed surveys. Qualitative research focuses on telling a story with the information collected, while quantitative research involves collecting, analyzing, and presenting hard datasets.

Data gathered through qualitative measurement describes traits or characteristics. You can collect it in different ways, including interviews and observation, and it can be in the form of descriptive words.

While gathering and analyzing data through qualitative measurement can be challenging, especially if you’re working with limited resources or a smaller team, the insights you get at the end of the project are often well worth the effort.

• What is qualitative measurement?

Qualitative measures can be particularly helpful in understanding how a phenomenon or action affects individuals and groups.

• Why is qualitative data important?

Through data, you can understand how to better serve your customers and employees and anticipate shifts in your business.

The data will provide a deeper understanding of your customers, empowering you to make decisions that positively benefit your company in the long run. Qualitative data helps you see patterns and trends so you can make actionable changes. It can also answer questions posed by your project so you can provide company stakeholders with helpful information and insights.

• How to collect qualitative data

Your ideal method for collecting qualitative data will depend on the resources you have at your disposal, the size of your team, and your project’s timeline.

You might select one method or a mixture of several. For instance, you could opt to send out surveys following a focus group session to receive additional feedback on one or two specific areas of interest.

Analyze your available resources and discuss options with project stakeholders before committing to one particular plan.

The following are some examples of the methods you could use:

Individual interviews

In-depth interviews are one of the most popular methods of collecting qualitative data. They are usually conducted in person, but you could also use video software.

During interviews, a researcher asks the person questions, logging their answers as they go.

Focus groups

Focus groups are a powerful way to observe and document a group of people, making them a common method for collecting qualitative data. They provide researchers with a direct way to interact with participants, listening to them while they share their insights and experiences and recording responses without the interference of software or third-party systems.

However, while focus groups and interviews are two of the most popular methods, they might not be right for every situation or company.

Direct observation

Direct observation allows researchers to see participants in their natural setting, offering an intriguing “real-life” angle to data collection . This method can provide rich, detailed information about the individuals or groups you are studying.

You can conduct surveys in person or online through web software or email. They can also be as detailed or general as your project requires. To get the most information from your surveys, use open-ended questions that encourage respondents to share their thoughts and opinions on the subject.

Diaries and journals

Product launches or employee experience initiatives are two examples of projects that could benefit from diaries and journals as a form of qualitative data gathering.

Diaries and journals enable participants to record their thoughts and feelings on a particular topic. By later examining the diary entries, project managers and stakeholders can better understand their reactions and opinions on the project and the questions asked.

• Examples of qualitative data

Qualitative data is non-numeric information. It’s descriptive, often including adjectives to paint a picture of a situation or object. Qualitative data can be used to describe a person or place, as you can see in the examples below:

The employee prefers black coffee to sweet beverages.

The cat is black and fluffy.

The brown leather couch is worn and faded.

There are many ways to collate qualitative data, but remember to use appropriate language when communicating it to other project stakeholders. Qualitative data isn’t flowery, but neither does it shy away from descriptors to comprehensively paint a picture.

• How to measure qualitative data

To measure qualitative data, define a clear project scope ahead of time. Know what questions you want answered and what people you need to speak to to make that happen. While not every result can be tallied, by understanding the questions and project scope well in advance, you’ll be better prepared to analyze what you’re querying.

Define the method you wish to use for your project. Whether you opt for surveys, focus groups, or a mixture of methods, employ the approach that will yield the most valuable data.

Work within your means and be realistic about the resources you can dedicate to data collection. For example, if you only have one or two employees to dedicate to the project, don’t commit to multiple focus group meetings with large groups of participants, as it might not be feasible.

• What’s the difference between qualitative and quantitative measurements?

Qualitative measurements are descriptive. You can’t measure them with a ruler, scale, or other numeric value, nor can you express them with a numeric value.

In contrast, quantitative measurements are numeric in nature and can be counted.

• When to use qualitative vs. quantitative measurements

Both qualitative and quantitative measurements can be valuable. Which to use greatly depends on the nature of your project.

If you’re looking to confirm a theory, such as determining which variety of body butter was sold most during a specific month, quantitative measurements will likely give you the answers you need.

To learn more about concepts and experiences, such as which advertising campaign your target customers prefer, opt for qualitative measurement.

You don’t have to commit to one or the other exclusively. Many businesses use a mixed-method approach to research, combining elements of both quantitative and qualitative measurements. Know the questions you want to answer and proceed accordingly with what makes the most sense for your goals.

• What are the best ways to communicate qualitative data?

Communicating the qualitative data you’ve gathered can be tricky. The information is subjective, and many project stakeholders or other involved parties may have an easier time understanding and reacting to numeric data.

To effectively communicate qualitative data, you’ll need to create a compelling storyline that offers context and relevant details.

It can also help to describe the data collection method you used. This not only helps set the stage for your story but gives those listening insight into research methodologies they may be unfamiliar with.

Finally, allow plenty of time for questions. Regardless of whether you’re speaking to your company’s CEO or a fellow project manager, you should be prepared to respond to questions with additional, relevant information.

How can qualitative measurement be expressed through data?

Qualitative data is non-numeric. It is most often expressed through descriptions since it is surveyed or observed rather than counted.

• Challenges associated with qualitative measurement

Any in-depth study or research project requires a time commitment. Depending on the research method you employ, other resources might be required. For instance, you might need to compensate the participants of a focus group in some way.

The time and resources required to undertake qualitative measurement could make it prohibitive for many companies, especially small ones with only a few employees. Outsourcing can also be expensive.

Conducting a cost–benefit analysis could help you decide if qualitative measurement is a worthwhile undertaking or one that should be delayed as you plan and prepare.

Should you be using a customer insights hub?

Do you want to discover previous research faster?

Do you share your research findings with others?

Do you analyze research data?

Start for free today, add your research, and get to key insights faster

Editor’s picks

Last updated: 24 October 2024

Last updated: 11 January 2024

Last updated: 17 January 2024

Last updated: 12 December 2023

Last updated: 30 April 2024

Last updated: 4 July 2024

Last updated: 12 October 2023

Last updated: 5 March 2024

Last updated: 6 March 2024

Last updated: 31 January 2024

Last updated: 23 January 2024

Last updated: 13 May 2024

Last updated: 20 December 2023

Latest articles

Related topics, decide what to build next, log in or sign up.

Get started for free

UpMetrics Blog

Expert insights, trends, and best practices around impact measurement and leveraging actionable data to drive meaningful change.

The Importance of Qualitative Measurement in Driving Social Good

Qualitative measurement is a broad and complicated field of approach necessary to determine the success of a social impact endeavor. To help ensure that your initiative is prosperous, we’ve compiled an overview of measuring qualitative data (and how it differs from quantitative data) and background on the many forms of qualitative data, techniques, and methodologies. Before embarking on qualitative measurement , here are a few key concepts to consider.

What is Qualitative Measurement?

Data acquired through a qualitative measure is a type of information that describes traits or characteristics. It's gathered through surveys, interviews, or observation, and it's usually presented as a story. The qualitative data might be in the form of descriptive words that can be analyzed for patterns or significance using coding. Coding helps the researcher categorize qualitative data and do quantitative analysis by identifying themes related to the research objectives.

How to Collect Qualitative Data

Foundations can collect data for qualitative research in a variety of methods. RCTs (randomized controlled trials) have long been considered the gold standard for measuring effect by government agencies. 

In-depth interviews, in which a researcher asks questions of a person or group touched by a topic, are one form of qualitative assessment. These are determined by the researcher's questions and the study's viability in terms of time and financial resources.  

Qualitative researchers often use direct observation to obtain data. This technique aids investigators in studying phenomena in precise settings as they occur in real life. Researchers can also use written resources to do qualitative research, such as books, periodicals, newspapers, and transcripts. 

Qualitative vs. Quantitative Measurements

Before comparing the differences, let’s first layout both the quantitative and qualitative measures definition :

What are Quantitative Measures?

Numbers and graphs are used in quantitative research. It's utilized to put ideas and assumptions to the test or validate them. This sort of study may be used by businesses to develop generalizable facts about a subject. Experiments, observations recorded as numbers, and surveys with closed-ended questions are all common quantitative approaches.

What are Qualitative Measures?

Qualitative measurements are written down for deciphering ideas, thoughts, and experiences. This sort of study allows you to learn more about issues that aren't well-understood. Interviews with open-ended inquiries, observations recorded in words, and literature reviews that investigate concepts and theories are all common qualitative approaches.

When Should You Use Qualitative vs. Quantitative Measurements

When determining whether to utilize qualitative or quantitative data, a good rule of thumb is:

• If you want to confirm or test a theory or hypothesis, use quantitative research.
• If you want to learn more about concepts, thoughts, experiences, conduct qualitative measurement .

You can use a qualitative, quantitative, or mixed-methods approach to most research issues. Which kind you pick is determined by several factors, including:

• Whether you're conducting inductive or deductive research
• Your research questions
• Whether you're conducting experimental, correlational, or descriptive research

Time, money, data availability, and responders' access are all practical issues that researchers should consider.

Although quantitative approaches generate data that can be pooled and analyzed to characterize and forecast correlations, a qualitative measurement may help explore and explain those relationships as well as contextual variations in their quality.

Qualitative research may use social analytical frameworks to understand observable patterns and trends, including the study of socially differentiated outcomes, and analyze poverty as a dynamic process rather than a static outcome.

How to Measure Qualitative Data

Results must be measured, but not every result can be tallied, recorded, or neatly fit into a framework. Some qualitative outcomes are intangible, such as "empowerment," "confidence," or "capacity." It doesn't imply you can't quantify these things just because they're tough to count. You'll need to measure them in various ways, including qualitative and mixed techniques.

Define the objective

You must first identify what you’re querying before deciding on a technique. What does "empowerment" mean in your program, for example, if it strives to empower women? Does this imply that women have some power over domestic decisions? Does this indicate that they attend community meetings? Or that they have the capability to leave a situation that makes them uncomfortable?

Select a method

To assess qualitative outcomes, you can employ a variety of techniques. Only a few are mentioned here. Combining multiple approaches to obtain diverse viewpoints might be beneficial. It would be ideal if you could alter or change techniques to fit your program's needs.

Interviews and surveys

Qualitative measurement examples are typically generated through the use of interviews and focus groups to meet with recipients and stakeholders directly to discuss their experiences and the program's results for qualitative approaches. Consider what you're measuring and whether a different approach might provide you with more valuable data.

Journals and logbooks

People in charge of a program, participants, and stakeholders can benefit from diaries, logs, and journals. Participants might also be given a journal to keep track of their experiences and ideas. Examining their diary entries may reveal if the training has influenced their thoughts or behaviors.

Photographs and art

Pictures and photographs make for excellent qualitative measures examples and may be used to assess quality, as many people find it simpler to convey changes graphically. Vulnerable youth, for example, can be encouraged to make a painting depicting their lives before and after joining the program.

The Best Ways to Communicate Qualitative Data 

Here are some methods foundations can employ to communicate the data they've acquired from qualitative data: 

Create separate outputs for each target audience

You may wish to reach out to various people, including the general public, legislators, and specialists in a specific sector. Consider producing a series of brief outputs for each of them. A summary prepared for a lay readership may lack the degree of information that a government body requires. You may also wish to highlight facts that are of particular relevance to specific readers.

Link to current events

Qualitative measurement frequently takes a highly in-depth approach to a single research issue but may also interact with broader but related themes. Consider concerns that aren't only part of a short-term media cycle but rather longer-term trends that are likely to resurface, such as housing prices or obesity, whenever feasible. It's not essential to twist your key results to make them fit; simply create a meaningful link.

Create a compelling narrative

Clearly communicating a story is the key to effectively communicating qualitative data. Individuals are more engaged by stories about people than they are by cold facts and numbers. Give your stories context and causality (this occurred to this person as a result of this), and you'll be following the same fundamental guidelines for excellent storytelling that screenwriters and novelists do.

Describe the method

When presenting qualitative data, consider that many people are unfamiliar with qualitative research and the methodologies you may have employed. Provide a brief description of your research methodology. If the reader is intrigued, provide them a means to learn more about it elsewhere, like a publication or a project website. It's preferable to tease the reader and make them desire more than to offer too much information up front. 

Unlock your organization's impact potential with an Impact Framework UpMetrics' next generation Impact Measurement and Management platform makes measuring, optimizing, and showcasing your organization's impact easier than ever before.

As part of our IMM Suite's free Starter Plan, we're excited to offer mission-driven organizations no-cost access to our new Impact Framework Builder functionality so that you can define how you're thinking about impact tied to your mission and vision. Click  here  to register and get started!  

UpMetrics Starter Plan

Access the essentials for free.

UpMetrics' Starter Plan gives you free, unlimited access to foundational features of our platform, perfect for getting started.

These include our impact framework builder, themed framework templates, and the ability to create and publish a public profile page.

Power of Qualitative Measurement: A Comprehensive Guide

Published on, what is qualitative measurement.

Ever feel like you're telling the same old story about your program's impact? Numbers are great, but they don't capture the whole picture, do they?   What if you could truly understand the "why" behind the "what"?

That's where qualitative measurement comes in.  It's about going beyond the numbers and diving deep into the rich tapestry of human experience.  Think interviews, focus groups, and observations – all designed to capture the thoughts, feelings, and motivations of the people your programs touch.

But here's the thing: the world is changing fast.  Those traditional methods, while valuable, can be slow and cumbersome.  They often rely on one-off evaluations, leaving you stuck with a static picture of your program's impact.   What if you could get real-time insights and continuously improve your programs?

Enter continuous learning with AI!   Imagine using cutting-edge technology to automate data collection and analysis.  This isn't just about fancy gadgets – it's about empowering you to:

• Hear stakeholder voices loud and clear:  Gain real-time insights into what matters most to your stakeholders, allowing you to adapt and improve your programs on the fly.
• Demonstrate program effectiveness in a whole new way:  Move beyond static reports and tell a compelling story about the ongoing impact your program is making.
• Secure stakeholder support with confidence:  Show them you're actively listening and continuously working to make a difference.

This article will be your guide to the modern approach to qualitative data collection. We'll break down the limitations of traditional methods and explore how AI can revolutionize how you gather and analyze insights.   By the end, you'll be equipped to harness the power of continuous learning and ensure your programs are constantly evolving to meet the needs of your stakeholders.

Ready to ditch the static and embrace continuous improvement? Let's dive in!

Quantitative vs. Qualitative Data in Impact Measurement

Quantitative data.

• Numerical and measurable
• Provides statistical insights
• Easily comparable across time periods
• Useful for tracking progress and trends

Qualitative Data

• Descriptive and contextual
• Provides depth and nuance
• Captures personal experiences and stories
• Helps understand the "why" behind the numbers

How They Complement Each Other

• Quantitative data provides the "what" and "how much", while qualitative data explains the "why" and "how"
• Combining both types offers a comprehensive view of impact
• Qualitative insights can guide the collection of relevant quantitative data
• Quantitative trends can be explained and enriched by qualitative feedback

Exploring Methodologies in Qualitative Measurement

Unveiling the "why".

We talked about ditching the static and embracing continuous program improvement through qualitative data collection (cue dramatic music!). But how do you actually uncover those rich human experiences that traditional methods often miss?

That's where different methodologies in qualitative measurement come into play. Each approach offers a unique lens for capturing the "why" behind the "what" of your program's impact.  Let's explore some of the most powerful tools in your continuous learning toolbox:

1. Living in Their World: Ethnography

Imagine becoming a fly on the wall, observing a community firsthand. That's the essence of ethnography . This method involves immersing yourself in the context you're studying, participating in activities, and observing behaviors.   By living alongside your stakeholders, you can gain a deeper understanding of their culture, values, and experiences with your program.

2. The Power of Storytelling: Narrative Analysis

People love stories, and for good reason!   Narrative analysis delves into the stories individuals tell about their experiences with your program.  By analyzing the content and structure of these narratives, you can uncover hidden themes, motivations, and emotional responses.  Think of it as unlocking the deeper meaning behind the words.

3. Walking in Their Shoes: Phenomenological Research

Ever wonder, "What's it like to be in their shoes?"   Phenomenological research helps you do just that.  Through in-depth interviews, you explore the lived experiences of your stakeholders in relation to your program. This method gets to the heart of their perceptions and how your program has impacted their lives.

4. Building Your Own Theory: Grounded Theory

Let's say you're starting with a hunch about your program's impact, but need more evidence.   Grounded theory is your friend!  This methodology involves iteratively collecting and analyzing data, allowing a theory to emerge organically from the ground up. You'll be surprised by the rich insights that can surface through this systematic approach.

5. Deep Dives: Case Studies

Need to understand a specific program outcome in detail?   Case studies offer a powerful lens.  By examining a single case or a few closely related cases, you can gain a comprehensive picture of the context, challenges, and successes of your program in action.  This detailed analysis allows you to learn from specific situations and improve your program for future iterations.

Remember, each methodology has its strengths and weaknesses. The best approach often involves combining these techniques to create a well-rounded understanding of your program's impact.

and Why We Need a Modern Approach

We talked about ditching the static and embracing continuous program improvement through qualitative data collection. But let's be honest, some traditional methods can feel...well, a bit traditional.  While ethnography, narrative analysis, and case studies offer valuable insights, they often come with limitations.

The Challenges of Traditional Methods:

• Third-Party Dependence:  These approaches often rely on external researchers, leading to a loss of control and potential for external biases creeping in.
• Time-Consuming Processes:  Traditional methods can take months, even years, to complete, hindering your ability to adapt and improve your program quickly.
• Limited Depth and Frequency:  In-depth interviews and focus groups, while valuable, are typically infrequent, potentially missing the full picture and failing to capture ongoing changes in stakeholder experiences.

The Future of Qualitative Measurement: Continuous Listening

Thankfully, the field of qualitative data collection is evolving!  The future lies in continuous listening to your stakeholders.  Imagine:

• Frequent data collection:  Gather insights regularly through SMS, WhatsApp, or email, keeping your finger on the pulse of stakeholder experiences.
• Shorter, more focused surveys:  Move away from lengthy questionnaires.  Use a mix of closed-ended and open-ended questions to capture both specific data and rich stakeholder perspectives.
• Faster data collection cycles:  Modern tools allow for quicker data collection and analysis, enabling you to adapt your program based on real-time insights.
• Advanced qualitative data analytics:  New technologies unlock deeper understanding by analyzing longitudinal results, combining qualitative and quantitative data, and delivering actionable recommendations.

This future is about gathering frequent, in-the-moment data directly from stakeholders, painting a more complete picture of their experiences and your program's impact.

The methodologies we've discussed are valuable foundations, but the future is continuous.  In the next article, we'll explore how to leverage modern tools and techniques to implement continuous listening and unlock the true power of qualitative data collection for your organization.

Qualitative Measurement Examples

Qualitative measurements provide rich, detailed data about experiences, behaviors, and perceptions that can't be easily quantified. These methods allow researchers to explore the depth and nuance of human experiences and social phenomena. The landscape of qualitative research is evolving, blending traditional techniques with cutting-edge AI-driven approaches.

The following visual guide illustrates seven qualitative measurement techniques, ranging from established methods to innovative AI-powered solutions. These examples showcase how qualitative data collection and analysis are being transformed across various fields, including education, healthcare, and social impact programs.

By understanding these methods, researchers and program managers can choose the most appropriate approach for gathering in-depth, contextual data to answer their research questions and drive meaningful improvements in their initiatives. The integration of AI and continuous data collection methods is opening new possibilities for more responsive, data-driven decision-making in social impact measurement.

The Bottom Line: A New Era of Understanding and Impact

AI-driven qualitative measurement isn't just an upgrade; it's a complete transformation. By embracing this revolution, organizations can ensure their strategies and actions are consistently aligned with evolving needs and perspectives, paving the way for more profound and lasting social change.

Ready to unlock the power of AI-driven qualitative data collection? Stay tuned for the next articles in this series, where we'll delve deeper into specific use cases and practical tips for implementation!

How to Measure Qualitative Data in the Modern Era

Qualitative data goes beyond the numbers, capturing the rich tapestry of human experience. It explores the "why" behind the "what," delving into user attitudes, motivations, and the stories behind the data points [1].  Unlike quantitative data, it's not easily measured with a ruler – but that doesn't mean it's any less valuable.

Modern Approaches to Qualitative Data Collection:

The good news?  Gone are the days of solely relying on time-consuming focus groups and interviews.  Modern qualitative data collection embraces a variety of methods, including:

• Short, Frequent Surveys:  Move away from lengthy questionnaires.  Use a mix of closed-ended and open-ended questions delivered via SMS, WhatsApp, or email to gather real-time insights from a wider range of stakeholders.
• Social Listening:  Tap into the wealth of data available on social media platforms.  Analyze online conversations to understand public sentiment and gain insights into program perception.
• In-App Feedback:  Gather feedback directly within your program or website.  Offer quick surveys or polls to capture user experiences while they're fresh in mind.
• User Testing:  Observe users interacting with your program or product.  See firsthand how they navigate the experience and identify areas for improvement.

Collect open-ended responses that reflect each voice and each experience

Extracting meaning from your data: ai-powered analysis.

Once you've collected your qualitative data, it's time to unlock its potential.  Here's where AI comes into play:

• ‍ Automated Coding and Analysis:  AI tools can quickly categorize and analyze large volumes of text data, identifying recurring themes and patterns that might be missed by human analysts. ‍
• Sentiment Analysis:  Go beyond the words and understand the emotions behind them.  AI can detect positive, negative, or neutral sentiment in your data, providing a deeper understanding of stakeholder attitudes. ‍
• Longitudinal Analysis:  Track changes in sentiment and perceptions over time.  AI helps you identify trends and measure the impact of your program on stakeholders.

How to tackle these challenges:

3. perform qualitative data analysis.

Gather diverse experiences and voices through surveys or interviews

Use AI to identify common themes and patterns (Few minutes)

Review identified themes and their frequencies

Identified Themes and Frequencies:

Popular qualitative data analysis techniques:.

While AI is revolutionizing the field, traditional qualitative data analysis techniques are still valuable tools.  Here are a few key approaches:

• Thematic Analysis:  Identify, analyze, and report patterns (themes) within the data.  This helps you understand the overarching ideas and experiences emerging from your qualitative research.
• Narrative Analysis:  Focus on the stories individuals tell about their experiences with your program.  By analyzing the content and structure of these narratives, you can uncover hidden meanings and motivations.
• Content Analysis:  Categorize and summarize your qualitative data to identify key concepts and trends.  This helps you gain a high-level overview of the main topics and ideas within your data set.

Remember, the best approach often involves a combination of these techniques.  By leveraging modern data collection methods and AI-powered analysis, you can unlock deeper insights from your qualitative data, leading to more informed decision-making and ultimately, greater social impact.

Qualitative measurements provide invaluable insights into human experiences, perceptions, and narratives. By employing methods such as interviews, focus groups, and observations, researchers can capture the rich, complex data that quantitative methods might miss. These insights are crucial for understanding the underlying motivations and challenges within various social phenomena. Integrating qualitative data into research enables a deeper, more nuanced analysis, fostering more informed and effective decision-making processes. Ultimately, qualitative measurements enrich our understanding and contribute to more holistic and impactful outcomes.

Looking for something else?

Still need help.

• UNC Libraries
• HSL Subject Research
• Qualitative Research Resources
• Assessing Qualitative Research

Qualitative Research Resources: Assessing Qualitative Research

Created by health science librarians.

• What is Qualitative Research?
• Qualitative Research Basics
• Special Topics
• Training Opportunities: UNC & Beyond
• Help at UNC
• Qualitative Software for Coding/Analysis
• Software for Audio, Video, Online Surveys
• Finding Qualitative Studies

About this Page

Legend (let evidence guide every new decision) assessment tools: cincinnati children's hospital, equator network: enhancing the quality and transparency of health research, other tools for assessing qualitative research.

• Writing Up Your Research
• Integrating Qualitative Research into Systematic Reviews
• Publishing Qualitative Research
• Presenting Qualitative Research
• Qualitative & Libraries: a few gems
• Data Repositories

Why is this information important?

• Qualitative research typically focuses on collecting very detailed information on a few cases and often addresses meaning, rather than objectively identifiable factors.
• This means that typical markers of research quality for quantitative studies, such as validity and reliability, cannot be used to assess qualitative research.

On this page you'll find:

The resources on this page will guide you to some of the alternative measures/tools or means you can use to assess qualitative research.

Evidence Evaluation Tools and Resources

This website has a number of resources for evaluating health sciences research across a variety of designs/study types, including an Evidence Appraisal form for qualitative research (in table), as well as forms for mixed methods studies from a variety of clinical question domains. The site includes information on the following:

• Evaluating the Evidence Algorithm (pdf download)
• Evidence Appraisal Forms ( see Domain of Clinical Questions Table )
• Table of Evidence Levels (pdf download)
• Grading a Body of Evidence (pdf download)
• Judging the Strength of a Recommendation (pdf download)
• LEGEND Glossary (pdf download)
• EQUATOR: Qualitative Research Reporting Guidelines
• EQUATOR Network Home

The EQUATOR Network is an ‘umbrella’ organisation that brings together researchers, medical journal editors, peer reviewers, developers of reporting guidelines, research funding bodies and other collaborators with mutual interest in improving the quality of research publications and of research itself. 

The EQUATOR Library contains a comprehensive searchable database of reporting guidelines for many study types--including qualitative--and also links to other resources relevant to research reporting:

• Library for health research reporting:  provides an up-to-date collection of guidelines and policy documents related to health research reporting. These are aimed mainly at authors of research articles, journal editors, peer reviewers and reporting guideline developers.
• Toolkits to support writing research, using guidelines, teaching research skills, selecting the appropriate reporting guideline
• Courses and events
• Librarian Network

Also see Articles box, below, some of which contain checklists or tools. 

Most checklists or tools are meant to help you think critically and systematically when appraising research.  Users should generally consult accompanying materials such as manuals, handbooks, and cited literature to use these tools appropriately.  Broad understanding of the variety and complexity of qualitative research is generally necessary, along with an understanding of the philosophical perspectives plus knowledge about specific qualitative research methods and their implementation.  

• CASP/Critical Assessment Skills Programme Tool for Evaluating Qualitative Research 2018
• CASP Knowledge Hub Includes critical appraisal checklists for key study designs; glossary of key research terms; key links related to evidence based healthcare, statistics, and research; a bibliography of articles and research papers about CASP and other critical appraisal tools and approaches 1993-2012.
• (Joanna Briggs Institute) Manual for Evidence Synthesis (2024) See the following chapters: Chapter 2: Systematic reviews of qualitative evidence. Includes appendices: • Appendix 2.1: Critical Appraisal Checklist for Qualitative Research • Appendix 2.2: Discussion of Qualitative critical appraisal criteria • Appendix 2.3 Qualitative data extraction tool Chapter 8: Mixed methods systematic reviews more... less... Aromataris E, Munn Z (Editors). JBI Manual for Evidence Synthesis. JBI, 2020. Available from https://synthesismanual.jbi.global. https://doi.org/10.46658/JBIMES-20-01
• McGill Mixed Methods Appraisal Tool (MMAT) Front Page Public wiki site for the MMAT: The MMAT is intended to be used as a checklist for concomitantly appraising and/or describing studies included in systematic mixed studies reviews (reviews including original qualitative, quantitative and mixed methods studies). The MMAT was first published in 2009. Since then, it has been validated in several studies testing its interrater reliability, usability and content validity. The latest version of the MMAT was updated in 2018.
• McGill Mixed Methods Appraisal Tool (MMAT) 2018 User Guide See full site (public wiki link above) for additional information, including FAQ's, references and resources, earlier versions, and more.
• McMaster University Critical Review Form & Guidelines for Qualitative Studies v2.0 Includes links to Qualitative Review Form (v2.0) and accompanying Guidelines from the Evidence Based Practice Research Group of McMaster University's School of Rehabilitation Science). Links are also provided for Spanish, German, and French versions.
• NICE Quality Appraisal Checklist-Qualitative Studies, 3rd ed, 2012, from UK National Institute for Health and Care Excellence Includes checklist and notes on its use. From Methods for the Development of NICE Public Health Guidance, 3rd edition. more... less... Produced by the National Institute for Health and Clinical Excellence © Copyright National Institute for Health and Clinical Excellence, 2006 (updated 2012). All rights reserved. This material may be freely reproduced for educational and not-for-profit purposes. No reproduction by or for commercial organisations, or for commercial purposes, is allowed without the express written permission of the Institute.
• NICE Quality Appraisal Checklist-Qualitative Studies, 3rd ed. (.pdf download) Appendix H Checklist and Notes download. © Copyright National Institute for Health and Clinical Excellence, 2006 (updated 2012). All rights reserved. This material may be freely reproduced for educational and not-for-profit purposes. No reproduction by or for commercial organisations, or for commercial purposes, is allowed without the express written permission of the Institute.
• Qualitative Research Review Guidelines, RATS
• SBU Swedish Agency for Health Technology Assessment and Assessment of Social Services Evaluation and synthesis of studies using qualitative methods of analysis, 2016. Appendix 2 of this document (at the end) contains a checklist for evaluating qualitative research. more... less... SBU. Evaluation and synthesis of studies using qualitative methods of analysis. Stockholm: Swedish Agency for Health Technology Assessment and Assessment of Social Services (SBU); 2016.
• Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice, 3rd ed (JAMA Evidence) Chapter 13.5 Qualitative Research
• Slides: Appraising Qualitative Research from Users' Guide to the Medical Literature, 3rd edition Click on the 'Related Content' tab to find the link to download the Appraising Qualitative Research slides.

These articles address a range of issues related to understanding and evaluating qualitative research; some  include checklists or tools.

Clissett, P. (2008) "Evaluating Qualitative Research." Journal of Orthopaedic Nursing 12: 99-105.

Cohen, Deborah J. and Benjamin F. Crabtree. (2008) "Evidence for Qualitative Research in Health Care: Controversies and Recommendations." Annals of Family Medicine 6(4): 331-339.

• Supplemental Appendix 1. Search Strategy for Criteria for Qualitative Research in Health Care
• Supplemental Appendix 2. Publications Analyzed: Health Care Journals and Frequently Referenced Books and Book Chapters (1980-2005) That Posited Criteria for "Good" Qualitative Research.

Dixon-Woods, M.,  R.L. Shaw, S. Agarwal, and J.A. Smith. (2004) "The Problem of Appraising Qualitative Research." Qual Safe Health Care 13: 223-225.

Fossey, E., C. Harvey, F. McDermott, and L. Davidson. (2002) "Understanding and Evaluating Qualitative Research." Australian and New Zealand Journal of Psychiatry 36(6): 717-732.

Hammarberg, K., M. Kirkman, S. de Lacey. (2016) "Qualitative Research Methods: When to Use and How to Judge them." Human Reproduction 31 (3): 498-501.

Lee, J. (2014) "Genre-Appropriate Judgments of Qualitative Research." Philosophy of the Social Sciences 44(3): 316-348. (This provides 3 strategies for evaluating qualitative research, 2 that the author is not crazy about and one that he considers more appropriate/accurate).

Majid, Umair and Vanstone,Meredith (2018). "Appraising Qualitative Research for Evidence Syntheses: A Compendium of Quality Appraisal Tools." Qualitative Health Research  28(13): 2115-2131.   PMID: 30047306 DOI:  10.1177/1049732318785358

Meyrick, Jane. (2006) "What is Good Qualitative Research? A First Step towards a Comprehensive Approach to Judging Rigour/Quality." Journal of Health Psychology 11(5): 799-808.

Miles, MB, AM Huberman, J Saldana. (2014) Qualitative Data Analysis.  Thousand Oaks, Califorinia, SAGE Publications, Inc. Chapter 11: Drawing and Verifying Conclusions . Check Availability of Print Book . 

Morse, JM. (1997) "Perfectly Healthy but Dead:"The Myth of Inter-Rater Reliability. Qualitative Health Research 7(4): 445-447.  

O’Brien BC, Harris IB, Beckman TJ, et al. (2014) Standards for reporting qualitative research: a synthesis of recommendations . Acad Med 89(9):1245–1251. DOI: 10.1097/ACM.0000000000000388 PMID: 24979285

The Standards for Reporting Qualitative Research (SRQR) consists of 21 items. The authors define and explain key elements of each item and provide examples from recently published articles to illustrate ways in which the standards can be met. The SRQR aims to improve the transparency of all aspects of qualitative research by providing clear standards for reporting qualitative research. These standards will assist authors during manuscript preparation, editors and reviewers in evaluating a manuscript for potential publication, and readers when critically appraising, applying, and synthesizing study findings.

Ryan, Frances, Michael Coughlin, and Patricia Cronin. (2007) "Step by Step Guide to Critiquing Research: Part 2, Qualitative Research." British Journal of Nursing 16(12): 738-744.

Stige, B, K. Malterud, and T. Midtgarden. (2009) "Toward an Agenda for Evaluation of Qualitative Research." Qualitative Health Research 19(10): 1504-1516.

Tong, Allison and Mary Amanda Dew. (2016-EPub ahead of print). "Qualitative Research in Transplantation: Ensuring Relevance and Rigor. "   Transplantation 

Allison Tong, Peter Sainsbury, Jonathan Craig; Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups , International Journal for Quality in Health Care , Volume 19, Issue 6, 1 December 2007, Pages 349–357, https://doi.org/10.1093/intqhc/mzm042

The criteria included in COREQ, a 32-item checklist, can help researchers to report important aspects of the research team, study methods, context of the study, findings, analysis and interpretations. Items most frequently included in the checklists related to sampling method, setting for data collection, method of data collection, respondent validation of findings, method of recording data, description of the derivation of themes and inclusion of supporting quotations. We grouped all items into three domains: (i) research team and reflexivity, (ii) study design and (iii) data analysis and reporting.

Tracy, Sarah (2010) “Qualitative Quality: Eight ‘Big-Tent’ Criteria for Excellent Qualitative Research.” Qualitative Inquiry 16(10):837-51

• Critical Appraisal Skills Programme
• IMPSCI (Implementation Science) Tutorials
• Johns Hopkins: Why Mixed Methods?
• Measuring, Learning, and Evaluation Project for the Urban Reproductive Health Initiative This project ran 2010-2015. Some project resources are still available.
• NIH OBSSR (Office of Behavioral & Social Sciences Research) Best Practices for Mixed Methods Research in Health Sciences, 2011 The OBSSR commissioned a team in 2010 to develop a resource that would provide guidance to NIH investigators on how to rigorously develop and evaluate mixed methods research applications. more... less... John W. Creswell, Ph.D., University of Nebraska-Lincoln Ann Carroll Klassen, Ph.D., Drexel University Vicki L. Plano Clark, Ph.D., University of Nebraska-Lincoln Katherine Clegg Smith, Ph.D., Johns Hopkins University With the Assistance of a Specially Appointed Working Group
• NIH OBSSR eSource: Introductory Social and Behavioral Science Training Materials eSource is a collection of online chapters that provide an introduction to selected behavioral and social science research approaches, including theory development and testing, survey methods, measurement, and study design. more... less... Link not working on OBSSR website, here https://obssr.od.nih.gov/about-us/publications/ Formerly: https://obssr-archive.od.nih.gov/pdf/Qualitative.PDF
• NSF Workshop on Interdisciplinary Standards for Systematic Qualitative Research On May 19-20, 2005, a workshop on Interdisciplinary Standards for Systematic Qualitative Research was held at the National Science Foundation (NSF) in Arlington, Virginia. The workshop was cofunded by a grant from four NSF Programs—Cultural Anthropology, Law and Social Science, Political Science, and Sociology… It is well recognized that each of the four disciplines have different research design and evaluation cultures as well as considerable variability in the emphasis on interpretation and explanation, commitment to constructivist and positivist epistemologies, and the degree of perceived consensus about the value and prominence of qualitative research methods. more... less... Within this multidisciplinary and multimethods context, twenty-four scholars from the four disciplines were charged to (1) articulate the standards used in their particular field to ensure rigor across the range of qualitative methodological approaches;1* (2) identify common criteria shared across the four disciplines for designing and evaluating research proposals and fostering multidisciplinary collaborations; and (3) develop an agenda for strengthening the tools, training, data, research design, and infrastructure for research using qualitative approaches.
• Technical Note: Mixed-Methods Evaluations (USAID) This open source resource from USAID (2013) discusses the mixing of qualitative and quantitative methods in mixed methods research.
• Qualitative Research Methods: A Data Collector's Field Guide (2005) From FHI 360/Family Health International with support from US AID. Natasha Mack, Cynthia Woodsong, Kathleen M. MacQueen, Greg Guest, and Emily Name. The guide is divided into five modules covering the following topics: Module 1 – Qualitative Research Methods Overview Module 2 – Participant Observation Module 3 – In-Depth Interviews Module 4 – Focus Groups Module 5 – Data Documentation and Management
• Robert Wood Johnson Foundation Guidelines for Designing, Analyzing, and Reporting Qualitative Research
• Robert Wood Johnson Foundation: Qualitative Research Guidelines Project

Qualitative Literacy

Not a checklist, this is a thorough discussion of assessing the scientific merit of a study based on in-depth interviews or participant observation, first by assessing exposure (e.g. time exposed in the field). Then, assuming sufficient exposure, the authors propose looking for signs of

• cognitive empathy, how those interviewed or observed perceive themselves and their social world, the meaning they attach to those perceptions, the motives they express for their actions 
• palpability: the evidence would be palpable or concrete rather than abstract or general
• heterogeneity: showing diversity across people, over time, among situations, or between contexts 
• follow-up: responding to the unexpected; following up on unanticipated statements or observations 
• self-awareness: showing that the author is explicitly aware of the impact of their presence on who was accessed and what they disclosed
• << Previous: Finding Qualitative Studies
• Next: Writing Up Your Research >>
• Last Updated: Oct 23, 2024 4:27 PM
• URL: https://guides.lib.unc.edu/qual