secondary research article

Root out friction in every digital experience, super-charge conversion rates, and optimize digital self-service

Uncover insights from any interaction, deliver AI-powered agent coaching, and reduce cost to serve

Increase revenue and loyalty with real-time insights and recommendations delivered to teams on the ground

Know how your people feel and empower managers to improve employee engagement, productivity, and retention

Take action in the moments that matter most along the employee journey and drive bottom line growth

Whatever they’re are saying, wherever they’re saying it, know exactly what’s going on with your people

Get faster, richer insights with qual and quant tools that make powerful market research available to everyone

Run concept tests, pricing studies, prototyping + more with fast, powerful studies designed by UX research experts

Track your brand performance 24/7 and act quickly to respond to opportunities and challenges in your market

Explore the platform powering Experience Management

• Free Account
• For Digital
• For Customer Care
• For Human Resources
• For Researchers
• Financial Services
• All Industries

Popular Use Cases

• Customer Experience
• Employee Experience
• Employee Exit Interviews
• Net Promoter Score
• Voice of Customer
• Customer Success Hub
• Product Documentation
• Training & Certification
• XM Institute
• Popular Resources
• Customer Stories

Market Research

• Artificial Intelligence
• Partnerships
• Marketplace

The annual gathering of the experience leaders at the world’s iconic brands building breakthrough business results, live in Salt Lake City.

• English/AU & NZ
• Español/Europa
• Español/América Latina
• Português Brasileiro
• REQUEST DEMO
• Experience Management
• Secondary Research

Try Qualtrics for free

Secondary research: definition, methods, & examples.

19 min read This ultimate guide to secondary research helps you understand changes in market trends, customers buying patterns and your competition using existing data sources.

In situations where you’re not involved in the data gathering process ( primary research ), you have to rely on existing information and data to arrive at specific research conclusions or outcomes. This approach is known as secondary research.

In this article, we’re going to explain what secondary research is, how it works, and share some examples of it in practice.

Free eBook: The ultimate guide to conducting market research

What is secondary research?

Secondary research, also known as desk research, is a research method that involves compiling existing data sourced from a variety of channels . This includes internal sources (e.g.in-house research) or, more commonly, external sources (such as government statistics, organizational bodies, and the internet).

Secondary research comes in several formats, such as published datasets, reports, and survey responses , and can also be sourced from websites, libraries, and museums.

The information is usually free — or available at a limited access cost — and gathered using surveys , telephone interviews, observation, face-to-face interviews, and more.

When using secondary research, researchers collect, verify, analyze and incorporate it to help them confirm research goals for the research period.

As well as the above, it can be used to review previous research into an area of interest. Researchers can look for patterns across data spanning several years and identify trends — or use it to verify early hypothesis statements and establish whether it’s worth continuing research into a prospective area.

How to conduct secondary research

There are five key steps to conducting secondary research effectively and efficiently:

1.    Identify and define the research topic

First, understand what you will be researching and define the topic by thinking about the research questions you want to be answered.

Ask yourself: What is the point of conducting this research? Then, ask: What do we want to achieve?

This may indicate an exploratory reason (why something happened) or confirm a hypothesis. The answers may indicate ideas that need primary or secondary research (or a combination) to investigate them.

2.    Find research and existing data sources

If secondary research is needed, think about where you might find the information. This helps you narrow down your secondary sources to those that help you answer your questions. What keywords do you need to use?

Which organizations are closely working on this topic already? Are there any competitors that you need to be aware of?

Create a list of the data sources, information, and people that could help you with your work.

3.    Begin searching and collecting the existing data

Now that you have the list of data sources, start accessing the data and collect the information into an organized system. This may mean you start setting up research journal accounts or making telephone calls to book meetings with third-party research teams to verify the details around data results.

As you search and access information, remember to check the data’s date, the credibility of the source, the relevance of the material to your research topic, and the methodology used by the third-party researchers. Start small and as you gain results, investigate further in the areas that help your research’s aims.

4.    Combine the data and compare the results

When you have your data in one place, you need to understand, filter, order, and combine it intelligently. Data may come in different formats where some data could be unusable, while other information may need to be deleted.

After this, you can start to look at different data sets to see what they tell you. You may find that you need to compare the same datasets over different periods for changes over time or compare different datasets to notice overlaps or trends. Ask yourself: What does this data mean to my research? Does it help or hinder my research?

5.    Analyze your data and explore further

In this last stage of the process, look at the information you have and ask yourself if this answers your original questions for your research. Are there any gaps? Do you understand the information you’ve found? If you feel there is more to cover, repeat the steps and delve deeper into the topic so that you can get all the information you need.

If secondary research can’t provide these answers, consider supplementing your results with data gained from primary research. As you explore further, add to your knowledge and update your findings. This will help you present clear, credible information.

Primary vs secondary research

Unlike secondary research, primary research involves creating data first-hand by directly working with interviewees, target users, or a target market. Primary research focuses on the method for carrying out research, asking questions, and collecting data using approaches such as:

• Interviews (panel, face-to-face or over the phone)
• Questionnaires or surveys
• Focus groups

Using these methods, researchers can get in-depth, targeted responses to questions, making results more accurate and specific to their research goals. However, it does take time to do and administer.

Unlike primary research, secondary research uses existing data, which also includes published results from primary research. Researchers summarize the existing research and use the results to support their research goals.

Both primary and secondary research have their places. Primary research can support the findings found through secondary research (and fill knowledge gaps), while secondary research can be a starting point for further primary research. Because of this, these research methods are often combined for optimal research results that are accurate at both the micro and macro level.

Sources of Secondary Research

There are two types of secondary research sources: internal and external. Internal data refers to in-house data that can be gathered from the researcher’s organization. External data refers to data published outside of and not owned by the researcher’s organization.

Internal data

Internal data is a good first port of call for insights and knowledge, as you may already have relevant information stored in your systems. Because you own this information — and it won’t be available to other researchers — it can give you a competitive edge . Examples of internal data include:

• Database information on sales history and business goal conversions
• Information from website applications and mobile site data
• Customer-generated data on product and service efficiency and use
• Previous research results or supplemental research areas
• Previous campaign results

External data

External data is useful when you: 1) need information on a new topic, 2) want to fill in gaps in your knowledge, or 3) want data that breaks down a population or market for trend and pattern analysis. Examples of external data include:

• Government, non-government agencies, and trade body statistics
• Company reports and research
• Competitor research
• Public library collections
• Textbooks and research journals
• Media stories in newspapers
• Online journals and research sites

Three examples of secondary research methods in action

How and why might you conduct secondary research? Let’s look at a few examples:

1.    Collecting factual information from the internet on a specific topic or market

There are plenty of sites that hold data for people to view and use in their research. For example, Google Scholar, ResearchGate, or Wiley Online Library all provide previous research on a particular topic. Researchers can create free accounts and use the search facilities to look into a topic by keyword, before following the instructions to download or export results for further analysis.

This can be useful for exploring a new market that your organization wants to consider entering. For instance, by viewing the U.S Census Bureau demographic data for that area, you can see what the demographics of your target audience are , and create compelling marketing campaigns accordingly.

2.    Finding out the views of your target audience on a particular topic

If you’re interested in seeing the historical views on a particular topic, for example, attitudes to women’s rights in the US, you can turn to secondary sources.

Textbooks, news articles, reviews, and journal entries can all provide qualitative reports and interviews covering how people discussed women’s rights. There may be multimedia elements like video or documented posters of propaganda showing biased language usage.

By gathering this information, synthesizing it, and evaluating the language, who created it and when it was shared, you can create a timeline of how a topic was discussed over time.

3.    When you want to know the latest thinking on a topic

Educational institutions, such as schools and colleges, create a lot of research-based reports on younger audiences or their academic specialisms. Dissertations from students also can be submitted to research journals, making these places useful places to see the latest insights from a new generation of academics.

Information can be requested — and sometimes academic institutions may want to collaborate and conduct research on your behalf. This can provide key primary data in areas that you want to research, as well as secondary data sources for your research.

Advantages of secondary research

There are several benefits of using secondary research, which we’ve outlined below:

• Easily and readily available data – There is an abundance of readily accessible data sources that have been pre-collected for use, in person at local libraries and online using the internet. This data is usually sorted by filters or can be exported into spreadsheet format, meaning that little technical expertise is needed to access and use the data.
• Faster research speeds – Since the data is already published and in the public arena, you don’t need to collect this information through primary research. This can make the research easier to do and faster, as you can get started with the data quickly.
• Low financial and time costs – Most secondary data sources can be accessed for free or at a small cost to the researcher, so the overall research costs are kept low. In addition, by saving on preliminary research, the time costs for the researcher are kept down as well.
• Secondary data can drive additional research actions – The insights gained can support future research activities (like conducting a follow-up survey or specifying future detailed research topics) or help add value to these activities.
• Secondary data can be useful pre-research insights – Secondary source data can provide pre-research insights and information on effects that can help resolve whether research should be conducted. It can also help highlight knowledge gaps, so subsequent research can consider this.
• Ability to scale up results – Secondary sources can include large datasets (like Census data results across several states) so research results can be scaled up quickly using large secondary data sources.

Disadvantages of secondary research

The disadvantages of secondary research are worth considering in advance of conducting research :

• Secondary research data can be out of date – Secondary sources can be updated regularly, but if you’re exploring the data between two updates, the data can be out of date. Researchers will need to consider whether the data available provides the right research coverage dates, so that insights are accurate and timely, or if the data needs to be updated. Also, fast-moving markets may find secondary data expires very quickly.
• Secondary research needs to be verified and interpreted – Where there’s a lot of data from one source, a researcher needs to review and analyze it. The data may need to be verified against other data sets or your hypotheses for accuracy and to ensure you’re using the right data for your research.
• The researcher has had no control over the secondary research – As the researcher has not been involved in the secondary research, invalid data can affect the results. It’s therefore vital that the methodology and controls are closely reviewed so that the data is collected in a systematic and error-free way.
• Secondary research data is not exclusive – As data sets are commonly available, there is no exclusivity and many researchers can use the same data. This can be problematic where researchers want to have exclusive rights over the research results and risk duplication of research in the future.

When do we conduct secondary research?

Now that you know the basics of secondary research, when do researchers normally conduct secondary research?

It’s often used at the beginning of research, when the researcher is trying to understand the current landscape . In addition, if the research area is new to the researcher, it can form crucial background context to help them understand what information exists already. This can plug knowledge gaps, supplement the researcher’s own learning or add to the research.

Secondary research can also be used in conjunction with primary research. Secondary research can become the formative research that helps pinpoint where further primary research is needed to find out specific information. It can also support or verify the findings from primary research.

You can use secondary research where high levels of control aren’t needed by the researcher, but a lot of knowledge on a topic is required from different angles.

Secondary research should not be used in place of primary research as both are very different and are used for various circumstances.

Questions to ask before conducting secondary research

Before you start your secondary research, ask yourself these questions:

• Is there similar internal data that we have created for a similar area in the past?

If your organization has past research, it’s best to review this work before starting a new project. The older work may provide you with the answers, and give you a starting dataset and context of how your organization approached the research before. However, be mindful that the work is probably out of date and view it with that note in mind. Read through and look for where this helps your research goals or where more work is needed.

• What am I trying to achieve with this research?

When you have clear goals, and understand what you need to achieve, you can look for the perfect type of secondary or primary research to support the aims. Different secondary research data will provide you with different information – for example, looking at news stories to tell you a breakdown of your market’s buying patterns won’t be as useful as internal or external data e-commerce and sales data sources.

• How credible will my research be?

If you are looking for credibility, you want to consider how accurate the research results will need to be, and if you can sacrifice credibility for speed by using secondary sources to get you started. Bear in mind which sources you choose — low-credibility data sites, like political party websites that are highly biased to favor their own party, would skew your results.

• What is the date of the secondary research?

When you’re looking to conduct research, you want the results to be as useful as possible , so using data that is 10 years old won’t be as accurate as using data that was created a year ago. Since a lot can change in a few years, note the date of your research and look for earlier data sets that can tell you a more recent picture of results. One caveat to this is using data collected over a long-term period for comparisons with earlier periods, which can tell you about the rate and direction of change.

• Can the data sources be verified? Does the information you have check out?

If you can’t verify the data by looking at the research methodology, speaking to the original team or cross-checking the facts with other research, it could be hard to be sure that the data is accurate. Think about whether you can use another source, or if it’s worth doing some supplementary primary research to replicate and verify results to help with this issue.

We created a front-to-back guide on conducting market research, The ultimate guide to conducting market research , so you can understand the research journey with confidence.

In it, you’ll learn more about:

• What effective market research looks like
• The use cases for market research
• The most important steps to conducting market research
• And how to take action on your research findings

Download the free guide for a clearer view on secondary research and other key research types for your business.

Related resources

Market intelligence 10 min read, marketing insights 11 min read, ethnographic research 11 min read, qualitative vs quantitative research 13 min read, qualitative research questions 11 min read, qualitative research design 12 min read, primary vs secondary research 14 min read, request demo.

Ready to learn more about Qualtrics?

Identifying Primary and Secondary Research Articles

• Primary and Secondary

Primary Research Articles

Primary research articles report on a single study. In the health sciences, primary research articles generally describe the following aspects of the study:

• The study's hypothesis or research question
• Some articles will include information on how participants were recruited or identified, as well as additional information about participants' sex, age, or race/ethnicity
• A "methods" or "methodology" section that describes how the study was performed and what the researchers did
• Results and conclusion section

Secondary Research Articles

Review articles are the most common type of secondary research article in the health sciences. A review article is a summary of previously published research on a topic. Authors who are writing a review article will search databases for previously completed research and summarize or synthesize those articles,  as opposed to recruiting participants and performing a new research study.

Specific types of review articles include:

• Systematic Reviews
• Meta-Analysis
• Narrative Reviews
• Integrative Reviews
• Literature Reviews

Review articles often report on the following:

• The hypothesis, research question, or review topic
• Databases searched-- authors should clearly describe where and how they searched for the research included in their reviews
• Systematic Reviews and Meta-Analysis should provide detailed information on the databases searched and the search strategy the authors used.Selection criteria-- the researchers should describe how they decided which articles to include
• A critical appraisal or evaluation of the quality of the articles included (most frequently included in systematic reviews and meta-analysis)
• Discussion, results, and conclusions

Determining Primary versus Secondary Using the Database Abstract

Information found in PubMed, CINAHL, Scopus, and other databases can help you determine whether the article you're looking at is primary or secondary.

Primary research article abstract

• Note that in the "Objectives" field, the authors describe their single, individual study.
• In the materials and methods section, they describe the number of patients included in the study and how those patients were divided into groups.
• These are all clues that help us determine this abstract is describing is a single, primary research article, as opposed to a literature review.
• Primary Article Abstract

Secondary research/review article abstract

• Note that the words "systematic review" and "meta-analysis" appear in the title of the article
• The objectives field also includes the term "meta-analysis" (a common type of literature review in the health sciences)
• The "Data Source" section includes a list of databases searched
• The "Study Selection" section describes the selection criteria
• These are all clues that help us determine that this abstract is describing a review article, as opposed to a single, primary research article.
• Secondary Research Article

• Primary vs. Secondary Worksheet

Full Text Challenge

Can you determine if the following articles are primary or secondary?

• Last Updated: Feb 17, 2024 5:25 PM
• URL: https://library.usfca.edu/primary-secondary

2130 Fulton Street San Francisco, CA 94117-1080 415-422-5555

• Facebook (link is external)
• Instagram (link is external)
• Twitter (link is external)
• YouTube (link is external)
• Consumer Information
• Privacy Statement
• Web Accessibility

Copyright © 2022 University of San Francisco

Secondary Research Guide: Definition, Methods, Examples

Apr 3, 2024

8 min. read

The internet has vastly expanded our access to information, allowing us to learn almost anything about everything. But not all market research is created equal , and this secondary research guide explains why.

There are two key ways to do research. One is to test your own ideas, make your own observations, and collect your own data to derive conclusions. The other is to use secondary research — where someone else has done most of the heavy lifting for you. 

Here’s an overview of secondary research and the value it brings to data-driven businesses.

Secondary Research Definition: What Is Secondary Research?

Primary vs Secondary Market Research

What Are Secondary Research Methods?

Advantages of secondary research, disadvantages of secondary research, best practices for secondary research, how to conduct secondary research with meltwater.

Secondary research definition: The process of collecting information from existing sources and data that have already been analyzed by others.

Secondary research (aka desk research ) provides a foundation to help you understand a topic, with the goal of building on existing knowledge. They often cover the same information as primary sources, but they add a layer of analysis and explanation to them.

Users can choose from several secondary research types and sources, including:

• Journal articles
• Research papers

With secondary sources, users can draw insights, detect trends , and validate findings to jumpstart their research efforts.

Primary vs. Secondary Market Research

We’ve touched a little on primary research , but it’s essential to understand exactly how primary and secondary research are unique.

Think of primary research as the “thing” itself, and secondary research as the analysis of the “thing,” like these primary and secondary research examples:

• An expert gives an interview (primary research) and a marketer uses that interview to write an article (secondary research).
• A company conducts a consumer satisfaction survey (primary research) and a business analyst uses the survey data to write a market trend report (secondary research).
• A marketing team launches a new advertising campaign across various platforms (primary research) and a marketing research firm, like Meltwater for market research , compiles the campaign performance data to benchmark against industry standards (secondary research).

In other words, primary sources make original contributions to a topic or issue, while secondary sources analyze, synthesize, or interpret primary sources.

Both are necessary when optimizing a business, gaining a competitive edge , improving marketing, or understanding consumer trends that may impact your business.

Secondary research methods focus on analyzing existing data rather than collecting primary data . Common examples of secondary research methods include:

• Literature review . Researchers analyze and synthesize existing literature (e.g., white papers, research papers, articles) to find knowledge gaps and build on current findings.
• Content analysis . Researchers review media sources and published content to find meaningful patterns and trends.
• AI-powered secondary research . Platforms like Meltwater for market research analyze vast amounts of complex data and use AI technologies like natural language processing and machine learning to turn data into contextual insights.

Researchers today have access to more secondary research companies and market research tools and technology than ever before, allowing them to streamline their efforts and improve their findings.

Want to see how Meltwater can complement your secondary market research efforts? Simply fill out the form at the bottom of this post, and we'll be in touch.

Conducting secondary research offers benefits in every job function and use case, from marketing to the C-suite. Here are a few advantages you can expect.

Cost and time efficiency

Using existing research saves you time and money compared to conducting primary research. Secondary data is readily available and easily accessible via libraries, free publications, or the Internet. This is particularly advantageous when you face time constraints or when a project requires a large amount of data and research.

Access to large datasets

Secondary data gives you access to larger data sets and sample sizes compared to what primary methods may produce. Larger sample sizes can improve the statistical power of the study and add more credibility to your findings.

Ability to analyze trends and patterns

Using larger sample sizes, researchers have more opportunities to find and analyze trends and patterns. The more data that supports a trend or pattern, the more trustworthy the trend becomes and the more useful for making decisions. 

Historical context

Using a combination of older and recent data allows researchers to gain historical context about patterns and trends. Learning what’s happened before can help decision-makers gain a better current understanding and improve how they approach a problem or project.

Basis for further research

Ideally, you’ll use secondary research to further other efforts . Secondary sources help to identify knowledge gaps, highlight areas for improvement, or conduct deeper investigations.

Tip: Learn how to use Meltwater as a research tool and how Meltwater uses AI.

Secondary research comes with a few drawbacks, though these aren’t necessarily deal breakers when deciding to use secondary sources.

Reliability concerns

Researchers don’t always know where the data comes from or how it’s collected, which can lead to reliability concerns. They don’t control the initial process, nor do they always know the original purpose for collecting the data, both of which can lead to skewed results.

Potential bias

The original data collectors may have a specific agenda when doing their primary research, which may lead to biased findings. Evaluating the credibility and integrity of secondary data sources can prove difficult.

Outdated information

Secondary sources may contain outdated information, especially when dealing with rapidly evolving trends or fields. Using outdated information can lead to inaccurate conclusions and widen knowledge gaps.

Limitations in customization

Relying on secondary data means being at the mercy of what’s already published. It doesn’t consider your specific use cases, which limits you as to how you can customize and use the data.

A lack of relevance

Secondary research rarely holds all the answers you need, at least from a single source. You typically need multiple secondary sources to piece together a narrative, and even then you might not find the specific information you need.

To make secondary market research your new best friend, you’ll need to think critically about its strengths and find ways to overcome its weaknesses. Let’s review some best practices to use secondary research to its fullest potential.

Identify credible sources for secondary research

To overcome the challenges of bias, accuracy, and reliability, choose secondary sources that have a demonstrated history of excellence . For example, an article published in a medical journal naturally has more credibility than a blog post on a little-known website.

Assess credibility based on peer reviews, author expertise, sampling techniques, publication reputation, and data collection methodologies. Cross-reference the data with other sources to gain a general consensus of truth.

The more credibility “factors” a source has, the more confidently you can rely on it. 

Evaluate the quality and relevance of secondary data

You can gauge the quality of the data by asking simple questions:

• How complete is the data? 
• How old is the data? 
• Is this data relevant to my needs?
• Does the data come from a known, trustworthy source?

It’s best to focus on data that aligns with your research objectives. Knowing the questions you want to answer and the outcomes you want to achieve ahead of time helps you focus only on data that offers meaningful insights.

Document your sources 

If you’re sharing secondary data with others, it’s essential to document your sources to gain others’ trust. They don’t have the benefit of being “in the trenches” with you during your research, and sharing your sources can add credibility to your findings and gain instant buy-in.

Secondary market research offers an efficient, cost-effective way to learn more about a topic or trend, providing a comprehensive understanding of the customer journey . Compared to primary research, users can gain broader insights, analyze trends and patterns, and gain a solid foundation for further exploration by using secondary sources.

Meltwater for market research speeds up the time to value in using secondary research with AI-powered insights, enhancing your understanding of the customer journey. Using natural language processing, machine learning, and trusted data science processes, Meltwater helps you find relevant data and automatically surfaces insights to help you understand its significance. Our solution identifies hidden connections between data points you might not know to look for and spells out what the data means, allowing you to make better decisions based on accurate conclusions. Learn more about Meltwater's power as a secondary research solution when you request a demo by filling out the form below:

Continue Reading

How To Do Market Research: Definition, Types, Methods

What Are Consumer Insights? Meaning, Examples, Strategy

Market Intelligence 101: What It Is & How To Use It

The 13 Best Market Research Tools

Consumer Intelligence: Definition & Examples

9 Top Consumer Insights Tools & Companies

What Is Desk Research? Meaning, Methodology, Examples

Top Secondary Market Research Companies | Desk Research Companies

Tutorial: Evaluating Information: Primary vs. Secondary Articles

• Evaluating Information
• Scholarly Literature Types
• Primary vs. Secondary Articles
• Peer Review
• Systematic Reviews & Meta-Analysis
• Gray Literature
• Evaluating Like a Boss
• Evaluating AV

Primary vs. Secondary Research Articles

In the sciences,  primary (or empirical) research articles :

• are original scientific reports of new research findings (Please note that an original scientific article does not include review articles, which summarize the research literature on a particular subject, or articles using meta-analyses, which analyze pre-published data.)
• usually include the following sections: Introduction , Methods , Results , Discussion, References
• are usually  peer-reviewed (examined by expert(s) in the field before publication). Please note that a peer-reviewed article is not the same as a review article, which summarizes the research literature on a particular subject

You may also choose to use some secondary sources (summaries or interpretations of original research) such as books (find these through the library catalog) or review articles (articles which organize and critically analyze the research of others on a topic). These secondary sources, particularly review articles, are often useful and easier-to-read summaries of research in an area. Additionally, you can use the listed references to find useful primary research articles.

Anatomy of a Scholarly Article

from NCSU Libraries' Anatomy of a Scholarly Article

Types of health studies

In the sciences, particularly the health sciences, there are a number of types of primary articles (the gold standard being randomized controlled trials ) and secondary articles (the gold standard being systematic reviews and meta-analysis ). The chart below summarizes their differences and the linked article gives more information.

Searching for Primary vs. Secondary Articles

Some scholarly databases will allow you to specific what kind of scholarly literature you're looking for.  However, be careful! Sometimes, depending on the database, the Review article type may mean book review instead of or as well as review article. You may also have to look under more or custom options to find these choices.

• << Previous: Scholarly Literature Types
• Next: Peer Review >>
• Last Updated: Oct 20, 2021 11:11 AM
• URL: https://guides.library.cornell.edu/evaluate

• Skip to main content
• Skip to primary sidebar
• Skip to footer
• QuestionPro

• Solutions Industries Gaming Automotive Sports and events Education Government Travel & Hospitality Financial Services Healthcare Cannabis Technology Use Case NPS+ Communities Audience Contactless surveys Mobile LivePolls Member Experience GDPR Positive People Science 360 Feedback Surveys
• Resources Blog eBooks Survey Templates Case Studies Training Help center

Home Market Research

Secondary Research: Definition, Methods and Examples.

In the world of research, there are two main types of data sources: primary and secondary. While primary research involves collecting new data directly from individuals or sources, secondary research involves analyzing existing data already collected by someone else. Today we’ll discuss secondary research.

One common source of this research is published research reports and other documents. These materials can often be found in public libraries, on websites, or even as data extracted from previously conducted surveys. In addition, many government and non-government agencies maintain extensive data repositories that can be accessed for research purposes.

LEARN ABOUT: Research Process Steps

While secondary research may not offer the same level of control as primary research, it can be a highly valuable tool for gaining insights and identifying trends. Researchers can save time and resources by leveraging existing data sources while still uncovering important information.

What is Secondary Research: Definition

Secondary research is a research method that involves using already existing data. Existing data is summarized and collated to increase the overall effectiveness of the research.

One of the key advantages of secondary research is that it allows us to gain insights and draw conclusions without having to collect new data ourselves. This can save time and resources and also allow us to build upon existing knowledge and expertise.

When conducting secondary research, it’s important to be thorough and thoughtful in our approach. This means carefully selecting the sources and ensuring that the data we’re analyzing is reliable and relevant to the research question . It also means being critical and analytical in the analysis and recognizing any potential biases or limitations in the data.

LEARN ABOUT: Level of Analysis

Secondary research is much more cost-effective than primary research , as it uses already existing data, unlike primary research, where data is collected firsthand by organizations or businesses or they can employ a third party to collect data on their behalf.

LEARN ABOUT: Data Analytics Projects

Secondary Research Methods with Examples

Secondary research is cost-effective, one of the reasons it is a popular choice among many businesses and organizations. Not every organization is able to pay a huge sum of money to conduct research and gather data. So, rightly secondary research is also termed “ desk research ”, as data can be retrieved from sitting behind a desk.

The following are popularly used secondary research methods and examples:

1. Data Available on The Internet

One of the most popular ways to collect secondary data is the internet. Data is readily available on the internet and can be downloaded at the click of a button.

This data is practically free of cost, or one may have to pay a negligible amount to download the already existing data. Websites have a lot of information that businesses or organizations can use to suit their research needs. However, organizations need to consider only authentic and trusted website to collect information.

2. Government and Non-Government Agencies

Data for secondary research can also be collected from some government and non-government agencies. For example, US Government Printing Office, US Census Bureau, and Small Business Development Centers have valuable and relevant data that businesses or organizations can use.

There is a certain cost applicable to download or use data available with these agencies. Data obtained from these agencies are authentic and trustworthy.

3. Public Libraries

Public libraries are another good source to search for data for this research. Public libraries have copies of important research that were conducted earlier. They are a storehouse of important information and documents from which information can be extracted.

The services provided in these public libraries vary from one library to another. More often, libraries have a huge collection of government publications with market statistics, large collection of business directories and newsletters.

4. Educational Institutions

Importance of collecting data from educational institutions for secondary research is often overlooked. However, more research is conducted in colleges and universities than any other business sector.

The data that is collected by universities is mainly for primary research. However, businesses or organizations can approach educational institutions and request for data from them.

5. Commercial Information Sources

Local newspapers, journals, magazines, radio and TV stations are a great source to obtain data for secondary research. These commercial information sources have first-hand information on economic developments, political agenda, market research, demographic segmentation and similar subjects.

Businesses or organizations can request to obtain data that is most relevant to their study. Businesses not only have the opportunity to identify their prospective clients but can also know about the avenues to promote their products or services through these sources as they have a wider reach.

Key Differences between Primary Research and Secondary Research

Understanding the distinction between primary research and secondary research is essential in determining which research method is best for your project. These are the two main types of research methods, each with advantages and disadvantages. In this section, we will explore the critical differences between the two and when it is appropriate to use them.

How to Conduct Secondary Research?

We have already learned about the differences between primary and secondary research. Now, let’s take a closer look at how to conduct it.

Secondary research is an important tool for gathering information already collected and analyzed by others. It can help us save time and money and allow us to gain insights into the subject we are researching. So, in this section, we will discuss some common methods and tips for conducting it effectively.

Here are the steps involved in conducting secondary research:

1. Identify the topic of research: Before beginning secondary research, identify the topic that needs research. Once that’s done, list down the research attributes and its purpose.

2. Identify research sources: Next, narrow down on the information sources that will provide most relevant data and information applicable to your research.

3. Collect existing data: Once the data collection sources are narrowed down, check for any previous data that is available which is closely related to the topic. Data related to research can be obtained from various sources like newspapers, public libraries, government and non-government agencies etc.

4. Combine and compare: Once data is collected, combine and compare the data for any duplication and assemble data into a usable format. Make sure to collect data from authentic sources. Incorrect data can hamper research severely.

4. Analyze data: Analyze collected data and identify if all questions are answered. If not, repeat the process if there is a need to dwell further into actionable insights.

Advantages of Secondary Research

Secondary research offers a number of advantages to researchers, including efficiency, the ability to build upon existing knowledge, and the ability to conduct research in situations where primary research may not be possible or ethical. By carefully selecting their sources and being thoughtful in their approach, researchers can leverage secondary research to drive impact and advance the field. Some key advantages are the following:

1. Most information in this research is readily available. There are many sources from which relevant data can be collected and used, unlike primary research, where data needs to collect from scratch.

2. This is a less expensive and less time-consuming process as data required is easily available and doesn’t cost much if extracted from authentic sources. A minimum expenditure is associated to obtain data.

3. The data that is collected through secondary research gives organizations or businesses an idea about the effectiveness of primary research. Hence, organizations or businesses can form a hypothesis and evaluate cost of conducting primary research.

4. Secondary research is quicker to conduct because of the availability of data. It can be completed within a few weeks depending on the objective of businesses or scale of data needed.

As we can see, this research is the process of analyzing data already collected by someone else, and it can offer a number of benefits to researchers.

Disadvantages of Secondary Research

On the other hand, we have some disadvantages that come with doing secondary research. Some of the most notorious are the following:

1. Although data is readily available, credibility evaluation must be performed to understand the authenticity of the information available.

2. Not all secondary data resources offer the latest reports and statistics. Even when the data is accurate, it may not be updated enough to accommodate recent timelines.

3. Secondary research derives its conclusion from collective primary research data. The success of your research will depend, to a greater extent, on the quality of research already conducted by primary research.

LEARN ABOUT: 12 Best Tools for Researchers

In conclusion, secondary research is an important tool for researchers exploring various topics. By leveraging existing data sources, researchers can save time and resources, build upon existing knowledge, and conduct research in situations where primary research may not be feasible.

There are a variety of methods and examples of secondary research, from analyzing public data sets to reviewing previously published research papers. As students and aspiring researchers, it’s important to understand the benefits and limitations of this research and to approach it thoughtfully and critically. By doing so, we can continue to advance our understanding of the world around us and contribute to meaningful research that positively impacts society.

QuestionPro can be a useful tool for conducting secondary research in a variety of ways. You can create online surveys that target a specific population, collecting data that can be analyzed to gain insights into consumer behavior, attitudes, and preferences; analyze existing data sets that you have obtained through other means or benchmark your organization against others in your industry or against industry standards. The software provides a range of benchmarking tools that can help you compare your performance on key metrics, such as customer satisfaction, with that of your peers.

Using QuestionPro thoughtfully and strategically allows you to gain valuable insights to inform decision-making and drive business success. Start today for free! No credit card is required.

LEARN MORE         FREE TRIAL

MORE LIKE THIS

21 Best Customer Advocacy Software for Customers in 2024

Apr 19, 2024

10 Quantitative Data Analysis Software for Every Data Scientist

Apr 18, 2024

11 Best Enterprise Feedback Management Software in 2024

17 Best Online Reputation Management Software in 2024

Apr 17, 2024

Other categories

• Academic Research
• Artificial Intelligence
• Assessments
• Brand Awareness
• Case Studies
• Communities
• Consumer Insights
• Customer effort score
• Customer Engagement
• Customer Experience
• Customer Loyalty
• Customer Research
• Customer Satisfaction
• Employee Benefits
• Employee Engagement
• Employee Retention
• Friday Five
• General Data Protection Regulation
• Insights Hub
• Life@QuestionPro
• Market Research
• Mobile diaries
• Mobile Surveys
• New Features
• Online Communities
• Question Types
• Questionnaire
• QuestionPro Products
• Release Notes
• Research Tools and Apps
• Revenue at Risk
• Survey Templates
• Training Tips
• Uncategorized
• Video Learning Series
• What’s Coming Up
• Workforce Intelligence

How To Do Secondary Research or a Literature Review

• Secondary Research
• Literature Review
• Step 1: Develop topic
• Step 2: Develop your search strategy
• Step 3. Document search strategy and organize results
• Systematic Literature Review Tips
• More Information

Search our FAQ

Make a research appointment.

Schedule a personalized one-on-one research appointment with one of Galvin Library's research specialists.

What is Secondary Research?

Secondary research, also known as a literature review , preliminary research , historical research , background research , desk research , or library research , is research that analyzes or describes prior research. Rather than generating and analyzing new data, secondary research analyzes existing research results to establish the boundaries of knowledge on a topic, to identify trends or new practices, to test mathematical models or train machine learning systems, or to verify facts and figures. Secondary research is also used to justify the need for primary research as well as to justify and support other activities. For example, secondary research may be used to support a proposal to modernize a manufacturing plant, to justify the use of newly a developed treatment for cancer, to strengthen a business proposal, or to validate points made in a speech.

Why Is Secondary Research Important?

Because secondary research is used for so many purposes in so many settings, all professionals will be required to perform it at some point in their careers. For managers and entrepreneurs, regardless of the industry or profession, secondary research is a regular part of worklife, although parts of the research, such as finding the supporting documents, are often delegated to juniors in the organization. For all these reasons, it is essential to learn how to conduct secondary research, even if you are unlikely to ever conduct primary research.

Secondary research is also essential if your main goal is primary research. Research funding is obtained only by using secondary research to show the need for the primary research you want to conduct. In fact, primary research depends on secondary research to prove that it is indeed new and original research and not just a rehash or replication of somebody else’s work.

• Next: Literature Review >>
• Last Updated: Dec 21, 2023 3:46 PM
• URL: https://guides.library.iit.edu/litreview

• Teesside University Student & Library Services
• Learning Hub Group

Research Methods

Secondary research.

• Primary Research

What is Secondary Research?

Advantages and disadvantages of secondary research, secondary research in literature reviews, secondary research - going beyond literature reviews, main stages of secondary research, useful resources, using material on this page.

• Quantitative Research This link opens in a new window
• Qualitative Research This link opens in a new window
• Being Critical This link opens in a new window
• Subject LibGuides This link opens in a new window

Secondary research

Secondary research uses research and data that has already been carried out. It is sometimes referred to as desk research. It is a good starting point for any type of research as it enables you to analyse what research has already been undertaken and identify any gaps. 

You may only need to carry out secondary research for your assessment or you may need to use secondary research as a starting point, before undertaking your own primary research .

Searching for both primary and secondary sources can help to ensure that you are up to date with what research has already been carried out in your area of interest and to identify the key researchers in the field.

"Secondary sources are the books, articles, papers and similar materials written or produced by others that help you to form your background understanding of the subject. You would use these to find out about experts’ findings, analyses or perspectives on the issue and decide whether to draw upon these explicitly in your research." (Cottrell, 2014, p. 123).

Examples of secondary research sources include:.

• journal articles
• official statistics, such as government reports or organisations which have collected and published data

Primary research  involves gathering data which has not been collected before. Methods to collect it can include interviews, focus groups, controlled trials and case studies. Secondary research often comments on and analyses this primary research.

Gopalakrishnan and Ganeshkumar (2013, p. 10) explain the difference between primary and secondary research:

"Primary research is collecting data directly from patients or population, while secondary research is the analysis of data already collected through primary research. A review is an article that summarizes a number of primary studies and may draw conclusions on the topic of interest which can be traditional (unsystematic) or systematic".

Secondary Data

As secondary data has already been collected by someone else for their research purposes, it may not cover all of the areas of interest for your research topic. This research will need to be analysed alongside other research sources and data in the same subject area in order to confirm, dispute or discuss the findings in a wider context.

"Secondary source data, as the name infers, provides second-hand information. The data come ‘pre-packaged’, their form and content reflecting the fact that they have been produced by someone other than the researcher and will not have been produced specifically for the purpose of the research project. The data, none the less, will have some relevance for the research in terms of the information they contain, and the task for the researcher is to extract that information and re-use it in the context of his/her own research project." (Denscombe, 2021, p. 268)

In the video below Dr. Benedict Wheeler (Senior Research Fellow at the European Center for Environment and Human Health at the University of Exeter Medical School) discusses secondary data analysis. Secondary data was used for his research on how the environment affects health and well-being and utilising this secondary data gave access to a larger data set.

As with all research, an important part of the process is to critically evaluate any sources you use. There are tools to help with this in the  Being Critical  section of the guide.

Louise Corti, from the UK Data Archive, discusses using secondary data  in the video below. T he importance of evaluating secondary research is discussed - this is to ensure the data is appropriate for your research and to investigate how the data was collected.

There are advantages and disadvantages to secondary research:

Advantages:

• Usually low cost
• Easily accessible
• Provides background information to clarify / refine research areas
• Increases breadth of knowledge
• Shows different examples of research methods
• Can highlight gaps in the research and potentially outline areas of difficulty
• Can incorporate a wide range of data
• Allows you to identify opposing views and supporting arguments for your research topic
• Highlights the key researchers and work which is being undertaken within the subject area
• Helps to put your research topic into perspective

Disadvantages

• Can be out of date
• Might be unreliable if it is not clear where or how the research has been collected - remember to think critically
• May not be applicable to your specific research question as the aims will have had a different focus

Literature reviews 

Secondary research for your major project may take the form of a literature review . this is where you will outline the main research which has already been written on your topic. this might include theories and concepts connected with your topic and it should also look to see if there are any gaps in the research., as the criteria and guidance will differ for each school, it is important that you check the guidance which you have been given for your assessment. this may be in blackboard and you can also check with your supervisor..

The videos below include some insights from academics regarding the importance of literature reviews.

Secondary research which goes beyond literature reviews

For some dissertations/major projects there might only be a literature review (discussed above ). For others there could be a literature review followed by primary research and for others the literature review might be followed by further secondary research. 

You may be asked to write a literature review which will form a background chapter to give context to your project and provide the necessary history for the research topic. However, you may then also be expected to produce the rest of your project using additional secondary research methods, which will need to produce results and findings which are distinct from the background chapter t o avoid repetition .

Remember, as the criteria and guidance will differ for each School, it is important that you check the guidance which you have been given for your assessment. This may be in Blackboard and you can also check with your supervisor.

Although this type of secondary research will go beyond a literature review, it will still rely on research which has already been undertaken. And,  "just as in primary research, secondary research designs can be either quantitative, qualitative, or a mixture of both strategies of inquiry" (Manu and Akotia, 2021, p. 4) .

Your secondary research may use the literature review to focus on a specific theme, which is then discussed further in the main project. Or it may use an alternative approach. Some examples are included below.  Remember to speak with your supervisor if you are struggling to define these areas.

Some approaches of how to conduct secondary research include:

• A systematic review is a structured literature review that involves identifying all of the relevant primary research using a rigorous search strategy to answer a focused research question.
• This involves comprehensive searching which is used to identify themes or concepts across a number of relevant studies. 
• The review will assess the q uality of the research and provide a summary and synthesis of all relevant available research on the topic.
• The systematic review  LibGuide goes into more detail about this process (The guide is aimed a PhD/Researcher students. However, students on other levels of study may find parts of the guide helpful too).
• Scoping reviews aim to identify and assess available research on a specific topic (which can include ongoing research). 
• They are "particularly useful when a body of literature has not yet been comprehensively reviewed, or exhibits a complex or heterogeneous nature not amenable to a more precise systematic review of the evidence. While scoping reviews may be conducted to determine the value and probable scope of a full systematic review, they may also be undertaken as exercises in and of themselves to summarize and disseminate research findings, to identify research gaps, and to make recommendations for the future research."  (Peters et al., 2015) .
• This is designed to  summarise the current knowledge and provide priorities for future research.
• "A state-of-the-art review will often highlight new ideas or gaps in research with no official quality assessment." (Baguss, 2020) .
• "Bibliometric analysis is a popular and rigorous method for exploring and analyzing large volumes of scientific data." (Donthu et al., 2021)
• Quantitative methods and statistics are used to analyse the bibliographic data of published literature. This can be used to measure the impact of authors, publications, or topics within a subject area.

The bibliometric analysis often uses the data from a citation source such as Scopus or Web of Science .

• This is a technique used to combine the statistic results of prior quantitative studies in order to increase precision and validity.
• "It goes beyond the parameters of a literature review, which assesses existing literature, to actually perform calculations based on the results collated, thereby coming up with new results" (Curtis and Curtis, 2011, p. 220)

(Adapted from: Grant and Booth, 2009, cited in Sarhan and Manu, 2021, p. 72 )

• Grounded Theory is used to create explanatory theory from data which has been collected.
• "Grounded theory data analysis strategies can be used with different types of data, including secondary data." ( Whiteside, Mills and McCalman, 2012 )
• This allows you to use a specific theory or theories which can then be applied to your chosen topic/research area.
• You could focus on one case study which is analysed in depth, or you could examine more than one in order to compare and contrast the important aspects of your research question.
• "Good case studies often begin with a predicament that is poorly comprehended and is inadequately explained or traditionally rationalised by numerous conflicting accounts. Therefore, the aim is to comprehend an existent problem and to use the acquired understandings to develop new theoretical outlooks or explanations."  ( Papachroni and Lochrie, 2015, p. 81 )

Main stages of secondary research for a dissertation/major project

In general, the main stages for conducting secondary research for your dissertation or major project will include:

Click on the image below to access the reading list which includes resources used in this guide as well as some additional useful resources.

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License .

• << Previous: Primary Research
• Next: Quantitative Research >>
• Last Updated: Aug 11, 2022 3:41 PM
• URL: https://libguides.tees.ac.uk/researchmethods

• Harvard Library
• Research Guides
• Faculty of Arts & Sciences Libraries

Library Research Guide for the History of Science: Introduction

• What is a Secondary Source?
• Senior Theses 2023
• Background and Context/Biography
• Exploring Your Topic
• Using HOLLIS

Page Contents

Recognizing secondary sources, find secondary sources, finding bibliographies.

• What is a Primary Source?
• Exploring the Special Collections at Harvard
• Citing Sources & Organizing Research
• Recognizing Secondary Sources
• Finding Secondary Sources

Secondary sources were created by someone who did not experience first-hand or participate in the events or conditions you’re researching. For a historical research project, secondary sources are generally scholarly books and articles.

A secondary source interprets and analyzes primary sources. These sources are one or more steps removed from the event. Secondary sources may contain pictures, quotes or graphics of primary sources.

Some types of secondary source include:  Textbooks; journal articles; histories; criticisms; commentaries; encyclopedias 

Examples of secondary sources include:

• A scholarly article about water and bathing in Mexico City, 1850-1920
• A book about the psychological effects of WWI
• A 2019 U.S. government document examining the work of African Americans at two Manhattan Project sites
• An NPR piece on race and vaccine skepticism

For a historical research project, secondary sources are usually scholarly books and articles, but as you can see from this list there are other possibilties.

History of Science, Technology and Medicine (Harvard Login)  (1975- ) is an index of books, book chapters, and journal articles. Some social sciences material is included. 

• HSTM is an amalgamation of four separate indexes with four different subject term systems; study the results of keyword searches to be sure that you know the proper subject terms for your topic in each of the, possibly four, relevant component databases. For example, the Wellcome Bibliography uses "Contraception" but the Isis Current Bibliography uses "Birth control".
• If you want to limit the coverage of your sources to a particular era, put one of these terms in a search box:  Antiquity or Ancient - “Greek and Roman” - “Middle Ages” or medieval - 13th  century - 14th century - 15 th  century - 16 th  century - 17th century - 18th century - 19th century - 20th century
• More detailed information about the use of this complex database.

ISISCB Bibliographic Resources in the History of Science, Technology and Medicine  indexes the Isis Current Bibliography.  Search results extend back to 1970.  It also offers a browse of the  Isis Cumulative Bibliographies  (1913-1975).  Search ISISCB Explore

PubMed (Harvard Login for full text access)  (1947- ) is the National Library of Medicine's index to biomedical journal articles.

• To limit to historical sources, attach the phrase (in"") "historical article" to your search. Example: "Psychology, clinical" and "historical article".
• Be sure to look for MeSH (Medical Subject Headings) on pertinent records by scrolling down past the abstracts. (Not all records in PubMed have MeSH terms.) Subject headings can help you get to more relevant records and/or can be helpful keyword suggestions.

America: History and Life (Harvard login)  is the primary bibliographic reference to the history of the United States and Canada from prehistory to the present.

Historical Abstracts (Harvard Login)  is a reference guide to the history of the world from 1450 to the present (excluding the United States and Canada, which are covered in America: History and Life, above).

• Both allow coverage date limitations as well as publication date limitations, Can usually use LC terms, not always 

Bibliography of British and Irish History (Harvard Login)  provides bibliographic data on historical writing dealing with the British Isles, and with the British empire and commonwealth, during all periods for which written documentation is available - from 55BC to the present.

• At the link above, choose BBIH from the top row of options.

The Forest History Society Research Portal  offers over 45,000 citations to published items on environmental history, over 30,000 photographs, and other material.

Web of Science Citation Indexes (Harvard Login)  (for historical articles1956- ) includes articles in all areas of science. You can use the Cited Reference Search in Web of Science to find secondary source articles that cite a specified secondary or primary source article or book. More information .

Library Guide to the History of Science Your guide to the History of Science at Harvard. It has more extensive lists of resources and tools than this introductory guide does.

There may already be a detailed list of sources, a bibliography, for your topic. Bibliographies don't always come at the end of a paper- many are independent works of their own, full of recommended sources on any given topic.

For example:

• Microbes and Minie Balls: An Annotated Bibliography of Civil War Medicine (Print Only) , by F. R. Freemon. Rutherford: Fairleigh Dickinson University Press, 1993, 253 pp.                          
• Annotated Bibliography on Medical Research in the South Pacific (Harvard login for HathiTrust full text), by Norman-Taylor, William, South Pacific Commission, 1963. 371pp.

Look for specialized subject bibliographies: search, e.g., <"science and state" AND China AND bibliography>   in HOLLIS  and WorldCat (advanced search). Note:The word Bibliography must be searched as a Subject keyword.

If you find an older article or book in a bibliography, you can use the Cited Reference Search in Web of Science to find more recent articles by seeing who has cited it.   If you have a bibliography of primary sources, then the Web of Science can be used to find secondary sources that cite a specified primary source. See Searching the Citation Indexes (Web of Science) .

• << Previous: Using HOLLIS
• Next: What is a Primary Source? >>
• Last Updated: Apr 14, 2024 9:25 PM
• URL: https://guides.library.harvard.edu/HistSciInfo

Harvard University Digital Accessibility Policy

• What is Secondary Research? + [Methods & Examples]

In some situations, the researcher may not be directly involved in the data gathering process and instead, would rely on already existing data in order to arrive at research outcomes. This approach to systematic investigation is known as secondary research. 

There are many reasons a researcher may want to make use of already existing data instead of collecting data samples, first-hand. In this article, we will share some of these reasons with you and show you how to conduct secondary research with Formplus. 

What is Secondary  Research?

Secondary research is a common approach to a systematic investigation in which the researcher depends solely on existing data in the course of the research process. This research design involves organizing, collating and analyzing these data samples for valid research conclusions. 

Secondary research is also known as desk research since it involves synthesizing existing data that can be sourced from the internet, peer-reviewed journals , textbooks, government archives, and libraries. What the secondary researcher does is to study already established patterns in previous researches and apply this information to the specific research context. 

Interestingly, secondary research often relies on data provided by primary research and this is why some researches combine both methods of investigation. In this sense, the researcher begins by evaluating and identifying gaps in existing knowledge before adopting primary research to gather new information that will serve his or her research. 

What are Secondary Research Methods?

As already highlighted, secondary research involves data assimilation from different sources, that is, using available research materials instead of creating a new pool of data using primary research methods. Common secondary research methods include data collection through the internet, libraries, archives, schools and organizational reports. 

• Online Data

Online data is data that is gathered via the internet. In recent times, this method has become popular because the internet provides a large pool of both free and paid research resources that can be easily accessed with the click of a button. 

While this method simplifies the data gathering process , the researcher must take care to depend solely on authentic sites when collecting information. In some way, the internet is a virtual aggregation for all other sources of secondary research data. 

• Data from Government and Non-government Archives

You can also gather useful research materials from government and non-government archives and these archives usually contain verifiable information that provides useful insights on varying research contexts. In many cases, you would need to pay a sum to gain access to these data. 

The challenge, however, is that such data is not always readily available due to a number of factors. For instance, some of these materials are described as classified information as such, it would be difficult for researchers to have access to them. 

• Data from Libraries

Research materials can also be accessed through public and private libraries. Think of a library as an information storehouse that contains an aggregation of important information that can serve as valid data in different research contexts. 

Typically, researchers donate several copies of dissertations to public and private libraries; especially in cases of academic research. Also, business directories, newsletters, annual reports and other similar documents that can serve as research data, are gathered and stored in libraries, in both soft and hard copies. 

• Data from Institutions of Learning

Educational facilities like schools, faculties, and colleges are also a great source of secondary data; especially in academic research. This is because a lot of research is carried out in educational institutions more than in other sectors. 

It is relatively easier to obtain research data from educational institutions because these institutions are committed to solving problems and expanding the body of knowledge. You can easily request research materials from educational facilities for the purpose of a literature review. 

Secondary research methods can also be categorized into qualitative and quantitative data collection methods . Quantitative data gathering methods include online questionnaires and surveys, reports about trends plus statistics about different areas of a business or industry.  

Qualitative research methods include relying on previous interviews and data gathered through focus groups which helps an organization to understand the needs of its customers and plan to fulfill these needs. It also helps businesses to measure the level of employee satisfaction with organizational policies. 

When Do We Conduct Secondary Research?

Typically, secondary research is the first step in any systematic investigation. This is because it helps the researcher to understand what research efforts have been made so far and to utilize this knowledge in mapping out a novel direction for his or her investigation. 

For instance, you may want to carry out research into the nature of a respiratory condition with the aim of developing a vaccine. The best place to start is to gather existing research material about the condition which would help to point your research in the right direction. 

When sifting through these pieces of information, you would gain insights into methods and findings from previous researches which would help you define your own research process. Secondary research also helps you to identify knowledge gaps that can serve as the name of your own research. 

Questions to ask before conducting Secondary Research

Since secondary research relies on already existing data, the researcher must take extra care to ensure that he or she utilizes authentic data samples for the research. Falsified data can have a negative impact on the research outcomes; hence, it is important to always carry out resource evaluation by asking a number of questions as highlighted below:

• What is the purpose of the research? Again, it is important for every researcher to clearly define the purpose of the research before proceeding with it. Usually, the research purpose determines the approach that would be adopted. 
• What is my research methodology? After identifying the purpose of the research, the next thing to do is outline the research methodology. This is the point where the researcher chooses to gather data using secondary research methods. 
• What are my expected research outcomes? 
• Who collected the data to be analyzed? Before going on to use secondary data for your research, it is necessary to ascertain the authenticity of the information. This usually affects the data reliability and determines if the researcher can trust the materials.  For instance, data gathered from personal blogs and websites may not be as credible as information obtained from an organization’s website. 
• When was the data collected? Data recency is another factor that must be considered since the recency of data can affect research outcomes. For instance, if you are carrying out research into the number of women who smoke in London, it would not be appropriate for you to make use of information that was gathered 5 years ago unless you plan to do some sort of data comparison. 
• Is the data consistent with other data available from other sources? Always compare and contrast your data with other available research materials as this would help you to identify inconsistencies if any.
• What type of data was collected? Take care to determine if the secondary data aligns with your research goals and objectives. 
• How was the data collected? 

Advantages of Secondary Research

• Easily Accessible With secondary research, data can easily be accessed in no time; especially with the use of the internet. Apart from the internet, there are different data sources available in secondary research like public libraries and archives which are relatively easy to access too. 
• Secondary research is cost-effective and it is not time-consuming. The researcher can cut down on costs because he or she is not directly involved in the data collection process which is also time-consuming. 
• Secondary research helps researchers to identify knowledge gaps which can serve as the basis of further systematic investigation. 
• It is useful for mapping out the scope of research thereby setting the stage for field investigations. When carrying out secondary research, the researchers may find that the exact information they were looking for is already available, thus eliminating the need and expense incurred in carrying out primary research in these areas. 

Disadvantages of Secondary Research  

• Questionable Data: With secondary research, it is hard to determine the authenticity of the data because the researcher is not directly involved in the research process. Invalid data can affect research outcomes negatively hence, it is important for the researcher to take extra care by evaluating the data before making use of it. 
• Generalization: Secondary data is unspecific in nature and may not directly cater to the needs of the researcher. There may not be correlations between the existing data and the research process. 
• Common Data: Research materials in secondary research are not exclusive to an individual or group. This means that everyone has access to the data and there is little or no “information advantage” gained by those who obtain the research.
• It has the risk of outdated research materials. Outdated information may offer little value especially for organizations competing in fast-changing markets.

How to Conduct Online Surveys with Formplus 

Follow these 5 steps to create and administer online surveys for secondary research: 

• Sign into Formplus

In the Formplus builder, you can easily create an online survey for secondary research by dragging and dropping preferred fields into your form. To access the Formplus builder, you will need to create an account on Formplus. 

Once you do this, sign in to your account and click on “Create Form ” to begin. 

• Edit Form Title

Click on the field provided to input your form title, for example, “Secondary Research Survey”.

• Click on the edit button to edit the form.

• Add Fields: Drag and drop preferred form fields into your form in the Formplus builder inputs column. There are several field input options for questionnaires in the Formplus builder. 
• Edit fields
• Click on “Save”
• Preview form. 
• Customize your Form

With the form customization options in the form builder, you can easily change the outlook of your form and make it more unique and personalized. Formplus allows you to change your form theme, add background images and even change the font according to your needs. 

• Multiple Sharing Options

Formplus offers multiple form sharing options which enables you to easily share your questionnaire with respondents. You can use the direct social media sharing buttons to share your form link to your organization’s social media pages. 

You can send out your survey form as email invitations to your research subjects too. If you wish, you can share your form’s QR code or embed it on your organization’s website for easy access. 

Why Use Formplus as a Secondary Research Tool?

• Simple Form Builder Solution

The Formplus form builder is easy to use and does not require you to have any knowledge in computer programming, unlike other form builders. For instance, you can easily add form fields to your form by dragging and dropping them from the inputs section in the builder. 

In the form builder, you can also modify your fields to be hidden or read-only and you can create smart forms with save and resume options, form lookup, and conditional logic. Formplus also allows you to customize your form by adding preferred background images and your organization’s logo. 

• Over 25 Form Fields

With over 25 versatile form fields available in the form builder, you can easily collect data the way you like. You can receive payments directly in your form by adding payment fields and you can also add file upload fields to allow you receive files in your form too. 

• Offline Form feature

With Formplus, you can collect data from respondents even without internet connectivity . Formplus automatically detects when there is no or poor internet access and allows forms to be filled out and submitted in offline mode. 

Offline form responses are automatically synced with the servers when the internet connection is restored. This feature is extremely useful for field research that may involve sourcing for data in remote and rural areas plus it allows you to scale up on your audience reach. 

• Team and Collaboration

 You can add important collaborators and team members to your shared account so that you all can work on forms and responses together. With the multiple users options, you can assign different roles to team members and you can also grant and limit access to forms and folders. 

This feature works with an audit trail that enables you to track changes and suggestions made to your form as the administrator of the shared account. You can set up permissions to limit access to the account while organizing and monitoring your form(s) effectively. 

• Embeddable Form

Formplus allows you to easily add your form with respondents with the click of a button. For instance, you can directly embed your form in your organization’s web pages by adding Its unique shortcode to your site’s HTML. 

You can also share your form to your social media pages using the social media direct sharing buttons available in the form builder. You can choose to embed the form as an iframe or web pop-up that is easy to fill. 

With Formplus, you can share your form with numerous form respondents in no time. You can invite respondents to fill out your form via email invitation which allows you to also track responses and prevent multiple submissions in your form. 

In addition, you can also share your form link as a QR code so that respondents only need to scan the code to access your form. Our forms have a unique QR code that you can add to your website or print in banners, business cards and the like. 

While secondary research can be cost-effective and time-efficient, it requires the researcher to take extra care in ensuring that the data is authentic and valid. As highlighted earlier, data in secondary research can be sourced through the internet, archives, and libraries, amongst other methods. 

Secondary research is usually the starting point of systematic investigation because it provides the researcher with a background of existing research efforts while identifying knowledge gaps to be filled. This type of research is typically used in science and education. 

It is, however, important to note that secondary research relies on the outcomes of collective primary research data in carrying out its systematic investigation. Hence, the success of your research will depend, to a greater extent, on the quality of data provided by primary research in relation to the research context.

Connect to Formplus, Get Started Now - It's Free!

• primary secondary research differences
• primary secondary research method
• secondary data collection
• secondary research examples
• busayo.longe

You may also like:

Exploratory Research: What are its Method & Examples?

Overview on exploratory research, examples and methodology. Shows guides on how to conduct exploratory research with online surveys

Recall Bias: Definition, Types, Examples & Mitigation

This article will discuss the impact of recall bias in studies and the best ways to avoid them during research.

What is Pure or Basic Research? + [Examples & Method]

Simple guide on pure or basic research, its methods, characteristics, advantages, and examples in science, medicine, education and psychology

Primary vs Secondary Research Methods: 15 Key Differences

Difference between primary and secondary research in definition, examples, data analysis, types, collection methods, advantages etc.

Formplus - For Seamless Data Collection

Collect data the right way with a versatile data collection tool. try formplus and transform your work productivity today..

• University of Northern Colorado Libraries

Q. How do I know if an article is a primary or secondary research article?

• 10 Archival Services
• 29 Articles
• 11 Bibliography
• 45 Borrowing
• 11 Campus Services
• 15 Campus Technology
• 16 Computing
• 5 Copy & Fax
• 3 Course Readings
• 8 Databases
• 10 Distance Learning/Extended Studies
• 1 Employment
• 1 Facilities reservations
• 8 Faculty Services
• 36 General services
• 10 Interlibrary Loan
• 8 Library Instruction
• 21 Library Policies
• 9 Media Materials
• 16 Online services
• 10 Prospector
• 26 Research
• 8 UNC History

Answered By: Jay Trask Last Updated: Oct 21, 2020     Views: 237506

A primary research article reports on an empirical research study conducted by the authors. It is almost always published in a peer-reviewed journal. This type of article:

• Asks a research question or states a hypothesis or hypotheses
• Identifies a research population
• Describes a specific research method
• Tests or measures something
• Includes a section called "method" or "methodology." This may only appear in the article, not the abstract.
• Includes a section called "results."

Words to look for as clues include: analysis, study, investigation, examination, experiment, numbers of people or objects analyzed, content analysis, or surveys.

To contrast, the following are not primary research articles (i.e., they are secondary sources):

• Literature reviews
• Meta-Analyses/Review articles (These are studies that arrive at conclusions based on research from many other studies.)
• Chapters in books
• Encyclopedia articles
• Speeches and interviews

Please note: if you are seeking information about primary and secondary sources for historical research, please find information here: https://libguides.unco.edu/history-primary-resources

• Share on Facebook

Was this helpful? Yes 335 No 137

Comments (0)

Related topics.

• Library Instruction

Research 101: Primary vs Secondary

• Research Strategies
• Find Research Articles for Any Subject
• Logos, Pathos, Ethos, and the Evaluation of Information
• Primary vs Secondary
• APA Citation Guide
• MLA Citation Guide
• Research help
• Hierarchy of sources

Where can I find primary sources

The library provides access to many scholarly articles containing original research. These can be found on our Find An Article page.

Additionally, many secondary sources will reference primary sources. If we do not have the book or article you are looking for, you can request it through our online form .

The Library of Congress also maintains an extensive online collection of primary sources from American History , covering a variety of topics, including arts, politics, science, sports, and more !

Are primary sources scholarly?

Some scholarly resources are primary and some are secondary. If an article talks about the authors original research, then it is primary . If a scholarly resource instead reviews or analyzes an event or the works of another author, then the work is secondary .

What is a primary source?

A primary source is a document or item that contains firsthand accounts of an event or original research on a topic. These can include any of the following:

• newspaper articles
• scholarly research articles
• photographs/films
• autobiographies

On the other hand, secondary sources are analysis or restatements of events that the author was not directly involved in. These can include:

• most books about a topic (including textbooks)
• articles that analyze and use research performed by other scholars

Tertiary sources are compilations of information from primary and secondary sources and are used to either provide an overview of a topic or help users to find more resources related to the topic. These typically include:

• Dictionaries and encyclopedias
• Bibliographies and indexes

When to use which?

Primary sources are useful when you want a view of what people saw and how they reacted at the time of an event.

Secondary sources often have a more complete view as more information has been found after the fact. Use them when you are looking for in-depth analysis.

Tertiary sources are used when you are looking for a quick overview or need to know where to find more information on a topic.

Ask a Librarian

• << Previous: Logos, Pathos, Ethos, and the Evaluation of Information
• Next: Plagiarism >>
• Last Updated: Apr 18, 2024 10:45 AM
• URL: https://bigsandy.libguides.com/researchprocess

• Login to Survey Tool Review Center

Secondary Research Advantages, Limitations, and Sources

Summary: secondary research should be a prerequisite to the collection of primary data, but it rarely provides all the answers you need. a thorough evaluation of the secondary data is needed to assess its relevance and accuracy..

5 minutes to read. By author Michaela Mora on January 25, 2022 Topics: Relevant Methods & Tips , Business Strategy , Market Research

Secondary research is based on data already collected for purposes other than the specific problem you have. Secondary research is usually part of exploratory market research designs.

The connection between the specific purpose that originates the research is what differentiates secondary research from primary research. Primary research is designed to address specific problems. However, analysis of available secondary data should be a prerequisite to the collection of primary data.

Advantages of Secondary Research

Secondary data can be faster and cheaper to obtain, depending on the sources you use.

Secondary research can help to:

• Answer certain research questions and test some hypotheses.
• Formulate an appropriate research design (e.g., identify key variables).
• Interpret data from primary research as it can provide some insights into general trends in an industry or product category.
• Understand the competitive landscape.

Limitations of Secondary Research

The usefulness of secondary research tends to be limited often for two main reasons:

Lack of relevance

Secondary research rarely provides all the answers you need. The objectives and methodology used to collect the secondary data may not be appropriate for the problem at hand.

Given that it was designed to find answers to a different problem than yours, you will likely find gaps in answers to your problem. Furthermore, the data collection methods used may not provide the data type needed to support the business decisions you have to make (e.g., qualitative research methods are not appropriate for go/no-go decisions).

Lack of Accuracy

Secondary data may be incomplete and lack accuracy depending on;

• The research design (exploratory, descriptive, causal, primary vs. repackaged secondary data, the analytical plan, etc.)
• Sampling design and sources (target audiences, recruitment methods)
• Data collection method (qualitative and quantitative techniques)
• Analysis point of view (focus and omissions)
• Reporting stages (preliminary, final, peer-reviewed)
• Rate of change in the studied topic (slowly vs. rapidly evolving phenomenon, e.g., adoption of specific technologies).
• Lack of agreement between data sources.

Criteria for Evaluating Secondary Research Data

Before taking the information at face value, you should conduct a thorough evaluation of the secondary data you find using the following criteria:

• Purpose : Understanding why the data was collected and what questions it was trying to answer will tell us how relevant and useful it is since it may or may not be appropriate for your objectives.
• Methodology used to collect the data : Important to understand sources of bias.
• Accuracy of data: Sources of errors may include research design, sampling, data collection, analysis, and reporting.
• When the data was collected : Secondary data may not be current or updated frequently enough for the purpose that you need.
• Content of the data : Understanding the key variables, units of measurement, categories used and analyzed relationships may reveal how useful and relevant it is for your purposes.
• Source reputation : In the era of purposeful misinformation on the Internet, it is important to check the expertise, credibility, reputation, and trustworthiness of the data source.

Secondary Research Data Sources

Compared to primary research, the collection of secondary data can be faster and cheaper to obtain, depending on the sources you use.

Secondary data can come from internal or external sources.

Internal sources of secondary data include ready-to-use data or data that requires further processing available in internal management support systems your company may be using (e.g., invoices, sales transactions, Google Analytics for your website, etc.).

Prior primary qualitative and quantitative research conducted by the company are also common sources of secondary data. They often generate more questions and help formulate new primary research needed.

However, if there are no internal data collection systems yet or prior research, you probably won’t have much usable secondary data at your disposal.

External sources of secondary data include:

• Published materials
• External databases
• Syndicated services.

Published Materials

Published materials can be classified as:

• General business sources: Guides, directories, indexes, and statistical data.
• Government sources: Census data and other government publications.

External Databases

In many industries across a variety of topics, there are private and public databases that can bed accessed online or by downloading data for free, a fixed fee, or a subscription.

These databases can include bibliographic, numeric, full-text, directory, and special-purpose databases. Some public institutions make data collected through various methods, including surveys, available for others to analyze.

Syndicated Services

These services are offered by companies that collect and sell pools of data that have a commercial value and meet shared needs by a number of clients, even if the data is not collected for specific purposes those clients may have.

Syndicated services can be classified based on specific units of measurements (e.g., consumers, households, organizations, etc.).

The data collection methods for these data may include:

• Surveys (Psychographic and Lifestyle, advertising evaluations, general topics)
• Household panels (Purchase and media use)
• Electronic scanner services (volume tracking data, scanner panels, scanner panels with Cable TV)
• Audits (retailers, wholesalers)
• Direct inquiries to institutions
• Clipping services tracking PR for institutions
• Corporate reports

You can spend hours doing research on Google in search of external sources, but this is likely to yield limited insights. Books, articles journals, reports, blogs posts, and videos you may find online are usually analyses and summaries of data from a particular perspective. They may be useful and give you an indication of the type of data used, but they are not the actual data. Whenever possible, you should look at the actual raw data used to draw your own conclusion on its value for your research objectives. You should check professionally gathered secondary research.

Here are some external secondary data sources often used in market research that you may find useful as starting points in your research. Some are free, while others require payment.

• Pew Research Center : Reports about the issues, attitudes, and trends shaping the world. It conducts public opinion polling, demographic research, media content analysis, and other empirical social science research.
• Data.Census.gov : Data dissemination platform to access demographic and economic data from the U.S. Census Bureau.
• Data.gov : The US. government’s open data source with almost 200,00 datasets ranges in topics from health, agriculture, climate, ecosystems, public safety, finance, energy, manufacturing, education, and business.
• Google Scholar : A web search engine that indexes the full text or metadata of scholarly literature across an array of publishing formats and disciplines.
• Google Public Data Explorer : Makes large, public-interest datasets easy to explore, visualize and communicate.
• Google News Archive : Allows users to search historical newspapers and retrieve scanned images of their pages.
• Mckinsey & Company : Articles based on analyses of various industries.
• Statista : Business data platform with data across 170+ industries and 150+ countries.
• Claritas : Syndicated reports on various market segments.
• Mintel : Consumer reports combining exclusive consumer research with other market data and expert analysis.
• MarketResearch.com : Data aggregator with over 350 publishers covering every sector of the economy as well as emerging industries.
• Packaged Facts : Reports based on market research on consumer goods and services industries.
• Dun & Bradstreet : Company directory with business information.

Related Articles

• What Is Market Research?
• Step by Step Guide to the Market Research Process
• How to Leverage UX and Market Research To Understand Your Customers
• Why Your Business Needs Discovery Research
• Your Market Research Plan to Succeed As a Startup
• Top Reason Why Businesses Fail & What To Do About It
• What To Value In A Market Research Vendor
• Don’t Let The Budget Dictate Your Market Research Approach
• How To Use Research To Find High-Order Brand Benefits
• How To Prioritize What To Research
• Don’t Just Trust Your Gut — Do Research
• Understanding the Pros and Cons of Mixed-Mode Research

Subscribe to our newsletter to get notified about future articles

Subscribe and don’t miss anything!

Recent Articles

• Re: Design/Growth Podcast – Researching User Experiences for Business Growth
• Why You Need Positioning Concept Testing in New Product Development
• Why Conjoint Analysis Is Best for Price Research
• The Rise of UX
• Making the Case Against the Van Westendorp Price Sensitivity Meter
• How to Future-Proof Experience Management and Your Business
• When Using Focus Groups Makes Sense
• How to Make Segmentation Research Actionable
• How To Integrate Market Research and UX Research for Desired Business Outcomes
• How To Get Value Out of Your Research Budget

Popular Articles

• Which Rating Scales Should I Use?
• What To Consider in Survey Design
• 6 Decisions To Make When Designing Product Concept Tests
• Write Winning Product Concepts To Get Accurate Results In Concept Tests
• How to Use Qualitative and Quantitative Research in Product Development
• The Opportunity of UX Research Webinar
• Myths & Misunderstandings About UX – MR Realities Podcast
• 12 Research Techniques to Solve Choice Overload
• Concept Testing for UX Researchers
• UX Research Geeks Podcast – Using Market Research for Better Context in UX
• A Researcher’s Path – Data Stories Leaders At Work Podcast
• How To Improve Racial and Gender Inclusion in Survey Design

• Privacy Overview
• Strictly Necessary Cookies

This website uses cookies so that we can provide you with the best user experience possible. Cookie information is stored in your browser and performs functions such as recognising you when you return to our website and helping our team to understand which sections of the website you find most interesting and useful.

Strictly Necessary Cookie should be enabled at all times so that we can save your preferences for cookie settings.

If you disable this cookie, we will not be able to save your preferences. This means that every time you visit this website you will need to enable or disable cookies again.

Numbers, Facts and Trends Shaping Your World

Read our research on:

Full Topic List

Regions & Countries

• Publications
• Our Methods
• Short Reads
• Tools & Resources

Read Our Research On:

America’s public school teachers are far less racially and ethnically diverse than their students

Elementary and secondary public school teachers in the United States are considerably less racially and ethnically diverse as a group than their students – and while the share of Black, Hispanic and Asian American teachers has increased in recent decades, it has not kept pace with the rapid growth in the racial and ethnic diversity of their students, according to data from the  National Center for Education Statistics  (NCES).

About eight-in-ten U.S. public school teachers (79%) identified as non-Hispanic White during the 2017-18 school year, the most recent year for which NCES has published demographic data about them. Fewer than one-in-ten teachers were either Black (7%), Hispanic (9%) or Asian American (2%). And fewer than 2% of teachers were either American Indian or Alaska Native, Pacific Islander, or of two or more races.

By comparison, 47% of all public elementary and secondary school students in the U.S. were White in 2018-19, according to the most recent data available. In that period, around a quarter of public school students were Hispanic (27%), 15% were Black and 5% were Asian. Those shares have increased over time as newer generations of young people entered the classroom. About 1% or fewer were Pacific Islander or identified as American Indian or Alaska Native, while 4% were of two or more races.

To assess the racial and ethnic differences in U.S. public elementary and secondary school teachers and the students they teach, this analysis uses federal data to study these two groups’ makeup and how they have changed over time.

The analysis draws on more than three decades of survey and demographic data from the National Center for Education Statistics  (NCES), including the National Teacher and Principal Survey (formerly called the Schools and Staffing Survey), the Public Elementary/Secondary School Universe Survey and the Education Demographic and Geographic Estimates (EDGE).

The analysis uses the most recent data available for U.S. teachers (the 2017-18 school year) and for students (the 2018-19 school year). Data was unavailable for some states due to low response rates or unmet data collection standards.

White, Black, Asian American, Pacific Islander and American Indian or Alaska Natives in this analysis are those who report being one race and are not Hispanic. Hispanics are of any race. Due to methodological changes, earlier NCES data does not include a category for two or more races, and Asian Americans are grouped with Pacific Islanders.

Not only do White teachers sharply outnumber educators of other races and ethnicities, but they also tend to work in different school environments, the NCES data shows:

• In schools where students come from households with lower incomes, greater shares of teachers are Black or Hispanic than at schools where students are wealthier. One measure of this is the share of students who qualify for free or reduced-price lunch, often used as a proxy measure  for students in lower-income households. In schools where more than 75% of students qualify for the program, 17% of teachers are Hispanic, 14% are Black and 63% are White. At schools where 25% or fewer students qualify, 4% of teachers are Hispanic, 2% are Black and nine-in-ten are White.
• Hispanic, Black and Asian teachers also make up larger shares of the teaching force in city schools (14%, 12% and 3%, respectively) than in rural ones, where 4% are Hispanic or Black and fewer than 1% are Asian American. White teachers, on the other hand, make up 69% of urban school teachers and 90% of rural ones, reflecting the broader  racial and ethnic makeup of America’s communities .
• At schools with bigger percentages of racial or ethnic minority students, larger shares of teachers were Hispanic, Black or Asian American while the reverse was true for schools with more White students. In schools where at least 90% of students were minorities, the share of teachers who were Hispanic was 28%, while 20% of teachers were Black, 5% were Asian and 43% were White. Across schools where at least 90% of students were White, nearly all teachers (97%) also were White. This is similar to the experience for students: Many students go to schools where at least half of their peers are of the same race or ethnicity .

Changes over time

The share of White teachers has decreased about 8 percentage points since the 1987-88 school year (the earliest period with comparable data), when about 87% of teachers were White. Both the number and share of Hispanic and Asian teachers grew during that time span. While the number of Black teachers also has increased since the late ’80s, they have declined as a share of teachers overall.

In the last 30 years, Hispanic teachers have overtaken Black educators as the second-largest racial or ethnic group among U.S. public school teachers. In 1987-88, there were almost three times as many Black public school teachers (191,000) as Hispanic ones (69,000). Since then, the number of Hispanic teachers increased nearly fivefold to 331,000, while the number of Black teachers increased by 25%, to 239,000.

The share of Asian public school teachers has also grown steadily. Between 1987-88 and 2017-18, the number of Asian teachers more than tripled, from 21,000 to 75,000. This happened during a time span (2000 to 2019) when Asian and Hispanic populations in the U.S. were the  fastest-growing racial or ethnic groups . 

Meanwhile, the pattern in racial and ethnic diversity among school principals is similar to that of teachers. In 2017-18, around eight-in-ten principals (78%) were White, compared with 11% who were Black, 9% who were Hispanic and 1% who were Asian American. This represents an increase since 1987-88, when Black, Hispanic and Asian or Pacific Islander principals accounted for 9%, 3% and less than 1% of principals, respectively.

During the same time span, growth in racial and ethnic diversity was much faster among U.S. students than among both teachers and principals. During the 1987-88 school year, 71% of American elementary and secondary school students were White, 14% were Black, 11% were Hispanic and 3% were Asian or Pacific Islander (the two categories were grouped together that year). By the 2018-19 school year, White children made up fewer than half of students (47%). Hispanic students accounted for the second-largest share at 27%, while 15% were Black, 5% were Asian American and 4% identified as multiracial. 

It’s worth pointing out that U.S. public school teachers look different from students not just in terms of race and ethnicity, but also in terms of gender. The most recent data shows that roughly three-quarters of all elementary and secondary school teachers were women, while students are nearly evenly split between boys and girls.

Geographic differences

The racial and ethnic makeup of teachers varies somewhat by state, but there is no state where teachers are less likely to be White than their students, according to separate NCES data for the 2017-18 school year (again, the only recent year where data for both groups was available).

Nationally, state public school teachers are on average 27 percentage points more likely to be White than their students. In Nevada, 77% of public school teachers are White but only one-third of public school students are – a 45-point difference and the widest gap of any state. In all, White teachers make up 79% of the nation’s public school teachers and they account for that share or greater of educators in 37 states. (There is no 2017-18 NCES data available for Maryland or the District of Columbia.)

The only state where White teachers are not the largest racial or ethnic group is Hawaii, where Asian teachers are 42% of the teaching force. White teachers account for about a quarter, and Native Hawaiians or Pacific Islanders comprise 10%.

In New Mexico, 36% of teachers are Hispanic – the largest share in any state. And the largest share of Black teachers is in Georgia, where a quarter of the teaching force is Black.

• Asian Americans
• Black Americans
• Hispanics/Latinos
• Race & Ethnicity
• Racial Bias & Discrimination
• Segregation

Key facts about Asian Americans living in poverty

Methodology: 2023 focus groups of asian americans, 1 in 10: redefining the asian american dream (short film), the hardships and dreams of asian americans living in poverty, key facts about asian american eligible voters in 2024, most popular.

1615 L St. NW, Suite 800 Washington, DC 20036 USA (+1) 202-419-4300 | Main (+1) 202-857-8562 | Fax (+1) 202-419-4372 |  Media Inquiries

Research Topics

• Age & Generations
• Coronavirus (COVID-19)
• Economy & Work
• Family & Relationships
• Gender & LGBTQ
• Immigration & Migration
• International Affairs
• Internet & Technology
• Methodological Research
• News Habits & Media
• Non-U.S. Governments
• Other Topics
• Politics & Policy
• Email Newsletters

ABOUT PEW RESEARCH CENTER  Pew Research Center is a nonpartisan fact tank that informs the public about the issues, attitudes and trends shaping the world. It conducts public opinion polling, demographic research, media content analysis and other empirical social science research. Pew Research Center does not take policy positions. It is a subsidiary of  The Pew Charitable Trusts .

Copyright 2024 Pew Research Center

Terms & Conditions

Privacy Policy

Cookie Settings

Reprints, Permissions & Use Policy

Skip to main page content

• AUTHOR INFO
• Institution: Google Indexer
• Sign In via User Name/Password

Inference of selective forces on house mouse genomes during secondary contact in East Asia

• Kazumichi Fujiwara 1 , 2 ,
• Shunpei Kubo 2 ,
• Toshinori Endo 2 ,
• Toyoyuki Takada 3 ,
• Toshihiko Shiroishi 4 ,
• Hitoshi Suzuki 5 and
• Naoki Osada 2
• 1 Mouse Genomics Resource Laboratory, National Institute of Genetics, Mishima 411-8540, Japan;
• 2 Graduate School of Information Science and Technology, Hokkaido University, Sapporo 060-0814, Japan;
• 3 Integrated BioResource Information Division, RIKEN BioResource Research Center, Tsukuba 305-0074, Japan;
• 4 RIKEN BioResource Research Center, Tsukuba 305-0074, Japan;
• 5 Graduate School of Environmental Science, Hokkaido University, Sapporo 060-0810, Japan
• Corresponding author: nosada{at}ist.hokudai.ac.jp

The house mouse ( Mus musculus ), which is commensal to humans, has spread globally via human activities, leading to secondary contact between genetically divergent subspecies. This pattern of genetic admixture can provide insights into the selective forces at play in this well-studied model organism. Our analysis of 163 house mouse genomes, with a particular focus on East Asia, revealed substantial admixture between the subspecies castaneus and musculus , particularly in Japan and southern China. We revealed, despite the different level of autosomal admixture among regions, that all Y Chromosomes in the East Asian samples belonged to the musculus -type haplogroup, potentially explained by genomic conflict under sex-ratio distortion owing to varying copy numbers of ampliconic genes on sex chromosomes, Slx and Sly . Our computer simulations, designed to replicate the observed scenario, show that the preferential fixation of musculus -type Y Chromosomes can be achieved with a slight increase in the male-to-female birth ratio. We also investigated the influence of selection on the posthybridization of the subspecies castaneus and musculus in Japan. Even though the genetic background of most Japanese samples closely resembles the subspecies musculus , certain genomic regions overrepresented the castaneus -like genetic components, particularly in immune-related genes. Furthermore, a large genomic block (∼2 Mbp) containing a vomeronasal/olfactory receptor gene cluster predominantly harbored castaneus -type haplotypes in the Japanese samples, highlighting the crucial role of olfaction-based recognition in shaping hybrid genomes.

[Supplemental material is available for this article.]

Article published online before print. Article, supplemental material, and publication date are at https://www.genome.org/cgi/doi/10.1101/gr.278828.123 .

Freely available online through the Genome Research Open Access option.

• Received December 7, 2023.
• Accepted March 15, 2024.
• © 2024 Fujiwara et al.; Published by Cold Spring Harbor Laboratory Press

This article, published in Genome Research , is available under a Creative Commons License (Attribution 4.0 International), as described at http://creativecommons.org/licenses/by/4.0/ .

What's this?

This Article

• Published in Advance March 20, 2024 , doi: 10.1101/gr.278828.123 Genome Res. 2024. © 2024 Fujiwara et al.; Published by Cold Spring Harbor Laboratory Press
• » Abstract Free
• Full Text (PDF) Free
• Supplemental Material
• gr.278828.123v1
• gr.278828.123v2 most recent

Article Category

• Alert me when this article is cited
• Alert me if a correction is posted
• Similar articles in this journal
• Article Metrics
• Similar articles in PubMed
• Download to citation manager
• Permissions

Citing Articles

• Load citing article information

Google Scholar

• Articles by Fujiwara, K.
• Articles by Osada, N.

PubMed/NCBI

• PubMed citation
• Profile for Fujiwara, K. http://orcid.org/0000-0002-7840-4676
• Profile for Endo, T. http://orcid.org/0000-0002-5711-2527
• Profile for Takada, T. http://orcid.org/0000-0001-6796-2085
• Profile for Shiroishi, T. http://orcid.org/0000-0003-0535-2016
• Profile for Suzuki, H. http://orcid.org/0000-0003-4426-385X
• Profile for Osada, N. http://orcid.org/0000-0003-0180-5372

Preprint Server

Current issue.

• February 2024, 34 (2)

From the Cover

• Long cell-free DNA in human plasma
• Sampling biases in single-cell/nucleus RNA-seq
• Transcriptional repression in response to DNA damage
• Z-RNA over intercalated motifs in lncRNA
• Plant genomics
• Alert me to new issues of Genome Research
• Advance Online Articles
• Submit a Manuscript
• GR in the News
• Editorial Board
• E-mail Alerts & RSS Feeds
• Recommend to Your Library
• Job Opportunities

• Author Info

Copyright © 2024 by Cold Spring Harbor Laboratory Press

• Print ISSN: 1088-9051
• Online ISSN: 1549-5469

Department of Economics

Aeasp 2024 fellow  le nguyen.

Le Thi Nguyen , a Ph.D. student in the Department of Economics at Howard University, brings a distinguished academic background from Vietnam, having secured the first rank in both B.S. and M.A. degrees. With an impressive 14 years of experience in research, teaching, and writing book chapters in microeconomics and macroeconomics, her expertise lies in analyzing the impact of fiscal and monetary policies on economic growth. She extends this knowledge to the micro-level, studying customer and firm behavior and researching policies to enhance export activities in Vietnam. Specializing in growth and development economics and international economics, she is proficient in analytical tools like SPSS, EVIEWS, STATA, and SQL. She is enthusiastic about exploring new software in data analytics. She actively contributes to academia as a teaching assistant at the Howard University Department of Economics and a research assistant at the Center of Excellence in Housing and Urban Research and Policy (CHURP). She aims to become a skilled macropolicy analyst, offering effective economic management consultation for firms and demonstrating her commitment to advancing economic analysis and policy implementation.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Iran J Public Health
• v.42(12); 2013 Dec

Secondary Data Analysis: Ethical Issues and Challenges

Research does not always involve collection of data from the participants. There is huge amount of data that is being collected through the routine management information system and other surveys or research activities. The existing data can be analyzed to generate new hypothesis or answer critical research questions. This saves lots of time, money and other resources. Also data from large sample surveys may be of higher quality and representative of the population. It avoids repetition of research & wastage of resources by detailed exploration of existing research data and also ensures that sensitive topics or hard to reach populations are not over researched ( 1 ). However, there are certain ethical issues pertaining to secondary data analysis which should be taken care of before handling such data.

Secondary data analysis

Secondary analysis refers to the use of existing research data to find answer to a question that was different from the original work ( 2 ). Secondary data can be large scale surveys or data collected as part of personal research. Although there is general agreement about sharing the results of large scale surveys, but little agreement exists about the second. While the fundamental ethical issues related to secondary use of research data remain the same, they have become more pressing with the advent of new technologies. Data sharing, compiling and storage have become much faster and easier. At the same time, there are fresh concerns about data confidentiality and security.

Issues in Secondary data analysis

Concerns about secondary use of data mostly revolve around potential harm to individual subjects and issue of return for consent. Secondary data vary in terms of the amount of identifying information in it. If the data has no identifying information or is completely devoid of such information or is appropriately coded so that the researcher does not have access to the codes, then it does not require a full review by the ethical board. The board just needs to confirm that the data is actually anonymous. However, if the data contains identifying information on participants or information that could be linked to identify participants, a complete review of the proposal will then be made by the board. The researcher will then have to explain why is it unavoidable to have identifying information to answer the research question and must also indicate how participants’ privacy and the confidentiality of the data will be protected. If the above said concerns are satisfactorily addressed, the researcher can then request for a waiver of consent.

If the data is freely available on the Internet, books or other public forum, permission for further use and analysis is implied. However, the ownership of the original data must be acknowledged. If the research is part of another research project and the data is not freely available, except to the original research team, explicit, written permission for the use of the data must be obtained from the research team and included in the application for ethical clearance.

However, there are certain other issues pertaining to the data that is procured for secondary analysis. The data obtained should be adequate, relevant but not excessive. In secondary data analysis, the original data was not collected to answer the present research question. Thus the data should be evaluated for certain criteria such as the methodology of data collection, accuracy, period of data collection, purpose for which it was collected and the content of the data. It shall be kept for no longer than is necessary for that purpose. It must be kept safe from unauthorized access, accidental loss or destruction. Data in the form of hardcopies should be kept in safe locked cabinets whereas softcopies should be kept as encrypted files in computers. It is the responsibility of the researcher conducting the secondary analysis to ensure that further analysis of the data conducted is appropriate. In some cases there is provision for analysis of secondary data in the original consent form with the condition that the secondary study is approved by the ethics review committee. According to the British Sociological Association’s Statement of Ethical Practice (2004) the researchers must inform participants regarding the use of data and obtain consent for the future use of the material as well. However it also says that consent is not a once-and-for-all event, but is subject to renegotiation over time ( 3 ). It appears that there are no guidelines about the specific conditions that require further consent.

Issues in Secondary analysis of Qualitative data

In qualitative research, the culture of data archiving is absent ( 4 ). Also, there is a concern that data archiving exposes subject’s personal views. However, the best practice is to plan anonymisation at the time of initial transcription. Use of pseudonyms or replacements can protect subject’s identity. A log of all replacements, aggregations or removals should be made and stored separately from the anonymised data files. But because of the circumstances, under which qualitative data is produced, their reinterpretation at some later date can be challenging and raises further ethical concerns.

There is a need for formulating specific guidelines regarding re-use of data, data protection and anonymisation and issues of consent in secondary data analysis.

Acknowledgements

The authors declare that there is no conflict of interest.

• Fielding NG, Fielding JL (2003). Resistance and adaptation to criminal identity: Using secondary analysis to evaluate classic studies of crime and deviance . Sociology , 34 ( 4 ): 671–689. [ Google Scholar ]
• Szabo V, Strang VR (1997). Secondary analysis of qualitative data . Advances in Nursing Science , 20 ( 2 ): 66–74. [ PubMed ] [ Google Scholar ]
• Statement of Ethical Practice for the British Sociological Association (2004). The British Sociological Association, Durham . Available at: http://www.york.ac.uk/media/abouttheuniversity/governanceandmanagement/governance/ethicscommittee/hssec/documents/BSA%20statement%20of%20ethical%20practice.pdf (Last accessed 24November2013)
• Archiving Qualitative Data: Prospects and Challenges of Data Preservation and Sharing among Australian Qualitative Researchers. Institute for Social Science Research, The University of Queensland, 2009 . Available at: http://www.assda.edu.au/forms/AQuAQualitativeArchiving_DiscussionPaper_FinalNov09.pdf (Last accessed 05September2013)

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Online First
• Locum doctor working and quality and safety: a qualitative study in English primary and secondary care
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0002-8129-8376 Jane Ferguson 1 ,
• http://orcid.org/0000-0001-9325-3362 Gemma Stringer 2 ,
• http://orcid.org/0000-0002-0696-480X Kieran Walshe 2 ,
• http://orcid.org/0000-0002-2972-7911 Thomas Allen 3 , 4 ,
• http://orcid.org/0000-0003-1621-8648 Christos Grigoroglou 3 ,
• http://orcid.org/0000-0002-2958-915X Darren M Ashcroft 5 ,
• http://orcid.org/0000-0001-6450-5815 Evangelos Kontopantelis 6 , 7
• 1 Health Services Management Centre, School of Social Policy , University of Birmingham , Birmingham , UK
• 2 Alliance Manchester Business School , University of Manchester , Manchester , UK
• 3 Manchester Centre for Health Economics, Division of Population Health, Health Services Research and Primary Care , University of Manchester , Manchester , UK
• 4 Danish Centre for Health Economics , University of Southern Denmark , Odense , Denmark
• 5 NIHR Greater Manchester Patient Safety Research Collaboration (PSRC), Division of Pharmacy and Optometry, Faculty of Biology Medicine and Health , University of Manchester , Manchester , UK
• 6 Division of Informatics, Imaging and Data Sciences , University of Manchester , Manchester , UK
• 7 NIHR School for Primary Care Research, Centre for Primary Care, Division of Population Health, Health Services Research and Primary Care , University of Manchester , Manchester , UK
• Correspondence to Dr Jane Ferguson, Health Services Management Centre, University of Birmingham, Birmingham, UK; j.ferguson.1{at}bham.ac.uk

Background The use of temporary doctors, known as locums, has been common practice for managing staffing shortages and maintaining service delivery internationally. However, there has been little empirical research on the implications of locum working for quality and safety. This study aimed to investigate the implications of locum working for quality and safety.

Methods Qualitative semi-structured interviews and focus groups were conducted with 130 participants, including locums, patients, permanently employed doctors, nurses and other healthcare professionals with governance and recruitment responsibilities for locums across primary and secondary healthcare organisations in the English NHS. Data were collected between March 2021 and April 2022. Data were analysed using reflexive thematic analysis and abductive analysis.

Results Participants described the implications of locum working for quality and safety across five themes: (1) ‘familiarity’ with an organisation and its patients and staff was essential to delivering safe care; (2) ‘balance and stability’ of services reliant on locums were seen as at risk of destabilisation and lacking leadership for quality improvement; (3) ‘discrimination and exclusion’ experienced by locums had negative implications for morale, retention and patient outcomes; (4) ‘defensive practice’ by locums as a result of perceptions of increased vulnerability and decreased support; (5) clinical governance arrangements, which often did not adequately cover locum doctors.

Conclusion Locum working and how locums were integrated into organisations posed some significant challenges and opportunities for patient safety and quality of care. Organisations should take stock of how they work with the locum workforce to improve not only quality and safety but also locum experience and retention.

• Health services research
• Patient safety
• Qualitative research
• Quality improvement

Data availability statement

No data are available.

This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See:  https://creativecommons.org/licenses/by/4.0/ .

https://doi.org/10.1136/bmjqs-2023-016699

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

WHAT IS ALREADY KNOWN ON THIS TOPIC

Despite longstanding policy concerns about the implications of locum working for quality and safety, there has been little empirical research. Understanding how organisations engage, support and work with locums and how locum doctors integrate and interact with the complex and changing systems in which they work is essential if quality and safety are to be improved.

WHAT THIS STUDY ADDS

This qualitative study examines the perspectives of locums, patients and people who work with locums to identify the implications of temporary medical working for quality and safety.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

Organisations should examine how they engage, support and work with locums. Organisations and locums need to reflect on whether their practices support a collective approach to patient safety and quality of care.

Introduction

Temporary doctors, often known as locums, are a vital resource that enable healthcare organisations to deliver care by flexing capacity and covering staffing gaps. In the United Kingdom, all doctors, other than those in their first year of training after qualifying, can work as a locum. Locum work can vary from very short-term (a single shift) to longer-term assignments (weeks, months or even sometimes years). Locums find work through various platforms, including locum agencies, online job platforms, professional networks or word of mouth. Locum agencies typically have some governance responsibilities (such as compliance with regulations and licensing requirements), but the extent of these responsibilities varies and the NHS in England has no oversight over how recruitment agencies operate. Despite concerns among policymakers, healthcare providers, professional associations and professional regulators about the implications of locum working for quality and safety and cost, 1–3 there is limited robust empirical research to evidence or support those concerns.

The workforce retention crisis is a significant challenge in healthcare internationally 4–6 and persistent understaffing poses a serious risk to patient safety. 7 8 In the UK, high doctor turnover has been linked to poorer service and health outcomes 9 and has led NHS trusts and general practices (GPs) to be ‘overly reliant’ 3 on temporary staff to fill rota gaps. 10 11 Expenditure on temporary staff in the NHS in England increased from £3.45 billion to £5.2 billion between 2021 and 2022. 3 12 The NHS Long Term Workforce Plan aims to reduce reliance on temporary staff and make substantive employment the most cost-effective and attractive option. 3 However, with the vacancy rate in the NHS projected to increase, 13 locums are likely to continue to be essential to maintaining service provision, especially in shortage specialities such as psychiatry. 14

An obvious implication of locum working is a reduced likelihood of organisational and team integration, 15 familiarity and a shared understanding of ‘the way things are done around here’. 16 Locums are likely to be less familiar with teams and other contextual factors relevant to providing safe and effective care 17 and more likely to be situated on the periphery of organisational structures, teams and governance systems 1 18 Teamwork represents a powerful process to improve patient care, 19 20 and trust, shared understanding, communication and collaboration have been associated with better patient outcomes. 21 22 The ability of healthcare teams to develop and maintain team situational awareness, or a shared perception, comprehension and subsequent projection of what is going on in complex and changing clinical environments, has been described as crucial for patient safety. 23 24 Through participation and working together, 25 teams gain an understanding of the roles, skills and competencies of others to demonstrate ‘collective competence’, 26 27 which is critical for healthcare delivery, 28 29 and existing research on locums suggests a need for better integration into teams to improve quality and safety. 30 31

Context matters for patient safety and quality improvement, 32 33 yet the limited evidence 17 relating to locums practice is largely ‘acontextual’ and tends to ignore the role of the organisation in the integration of temporary staff, focusing instead on the potential risks locums present as individual clinicians, 17 30 which is perhaps unsurprising given the liminal space locums occupy. In the UK, responsibility for the quality and safety of healthcare services is shared primarily between organisations and the individual professionals working within them. 34 Organisations are responsible for creating systems and environments that promote and protect clinical governance and enable all doctors to meet their professional obligations, while doctors are expected to participate in the systems and processes put in place by regulators and organisations to protect and improve patient care. 35 However, NHS trusts and primary care organisations procure the services of locum doctors without assuming the responsibilities normally associated with an employer–employee relationship 30 and locums often struggle to participate in teams and governance systems that were designed for doctors working in conventional employment relationships. 18 36

There is longstanding debate about the role of individual accountability in patient safety and how responsibility is distributed between organisations and individuals. 37 A systems approach reasons that adverse events are likely to occur as a result of system failures rather than individual failures, 38 and patients are protected from mistakes by well designed systems and environments that promote safety cultures. 39 But locums are often positioned at the periphery of these systems, 30 and doctors who are new to and also peripheral to organisations, and organisations who are inexperienced with and unsupportive of locums are unlikely to be able to perform optimally. 40

The aim of this research was to provide evidence on how locum working arrangements impact quality and safety and the implications of locum working for patients, locums and health service organisations in primary and secondary care in the English NHS. Locum doctors are an essential and growing part of the healthcare workforce 1 who have been largely ignored in healthcare workforce research. This research addresses a gap in the empirical evidence base on how locum doctor working arrangements affect quality and safety, and provides, for the first time, an in-depth exploration that includes perspectives from patients, locums and the people they work with.

Study design and setting

A qualitative semi-structured interview and focus group study was conducted with locums, people working with locums, and patients with experience of being treated by locums. Participants were purposively sampled through 11 organisations, including NHS trusts, primary care practices, statutory NHS bodies and locum agencies. Locum doctor participants were recruited through these organisations, locum recruitment agencies and networks. We used purposive, snowball and convenience sampling, drawing on intelligence from stakeholders, including our project advisory group, to identify and recruit organisations and participants. Patient participants were recruited through patient and contributor forums. The forum involved active partnership between patients and researchers in the research process to develop research which is relevant and useful to patient and public needs. Participant demographics were monitored to ensure representation across a broad range of roles in primary and secondary care and to increase diversity in terms of gender and ethnicity (see table 1 ).

• View inline

Characteristics of study participants

Data collection

Three semi-structured interview and focus group guides were developed for use with locums, people working with locums and patients with experience of being treated by locums (as shown in online supplemental files 1-3 ). Our previous review of the literature relating to quality and safety and locum work 17 informed the schedules as well as the initial coding and thematic development. Schedules were also refined and informed by our patient and public involvement (PPI) forum and our project advisory group. Each schedule was intended to explore locum doctor working arrangements with a particular focus on understanding how locum doctor working may affect the safety and quality of care and what strategies or systems organisations and individuals used to assure or improve quality and safety. The topic guides for locums and people working with locums also covered governance and support, the impact of the COVID pandemic and policies and initiatives used to support locums.

Supplemental material

Interviews and focus groups were transcribed verbatim by a professional transcription company and organised into codes and themes using the software package NVivo. 41 Reflexive thematic analysis (RTA) 42 was used and involved familiarisation with the data by reading and re-reading the transcripts and field notes; coding the dataset and collating all relevant data extracts; generating initial themes by examining the codes and collated data to identify significant broader patterns of meaning across the dataset; reviewing themes by questioning whether themes answered the research question and told a convincing story of the data and combining, splitting and discarding themes as necessary; defining and naming themes by developing a detailed analysis of each theme; and finally the analytical write up which positioned the analysis in relation to existing literature. 43 RTA acknowledges the active role of researchers in knowledge production and the researcher’s subjectivity as the analytic resource. 42 RTA recognises interpretive variability between researchers based on differences in their knowledge and skills, theoretical assumptions and differences in how they responded to the dataset is acknowledged and expected. 42 The research team worked reflexively discussing their personal biases and their potential impact on the research at regular meetings throughout the data collection and analysis period. Our PPI forum were also involved in data collection and analysis, and offered a form of triangulation to enhance rigour, challenge and alternative interpretations of the findings. 44 Analysis adopted a constructionist epistemology, in that while we acknowledged the importance of recurrence in generating themes, meaning and meaningfulness were the central criteria in the coding process. 42

After themes were developed, an abductive approach was taken to position findings against a background of existing theory and knowledge. 17 30 This provided a way of constructing empirically based theorisations without confining theory to predefined concepts. 45 This approach integrated inductive data-driven coding with deductive theory-driven interpretation; aiming to find a middle ground between inductive and deductive methods and the most logical solution and useful explanation for phenomena. 45

We conducted 130 interviews with 88 participants who worked in healthcare and 42 patients took part in focus groups and one-to-one interviews. Participants included locums, permanently employed doctors; nurses and other health professionals; medical directors/clinical leaders; responsible officers (ROs are accountable for local clinical governance processes and focus on the performance of doctors) and appraisers; leads for medical staffing and clinical governance and practice managers (see table 2 ). Three experienced qualitative researchers (JF, GS and KW) and two members of the PPI forum (MM and MS) carried out five focus groups with 30 patients, and JF and GS carried out 12 one-to-one interviews. Data were collected between March 2021 and April 2022 during the COVID pandemic using video conferencing software (n=126) or over the phone (n=4) at a time convenient to participants. Interviews and focus groups ranged in length from 23 to 171 min, with the average interview being 59 min.

Healthcare organisations and participant roles

Thematic framework

Our findings are presented under five broad and interrelated themes that examine how locum work relates to and impacts quality and safety: ‘familiarity’ with an organisation and its patients and staff; ‘balance and stability’ in services with lots of locums; ‘discrimination and exclusion’ towards locums and their effects; ‘defensive practice’ by locums; and the positioning of locums outside clinical governance arrangements.

Familiarity: knowing who, where and how

Locums described often working in unfamiliar environments, sometimes with minimal induction and varying levels of support. Unfamiliarity, lack of access to or other restrictions on computer systems, policies, procedures and buildings meant that locums were not always able to do their job safely, productively or effectively.

That’s probably the biggest sort of safety aspect that sticks in my mind, is that it is unbelievably frustrating to have to learn a whole new set of patients from day to day … when I was signed up to four different hospitals, plus the locum agencies, I very quickly realised that not only is it the fact that you don’t know the patients from day to day, if you’re chopping and changing site the whole time, then store cupboards are laid out differently, ways of contacting relevant staff members are different, you’ve got to recognise what code to put in to bleep someone that’s different at every single site. (Interview 23, locum, secondary care)

Locum working sometimes created extra work for permanent staff who were responsible for inducting, training and supervising locums. The amount of additional workload was dependent on contextual factors, such as the experience of the locum, organisational support and length of placement, access to systems and what terms and conditions locums or organisations had negotiated. Locum reliance on permanent staff meant that care could be delayed, partially completed or not completed at all, which sometimes caused resentment.

Some of the things that we don’t … like, for example, procedures of limited clinical value that we don’t refer in for, they won’t know about those in our areas … So they’ll do referrals that we then will get pulled on. They’ll maybe prescribe medications that are not first line medications within our own formulary. So we see quite a bit of that, you know, there’s quite a lot of tidying up to be done afterwards or work. They generate that. So whilst we meet the patient numbers, they create a lot of work for the rest of the team. (Interview 3, practice manager, primary care)

Locums mitigated risks related to working in unfamiliar environments by avoiding organisations considered chaotic or unsafe, working below their grade to avoid having responsibility in unfamiliar organisations where they may not be supported or included in the team or working in a limited number of organisations to increase familiarity.

Most locums take jobs, locum work below their grade. So a person who’s at a registrar level would take a locum work as an SHO (senior house officer), because they don't know the trust that well. (Interview 55, locum, secondary care)

However, lack of familiarity and discontinuity could at times be beneficial for patients and organisations as fresh perspectives offered by locums led to different routes of treatment or management, and could alter organisational cultures or practices.

So that [locum] doctor, through that line of questioning and not having any sort of prior history … ordered the right tests and didn’t feel constrained in that practice about what tests that they could order. And someone subsequently … because when you get referred to hospital, the consultant said that that doctor was very much on the ball. And, of course, that’s a change to lifelong medication. And literally within a month of the medication kicking in, it transformed my life. (Focus group A, patient 1)

Balance and stability

The balance between locum and permanent staff had implications for quality and safety, organisational leadership, long-term planning and governance. Locums were often employed to deliver immediate services and consequently were less likely to be involved in team and organisational development. Locums recognised that having ‘an NHS run by locums’ was detrimental to quality and safety, and some avoided organisations that were locum dependent for this reason. Well functioning established teams were regarded as better able to incorporate a small number of locums without being significantly impacted.

Locum work, my view on it is they’re there to fill a gap. They shouldn’t be relied upon to deliver a service Monday to Friday, day in, day out, week in, week out. And unfortunately my trust see it as that, though, that’s my worry that they feel they’re not just plugging a gap, they’re almost as a workforce … (Interview 84, lead GP, primary and secondary care)

Departments that were disproportionately locum dependent were often perceived to lack clinical leadership and direction. An absence of consistent medical leadership meant that quality improvement was slower or less likely to happen, and trusting relationships between staff were harder to establish.

If you get a department that is disproportionately locum dependent, then it stagnates, it doesn't progress. Things like implementation of new NICE guidance, for example, that sort of thing tends not to happen or happen less well, less quickly. (Interview 30, responsible officer, secondary care)

Discrimination and exclusion

Most locums described negative behaviours and attitudes from staff and some patients, which impacted their involvement, inclusion and experiences in organisations. Negative attitudes and behaviours towards locums could affect turnover, locum well-being, team dynamics and potentially patient safety. Perceived disparities between pay, workload, competence and organisational and team commitment between locums and permanent staff could be sources of resentment and influenced how locums were treated and viewed. This compromised staff communication and reduced the sharing of important patient information.

I guess like any temporary post really, you struggle to invest in them, don't necessarily see them as being part of the team. Not very positive about them, particularly junior staff, particularly in the acute trusts. We'd have locums refusing to come back because of the treatment of the midwives. (Interview 86, clinical lead, secondary care)

Negative perceptions of competency and safety meant that locums were often stigmatised, marginalised and excluded. The identity of locum intersected and overlapped with other identities and was described as ‘layering up’ with ethnicity and gender to further exacerbate discrimination.

Oh, doctors coming over from Germany. There was one locum … that administered a dose of something and the patient died, and then there’s this whole layer of extra negativity attached to locum doctors in general because of what one doctor did, and that doctor happened to be someone from a different ethnicity … As a UK born and qualified doctor I can see that those overseas get it but I can also see that I have experienced that as well. So yeah, it can layer up with the whole locum thing. (Interview 59, locum GP, primary care)

A sense of othering and being seen as less was particularly evident during the COVID pandemic when resources were limited. Some locums described how they were not afforded the same protections as permanent members of staff and were sometimes expected to take on riskier work.

I’ve worked in another practice where, because they live on locums and they live on ad hoc locums, you’re a piece of dirt under the shoe. You don’t get gloves, you didn’t have aprons, you didn’t have a face visor, you didn’t have safety specs, you have to ask for a mask. Not only are you not treated as a service provider, you’re not treated as a colleague, someone with knowledge. (Interview 44, locum GP)

Defensive practice by locums

Locums recognised that they were likely to be scapegoated if things went wrong, and some locums described being more likely to practice defensively. Defensive practice has been defined as deviation from standard practice to avoid litigation, complaints or criticism. 46 Participants reported instances of defensive practice which involved providing services (eg, tests, referrals) or avoiding high-risk decisions, usually to reduce the risk of adverse outcomes such as patient complaints or potential termination of contract at short notice. Locums described practicing defensively because they were attempting to practice as safely as possible in complex unfamiliar environments where they were professionally isolated and perceived negatively. Permanent members of staff could perceive that locums practiced defensively because they lacked confidence in their abilities. The diversion of resources away from more clinically relevant activities placed additional burden on teams, who were already facing significant workload challenges.

Being risk averse and practising defensive medicine usually means more tests, more referrals, whereas holding risk tends to be disadvantageous for you as a locum because what’s the benefit to you of not doing that. You’re benefiting the system by rationing resource, the patient won’t thank you. (Interview 35, locum GP)

Locums described avoiding making decisions when risks to employment or medical licenses were perceived as high. Locums felt they were more vulnerable to criticisms of their clinical competence and disempowered to make decisions. Others felt that some locums were simply avoiding work and evaded responsibility for patients by pushing work onto others or into the future.

You don’t interfere, very simple. Over time locums have learned that if you interfere, if you participate in the team, you participate in patient care, [and this] is when you get into trouble … Well most of the locums that I know will just say, okay, there’s already somebody else who’s made a decision, it’s not my job to make a decision, I just follow through. If things go wrong, call the senior person and be done with it, that’s the end of my role. Actually doing something to protect a patient is not important for a locum because the risk is too high. (Interview 55, locum, secondary care)

Locums fall outside clinical governance arrangements

Governance practices in relation to locums varied widely and were not generally regarded as being as robust in comparison to permanently employed doctors. Responsibility for involving locum doctors in performance feedback, supervision, educational opportunities, appraisal and quality improvement was unclear. While some organisations included locums in their governance activities, others regarded locum work as transactional; where the locum was there to provide a finite service and the organisation assumed no responsibilities for their performance, development or oversight. There were concerns that governance structures were modelled on and designed for permanently employed doctors and did not work for locums. When deficits in performance were undetected or unaddressed, doctor performance and patient safety could be jeopardised.

I think it’s a remote world. It’s like a cloud, you know, it’s like the cloud. We talk about the cloud when it comes to storing information. And I think locum world is a bit like that … And I don’t know the doctors anywhere like as much as I did when I was an RO in the NHS, I knew them all personally. If I used to have a problem, I used to get them in my office there and then, chat it all through, sort it. Can’t do that in locum world, it might take me four days to get hold of the doctor, some of them won’t respond immediately … They don’t know me and I don’t know them. (Interview 51, responsible officer, locum agency)

The absence of typical recruitment processes (involving meeting a doctor, carrying out an interview and following up on references) meant that healthcare organisations were reliant on partial information from locum agencies, which made it difficult to determine competency, scope of practice and suitability for a role. However, staff shortages and a requirement to meet safe staffing ratios meant that organisational leaders had little recourse of action if they were unsure about a doctor’s capability, which caused anxiety and frustration. This suggests that the provision of healthcare superseded ensuring safety standards and necessitated accepting one of two objectionable alternatives; accepting gaps in staffing that may jeopardise patient safety or accepting unknown doctors; each of which may compromise patient safety.

If a locum turns up and I have serious doubts about their ability to do the job to the required standard, I don’t have any recourse … And therefore I’m in a position where either I accept this locum or I don’t. There’s not much in the way of middle ground. Not accepting them is a really unpalatable choice because if I say look, I’m sorry, I don’t think you’re up to this, I think you should go home, that leaves me with a gap. (Interview 30, consultant and responsible officer, secondary care)

Similar governance and information sharing problems were described by locum agencies and NHS organisations; both described difficulties in gathering and sharing feedback. When concerns were raised, participants were often uncertain as to what happened to the information they provided and whether it was shared or acted on. Locums often did not get to hear about concerns raised about them, meaning learning opportunities were missed.

It would give you more confidence if you heard back. And sometimes I'll pick up the phone and you try to do the best you can to make sure this information gets passed on. But I just have this nagging doubt that I'm not always convinced it does. (Interview 30, responsible officer, secondary care)

There was also a perception from some locum agency responsible officers that while most locum doctors were excellent, there were some locums who were isolated and in need of organisational and professional support.

You have to accept that whilst within the agency world, 80 per cent of the doctors we place are excellent, and have no problems, and do a great job, perhaps 20 per cent are those that have shaken down to that 20 per cent in the agency world, because they’ve not succeeded in the NHS, they’ve not got a substantive place, they are lost souls. And they are less able to cope with the vicissitudes of busy clinical life and professional life within a large organisation such as the NHS. (Interview 47, responsible officer, locum agency)

Our findings provide some profound and concerning insights for patient safety and quality of care. The ways in which locums were recruited, inducted, deployed and integrated, and supported by organisations undoubtedly affected quality and safety. Our findings indicate that regardless of their level of experience, it was unlikely that locum doctors would be able to function optimally in unfamiliar environments; and organisations who had poor supportive infrastructure and governance mechanisms for locums were less likely to deliver high-quality safe services.

Locums were often regarded as organisational outsiders—positioned at the periphery of the team and the organisation. The implications of transience and peripheral participation were weaker relationships with organisations, teams, peers and patients, leading some to suggest locum working is better suited to experienced doctors. 47 Consistent with previous research, 48 frequent variation in process, systems and equipment, combined with disruption in relationships and a lack of mutual awareness of team skills and competencies, decreased collective competence, placed additional burden on the wider healthcare team and reduced patient safety. As others have found in research on safe staffing and nursing, 49 temporary staff are not effective substitutes for staff who regularly work in the organisation. Safe medical staffing is not just achieved by filling rota gaps, but also team composition and doctors’ familiarity with the team and organisation must be taken into account. Regulatory agencies should consider locum usage in their inspections and perhaps be particularly concerned when organisations have ‘services run on locums’.

Our research found, as others have, 18 that organisations and doctors sometimes struggled to meet their governance obligations and that governance activities differed based on contractual status and organisational policies and norms, with systems being less robust for locums. This research has highlighted that much still needs to be done to develop governance systems that promote and protect the interests of patients and create an environment which supports locum doctors in meeting their professional obligations.

More positively, locum doctors are a potentially valuable source of information about safety concerns, faulty systems or poor conduct. 50 Locums move between organisations, have broad systems knowledge and are perhaps better placed to identify some quality and safety issues than permanent doctors. However, findings indicate opportunities for shared learning were often missed. Locums recognised their precarity and vulnerability when offering second opinions, sharing improvement ideas or voicing safety concerns; meaning opinions were not always offered and concerns were not always raised. Failure to voice concerns is a persistent problem in healthcare, 51 and locums may be even less inclined to offer potentially valuable information about safety concerns because of their perceptions of unsupportive organisational climates.

Our findings shed light on how temporary doctors fit into the enduring debate 37 around how responsibility between organisational systems and individual professionals is distributed. Locums appear to represent a subsection of the medical profession for whom the wider paradigm shift from a focus on individual blame to a systems approach 52 appeared not to have been made. Locums were often not regarded as a part of the organisation, and therefore the system, and not afforded the same protections as permanent staff when things went wrong. Blaming locums when things go wrong and punishing or sanctioning individuals who make errors in contexts that were not designed to incorporate temporary workers may divert attention from understanding inadequately designed, poorly functioning systems, or indeed the individual practice of other doctors. While we should take into account systemic factors that impede locums from performing safely, we should expect high standards of healthcare professionals, be cognisant of individual agency and recognise the distinction between blaming someone and holding them responsible. 53

Strengths and limitations

This large qualitative study explores locum working and quality and safety in an under-researched, yet growing area of the medical workforce. However, sites were all based in England, which means caution should be taken when extrapolating findings. Similar research in other countries and contexts to understand more about locum doctor working and quality and safety is therefore important. It is possible that our sample may have been skewed towards locums, healthcare professionals and patients who had more negative perceptions and experiences, although accounts resonate with previous research 30 and patient perspectives were generally positive. Our data were collected during the COVID pandemic, which may have affected findings as there was a reduction in locum working during that time 10 11 ; it also meant we were unable to carry out observations, which would have strengthened our findings and mitigated some of the inherent limitations of interviews, such as recall bias. We used both one-to-one interviews and focus groups in data collection. Although flexibility in data collection meant that participants had the option to take part in an interview or a focus group, these methods are used for different reasons and produce different data. There may have been differences in what participants disclosed depending on the method

Our findings show that the way in which doctors who worked on a temporary basis were integrated into organisations posed some significant challenges and opportunities for patient safety and quality of care, and that both organisations and locums had a part to play in improvement. Doctors working as locums are a heterogeneous group with differing backgrounds, experiences, skills and capabilities that likely reflect the variability seen in the wider population of doctors. Locums are working in the same pressured and imperfect systems as other health workers; it is vital that systemic problems are not mistaken for problems about individuals and important to recognise that a locum is not a type of doctor but a way of working. Our findings are a call to action for organisations to take stock of how they engage, support and work with locums, and asks both locums and organisations to reflect on whether their practices support a collective approach to patient safety and quality of care.

Ethics statements

Patient consent for publication.

Not applicable.

Ethics approval

This study involves human participants and was approved by the Health Research Authority North West—Haydock Research Ethics Committee 20/NW/0386. Participants gave informed consent to participate in the study before taking part.

• General Medical Council
• NHS England
• World Health Organization
• Health Social Care Committee
• Harvey PR ,
• Trudgill NJ
• Bower P , et al
• Grigoroglou C ,
• Kontopantelis E , et al
• Shembavnekar N ,
• Bazeer N , et al
• Glatter R ,
• Papadakos P ,
• Ferguson J ,
• Schmutz JB ,
• DiazGranados D ,
• Dietz AS , et al
• Sutradhar R ,
• Jerath A , et al
• Oeppen RS , et al
• Pollack AH ,
• Mishra SR ,
• Apodaca C , et al
• McLaney E ,
• Morassaei S ,
• Hughes L , et al
• Firth-Cozens J
• Tazzyman A ,
• Walshe K , et al
• Pouwels KB ,
• van Hecke O , et al
• Aveling E-L ,
• Dixon-Woods M
• Blumenthal DM ,
• Olenski AR ,
• Tsugawa Y , et al
• QSR International Pty Ltd
• Stewart D ,
• Tierney T , et al
• Timmermans S
• Baungaard N ,
• Skovvang PL ,
• Assing Hvidt E , et al
• Lewis L , et al
• Liberati EG ,
• Tarrant C ,
• Willars J , et al
• Zaranko B ,
• Sanford NJ ,
• Kelly E , et al
• Martin GP ,
• Dixon-Woods M ,
• Aveling EL ,
• Campbell A , et al
• Karsh B-T ,
• Holden RJ ,
• Alper SJ , et al

Supplementary materials

Supplementary data.

This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

• Data supplement 1
• Data supplement 2
• Data supplement 3
• Data supplement 4

Twitter @janefergo, @@kieran_walshe

Contributors JF, KW, DA, TA and EK conceived the study. Recruitment was led by JF and supported by GS. JF, GS and KW conducted the interviews, reviewed and analysed the transcripts, and JF wrote the first version of the manuscript. Two members of the patient and public involvement (PPI) forum also assisted with focus groups. JF conducted data analysis with input from KW and GS, the PPI forum, and review by all authors. JF and KW were involved in initial critical review and revision of the manuscript, followed by all authors. All authors read and approved the final version of the manuscript. JF is the guarantor.

Funding This study was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme (NIHR128349), and the NIHR Greater Manchester Patient Safety Research Collaboration (PSRC). The views expressed are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care.

Competing interests None declared.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Linked Articles

• Editorial Locums: threat or opportunity Richard Lilford BMJ Quality & Safety 2024; - Published Online First: 16 Apr 2024. doi: 10.1136/bmjqs-2023-016951

Read the full text or download the PDF:

• Open access
• Published: 18 April 2024

Implementation of an audit and feedback module targeting low-value clinical practices in a provincial trauma quality assurance program: a cost-effectiveness study

• Blanchard Conombo 1 , 2 ,
• Jason R. Guertin 1 ,
• Jeffrey S. Hoch 3 ,
• Jeremy Grimshaw 4 ,
• Mélanie Bérubé 2 , 5 ,
• Christian Malo 5 ,
• Simon Berthelot 2 , 6 , 7 ,
• François Lauzier 8 ,
• Henry T. Stelfox 9 ,
• Alexis F. Turgeon 2 , 8 ,
• Patrick Archambault 10 , 6 ,
• Amina Belcaid 2 &
• Lynne Moore 1 , 2  

BMC Health Services Research volume  24 , Article number:  479 ( 2024 ) Cite this article

112 Accesses

Metrics details

Audit and Feedback (A&F) interventions based on quality indicators have been shown to lead to significant improvements in compliance with evidence-based care including de-adoption of low-value practices (LVPs). Our primary aim was to evaluate the cost-effectiveness of adding a hypothetical A&F module targeting LVPs for trauma admissions to an existing quality assurance intervention targeting high-value care and risk-adjusted outcomes. A secondary aim was to assess how certain A&F characteristics might influence its cost-effectiveness.

We conducted a cost-effectiveness analysis using a probabilistic static decision analytic model in the Québec trauma care continuum. We considered the Québec Ministry of Health perspective. Our economic evaluation compared a hypothetical scenario in which the A&F module targeting LVPs is implemented in a Canadian provincial trauma quality assurance program to a status quo scenario in which the A&F module is not implemented. In scenarios analyses we assessed the impact of A&F characteristics on its cost-effectiveness. Results are presented in terms of incremental costs per LVP avoided.

Results suggest that the implementation of A&F module (Cost = $1,480,850; Number of LVPs = 6,005) is associated with higher costs and higher effectiveness compared to status quo (Cost = $1,124,661; Number of LVPs = 8,228). The A&F module would cost $160 per LVP avoided compared to status quo. The A&F module becomes more cost-effective with the addition of facilitation visits; more frequent evaluation; and when only high-volume trauma centers are considered.

A&F module targeting LVPs is associated with higher costs and higher effectiveness than status quo and has the potential to be cost-effective if the decision-makers’ willingness-to-pay is at least $160 per LVP avoided. This likely represents an underestimate of true ICER due to underestimated costs or missed opportunity costs. Results suggest that virtual facilitation visits, frequent evaluation, and implementing the module in high-volume centers can improve cost-effectiveness.

Peer Review reports

Introduction

Low-value practices (LVPs) are tests and treatments that are not supported by evidence and may expose patients to physical and psychological harm [ 1 , 2 ]. They have been estimated to consume up to 30% of healthcare resources in Canada [ 3 ] and in the US [ 4 ]. In 2013, an estimated $270 billion was wasted on excess healthcare services in the US [ 2 ]. From a patient and caregiver perspective, LVPs expose patients to physical and psychological harms, delays to effective treatment, and direct and indirect expenses [ 2 , 5 , 6 , 7 , 8 ]. From a healthcare system perspective, they put strain on tight healthcare budgets and decrease the availability of scarce resources.

Recent literature suggests that interventions targeting the de-implementation of ineffective or harmful health interventions have the potential to reduce overuse and improve clinically important outcomes [ 9 ]. Among these are Audit and Feedback (A&F) interventions, defined as ‘a summary of clinical performance of healthcare over a specified period aimed at providing information to health professionals to allow them to assess and adjust their performance’ [ 10 ]. We now have extensive evidence of the effectiveness of A&F interventions, including those targeting de-implementation of LVPs. A systematic review including 140 randomized controlled trials (RCTs) estimated that A&F interventions resulted in close to 4.3% absolute increase in adherence to evidence-based care (IQR 0.5% to 16.0%) [ 11 ]. The effect of an A&F intervention appears to be larger when it targets de-implementation of low-value practices (absolute decrease of 10.5%). This review also revealed that A&F effectiveness is influenced by its design and delivery [ 11 ]. The World Health Organisation recently expressed concern about the major knowledge gap on the cost and cost-effectiveness of A&F interventions [ 12 ], and recommended that implementation of these interventions be informed by data on their cost-effectiveness [ 13 ]. Despite this, most A&F interventions, including those used across Canadian trauma systems, are implemented without evidence on their cost-effectiveness [ 12 , 14 ]. A 2022 systematic review summarized evidence on the economic value of A&F interventions in healthcare [ 15 ] and found that they have a high potential to be cost-effective. However, authors only identified economic evaluations for 6% of A&F trials, methodological quality of these evaluations was low, and authors concluded that model-based simulations were urgently needed to assess the impact of A&F characteristics on cost-effectiveness to inform optimal A&F design.

Trauma systems are a favorable setting for de-implementation interventions as they possess many documented facilitators including quality improvement teams with medical leadership, routinely-collected clinical data, and performance linked to accreditation [ 16 ]. Furthermore, potential gains are huge due to the resource-intensive nature of trauma care. Trauma systems are thus the ideal setting to advance knowledge on de-implementation. Our research team recently published a list of quality indicators targeting LVPs in acute trauma care [ 17 , 18 ]. We aim to evaluate the cost-effectiveness of an A&F module targeting the de-implementation of these LVPs in an integrated Canadian trauma system and to assess the impact of A&F characteristics on cost-effectiveness.

We conducted an economic evaluation according to the Canadian guidelines for the Economic Evaluation of Health Technologies [ 19 ], and results are reported following the CHEERS 2022 statement [ 20 ]. The study protocol was developed with a project advisory committee including two emergency physicians (CM, EM), two trauma surgeons (TR, NY), three critical care physicians (FL, AFT, HTS), a neurosurgeon (PLB), a spine surgeon (JP), an orthopedic surgeon (ML), two trauma service managers (MB, CR), a trauma registry co-ordinator (AB), and epidemiologist (LM), and two health economists (JRG, JSH). The protocol was approved a priori by all co-authors, members of the advisory committee, a granting agency peer-review committee (Canadian Institutes of Health Research project #353374) and the CHU-de Québec – Université Laval research ethics committee.

Our economic evaluation is based on a hypothetical A&F module embedded in the Québec Trauma Care Continuum , a provincial regionalized trauma system comprising 57 adult trauma centers of which 3 are level I (highly specialized urban centers), 5 are level II (similar capacity to level I but in smaller cities), 21 are level III (hospitals in small towns transferring most major trauma to level I/II centers after stabilization), and 28 are level IV (rural community hospitals). All centers undergo mandatory, periodic verifications in line with designation, conducted by the provincial healthcare quality agency, Institut national d’excellence en santé et services sociaux ( INESSS ) and overseen by the Ministry of Health and Social Services [ 21 ]. Verification includes A&F on adherence to high-value care and risk-adjusted outcomes. Local trauma committees in each center are required to ensure the quality of the trauma program according to designation requirements. Committees include the program medical director (Chair), the program manager, heads of critical care, emergency and surgical departments, heads of multidisciplinary services, and a hospital administrator. Quality improvement activities include trimestral committee meetings with chart review, development of local care protocols, and discussions with clinical and administrative leads locally and at referring centers to identify improvement strategies. Formal letters of agreement are signed by heads of clinical departments to operate changes in their services when required.

Intervention and comparator

We compared a hypothetical scenario in which an A&F module targeting LVPs is implemented in the Québec trauma system to a status quo scenario in which the A&F module is not implemented.

Comparator (status quo scenario)

The study comparator is the A&F intervention currently in place in the Québec Trauma Care Continuum, designed by the provincial healthcare quality agency using the US Agency for Healthcare Research and Quality guidelines [ 22 ]. This A&F intervention targets trauma committees in each trauma center and, as explained above, currently includes modules for adherence to high-value practices (15 quality indicators) and optimal outcomes (3 quality indicators). The A&F intervention currently in place consists of:

Quality reports disseminated via a Web platform to local trauma committees and hospital boards of directors produced using trauma registry data.

Web links to user-friendly information sheets including definitions for quality indicators and references supporting each indicator.

Information sheets and Web capsules with guidelines on how the results should be interpreted and acted upon.

A case revision tool integrated into the trauma registry.

Within 6 months of reception of the report, committees are required to submit an action plan proposing improvement strategy for quality indicators for which they are identified as negative outliers.

Intervention

The study intervention is an A&F module targeting LVPs (6 quality indicators) ( http://www.ohri.ca/auditfeedback/laboratories/ ). The 6 quality indicators were selected using the results of an expert consensus study [ 23 ] and an indicator validation study using data from the Quebec trauma registy [ 24 ].

In the base case scenario, the module includes the components already in place described in the status quo scenario, applied to quality indicators on LVPs. We attributed a 5-year lifespan to the A&F module as current literature recommends that quality indicators be updated every five years [ 25 ]. To account for the 5-year lifespan of the A&F module and its potential benefits one year beyond its lifespan, we used a 6-year time horizon.

Type of economic evaluation

For this early economic evaluation, i.e., an evaluation prior to the implementation of the module, a probabilistic static decision analytic model was developed to estimate the incremental cost-effectiveness ratio (ICER) of the A&F module compared with status quo scenario in which the A&F module is not implemented for patients with acute injury (Fig.  1 ). We considered the Québec Ministry of Health perspective.

Decision-analytic model. In the status quo scenario, there is no implementation of the A&F module targeting LVPs. In the intervention scenario, an A&F module targeting LVPs is implemented at baseline (at the beginning of the 1st year). “Use data from year 1” means that data on the effectiveness and costs of A&F module from the 1st year are available at the beginning of the second year and so on for the following years

Effectiveness

The incremental effectiveness of the A&F module was estimated as the incremental number of LVPs avoided. Plausible ranges of percent reductions in LVPs were obtained from the 2012 Cochrane review, which presented effectiveness of A&F interventions as medians and interquartile ranges [ 26 ]. Specifically, we used the pooled estimate of effectiveness specific to deimplementation interventions based on 29 studies. For the purposes of our analysis, these values were used to estimate mean effectiveness and associated standard errors using a method based on highly-cited recommendations [ 27 , 28 ].

The incremental costs of the A&F module over the comparator were estimated by summing the implementation costs of the A&F module over its 5-year lifespan and the potential reduction in resource utilization for all LVPs, valued in costs and estimated between years 2 and 6. The implementation costs were determined by identifying all non-recurrent and recurrent costs related to the implementation of the A&F module including data validation and analyses, report production and validation, administrative costs, and follow-up in local trauma committees (Table  1 ). The potential reduction in resource utilization was estimated by multiplying the hypothesized reduction in the frequency of the LVPs by their average costs. Detailed information on how practices were costed are available elsewhere [ 29 ]. Briefly, we estimated direct healthcare costs for each LVP from the Ministry of Health perspective using an activity-based costing approach. Activity-based costing involves multiplying unit costs of specific activity centres by the corresponding units of resources used. This method provides an estimate of hospital resource use by activity center, consistent with Grading of Recommendations Assessment, Development and Evaluations guidelines [ 30 , 31 , 32 ]. All costs are expressed in 2020 Canadian dollars. We report our study following the Consolidated Health Economic Evaluation Reporting Standards statement [ 29 ].

Incremental cost-effectiveness ratio

The incremental cost-effectiveness ratio (ICER) was estimated by dividing the incremental costs (or savings) of the A&F module by its incremental effectiveness. Results are reported as the incremental cost per LVP avoided.

Discount rate

All future costs and benefits were discounted at a rate of 1.5% as recommended by current Canadian guidelines [ 19 ].

Scenario analyses

Our advisory committee identified 6 scenario-specific sensitivity analyses based on published evidence of A&F effectiveness and context-specific considerations (#1, 2, 3, 6) [ 24 , 26 ] as well as Canadian guidelines on economic evaluation (#4, 5):

Adding a virtual facilitation visit once per cycle to help trauma committees identify barriers and facilitators and use them to identify improvement strategies for their action plan; [ 15 , 26 ]

Increasing feedback frequency from annually to monthly, as assessed in the systematic review; [ 26 ]

Implementing the module only in high-volume trauma centers (i.e., level I and II);

Varying the discount rate between 0 and 5%, as recommended by current Canadian guidelines; [ 19 ]

Increasing the lifespan of the A&F module to 10 years (Supplementary Fig.  1 ) to take into account the effect of time on module effectiveness.

Increasing the costs of LVPs by 100% to account for lack of complete data on physician billing and unit costs that underestimate market prices. This is based on evidence that physician billing represents approximately 56% of hospital costs in Canada ( https://www.cihi.ca/sites/default/files/document/nhex-trends-2020-narrative-report-en.pdf ).

We present the ICER based on the results of probabilistic sensitivity analysis (PSA) as recommended in the Canadian Guidelines for the Economic Evaluation of Health Technologies [ 19 ]. In the PSA, model parameters were represented by distributions of possible values rather than point estimates to address parameter uncertainty. All parameters and their distributions are presented in Table  2 . Parameter distributions were randomly sampled 10,000 times. Results were summarized using cost-effectiveness acceptability curves (CEACs) and the cost-effectiveness acceptability frontier (CEAF) [ 33 ]. We used Excel software (Microsoft Office 2019 Professional Plus) to construct a decision model, to analyze base case results, and conduct PSA.

The mean costs of the A&F module and status quo scenario were $1,480,850 and $1,124,661 respectively. The associated average number of LVPs were 6,005 for the A&F module and 8,228 for status quo scenario. The implementation of the A&F module is associated with a reduction of approximately 2,223 LVPs. The ICER for the A&F module versus status quo scenario was $160 per LVP avoided (Table  3 ). The results of the PSA plotted on a cost-effectiveness plane (Fig.  2 ) show that most of the points in the scatter plot are located in the Northeast quadrant, indicating that the A&F module has a potential to be cost-effective given a decision maker’s willingness-to-pay (WTP). The cost-effectiveness acceptability frontier indicates that A&F module is cost-effective in 50% of our iterations at a WTP of $160 per LVP avoided (Fig.  3 ).

Probabilistic sensitivity analysis comparing A&F module and status quo scenario (no A&F module targeting LVPs). The x-axis represents the incremental effectiveness, number of LVPs avoided. The y-axis represents the incremental costs between A&F module and status quo scenario. Each circle represents a single simulation for a total of 10,000 simulations

Cost-effectiveness acceptability curve (CEAC) between A&F module and status quo scenario and cost-effectiveness acceptability frontier (CEAF). The x-axis represents the willingness-to-pay (WTP) for each LVP avoided. The y-axis represents the percentage of simulations in which the A&F module is cost-effectiveness relative to the status quo scenario at different WTP threshold. The switch point where the A&F module became a cost-effective intervention corresponds to $160 per LVP avoided, equal to the ICER estimate. A&F module became 100% cost-effective at a WTP of $1000 per LVP avoided. The A&F module had the highest expected net benefit, for all values of WTP greater than the ICER. At our ICER estimate, 50% of the distribution of ICERs were cost-effective

Adding a virtual facilitation visit to the A&F module (one visit per A&F cycle) would reduce the estimated ICER (improve its cost-effectiveness profile compared to the base case scenario) to $108 per LVP avoided (Table  4 ). More frequent feedback (monthly) is associated with a slight improvement in its cost-effectiveness profile ($154 per LVP avoided). The A&F module is more cost-effective ($48 per LVP avoided) when only high-volume trauma centers are considered for the implementation of the module. Similarly, an increase in the costs of LVPs by 100% and a longer time horizon would lead to a reduction in the ICER to $10 and $106 per LVP avoided, respectively. On the other hand, a discount rate of 5% increases the ICER to $199 per LVP avoided (Table  4 ).

The results of this early economic evaluation suggest that the addition of an A&F module targeting LVPs to a provincial trauma quality assurance program over a time horizon of 6 years is associated with an ICER of $160 per LVP avoided. In analyses that simultaneously accounted for uncertainty in all key model parameters, 50% of simulations were cost-effective at a WTP of $160 per LVP avoided. The A&F module is more cost-effective with the addition of facilitation visits, frequent evaluation and if restricted to high-volume trauma centers.

Our study fills a major knowledge gap on the potential cost-effectiveness of A&F interventions to de-implement low-value care. Comparison of our results with the literature on acute trauma care is difficult, because there are no studies that have assessed the cost-effectiveness profile of A&F interventions in the context of acute injury care. However, a 2022 systematic review on the economic value of A&F interventions in various health areas summarized results of 35 studies that compared different A&F strategies targeting health professionals compliance with desire practices or patient health outcomes [ 15 ]. The results of this systematic review mirror our findings. Of 14 cost-effectiveness analyses based on changes in compliance to desired practice from the public healthcare payer perspective, 12 (86%) studies found that the A&F interventions were more costly but more effective than the comparator [ 15 ]. From studies assessing de-implementation of LVPs [ 34 , 35 ], A&F interventions were associated with a reduction in the overuse of LVPs and had the potential to be cost-effective [ 34 , 35 ]. Four (28%) studies included in the review conducted simulations to assess the influence of A&F characteristics on cost-effectiveness in scenario analyses [ 34 , 35 , 36 , 37 ]. Despite having different comparator groups (do-nothing scenario), these studies also observed improved cost-effectiveness when facilitation visits are added to A&F intervention [ 35 ] and the time horizon of the intervention is increased to 9 months (4 to 9 months) [ 34 ]. In addition, our study provides evidence that cost-effectiveness of A&F interventions may be improved by increasing frequency and restricting the intervention to high volume hospitals.

Strengths and limitations

Our study is based on effectiveness parameters from a meta-analysis on over 140 RCTs on the effectiveness of A&F interventions in different healthcare settings [ 11 ], on observed data on the frequency of LVPs [ 24 ], and on costs based on a mature, province-wide quality assurance program ( https://www.donneesquebec.ca/recherche/dataset/as-471-rapports-financiers-annuels-des-etablissements ). In addition, we conducted extensive sensitivity analyses and a range of scenario analyses to evaluate the robustness of our results and to assess the influence of key A&F characteristics on its cost-effectiveness profile. Despite these strengths, our results should be interpreted within the context of the study’s limitations. First, our evaluation is based on estimates of effectiveness from a meta-analysis published in 2012. While this study represents the most up-to-date evidence synthesis available (the Cochrane review is yet to be updated [ 38 ] and systematic reviews published more recently have not included meta-analyses) [ 39 ], it does not include the most recent evidence. Furthermore, while we used estimates specific to deimplementation interventions from the review, none of the studies were specific to trauma, none evaluated an intervention delivered in the context of accreditation, and none compared a deimplementation module in a system with an A&F intervention already in place. Furthermore, risk of bias was low for only 31% of included studies. The estimate used may therefore represent an underestimate or overestimate of the true effectiveness. Second, we conducted an early economic evaluation to assess if a hypothetical A&F module could be cost-effective and, if so, under which conditions. As such, the results of our economic evaluation provide encouragement that the true ICER of the intervention were it to be designed and implemented in the Québec trauma system might be promising as well. We used a broad range of scenarios and parameter values within our probabilistic sensitivity analyses but attributed the same weight to all scenarios analyzed. The base-case scenario will not necessarily be the one that will be implemented. However, intervention costs were based on resources currently used in the Québec trauma care continuum and opportunity costs related to LVPs avoided were based on observed baseline frequencies. We plan to conduct an economic evaluation after our cluster randomized trial (funded and currently underway) to assess the true (observed) cost-effectiveness of the intervention in a pragmatic setting. Third, opportunity costs of LVPs avoided did not fully account for physician fees and were based on unit costs that are known to underestimate their true costs. In addition, we did not account for potential resource repercussions of LVPs, for example, re-imaging due to uncertain findings or treatment of clinically nonsignificant incidental findings. Our scenario analysis where costs associated with LVPs were increased by 100% probably better reflects the Québec Ministry of Health perspective; the large decrease in the ICER ($160 to $10) suggest that opportunity costs related to LVPs are an important determinant of the cost-effectiveness of an A&F module targeting de-implementation. Furthermore, we only considered direct healthcare costs associated with the two competing strategies and did not factor in the effects of indirect costs (e.g., time off work for patients) from LVPs, which would also have led to an underestimation of the intervention’s cost-effectiveness. Fourth, our study is based on the single healthcare payer model, and it is uncertain if our findings would be applicable to other jurisdictions with alternate payer systems. Also, physicians in Canada receive payments based on fee for service that is periodically negotiated [ 40 ], so our results are dependent on current unit costs in our system and may not apply well in non-universal health systems or other jurisdictions with different structures. Fifth, in the absence of evidence indicating otherwise, our base case scenario was based on the strong assumption that effectiveness was the same for all 6 indicators. However, in probabilistic sensitivity analyses, we allowed the effectiveness of LVP to vary independently. Sixth, we were unable to take account of the uncertainty of the cost estimates of implementing the A&F module, derived from expert consultation, which we anticipate may have been underestimated. Finally, we deliberately focused on adherence to desired practice (LVPs avoided) rather than health outcomes (e.g., adverse events) due to lack of available data associated with utility/disutility of LVPs for trauma patients. Nevertheless, a strong argument can be made for focus on the measurement of LVPs avoided for assessment of the quality of our A&F intervention, as they relate most closely to actions that are within the control of healthcare professionals. Indeed, economic evaluations of similar A&F interventions have obtained more meaningful results with similar intermediate outcomes than with Quality-Adjusted Life Years (QALYs) [ 41 ]. Studies have also demonstrated that reducing LVPs will reduce physical harms and adverse events [ 42 , 43 , 44 ]. However, this probably led to an underestimation of the true cost-effectiveness profile of our A&F module as health outcomes or negative health consequences of LVPs are not considered in the measure of effectiveness [ 15 ].

Potential impact

The outcome parameter used in decision model (LVPs avoided) is unique and does not have an explicit cost-effectiveness threshold associated with it. Therefore, the decision to invest in the intervention will be based on the decision-makers willingness-to-pay, i.e., would they be prepared to invest 160$ per LVP avoided? However, the decision should also be based on other considerations, e.g., opportunity costs are likely to be greater than those estimated, cost-effectiveness may be increased if virtual facilitation visits are added, if the frequency of evaluations are increased, and if the intervention is restricted to high-volume trauma centers (level I and II). The intervention has the potential to lead to a global awareness of healthcare overuse and therefore a decrease in other LVPs [ 24 ].

Our economic evaluation suggests that an A&F module targeting de-implementation, integrated into a provincial quality-assurance program, has a high potential to reduce LVPs while increasing total healthcare costs, with an ICER of $160 per LVP avoided. Results suggest that virtual facilitation visits, frequent evaluation and implementing the intervention only in high-volume centers increase cost-effectiveness. However, its economic potential is likely underestimated in this study due to opportunity costs that were underestimated (costs of LVPs) or not accounted for (indirect costs, health outcomes, and long-term consequences). The findings of the present study may inform the development of A&F interventions targeting de-implementation and they demonstrate the feasibility of conducting early economic evaluations to inform optimal A&F intervention design.

Availability of data and materials

Quebec Trauma Registry is subject to a third-party restriction (Quebec Ministry of Health and Social Services).

Abbreviations

Low-value practices

Audit and Feedback

Randomized Controlled Trial

Willingness-to-pay

Computed tomography

Interquartile Range

Cost-effectiveness acceptability curve

Cost-effectiveness acceptability frontier

Probability sensitivity analysis

Choosing Wisely. Home page. Accessed February 21, 2022. https://www.choosingwisely.org/ .

Brownlee S, Chalkidou K, Doust J, et al. Evidence for overuse of medical services around the world. Lancet. 2017;390(10090):156–68.

Article   PubMed   PubMed Central   Google Scholar  

Canadian Institute for Health Information. CIHI. unnecessary care in Canada: technical report. Ottawa, ON: CIHI, 2017.

Lown Institute. Low-value care. https://lowninstitute.org/lown-issues/low-value-care/ . Accessed 20 Dec 2022.

Berwick DM, Hackbarth AD. Eliminating waste in US health care. JAMA. 2012;307(14):1513–6.

Article   CAS   PubMed   Google Scholar  

Boat TF, Chao SM, O’Neill PH. From waste to value in health care. JAMA. 2008;299(5):568–71.

Reilly BM, Evans AT. Much ado about (doing) nothing. Ann Intern Med. 2009;150(4):270–1.

Article   PubMed   Google Scholar  

Hauser CJ, Adams CA Jr, Eachempati SR. Surgical Infection Society guideline: prophylactic antibiotic use in open fractures: an evidence-based guideline. Surg infect. 2006;7(4):379–405.

Article   Google Scholar  

Niven DJ, Mrklas KJ, Holodinsky JK, et al. Towards understanding the de-adoption of low-value clinical practices: a scoping review. BMC Med. 2015;13:255.

World Health Organization. Using audit and feedback to health professionals to improve the quality and safety of health care. https://apps.who.int/iris/handle/10665/332014 . Accessed 20 Dec 2022.

Noah I. Optimizing Audit and Feedback Interventions to Improve Quality in Primary Care. 2014.

Flottorp S, Jamtvedt G, Gibis B, McKee M. Using audit and feedback to health professionals to improve the quality and safety of health care. http://www.euro.who.int/__data/assets/pdf_file/0003/124419/e94296.pdf . Published 2010. Accessed.

Soleymani F, Rashidian A, Dinarvand R, Kebriaeezade A, Hosseini M, Abdollahi M. Assessing the effectiveness and cost-effectiveness of audit and feedback on physician’s prescribing indicators: study protocol of a randomized controlled trial with economic evaluation. Daru. 2012;20(1):88–88.

Avery AJ, Rodgers S, Cantrill JA, et al. A pharmacist-led information technology intervention for medication errors (PINCER): a multicentre, cluster randomised, controlled trial and cost-effectiveness analysis. Lancet. 2012;379(9823):1310–9.

Moore L, Guertin JR, Tardif P-A, et al. Economic evaluations of audit and feedback interventions: a systematic review. BMJ Qual Saf. 2022;31(10):754.

Squires JE, Sullivan K, Eccles MP, Worswick J, Grimshaw JM. Are multifaceted interventions more effective than single-component interventions in changing health-care professionals’ behaviours? An overview of systematic reviews. Implement Sci. 2014;9(1):152.

Fitch K, Bernstein JS, Aguilar MD, et al. The RAND/UCLA Appropriateness Method User's Manual. https://www.rand.org/pubs/monograph_reports/MR1269.html . Published 2001. Accessed 17 Sept 2018.

Heyworth J. Trauma services in a district general hospital. BMJ (Clinical research ed). 1990;300(6728):876–7.

CADTH. Guidelines for the economic evaluation of health technologies: Canada. 4th edition. Ottawa: CADTH; 2017.

Husereau D, Drummond M, Augustovski F, et al. Consolidated Health Economic Evaluation Reporting Standards 2022 (CHEERS 2022) statement: updated reporting guidance for health economic evaluations. BMJ (Clinical research ed). 2022;376:e067975.

PubMed   Google Scholar  

Ministère de la Santé et des Services sociaux. Trajectoire - Traumatologie. https://www.msss.gouv.qc.ca/professionnels/soins-et-services/guide-urgences-trajectoire-traumatologie/ . Published 2022. Accessed 13 Sept 2022.

Developing a framework and research agenda for overuse and appropriateness measures. Agency for Healthcare Research and Quality Appropriateness Small Conference. Published 2009. Accessed 17 Sept 2018.

Moore L, Bérubé M, Tardif PA, et al. Quality Indicators targeting low-value clinical practices in trauma care. JAMA Surg. 2022;157(6):507–14.

Moore L, Bérubé M, Tardif PA, et al. Validation of quality indicators targeting low-value trauma care. JAMA Surg. 2022;157(11):1008–16.

Stelfox HT, Straus SE. Measuring quality of care: considering measurement frameworks and needs assessment to guide quality indicator development. J Clin Epidemiol. 2013;66(12):1320–7.

Ivers N, Jamtvedt G, Flottorp S, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Datab Syst Rev. 2012;6:Cd000259.

Google Scholar  

Hozo SP, Djulbegovic B, Hozo I. Estimating the mean and variance from the median, range, and the size of a sample. BMC Med Res Methodol. 2005;5(1):13.

Wan X, Wang W, Liu J, Tong T. Estimating the sample mean and standard deviation from the sample size, median, range and/or interquartile range. BMC Med Res Methodol. 2014;14(1):135.

Conombo B, Guertin JR, Hoch JS, et al. Potential avoidable costs of low-value clinical practices in acute injury care in an integrated canadian provincial trauma system. JAMA Surg. 2023;158(9):977–9.

Canby JBT. Applying activity-based costing to healthcare settings. Healthc Financ Manag. 1995;49(2):50–2 54-56.

Chan YC. Improving hospital cost accounting with activity-based costing. Health Care Manage Rev. 1993;18(1):71–7.

Guyatt GH, Oxman AD, Kunz R, et al. Incorporating considerations of resources use into grading recommendations. BMJ. 2008;336(7654):1170–3.

Fenwick E, Claxton K, Sculpher M. Representing uncertainty: the role of cost-effectiveness acceptability curves. Health Econ. 2001;10(8):779–87.

Ling R, Giles M, Searles A. Administration of indwelling urinary catheters in four Australian Hospitals: cost-effectiveness analysis of a multifaceted nurse-led intervention. BMC Health Serv Res. 2021;21(1):897.

Gandjour A, Lauterbach KW. When is it worth reducing overuse of health care services? The example of prescribing expensive antihypertensives. Med Klin (Munich). 2005;100(9):535–41.

Barasa EW, Ayieko P, Cleary S, English M. A multifaceted intervention to improve the quality of care of children in district hospitals in Kenya: a cost-effectiveness analysis. PLoS Med. 2012;9(6):e1001238.

Huis A, Schoonhoven L, Grol R, Donders R, Hulscher M, van Achterberg T. Impact of a team and leaders-directed strategy to improve nurses’ adherence to hand hygiene guidelines: a cluster randomised trial. Int J Nurs Stud. 2013;50(4):464–74.

Crawshaw J, Meyer C, Antonopoulou V, et al. Identifying behaviour change techniques in 287 randomized controlled trials of audit and feedback interventions targeting practice change among healthcare professionals. Implement Sci. 2023;18(1):63.

Dunsmore J, Duncan E, MacLennan S, N’Dow J, MacLennan S. Effectiveness of de-implementation strategies for low-value prescribing in secondary care: a systematic review. Implement Sci Commun. 2023;4(1):115.

Ridic G, Gleason S, Ridic O. Comparisons of health care systems in the United States. Germany Canada Mater Sociomed. 2012;24(2):112–20.

Elliott RA, Putman Kd Fau - Franklin M, Franklin M Fau - Annemans L, et al. Cost effectiveness of a pharmacist-led information technology intervention for reducing rates of clinically important errors in medicines management in general practices (PINCER). (1179–2027 (Electronic)).

Elliott RA, Putman KD, Franklin M, et al. Cost effectiveness of a pharmacist-led information technology intervention for reducing rates of clinically important errors in medicines management in general practices (PINCER). Pharmacoeconomics. 2014;32(6):573–90.

Johri M, Ng ESW, Bermudez-Tamayo C, Hoch JS, Ducruet T, Chaillet N. A cluster-randomized trial to reduce caesarean delivery rates in Quebec: cost-effectiveness analysis. BMC Med. 2017;15(1):96.

Rodriguez-Martinez CE, Sossa-Briceño MP, Castro-Rodriguez JA. Cost-effectiveness of the utilization of “good practice” or the lack thereof according to a bronchiolitis evidence-based clinical practice guideline. J Eval Clin Pract. 2019;25(4):682–8.

Download references

Acknowledgements

We thank Natalie Yanchar, Éric Mercier, Jérôme Paquet, Tarek Razek, Martin Lesieur, Paule Lessard Boneaventure and Christine Rizzo for their role as members of the advisory committee.

Role of funder/sponsor

The funders had no role in developing this manuscript.

This research was supported by the Canadian Institutes of Health Research (Foundation grant, #353374).

Author information

Authors and affiliations.

Department of Social and Preventative Medicine, Université Laval, Québec City, Québec, Canada

Blanchard Conombo, Jason R. Guertin & Lynne Moore

Population Health and Optimal Health Practices Research Unit, Trauma – Emergency – Critical Care Medicine, Quebec University Hospital, Centre de Recherche du CHU de Québec-Université Laval, 18E Rue, Local H-012a, Québec City, Québec, 1401G1J 1Z4, Canada

Blanchard Conombo, Mélanie Bérubé, Simon Berthelot, Alexis F. Turgeon, Amina Belcaid & Lynne Moore

Division of Health Policy and Management, Department of Public Health Sciences, University of California at Davis, Davis, CA, USA

Jeffrey S. Hoch

Department of Medicine, University of Ottawa, Ottawa, ON, Canada

Jeremy Grimshaw

Faculty of Nursing, Université Laval, Québec City, Québec, Canada

Mélanie Bérubé & Christian Malo

Department of Family Medicine and Emergency Medicine, Université Laval, Québec City, Québec, Canada

Simon Berthelot & Patrick Archambault

Centre de Recherche Intégrée Pour Un Système Apprenant en Santé Et Services Sociaux, Centre Intégré de Santé Et de Services Sociaux de Chaudière-Appalaches, Lévis, Québec, Canada

Simon Berthelot

Department of Anesthesiology and Critical Care Medicine, Division of Critical Care Medicine, Université Laval, Québec City, Québec, Canada

François Lauzier & Alexis F. Turgeon

Department of Critical Care Medicine, Medicine and Community Health Sciences, O’Brien Institute for Public Health, University of Calgary, Calgary, AB, Canada

Henry T. Stelfox

VITAM-Centre de Recherche en Santé Durable, Québec City, Québec, Canada

Patrick Archambault

You can also search for this author in PubMed   Google Scholar

Contributions

BC, LM, JRG, JSH, HTS, PA, SB, FL, AFT, JG, MB, CM and AB developed the original research concept and developed the study design. BC, LM, FL, AFT, JSH, HTS, AB, MB and JRG have contributed to the data acquisition, analysis, or interpretation. JH, HTS and SB oversaw the analysis and provided feedback. BC and LM developed the draft manuscript. All authors made substantial contributions to the manuscript development, critical revision and approved the final version of the manuscript.

Corresponding author

Correspondence to Lynne Moore .

Ethics declarations

Ethics approval and consent to participate.

This study was approved by the CHU-de Québec – Université Laval research ethics committee and patient consent is not required.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Conombo, B., Guertin, J.R., Hoch, J.S. et al. Implementation of an audit and feedback module targeting low-value clinical practices in a provincial trauma quality assurance program: a cost-effectiveness study. BMC Health Serv Res 24 , 479 (2024). https://doi.org/10.1186/s12913-024-10969-2

Download citation

Received : 09 October 2023

Accepted : 09 April 2024

Published : 18 April 2024

DOI : https://doi.org/10.1186/s12913-024-10969-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Audit and feedback
• Cost-effectiveness analysis
• Low-value care

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

Skip to Content

College of Arts and Sciences names 2024 Van Ek Scholars

• Share via Twitter
• Share via Facebook
• Share via LinkedIn
• Share via E-mail

Twenty-three students receive one of college's highest honors, recognized for academic achievement and depth of service

The College of Arts and Sciences has awarded the Jacob Van Ek Scholarship—one of the college’s highest honors—to 23 exceptional undergraduates.

The award is named for Jacob Van Ek (1896-1999), who arrived at CU as a young assistant professor shortly after earning his doctorate in 1925 at what is now known as Iowa State University. Within three years, he was a full professor and, by 1929, he was dean of the College of Liberal Arts, serving until 1959. 

The following students are this year’s Jakob Van Ek Scholar Award recipients:

• Emily Aguirre, English/Spanish
• Simon Bantugan, Geology
• Seleny Banuelas, BFA Ceramics/BA Spanish
• Aaron Barrios, Physics/Astronomy/Math
• Mariana Bastias, English/Psychology and Neuroscience
• Auburn Berry, Integrative Physiology
• Blanca Cerda, Neuroscience
• Rachel Coppock, Anthropology
• Sarah Coronna, Anthropology
• Adriana Espinoza, Molecular, Cellular and Developmental Biology
• Samantha Haygood, Philosophy and Sociology
• Lucas Hendricks, Chinese
• Taylor Johnson, Philosophy
• Emma Judge, Biochemistry
• Samantha Lane, Sociology
• Isabella (Bella) Malherbe, Art History
• Grant Mannino, Psychology and Neuroscience
• Miles Moore, Ecology and Evolutionary Biology
• Rumi Natanzi, Ethnic Studies
• Callie Noar, Integrative Physiology
• Alexandra Plocki, Physics
• Hayley Thomas, International Affairs
• Grace Tuzizila, Integrative Physiology

Related Articles

Twenty-six students are named 2023 Van Ek Scholars

Awards recognize innovation in research, education and inclusivity

New scholarship celebrates, supports ‘ANDers’

• College of Arts and Sciences
• Scholarships
• Undergraduate Students