what makes a marketing research study valid

• Privacy Policy

Home » Validity – Types, Examples and Guide

Validity – Types, Examples and Guide

Table of Contents

Validity is a fundamental concept in research, referring to the extent to which a test, measurement, or study accurately reflects or assesses the specific concept that the researcher is attempting to measure. Ensuring validity is crucial as it determines the trustworthiness and credibility of the research findings.

Research Validity

Research validity pertains to the accuracy and truthfulness of the research. It examines whether the research truly measures what it claims to measure. Without validity, research results can be misleading or erroneous, leading to incorrect conclusions and potentially flawed applications.

How to Ensure Validity in Research

Ensuring validity in research involves several strategies:

• Clear Operational Definitions : Define variables clearly and precisely.
• Use of Reliable Instruments : Employ measurement tools that have been tested for reliability.
• Pilot Testing : Conduct preliminary studies to refine the research design and instruments.
• Triangulation : Use multiple methods or sources to cross-verify results.
• Control Variables : Control extraneous variables that might influence the outcomes.

Types of Validity

Validity is categorized into several types, each addressing different aspects of measurement accuracy.

Internal Validity

Internal validity refers to the degree to which the results of a study can be attributed to the treatments or interventions rather than other factors. It is about ensuring that the study is free from confounding variables that could affect the outcome.

External Validity

External validity concerns the extent to which the research findings can be generalized to other settings, populations, or times. High external validity means the results are applicable beyond the specific context of the study.

Construct Validity

Construct validity evaluates whether a test or instrument measures the theoretical construct it is intended to measure. It involves ensuring that the test is truly assessing the concept it claims to represent.

Content Validity

Content validity examines whether a test covers the entire range of the concept being measured. It ensures that the test items represent all facets of the concept.

Criterion Validity

Criterion validity assesses how well one measure predicts an outcome based on another measure. It is divided into two types:

• Predictive Validity : How well a test predicts future performance.
• Concurrent Validity : How well a test correlates with a currently existing measure.

Face Validity

Face validity refers to the extent to which a test appears to measure what it is supposed to measure, based on superficial inspection. While it is the least scientific measure of validity, it is important for ensuring that stakeholders believe in the test’s relevance.

Importance of Validity

Validity is crucial because it directly affects the credibility of research findings. Valid results ensure that conclusions drawn from research are accurate and can be trusted. This, in turn, influences the decisions and policies based on the research.

Examples of Validity

• Internal Validity : A randomized controlled trial (RCT) where the random assignment of participants helps eliminate biases.
• External Validity : A study on educational interventions that can be applied to different schools across various regions.
• Construct Validity : A psychological test that accurately measures depression levels.
• Content Validity : An exam that covers all topics taught in a course.
• Criterion Validity : A job performance test that predicts future job success.

Where to Write About Validity in A Thesis

In a thesis, the methodology section should include discussions about validity. Here, you explain how you ensured the validity of your research instruments and design. Additionally, you may discuss validity in the results section, interpreting how the validity of your measurements affects your findings.

Applications of Validity

Validity has wide applications across various fields:

• Education : Ensuring assessments accurately measure student learning.
• Psychology : Developing tests that correctly diagnose mental health conditions.
• Market Research : Creating surveys that accurately capture consumer preferences.

Limitations of Validity

While ensuring validity is essential, it has its limitations:

• Complexity : Achieving high validity can be complex and resource-intensive.
• Context-Specific : Some validity types may not be universally applicable across all contexts.
• Subjectivity : Certain types of validity, like face validity, involve subjective judgments.

By understanding and addressing these aspects of validity, researchers can enhance the quality and impact of their studies, leading to more reliable and actionable results.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Split-Half Reliability – Methods, Examples and...

Parallel Forms Reliability – Methods, Example...

Inter-Rater Reliability – Methods, Examples and...

Alternate Forms Reliability – Methods, Examples...

Face Validity – Methods, Types, Examples

Construct Validity – Types, Threats and Examples

Validity & Reliability In Research

A Plain-Language Explanation (With Examples)

By: Derek Jansen (MBA) | Expert Reviewer: Kerryn Warren (PhD) | September 2023

Validity and reliability are two related but distinctly different concepts within research. Understanding what they are and how to achieve them is critically important to any research project. In this post, we’ll unpack these two concepts as simply as possible.

This post is based on our popular online course, Research Methodology Bootcamp . In the course, we unpack the basics of methodology  using straightfoward language and loads of examples. If you’re new to academic research, you definitely want to use this link to get 50% off the course (limited-time offer).

Overview: Validity & Reliability

• The big picture
• Validity 101
• Reliability 101 
• Key takeaways

First, The Basics…

First, let’s start with a big-picture view and then we can zoom in to the finer details.

Validity and reliability are two incredibly important concepts in research, especially within the social sciences. Both validity and reliability have to do with the measurement of variables and/or constructs – for example, job satisfaction, intelligence, productivity, etc. When undertaking research, you’ll often want to measure these types of constructs and variables and, at the simplest level, validity and reliability are about ensuring the quality and accuracy of those measurements .

As you can probably imagine, if your measurements aren’t accurate or there are quality issues at play when you’re collecting your data, your entire study will be at risk. Therefore, validity and reliability are very important concepts to understand (and to get right). So, let’s unpack each of them.

What Is Validity?

In simple terms, validity (also called “construct validity”) is all about whether a research instrument accurately measures what it’s supposed to measure .

For example, let’s say you have a set of Likert scales that are supposed to quantify someone’s level of overall job satisfaction. If this set of scales focused purely on only one dimension of job satisfaction, say pay satisfaction, this would not be a valid measurement, as it only captures one aspect of the multidimensional construct. In other words, pay satisfaction alone is only one contributing factor toward overall job satisfaction, and therefore it’s not a valid way to measure someone’s job satisfaction.

Oftentimes in quantitative studies, the way in which the researcher or survey designer interprets a question or statement can differ from how the study participants interpret it . Given that respondents don’t have the opportunity to ask clarifying questions when taking a survey, it’s easy for these sorts of misunderstandings to crop up. Naturally, if the respondents are interpreting the question in the wrong way, the data they provide will be pretty useless . Therefore, ensuring that a study’s measurement instruments are valid – in other words, that they are measuring what they intend to measure – is incredibly important.

There are various types of validity and we’re not going to go down that rabbit hole in this post, but it’s worth quickly highlighting the importance of making sure that your research instrument is tightly aligned with the theoretical construct you’re trying to measure .  In other words, you need to pay careful attention to how the key theories within your study define the thing you’re trying to measure – and then make sure that your survey presents it in the same way.

For example, sticking with the “job satisfaction” construct we looked at earlier, you’d need to clearly define what you mean by job satisfaction within your study (and this definition would of course need to be underpinned by the relevant theory). You’d then need to make sure that your chosen definition is reflected in the types of questions or scales you’re using in your survey . Simply put, you need to make sure that your survey respondents are perceiving your key constructs in the same way you are. Or, even if they’re not, that your measurement instrument is capturing the necessary information that reflects your definition of the construct at hand.

If all of this talk about constructs sounds a bit fluffy, be sure to check out Research Methodology Bootcamp , which will provide you with a rock-solid foundational understanding of all things methodology-related. Remember, you can take advantage of our 60% discount offer using this link.

Need a helping hand?

What Is Reliability?

As with validity, reliability is an attribute of a measurement instrument – for example, a survey, a weight scale or even a blood pressure monitor. But while validity is concerned with whether the instrument is measuring the “thing” it’s supposed to be measuring, reliability is concerned with consistency and stability . In other words, reliability reflects the degree to which a measurement instrument produces consistent results when applied repeatedly to the same phenomenon , under the same conditions .

As you can probably imagine, a measurement instrument that achieves a high level of consistency is naturally more dependable (or reliable) than one that doesn’t – in other words, it can be trusted to provide consistent measurements . And that, of course, is what you want when undertaking empirical research. If you think about it within a more domestic context, just imagine if you found that your bathroom scale gave you a different number every time you hopped on and off of it – you wouldn’t feel too confident in its ability to measure the variable that is your body weight 🙂

It’s worth mentioning that reliability also extends to the person using the measurement instrument . For example, if two researchers use the same instrument (let’s say a measuring tape) and they get different measurements, there’s likely an issue in terms of how one (or both) of them are using the measuring tape. So, when you think about reliability, consider both the instrument and the researcher as part of the equation.

As with validity, there are various types of reliability and various tests that can be used to assess the reliability of an instrument. A popular one that you’ll likely come across for survey instruments is Cronbach’s alpha , which is a statistical measure that quantifies the degree to which items within an instrument (for example, a set of Likert scales) measure the same underlying construct . In other words, Cronbach’s alpha indicates how closely related the items are and whether they consistently capture the same concept . 

Recap: Key Takeaways

Alright, let’s quickly recap to cement your understanding of validity and reliability:

• Validity is concerned with whether an instrument (e.g., a set of Likert scales) is measuring what it’s supposed to measure
• Reliability is concerned with whether that measurement is consistent and stable when measuring the same phenomenon under the same conditions.

In short, validity and reliability are both essential to ensuring that your data collection efforts deliver high-quality, accurate data that help you answer your research questions . So, be sure to always pay careful attention to the validity and reliability of your measurement instruments when collecting and analysing data. As the adage goes, “rubbish in, rubbish out” – make sure that your data inputs are rock-solid.

Psst… there’s more!

This post is an extract from our bestselling short course, Methodology Bootcamp . If you want to work smart, you don't want to miss this .

You Might Also Like:

THE MATERIAL IS WONDERFUL AND BENEFICIAL TO ALL STUDENTS.

THE MATERIAL IS WONDERFUL AND BENEFICIAL TO ALL STUDENTS AND I HAVE GREATLY BENEFITED FROM THE CONTENT.

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Methodology
• Reliability vs Validity in Research | Differences, Types & Examples

Reliability vs Validity in Research | Differences, Types & Examples

Published on 3 May 2022 by Fiona Middleton . Revised on 10 October 2022.

Reliability and validity are concepts used to evaluate the quality of research. They indicate how well a method , technique, or test measures something. Reliability is about the consistency of a measure, and validity is about the accuracy of a measure.

It’s important to consider reliability and validity when you are creating your research design , planning your methods, and writing up your results, especially in quantitative research .

Table of contents

Understanding reliability vs validity, how are reliability and validity assessed, how to ensure validity and reliability in your research, where to write about reliability and validity in a thesis.

Reliability and validity are closely related, but they mean different things. A measurement can be reliable without being valid. However, if a measurement is valid, it is usually also reliable.

What is reliability?

Reliability refers to how consistently a method measures something. If the same result can be consistently achieved by using the same methods under the same circumstances, the measurement is considered reliable.

What is validity?

Validity refers to how accurately a method measures what it is intended to measure. If research has high validity, that means it produces results that correspond to real properties, characteristics, and variations in the physical or social world.

High reliability is one indicator that a measurement is valid. If a method is not reliable, it probably isn’t valid.

However, reliability on its own is not enough to ensure validity. Even if a test is reliable, it may not accurately reflect the real situation.

Validity is harder to assess than reliability, but it is even more important. To obtain useful results, the methods you use to collect your data must be valid: the research must be measuring what it claims to measure. This ensures that your discussion of the data and the conclusions you draw are also valid.

Prevent plagiarism, run a free check.

Reliability can be estimated by comparing different versions of the same measurement. Validity is harder to assess, but it can be estimated by comparing the results to other relevant data or theory. Methods of estimating reliability and validity are usually split up into different types.

Types of reliability

Different types of reliability can be estimated through various statistical methods.

Types of validity

The validity of a measurement can be estimated based on three main types of evidence. Each type can be evaluated through expert judgement or statistical methods.

To assess the validity of a cause-and-effect relationship, you also need to consider internal validity (the design of the experiment ) and external validity (the generalisability of the results).

The reliability and validity of your results depends on creating a strong research design , choosing appropriate methods and samples, and conducting the research carefully and consistently.

Ensuring validity

If you use scores or ratings to measure variations in something (such as psychological traits, levels of ability, or physical properties), it’s important that your results reflect the real variations as accurately as possible. Validity should be considered in the very earliest stages of your research, when you decide how you will collect your data .

• Choose appropriate methods of measurement

Ensure that your method and measurement technique are of high quality and targeted to measure exactly what you want to know. They should be thoroughly researched and based on existing knowledge.

For example, to collect data on a personality trait, you could use a standardised questionnaire that is considered reliable and valid. If you develop your own questionnaire, it should be based on established theory or the findings of previous studies, and the questions should be carefully and precisely worded.

• Use appropriate sampling methods to select your subjects

To produce valid generalisable results, clearly define the population you are researching (e.g., people from a specific age range, geographical location, or profession). Ensure that you have enough participants and that they are representative of the population.

Ensuring reliability

Reliability should be considered throughout the data collection process. When you use a tool or technique to collect data, it’s important that the results are precise, stable, and reproducible.

• Apply your methods consistently

Plan your method carefully to make sure you carry out the same steps in the same way for each measurement. This is especially important if multiple researchers are involved.

For example, if you are conducting interviews or observations, clearly define how specific behaviours or responses will be counted, and make sure questions are phrased the same way each time.

• Standardise the conditions of your research

When you collect your data, keep the circumstances as consistent as possible to reduce the influence of external factors that might create variation in the results.

For example, in an experimental setup, make sure all participants are given the same information and tested under the same conditions.

It’s appropriate to discuss reliability and validity in various sections of your thesis or dissertation or research paper. Showing that you have taken them into account in planning your research and interpreting the results makes your work more credible and trustworthy.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

Middleton, F. (2022, October 10). Reliability vs Validity in Research | Differences, Types & Examples. Scribbr. Retrieved 31 May 2024, from https://www.scribbr.co.uk/research-methods/reliability-or-validity/

Is this article helpful?

Fiona Middleton

Other students also liked, the 4 types of validity | types, definitions & examples, a quick guide to experimental design | 5 steps & examples, sampling methods | types, techniques, & examples.

• school Campus Bookshelves
• menu_book Bookshelves
• perm_media Learning Objects
• login Login
• how_to_reg Request Instructor Account
• hub Instructor Commons

Margin Size

• Download Page (PDF)
• Download Full Book (PDF)
• Periodic Table
• Physics Constants
• Scientific Calculator
• Reference & Cite
• Tools expand_more
• Readability

selected template will load here

This action is not available.

6.3: Steps in a Successful Marketing Research Plan

• Last updated
• Save as PDF
• Page ID 99245

\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)

\( \newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\)

( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\)

\( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)

\( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\)

\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)

\( \newcommand{\Span}{\mathrm{span}}\)

\( \newcommand{\id}{\mathrm{id}}\)

\( \newcommand{\kernel}{\mathrm{null}\,}\)

\( \newcommand{\range}{\mathrm{range}\,}\)

\( \newcommand{\RealPart}{\mathrm{Re}}\)

\( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)

\( \newcommand{\Argument}{\mathrm{Arg}}\)

\( \newcommand{\norm}[1]{\| #1 \|}\)

\( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\AA}{\unicode[.8,0]{x212B}}\)

\( \newcommand{\vectorA}[1]{\vec{#1}}      % arrow\)

\( \newcommand{\vectorAt}[1]{\vec{\text{#1}}}      % arrow\)

\( \newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

\( \newcommand{\vectorC}[1]{\textbf{#1}} \)

\( \newcommand{\vectorD}[1]{\overrightarrow{#1}} \)

\( \newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}} \)

\( \newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}} \)

Learning Objectives

By the end of this section, you will be able to:

• Identify and describe the steps in a marketing research plan.
• Discuss the different types of data research.
• Explain how data is analyzed.
• Discuss the importance of effective research reports.

Define the Problem

There are seven steps to a successful marketing research project (see Figure 6.3). Each step will be explained as we investigate how a marketing research project is conducted.

The first step, defining the problem, is often a realization that more information is needed in order to make a data-driven decision. Problem definition is the realization that there is an issue that needs to be addressed. An entrepreneur may be interested in opening a small business but must first define the problem that is to be investigated. A marketing research problem in this example is to discover the needs of the community and also to identify a potentially successful business venture.

Many times, researchers define a research question or objectives in this first step. Objectives of this research study could include: identify a new business that would be successful in the community in question, determine the size and composition of a target market for the business venture, and collect any relevant primary and secondary data that would support such a venture. At this point, the definition of the problem may be “Why are cat owners not buying our new cat toy subscription service?”

Additionally, during this first step we would want to investigate our target population for research. This is similar to a target market, as it is the group that comprises the population of interest for the study. In order to have a successful research outcome, the researcher should start with an understanding of the problem in the current situational environment.

Develop the Research Plan

Step two is to develop the research plan. What type of research is necessary to meet the established objectives of the first step? How will this data be collected? Additionally, what is the time frame of the research and budget to consider? If you must have information in the next week, a different plan would be implemented than in a situation where several months were allowed. These are issues that a researcher should address in order to meet the needs identified.

Research is often classified as coming from one of two types of data: primary and secondary. Primary data is unique information that is collected by the specific researcher with the current project in mind. This type of research doesn’t currently exist until it is pulled together for the project. Examples of primary data collection include survey, observation, experiment, or focus group data that is gathered for the current project.

Secondary data is any research that was completed for another purpose but can be used to help inform the research process. Secondary data comes in many forms and includes census data, journal articles, previously collected survey or focus group data of related topics, and compiled company data. Secondary data may be internal, such as the company’s sales records for a previous quarter, or external, such as an industry report of all related product sales. Syndicated data , a type of external secondary data, is available through subscription services and is utilized by many marketers. As you can see in Table 6.1, primary and secondary data features are often opposite—the positive aspects of primary data are the negative side of secondary data.

There are four research types that can be used: exploratory, descriptive, experimental, and ethnographic research designs (see Figure 6.4). Each type has specific formats of data that can be collected. Qualitative research can be shared through words, descriptions, and open-ended comments. Qualitative data gives context but cannot be reduced to a statistic. Qualitative data examples are categorical and include case studies, diary accounts, interviews, focus groups, and open-ended surveys. By comparison, quantitative data is data that can be reduced to number of responses. The number of responses to each answer on a multiple-choice question is quantitative data. Quantitative data is numerical and includes things like age, income, group size, and height.

Exploratory research is usually used when additional general information in desired about a topic. When in the initial steps of a new project, understanding the landscape is essential, so exploratory research helps the researcher to learn more about the general nature of the industry. Exploratory research can be collected through focus groups, interviews, and review of secondary data. When examining an exploratory research design, the best use is when your company hopes to collect data that is generally qualitative in nature. 7

For instance, if a company is considering a new service for registered users but is not quite sure how well the new service will be received or wants to gain clarity of exactly how customers may use a future service, the company can host a focus group. Focus groups and interviews will be examined later in the chapter. The insights collected during the focus group can assist the company when designing the service, help to inform promotional campaign options, and verify that the service is going to be a viable option for the company.

Descriptive research design takes a bigger step into collection of data through primary research complemented by secondary data. Descriptive research helps explain the market situation and define an “opinion, attitude, or behavior” of a group of consumers, employees, or other interested groups. 8 The most common method of deploying a descriptive research design is through the use of a survey. Several types of surveys will be defined later in this chapter. Descriptive data is quantitative in nature, meaning the data can be distilled into a statistic, such as in a table or chart.

Again, descriptive data is helpful in explaining the current situation. In the opening example of LEGO , the company wanted to describe the situation regarding children’s use of its product. In order to gather a large group of opinions, a survey was created. The data that was collected through this survey allowed the company to measure the existing perceptions of parents so that alterations could be made to future plans for the company.

Experimental research , also known as causal research , helps to define a cause-and-effect relationship between two or more factors. This type of research goes beyond a correlation to determine which feature caused the reaction. Researchers generally use some type of experimental design to determine a causal relationship. An example is A/B testing, a situation where one group of research participants, group A, is exposed to one treatment and then compared to the group B participants, who experience a different situation. An example might be showing two different television commercials to a panel of consumers and then measuring the difference in perception of the product. Another example would be to have two separate packaging options available in different markets. This research would answer the question “Does one design sell better than the other?” Comparing that to the sales in each market would be part of a causal research study. 9

The final method of collecting data is through an ethnographic design. Ethnographic research is conducted in the field by watching people interact in their natural environment. For marketing research, ethnographic designs help to identify how a product is used, what actions are included in a selection, or how the consumer interacts with the product. 10

Examples of ethnographic research would be to observe how a consumer uses a particular product, such as baking soda. Although many people buy baking soda, its uses are vast. So are they using it as a refrigerator deodorizer, a toothpaste, to polish a belt buckle, or to use in baking a cake?

Select the Data Collection Method

Data collection is the systematic gathering of information that addresses the identified problem. What is the best method to do that? Picking the right method of collecting data requires that the researcher understand the target population and the design picked in the previous step. There is no perfect method; each method has both advantages and disadvantages, so it’s essential that the researcher understand the target population of the research and the research objectives in order to pick the best option.

Sometimes the data desired is best collected by watching the actions of consumers. For instance, how many cars pass a specific billboard in a day? What website led a potential customer to the company’s website? When are consumers most likely to use the snack vending machines at work? What time of day has the highest traffic on a social media post? What is the most streamed television program this week? Observational research is the collecting of data based on actions taken by those observed. Many data observations do not require the researched individuals to participate in the data collection effort to be highly valuable. Some observation requires an individual to watch and record the activities of the target population through personal observations .

Unobtrusive observation happens when those being observed aren’t aware that they are being watched. An example of an unobtrusive observation would be to watch how shoppers interact with a new stuffed animal display by using a one-way mirror. Marketers can identify which products were handled more often while also determining which were ignored.

Other methods can use technology to collect the data instead. Instances of mechanical observation include the use of vehicle recorders, which count the number of vehicles that pass a specific location. Computers can also assess the number of shoppers who enter a store, the most popular entry point for train station commuters, or the peak time for cars to park in a parking garage.

When you want to get a more in-depth response from research participants, one method is to complete a one-on-one interview . One-on-one interviews allow the researcher to ask specific questions that match the respondent’s unique perspective as well as follow-up questions that piggyback on responses already completed. An interview allows the researcher to have a deeper understanding of the needs of the respondent, which is another strength of this type of data collection. The downside of personal interviews it that a discussion can be very time-consuming and results in only one respondent’s answers. Therefore, in order to get a large sample of respondents, the interview method may not be the most efficient method.

Taking the benefits of an interview and applying them to a small group of people is the design of a focus group . A focus group is a small number of people, usually 8 to 12, who meet the sample requirements. These individuals together are asked a series of questions where they are encouraged to build upon each other’s responses, either by agreeing or disagreeing with the other group members. Focus groups are similar to interviews in that they allow the researcher, through a moderator, to get more detailed information from a small group of potential customers (see Figure 6.5).

Link to Learning: Focus Groups

Focus groups are a common method for gathering insights into consumer thinking and habits. Companies will use this information to develop or shift their initiatives. The best way to understand a focus group is to watch a few examples or explanations. TED-Ed has this video that explains how focus groups work.

You might be asking when it is best to use a focus group or a survey. Learn the differences, the pros and cons of each, and the specific types of questions you ask in both situations in this article .

Preparing for a focus group is critical to success. It requires knowing the material and questions while also managing the group of people. Watch this video to learn more about how to prepare for a focus group and the types of things to be aware of.

One of the benefits of a focus group over individual interviews is that synergy can be generated when a participant builds on another’s ideas. Additionally, for the same amount of time, a researcher can hear from multiple respondents instead of just one. 11 Of course, as with every method of data collection, there are downsides to a focus group as well. Focus groups have the potential to be overwhelmed by one or two aggressive personalities, and the format can discourage more reserved individuals from speaking up. Finally, like interviews, the responses in a focus group are qualitative in nature and are difficult to distill into an easy statistic or two.

Combining a variety of questions on one instrument is called a survey or questionnaire . Collecting primary data is commonly done through surveys due to their versatility. A survey allows the researcher to ask the same set of questions of a large group of respondents. Response rates of surveys are calculated by dividing the number of surveys completed by the total number attempted. Surveys are flexible and can collect a variety of quantitative and qualitative data. Questions can include simplified yes or no questions, select all that apply, questions that are on a scale, or a variety of open-ended types of questions. There are four types of surveys (see Table 6.2) we will cover, each with strengths and weaknesses defined.

Let’s start off with mailed surveys —surveys that are sent to potential respondents through a mail service. Mailed surveys used to be more commonly used due to the ability to reach every household. In some instances, a mailed survey is still the best way to collect data. For example, every 10 years the United States conducts a census of its population (see Figure 6.6). The first step in that data collection is to send every household a survey through the US Postal Service (USPS). The benefit is that respondents can complete and return the survey at their convenience. The downside of mailed surveys are expense and timeliness of responses. A mailed survey requires postage, both when it is sent to the recipient and when it is returned. That, along with the cost of printing, paper, and both sending and return envelopes, adds up quickly. Additionally, physically mailing surveys takes time. One method of reducing cost is to send with bulk-rate postage, but that slows down the delivery of the survey. Also, because of the convenience to the respondent, completed surveys may be returned several weeks after being sent. Finally, some mailed survey data must be manually entered into the analysis software, which can cause delays or issues due to entry errors.

Phone surveys are completed during a phone conversation with the respondent. Although the traditional phone survey requires a data collector to talk with the participant, current technology allows for computer-assisted voice surveys or surveys to be completed by asking the respondent to push a specific button for each potential answer. Phone surveys are time intensive but allow the respondent to ask questions and the surveyor to request additional information or clarification on a question if warranted. Phone surveys require the respondent to complete the survey simultaneously with the collector, which is a limitation as there are restrictions for when phone calls are allowed. According to Telephone Consumer Protection Act , approved by Congress in 1991, no calls can be made prior to 8:00 a.m. or after 9:00 p.m. in the recipient’s time zone. 12 Many restrictions are outlined in this original legislation and have been added to since due to ever-changing technology.

In-person surveys are when the respondent and data collector are physically in the same location. In-person surveys allow the respondent to share specific information, ask questions of the surveyor, and follow up on previous answers. Surveys collected through this method can take place in a variety of ways: through door-to-door collection, in a public location, or at a person’s workplace. Although in-person surveys are time intensive and require more labor to collect data than some other methods, in some cases it’s the best way to collect the required data. In-person surveys conducted through a door-to-door method is the follow-up used for the census if respondents do not complete the mailed survey. One of the downsides of in-person surveys is the reluctance of potential respondents to stop their current activity and answer questions. Furthermore, people may not feel comfortable sharing private or personal information during a face-to-face conversation.

Electronic surveys are sent or collected through digital means and is an opportunity that can be added to any of the above methods as well as some new delivery options. Surveys can be sent through email, and respondents can either reply to the email or open a hyperlink to an online survey (see Figure 6.7). Additionally, a letter can be mailed that asks members of the survey sample to log in to a website rather than to return a mailed response. Many marketers now use links, QR codes, or electronic devices to easily connect to a survey. Digitally collected data has the benefit of being less time intensive and is often a more economical way to gather and input responses than more manual methods. A survey that could take months to collect through the mail can be completed within a week through digital means.

Design the Sample

Although you might want to include every possible person who matches your target market in your research, it’s often not a feasible option, nor is it of value. If you did decide to include everyone, you would be completing a census of the population. Getting everyone to participate would be time-consuming and highly expensive, so instead marketers use a sample , whereby a portion of the whole is included in the research. It’s similar to the samples you might receive at the grocery store or ice cream shop; it isn’t a full serving, but it does give you a good taste of what the whole would be like.

So how do you know who should be included in the sample? Researchers identify parameters for their studies, called sample frames . A sample frame for one study may be college students who live on campus; for another study, it may be retired people in Dallas, Texas, or small-business owners who have fewer than 10 employees. The individual entities within the sampling frame would be considered a sampling unit . A sampling unit is each individual respondent that would be considered as matching the sample frame established by the research. If a researcher wants businesses to participate in a study, then businesses would be the sampling unit in that case.

The number of sampling units included in the research is the sample size . Many calculations can be conducted to indicate what the correct size of the sample should be. Issues to consider are the size of the population, the confidence level that the data represents the entire population, the ease of accessing the units in the frame, and the budget allocated for the research.

There are two main categories of samples: probability and nonprobability (see Figure 6.8). Probability samples are those in which every member of the sample has an identified likelihood of being selected. Several probability sample methods can be utilized. One probability sampling technique is called a simple random sample , where not only does every person have an identified likelihood of being selected to be in the sample, but every person also has an equal chance of exclusion. An example of a simple random sample would be to put the names of all members of a group into a hat and simply draw out a specific number to be included. You could say a raffle would be a good example of a simple random sample.

Another probability sample type is a stratified random sample , where the population is divided into groups by category and then a random sample of each category is selected to participate. For instance, if you were conducting a study of college students from your school and wanted to make sure you had all grade levels included, you might take the names of all students and split them into different groups by grade level—freshman, sophomore, junior, and senior. Then, from those categories, you would draw names out of each of the pools, or strata.

A nonprobability sample is a situation in which each potential member of the sample has an unknown likelihood of being selected in the sample. Research findings that are from a nonprobability sample cannot be applied beyond the sample. Several examples of nonprobability sampling are available to researchers and include two that we will look at more closely: convenience sampling and judgment sampling.

The first nonprobability sampling technique is a convenience sample . Just like it sounds, a convenience sample is when the researcher finds a group through a nonscientific method by picking potential research participants in a convenient manner. An example might be to ask other students in a class you are taking to complete a survey that you are doing for a class assignment or passing out surveys at a basketball game or theater performance.

A judgment sample is a type of nonprobability sample that allows the researcher to determine if they believe the individual meets the criteria set for the sample frame to complete the research. For instance, you may be interested in researching mothers, so you sit outside a toy store and ask an individual who is carrying a baby to participate.

Collect the Data

Now that all the plans have been established, the instrument has been created, and the group of participants has been identified, it is time to start collecting data. As explained earlier in this chapter, data collection is the process of gathering information from a variety of sources that will satisfy the research objectives defined in step one. Data collection can be as simple as sending out an email with a survey link enclosed or as complex as an experiment with hundreds of consumers. The method of collection directly influences the length of this process. Conducting personal interviews or completing an experiment, as previously mentioned, can add weeks or months to the research process, whereas sending out an electronic survey may allow a researcher to collect the necessary data in a few days. 13

Analyze and Interpret the Data

Once the data has been collected, the process of analyzing it may begin. Data analysis is the distillation of the information into a more understandable and actionable format. The analysis itself can take many forms, from the use of basic statistics to a more comprehensive data visualization process. First, let’s discuss some basic statistics that can be used to represent data.

The first is the mean of quantitative data. A mean is often defined as the arithmetic average of values. The formula is:

A common use of the mean calculation is with exam scores. Say, for example, you have earned the following scores on your marketing exams: 72, 85, 68, and 77. To find the mean, you would add up the four scores for a total of 302. Then, in order to generate a mean, that number needs to be divided by the number of exam scores included, which is 4. The mean would be 302 divided by 4, for a mean test score of 75.5. Understanding the mean can help to determine, with one number, the weight of a particular value.

Another commonly used statistic is median. The median is often referred to as the middle number. To generate a median, all the numeric answers are placed in order, and the middle number is the median. Median is a common statistic when identifying the income level of a specific geographic region. 14 For instance, the median household income for Albuquerque, New Mexico, between 2015 and 2019 was $52,911. 15 In this case, there are just as many people with an income above the amount as there are below.

Mode is another statistic that is used to represent data of all types, as it can be used with quantitative or qualitative data and represents the most frequent answer. Eye color, hair color, and vehicle color can all be presented with a mode statistic. Additionally, some researchers expand on the concept of mode and present the frequency of all responses, not just identifying the most common response. Data such as this can easily be presented in a frequency graph, 16 such as the one in Figure 6.9.

Additionally, researchers use other analyses to represent the data rather than to present the entirety of each response. For example, maybe the relationship between two values is important to understand. In this case, the researcher may share the data as a cross tabulation (see Figure 6.10). Below is the same data as above regarding social media use cross tabulated with gender—as you can see, the data is more descriptive when you can distinguish between the gender identifiers and how much time is spent per day on social media.

Not all data can be presented in a graphical format due to the nature of the information. Sometimes with qualitative methods of data collection, the responses cannot be distilled into a simple statistic or graph. In that case, the use of quotations, otherwise known as verbatims , can be used. These are direct statements presented by the respondents. Often you will see a verbatim statement when reading a movie or book review. The critic’s statements are used in part or in whole to represent their feelings about the newly released item.

Link to Learning: Infographics

As they say, a picture is worth a thousand words. For this reason, research results are often shown in a graphical format in which data can be taken in quickly, called an infographic .

Check out this infographic on what components make for a good infographic. As you can see, a good infographic needs four components: data, design, a story, and the ability to share it with others. Without all four pieces, it is not as valuable a resource as it could be. The ultimate infographic is represented as the intersection of all four.

Infographics are particularly advantageous online. Refer to this infographic on why they are beneficial to use online .

Prepare the Research Report

The marketing research process concludes by sharing the generated data and makes recommendations for future actions. What starts as simple data must be interpreted into an analysis. All information gathered should be conveyed in order to make decisions for future marketing actions. One item that is often part of the final step is to discuss areas that may have been missed with the current project or any area of further study identified while completing it. Without the final step of the marketing research project, the first six steps are without value. It is only after the information is shared, through a formal presentation or report, that those recommendations can be implemented and improvements made. The first six steps are used to generate information, while the last is to initiate action. During this last step is also when an evaluation of the process is conducted. If this research were to be completed again, how would we do it differently? Did the right questions get answered with the survey questions posed to the respondents? Follow-up on some of these key questions can lead to additional research, a different study, or further analysis of data collected.

Methods of Quantifying Marketing Research

One of the ways of sharing information gained through marketing research is to quantify the research . Quantifying the research means to take a variety of data and compile into a quantity that is more easily understood. This is a simple process if you want to know how many people attended a basketball game, but if you want to quantify the number of students who made a positive comment on a questionnaire, it can be a little more complicated. Researchers have a variety of methods to collect and then share these different scores. Below are some of the most common types used in business.

Is a customer aware of a product, brand, or company? What is meant by awareness? Awareness in the context of marketing research is when a consumer is familiar with the product, brand, or company. It does not assume that the consumer has tried the product or has purchased it. Consumers are just aware. That is a measure that many businesses find valuable. There are several ways to measure awareness. For instance, the first type of awareness is unaided awareness . This type of awareness is when no prompts for a product, brand, or company are given. If you were collecting information on fast-food restaurants, you might ask a respondent to list all the fast-food restaurants that serve a chicken sandwich. Aided awareness would be providing a list of products, brands, or companies and the respondent selects from the list. For instance, if you give a respondent a list of fast-food restaurants and ask them to mark all the locations with a chicken sandwich, you are collecting data through an aided method. Collecting these answers helps a company determine how the business location compares to those of its competitors. 17

Customer Satisfaction (CSAT)

Have you ever been asked to complete a survey at the end of a purchase? Many businesses complete research on buying, returning, or other customer service processes. A customer satisfaction score , also known as CSAT, is a measure of how satisfied customers are with the product, brand, or service. A CSAT score is usually on a scale of 0 to 100 percent. 18 But what constitutes a “good” CSAT score? Although what is identified as good can vary by industry, normally anything in the range from 75 to 85 would be considered good. Of course, a number higher than 85 would be considered exceptional. 19

Customer Acquisition Cost (CAC) and Customer Effort Score (CES)

Other metrics often used are a customer acquisition cost (CAC) and customer effort score (CES). How much does it cost a company to gain customers? That’s the purpose of calculating the customer acquisition cost. To calculate the customer acquisition cost , a company would need to total all expenses that were accrued to gain new customers. This would include any advertising, public relations, social media postings, etc. When a total cost is determined, it is divided by the number of new customers gained through this campaign.

The final score to discuss is the customer effort score , also known as a CES. The CES is a “survey used to measure the ease of service experience with an organization.” 20 Companies that are easy to work with have a better CES than a company that is notorious for being difficult. An example would be to ask a consumer about the ease of making a purchase online by incorporating a one-question survey after a purchase is confirmed. If a number of responses come back negative or slightly negative, the company will realize that it needs to investigate and develop a more user-friendly process.

Knowledge Check

It’s time to check your knowledge on the concepts presented in this section. Refer to the Answer Key at the end of the book for feedback.

Sagar is completing a marketing research project and is at the stage where he must decide who will be sent the survey. What stage of the marketing research plan is Sagar currently on?

• Defining the problem
• Developing the research plan
• Selecting a data collection method
• Designing the sample

A strength of mailing a survey is that ________.

• you are able to send it to all households in an area
• it is inexpensive
• responses are automatically loaded into the software
• the data comes in quickly

Bartlett is considering the different types of data that can be pulled together for a research project. Currently they have collected journal articles, survey data, and syndicated data and completed a focus group. What type(s) of data have they collected?

• Primary data
• Secondary data
• Secondary and primary data
• Professional data

Which statistic can be used to show how many people responded to a survey question with “strongly agree”?

Why would a researcher want to use a cross tabulation?

• It shows how respondents answered two variables in relation to each other and can help determine patterns by different groups of respondents.
• By presenting the data in the form of a picture, the information is easier for the reader to understand.
• It is an easy way to see how often one answer is selected by the respondents.
• This analysis can used to present interview or focus group data.

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

10.2 Steps in the Marketing Research Process

Learning objective.

• Describe the basic steps in the marketing research process and the purpose of each step.

The basic steps used to conduct marketing research are shown in Figure 10.6 “Steps in the Marketing Research Process” . Next, we discuss each step.

Figure 10.6 Steps in the Marketing Research Process

Step 1: Define the Problem (or Opportunity)

There’s a saying in marketing research that a problem half defined is a problem half solved. Defining the “problem” of the research sounds simple, doesn’t it? Suppose your product is tutoring other students in a subject you’re a whiz at. You have been tutoring for a while, and people have begun to realize you’re darned good at it. Then, suddenly, your business drops off. Or it explodes, and you can’t cope with the number of students you’re being asked help. If the business has exploded, should you try to expand your services? Perhaps you should subcontract with some other “whiz” students. You would send them students to be tutored, and they would give you a cut of their pay for each student you referred to them.

Both of these scenarios would be a problem for you, wouldn’t they? They are problems insofar as they cause you headaches. But are they really the problem? Or are they the symptoms of something bigger? For example, maybe your business has dropped off because your school is experiencing financial trouble and has lowered the number of scholarships given to incoming freshmen. Consequently, there are fewer total students on campus who need your services. Conversely, if you’re swamped with people who want you to tutor them, perhaps your school awarded more scholarships than usual, so there are a greater number of students who need your services. Alternately, perhaps you ran an ad in your school’s college newspaper, and that led to the influx of students wanting you to tutor them.

Businesses are in the same boat you are as a tutor. They take a look at symptoms and try to drill down to the potential causes. If you approach a marketing research company with either scenario—either too much or too little business—the firm will seek more information from you such as the following:

• In what semester(s) did your tutoring revenues fall (or rise)?
• In what subject areas did your tutoring revenues fall (or rise)?
• In what sales channels did revenues fall (or rise): Were there fewer (or more) referrals from professors or other students? Did the ad you ran result in fewer (or more) referrals this month than in the past months?
• Among what demographic groups did your revenues fall (or rise)—women or men, people with certain majors, or first-year, second-, third-, or fourth-year students?

The key is to look at all potential causes so as to narrow the parameters of the study to the information you actually need to make a good decision about how to fix your business if revenues have dropped or whether or not to expand it if your revenues have exploded.

The next task for the researcher is to put into writing the research objective. The research objective is the goal(s) the research is supposed to accomplish. The marketing research objective for your tutoring business might read as follows:

To survey college professors who teach 100- and 200-level math courses to determine why the number of students referred for tutoring dropped in the second semester.

This is admittedly a simple example designed to help you understand the basic concept. If you take a marketing research course, you will learn that research objectives get a lot more complicated than this. The following is an example:

“To gather information from a sample representative of the U.S. population among those who are ‘very likely’ to purchase an automobile within the next 6 months, which assesses preferences (measured on a 1–5 scale ranging from ‘very likely to buy’ to ‘not likely at all to buy’) for the model diesel at three different price levels. Such data would serve as input into a forecasting model that would forecast unit sales, by geographic regions of the country, for each combination of the model’s different prices and fuel configurations (Burns & Bush, 2010).”

Now do you understand why defining the problem is complicated and half the battle? Many a marketing research effort is doomed from the start because the problem was improperly defined. Coke’s ill-fated decision to change the formula of Coca-Cola in 1985 is a case in point: Pepsi had been creeping up on Coke in terms of market share over the years as well as running a successful promotional campaign called the “Pepsi Challenge,” in which consumers were encouraged to do a blind taste test to see if they agreed that Pepsi was better. Coke spent four years researching “the problem.” Indeed, people seemed to like the taste of Pepsi better in blind taste tests. Thus, the formula for Coke was changed. But the outcry among the public was so great that the new formula didn’t last long—a matter of months—before the old formula was reinstated. Some marketing experts believe Coke incorrectly defined the problem as “How can we beat Pepsi in taste tests?” instead of “How can we gain market share against Pepsi?” (Burns & Bush, 2010)

New Coke Is It! 1985

(click to see video)

This video documents the Coca-Cola Company’s ill-fated launch of New Coke in 1985.

1985 Pepsi Commercial—“They Changed My Coke”

This video shows how Pepsi tried to capitalize on the blunder.

Step 2: Design the Research

The next step in the marketing research process is to do a research design. The research design is your “plan of attack.” It outlines what data you are going to gather and from whom, how and when you will collect the data, and how you will analyze it once it’s been obtained. Let’s look at the data you’re going to gather first.

There are two basic types of data you can gather. The first is primary data. Primary data is information you collect yourself, using hands-on tools such as interviews or surveys, specifically for the research project you’re conducting. Secondary data is data that has already been collected by someone else, or data you have already collected for another purpose. Collecting primary data is more time consuming, work intensive, and expensive than collecting secondary data. Consequently, you should always try to collect secondary data first to solve your research problem, if you can. A great deal of research on a wide variety of topics already exists. If this research contains the answer to your question, there is no need for you to replicate it. Why reinvent the wheel?

Sources of Secondary Data

Your company’s internal records are a source of secondary data. So are any data you collect as part of your marketing intelligence gathering efforts. You can also purchase syndicated research. Syndicated research is primary data that marketing research firms collect on a regular basis and sell to other companies. J.D. Power & Associates is a provider of syndicated research. The company conducts independent, unbiased surveys of customer satisfaction, product quality, and buyer behavior for various industries. The company is best known for its research in the automobile sector. One of the best-known sellers of syndicated research is the Nielsen Company, which produces the Nielsen ratings. The Nielsen ratings measure the size of television, radio, and newspaper audiences in various markets. You have probably read or heard about TV shows that get the highest (Nielsen) ratings. (Arbitron does the same thing for radio ratings.) Nielsen, along with its main competitor, Information Resources, Inc. (IRI), also sells businesses scanner-based research . Scanner-based research is information collected by scanners at checkout stands in stores. Each week Nielsen and IRI collect information on the millions of purchases made at stores. The companies then compile the information and sell it to firms in various industries that subscribe to their services. The Nielsen Company has also recently teamed up with Facebook to collect marketing research information. Via Facebook, users will see surveys in some of the spaces in which they used to see online ads (Rappeport, Gelles, 2009).

By contrast, MarketResearch.com is an example of a marketing research aggregator. A marketing research aggregator is a marketing research company that doesn’t conduct its own research and sell it. Instead, it buys research reports from other marketing research companies and then sells the reports in their entirety or in pieces to other firms. Check out MarketResearch.com’s Web site. As you will see there are a huge number of studies in every category imaginable that you can buy for relatively small amounts of money.

Figure 10.7

Market research aggregators buy research reports from other marketing research companies and then resell them in part or in whole to other companies so they don’t have to gather primary data.

Source: http://www.marketresearch.com .

Your local library is a good place to gather free secondary data. It has searchable databases as well as handbooks, dictionaries, and books, some of which you can access online. Government agencies also collect and report information on demographics, economic and employment data, health information, and balance-of-trade statistics, among a lot of other information. The U.S. Census Bureau collects census data every ten years to gather information about who lives where. Basic demographic information about sex, age, race, and types of housing in which people live in each U.S. state, metropolitan area, and rural area is gathered so that population shifts can be tracked for various purposes, including determining the number of legislators each state should have in the U.S. House of Representatives. For the U.S. government, this is primary data. For marketing managers it is an important source of secondary data.

The Survey Research Center at the University of Michigan also conducts periodic surveys and publishes information about trends in the United States. One research study the center continually conducts is called the “Changing Lives of American Families” ( http://www.isr.umich.edu/home/news/research-update/2007-01.pdf ). This is important research data for marketing managers monitoring consumer trends in the marketplace. The World Bank and the United Nations are two international organizations that collect a great deal of information. Their Web sites contain many free research studies and data related to global markets. Table 10.1 “Examples of Primary Data Sources versus Secondary Data Sources” shows some examples of primary versus secondary data sources.

Table 10.1 Examples of Primary Data Sources versus Secondary Data Sources

Gauging the Quality of Secondary Data

When you are gathering secondary information, it’s always good to be a little skeptical of it. Sometimes studies are commissioned to produce the result a client wants to hear—or wants the public to hear. For example, throughout the twentieth century, numerous studies found that smoking was good for people’s health. The problem was the studies were commissioned by the tobacco industry. Web research can also pose certain hazards. There are many biased sites that try to fool people that they are providing good data. Often the data is favorable to the products they are trying to sell. Beware of product reviews as well. Unscrupulous sellers sometimes get online and create bogus ratings for products. See below for questions you can ask to help gauge the credibility of secondary information.

Gauging the Credibility of Secondary Data: Questions to Ask

• Who gathered this information?
• For what purpose?
• What does the person or organization that gathered the information have to gain by doing so?
• Was the information gathered and reported in a systematic manner?
• Is the source of the information accepted as an authority by other experts in the field?
• Does the article provide objective evidence to support the position presented?

Types of Research Design

Now let’s look specifically at the types of research designs that are utilized. By understanding different types of research designs, a researcher can solve a client’s problems more quickly and efficiently without jumping through more hoops than necessary. Research designs fall into one of the following three categories:

• Exploratory research design
• Descriptive research design
• Causal research design (experiments)

An exploratory research design is useful when you are initially investigating a problem but you haven’t defined it well enough to do an in-depth study of it. Perhaps via your regular market intelligence, you have spotted what appears to be a new opportunity in the marketplace. You would then do exploratory research to investigate it further and “get your feet wet,” as the saying goes. Exploratory research is less structured than other types of research, and secondary data is often utilized.

One form of exploratory research is qualitative research. Qualitative research is any form of research that includes gathering data that is not quantitative, and often involves exploring questions such as why as much as what or how much . Different forms, such as depth interviews and focus group interviews, are common in marketing research.

The depth interview —engaging in detailed, one-on-one, question-and-answer sessions with potential buyers—is an exploratory research technique. However, unlike surveys, the people being interviewed aren’t asked a series of standard questions. Instead the interviewer is armed with some general topics and asks questions that are open ended, meaning that they allow the interviewee to elaborate. “How did you feel about the product after you purchased it?” is an example of a question that might be asked. A depth interview also allows a researcher to ask logical follow-up questions such as “Can you tell me what you mean when you say you felt uncomfortable using the service?” or “Can you give me some examples?” to help dig further and shed additional light on the research problem. Depth interviews can be conducted in person or over the phone. The interviewer either takes notes or records the interview.

Focus groups and case studies are often utilized for exploratory research as well. A focus group is a group of potential buyers who are brought together to discuss a marketing research topic with one another. A moderator is used to focus the discussion, the sessions are recorded, and the main points of consensus are later summarized by the market researcher. Textbook publishers often gather groups of professors at educational conferences to participate in focus groups. However, focus groups can also be conducted on the telephone, in online chat rooms, or both, using meeting software like WebEx. The basic steps of conducting a focus group are outlined below.

The Basic Steps of Conducting a Focus Group

• Establish the objectives of the focus group. What is its purpose?
• Identify the people who will participate in the focus group. What makes them qualified to participate? How many of them will you need and what they will be paid?
• Obtain contact information for the participants and send out invitations (usually e-mails are most efficient).
• Develop a list of questions.
• Choose a facilitator.
• Choose a location in which to hold the focus group and the method by which it will be recorded.
• Conduct the focus group. If the focus group is not conducted electronically, include name tags for the participants, pens and notepads, any materials the participants need to see, and refreshments. Record participants’ responses.
• Summarize the notes from the focus group and write a report for management.

A case study looks at how another company solved the problem that’s being researched. Sometimes multiple cases, or companies, are used in a study. Case studies nonetheless have a mixed reputation. Some researchers believe it’s hard to generalize, or apply, the results of a case study to other companies. Nonetheless, collecting information about companies that encountered the same problems your firm is facing can give you a certain amount of insight about what direction you should take. In fact, one way to begin a research project is to carefully study a successful product or service.

Two other types of qualitative data used for exploratory research are ethnographies and projective techniques. In an ethnography , researchers interview, observe, and often videotape people while they work, live, shop, and play. The Walt Disney Company has recently begun using ethnographers to uncover the likes and dislikes of boys aged six to fourteen, a financially attractive market segment for Disney, but one in which the company has been losing market share. The ethnographers visit the homes of boys, observe the things they have in their rooms to get a sense of their hobbies, and accompany them and their mothers when they shop to see where they go, what the boys are interested in, and what they ultimately buy. (The children get seventy-five dollars out of the deal, incidentally.) (Barnes, 2009)

Projective techniques are used to reveal information research respondents might not reveal by being asked directly. Asking a person to complete sentences such as the following is one technique:

People who buy Coach handbags __________.

(Will he or she reply with “are cool,” “are affluent,” or “are pretentious,” for example?)

KFC’s grilled chicken is ______.

Or the person might be asked to finish a story that presents a certain scenario. Word associations are also used to discern people’s underlying attitudes toward goods and services. Using a word-association technique, a market researcher asks a person to say or write the first word that comes to his or her mind in response to another word. If the initial word is “fast food,” what word does the person associate it with or respond with? Is it “McDonald’s”? If many people reply that way, and you’re conducting research for Burger King, that could indicate Burger King has a problem. However, if the research is being conducted for Wendy’s, which recently began running an advertising campaign to the effect that Wendy’s offerings are “better than fast food,” it could indicate that the campaign is working.

Completing cartoons is yet another type of projective technique. It’s similar to finishing a sentence or story, only with the pictures. People are asked to look at a cartoon such as the one shown in Figure 10.8 “Example of a Cartoon-Completion Projective Technique” . One of the characters in the picture will have made a statement, and the person is asked to fill in the empty cartoon “bubble” with how they think the second character will respond.

Figure 10.8 Example of a Cartoon-Completion Projective Technique

In some cases, your research might end with exploratory research. Perhaps you have discovered your organization lacks the resources needed to produce the product. In other cases, you might decide you need more in-depth, quantitative research such as descriptive research or causal research, which are discussed next. Most marketing research professionals advise using both types of research, if it’s feasible. On the one hand, the qualitative-type research used in exploratory research is often considered too “lightweight.” Remember earlier in the chapter when we discussed telephone answering machines and the hit TV sitcom Seinfeld ? Both product ideas were initially rejected by focus groups. On the other hand, relying solely on quantitative information often results in market research that lacks ideas.

The Stone Wheel—What One Focus Group Said

Watch the video to see a funny spoof on the usefulness—or lack of usefulness—of focus groups.

Descriptive Research

Anything that can be observed and counted falls into the category of descriptive research design. A study using a descriptive research design involves gathering hard numbers, often via surveys, to describe or measure a phenomenon so as to answer the questions of who , what , where , when , and how . “On a scale of 1–5, how satisfied were you with your service?” is a question that illustrates the information a descriptive research design is supposed to capture.

Physiological measurements also fall into the category of descriptive design. Physiological measurements measure people’s involuntary physical responses to marketing stimuli, such as an advertisement. Elsewhere, we explained that researchers have gone so far as to scan the brains of consumers to see what they really think about products versus what they say about them. Eye tracking is another cutting-edge type of physiological measurement. It involves recording the movements of a person’s eyes when they look at some sort of stimulus, such as a banner ad or a Web page. The Walt Disney Company has a research facility in Austin, Texas, that it uses to take physical measurements of viewers when they see Disney programs and advertisements. The facility measures three types of responses: people’s heart rates, skin changes, and eye movements (eye tracking) (Spangler, 2009).

Figure 10.9

A woman shows off her headgear for an eye-tracking study. The gear’s not exactly a fashion statement but . . .

lawrencegs – Google Glass – CC BY 2.0.

A strictly descriptive research design instrument—a survey, for example—can tell you how satisfied your customers are. It can’t, however, tell you why. Nor can an eye-tracking study tell you why people’s eyes tend to dwell on certain types of banner ads—only that they do. To answer “why” questions an exploratory research design or causal research design is needed (Wagner, 2007).

Causal Research

Causal research design examines cause-and-effect relationships. Using a causal research design allows researchers to answer “what if” types of questions. In other words, if a firm changes X (say, a product’s price, design, placement, or advertising), what will happen to Y (say, sales or customer loyalty)? To conduct causal research, the researcher designs an experiment that “controls,” or holds constant, all of a product’s marketing elements except one (or using advanced techniques of research, a few elements can be studied at the same time). The one variable is changed, and the effect is then measured. Sometimes the experiments are conducted in a laboratory using a simulated setting designed to replicate the conditions buyers would experience. Or the experiments may be conducted in a virtual computer setting.

You might think setting up an experiment in a virtual world such as the online game Second Life would be a viable way to conduct controlled marketing research. Some companies have tried to use Second Life for this purpose, but the results have been somewhat mixed as to whether or not it is a good medium for marketing research. The German marketing research firm Komjuniti was one of the first “real-world” companies to set up an “island” in Second Life upon which it could conduct marketing research. However, with so many other attractive fantasy islands in which to play, the company found it difficult to get Second Life residents, or players, to voluntarily visit the island and stay long enough so meaningful research could be conducted. (Plus, the “residents,” or players, in Second Life have been known to protest corporations invading their world. When the German firm Komjuniti created an island in Second Life to conduct marketing research, the residents showed up waving signs and threatening to boycott the island.) (Wagner, 2007)

Why is being able to control the setting so important? Let’s say you are an American flag manufacturer and you are working with Walmart to conduct an experiment to see where in its stores American flags should be placed so as to increase their sales. Then the terrorist attacks of 9/11 occur. In the days afterward, sales skyrocketed—people bought flags no matter where they were displayed. Obviously, the terrorist attacks in the United States would have skewed the experiment’s data.

An experiment conducted in a natural setting such as a store is referred to as a field experiment . Companies sometimes do field experiments either because it is more convenient or because they want to see if buyers will behave the same way in the “real world” as in a laboratory or on a computer. The place the experiment is conducted or the demographic group of people the experiment is administered to is considered the test market . Before a large company rolls out a product to the entire marketplace, it will often place the offering in a test market to see how well it will be received. For example, to compete with MillerCoors’ sixty-four-calorie beer MGD 64, Anheuser-Busch recently began testing its Select 55 beer in certain cities around the country (McWilliams, 2009).

Figure 10.10

Select 55 beer: Coming soon to a test market near you? (If you’re on a diet, you have to hope so!)

Martine – Le champagne – CC BY-NC 2.0.

Many companies use experiments to test all of their marketing communications. For example, the online discount retailer O.co (formerly called Overstock.com) carefully tests all of its marketing offers and tracks the results of each one. One study the company conducted combined twenty-six different variables related to offers e-mailed to several thousand customers. The study resulted in a decision to send a group of e-mails to different segments. The company then tracked the results of the sales generated to see if they were in line with the earlier experiment it had conducted that led it to make the offer.

Step 3: Design the Data-Collection Forms

If the behavior of buyers is being formally observed, and a number of different researchers are conducting observations, the data obviously need to be recorded on a standardized data-collection form that’s either paper or electronic. Otherwise, the data collected will not be comparable. The items on the form could include a shopper’s sex; his or her approximate age; whether the person seemed hurried, moderately hurried, or unhurried; and whether or not he or she read the label on products, used coupons, and so forth.

The same is true when it comes to surveying people with questionnaires. Surveying people is one of the most commonly used techniques to collect quantitative data. Surveys are popular because they can be easily administered to large numbers of people fairly quickly. However, to produce the best results, the questionnaire for the survey needs to be carefully designed.

Questionnaire Design

Most questionnaires follow a similar format: They begin with an introduction describing what the study is for, followed by instructions for completing the questionnaire and, if necessary, returning it to the market researcher. The first few questions that appear on the questionnaire are usually basic, warm-up type of questions the respondent can readily answer, such as the respondent’s age, level of education, place of residence, and so forth. The warm-up questions are then followed by a logical progression of more detailed, in-depth questions that get to the heart of the question being researched. Lastly, the questionnaire wraps up with a statement that thanks the respondent for participating in the survey and information and explains when and how they will be paid for participating. To see some examples of questionnaires and how they are laid out, click on the following link: http://cas.uah.edu/wrenb/mkt343/Project/Sample%20Questionnaires.htm .

How the questions themselves are worded is extremely important. It’s human nature for respondents to want to provide the “correct” answers to the person administering the survey, so as to seem agreeable. Therefore, there is always a hazard that people will try to tell you what you want to hear on a survey. Consequently, care needs to be taken that the survey questions are written in an unbiased, neutral way. In other words, they shouldn’t lead a person taking the questionnaire to answer a question one way or another by virtue of the way you have worded it. The following is an example of a leading question.

Don’t you agree that teachers should be paid more ?

The questions also need to be clear and unambiguous. Consider the following question:

Which brand of toothpaste do you use ?

The question sounds clear enough, but is it really? What if the respondent recently switched brands? What if she uses Crest at home, but while away from home or traveling, she uses Colgate’s Wisp portable toothpaste-and-brush product? How will the respondent answer the question? Rewording the question as follows so it’s more specific will help make the question clearer:

Which brand of toothpaste have you used at home in the past six months? If you have used more than one brand, please list each of them 1 .

Sensitive questions have to be asked carefully. For example, asking a respondent, “Do you consider yourself a light, moderate, or heavy drinker?” can be tricky. Few people want to admit to being heavy drinkers. You can “soften” the question by including a range of answers, as the following example shows:

How many alcoholic beverages do you consume in a week ?

• __0–5 alcoholic beverages
• __5–10 alcoholic beverages
• __10–15 alcoholic beverages

Many people don’t like to answer questions about their income levels. Asking them to specify income ranges rather than divulge their actual incomes can help.

Other research question “don’ts” include using jargon and acronyms that could confuse people. “How often do you IM?” is an example. Also, don’t muddy the waters by asking two questions in the same question, something researchers refer to as a double-barreled question . “Do you think parents should spend more time with their children and/or their teachers?” is an example of a double-barreled question.

Open-ended questions , or questions that ask respondents to elaborate, can be included. However, they are harder to tabulate than closed-ended questions , or questions that limit a respondent’s answers. Multiple-choice and yes-and-no questions are examples of closed-ended questions.

Testing the Questionnaire

You have probably heard the phrase “garbage in, garbage out.” If the questions are bad, the information gathered will be bad, too. One way to make sure you don’t end up with garbage is to test the questionnaire before sending it out to find out if there are any problems with it. Is there enough space for people to elaborate on open-ended questions? Is the font readable? To test the questionnaire, marketing research professionals first administer it to a number of respondents face to face. This gives the respondents the chance to ask the researcher about questions or instructions that are unclear or don’t make sense to them. The researcher then administers the questionnaire to a small subset of respondents in the actual way the survey is going to be disseminated, whether it’s delivered via phone, in person, by mail, or online.

Getting people to participate and complete questionnaires can be difficult. If the questionnaire is too long or hard to read, many people won’t complete it. So, by all means, eliminate any questions that aren’t necessary. Of course, including some sort of monetary incentive for completing the survey can increase the number of completed questionnaires a market researcher will receive.

Step 4: Specify the Sample

Once you have created your questionnaire or other marketing study, how do you figure out who should participate in it? Obviously, you can’t survey or observe all potential buyers in the marketplace. Instead, you must choose a sample. A sample is a subset of potential buyers that are representative of your entire target market, or population being studied. Sometimes market researchers refer to the population as the universe to reflect the fact that it includes the entire target market, whether it consists of a million people, a hundred thousand, a few hundred, or a dozen. “All unmarried people over the age of eighteen who purchased Dirt Devil steam cleaners in the United States during 2011” is an example of a population that has been defined.

Obviously, the population has to be defined correctly. Otherwise, you will be studying the wrong group of people. Not defining the population correctly can result in flawed research, or sampling error. A sampling error is any type of marketing research mistake that results because a sample was utilized. One criticism of Internet surveys is that the people who take these surveys don’t really represent the overall population. On average, Internet survey takers tend to be more educated and tech savvy. Consequently, if they solely constitute your population, even if you screen them for certain criteria, the data you collect could end up being skewed.

The next step is to put together the sampling frame , which is the list from which the sample is drawn. The sampling frame can be put together using a directory, customer list, or membership roster (Wrenn et. al., 2007). Keep in mind that the sampling frame won’t perfectly match the population. Some people will be included on the list who shouldn’t be. Other people who should be included will be inadvertently omitted. It’s no different than if you were to conduct a survey of, say, 25 percent of your friends, using friends’ names you have in your cell phone. Most of your friends’ names are likely to be programmed into your phone, but not all of them. As a result, a certain degree of sampling error always occurs.

There are two main categories of samples in terms of how they are drawn: probability samples and nonprobability samples. A probability sample is one in which each would-be participant has a known and equal chance of being selected. The chance is known because the total number of people in the sampling frame is known. For example, if every other person from the sampling frame were chosen, each person would have a 50 percent chance of being selected.

A nonprobability sample is any type of sample that’s not drawn in a systematic way. So the chances of each would-be participant being selected can’t be known. A convenience sample is one type of nonprobability sample. It is a sample a researcher draws because it’s readily available and convenient to do so. Surveying people on the street as they pass by is an example of a convenience sample. The question is, are these people representative of the target market?

For example, suppose a grocery store needed to quickly conduct some research on shoppers to get ready for an upcoming promotion. Now suppose that the researcher assigned to the project showed up between the hours of 10 a.m. and 12 p.m. on a weekday and surveyed as many shoppers as possible. The problem is that the shoppers wouldn’t be representative of the store’s entire target market. What about commuters who stop at the store before and after work? Their views wouldn’t be represented. Neither would people who work the night shift or shop at odd hours. As a result, there would be a lot of room for sampling error in this study. For this reason, studies that use nonprobability samples aren’t considered as accurate as studies that use probability samples. Nonprobability samples are more often used in exploratory research.

Lastly, the size of the sample has an effect on the amount of sampling error. Larger samples generally produce more accurate results. The larger your sample is, the more data you will have, which will give you a more complete picture of what you’re studying. However, the more people surveyed or studied, the more costly the research becomes.

Statistics can be used to determine a sample’s optimal size. If you take a marketing research or statistics class, you will learn more about how to determine the optimal size.

Of course, if you hire a marketing research company, much of this work will be taken care of for you. Many marketing research companies, like ResearchNow, maintain panels of prescreened people they draw upon for samples. In addition, the marketing research firm will be responsible for collecting the data or contracting with a company that specializes in data collection. Data collection is discussed next.

Step 5: Collect the Data

As we have explained, primary marketing research data can be gathered in a number of ways. Surveys, taking physical measurements, and observing people are just three of the ways we discussed. If you’re observing customers as part of gathering the data, keep in mind that if shoppers are aware of the fact, it can have an effect on their behavior. For example, if a customer shopping for feminine hygiene products in a supermarket aisle realizes she is being watched, she could become embarrassed and leave the aisle, which would adversely affect your data. To get around problems such as these, some companies set up cameras or two-way mirrors to observe customers. Organizations also hire mystery shoppers to work around the problem. A mystery shopper is someone who is paid to shop at a firm’s establishment or one of its competitors to observe the level of service, cleanliness of the facility, and so forth, and report his or her findings to the firm.

Make Extra Money as a Mystery Shopper

Watch the YouTube video to get an idea of how mystery shopping works.

Survey data can be collected in many different ways and combinations of ways. The following are the basic methods used:

• Face-to-face (can be computer aided)
• Telephone (can be computer aided or completely automated)
• Mail and hand delivery
• E-mail and the Web

A face-to-face survey is, of course, administered by a person. The surveys are conducted in public places such as in shopping malls, on the street, or in people’s homes if they have agreed to it. In years past, it was common for researchers in the United States to knock on people’s doors to gather survey data. However, randomly collected door-to-door interviews are less common today, partly because people are afraid of crime and are reluctant to give information to strangers (McDaniel & Gates, 1998).

Nonetheless, “beating the streets” is still a legitimate way questionnaire data is collected. When the U.S. Census Bureau collects data on the nation’s population, it hand delivers questionnaires to rural households that do not have street-name and house-number addresses. And Census Bureau workers personally survey the homeless to collect information about their numbers. Face-to-face surveys are also commonly used in third world countries to collect information from people who cannot read or lack phones and computers.

A plus of face-to-face surveys is that they allow researchers to ask lengthier, more complex questions because the people being surveyed can see and read the questionnaires. The same is true when a computer is utilized. For example, the researcher might ask the respondent to look at a list of ten retail stores and rank the stores from best to worst. The same question wouldn’t work so well over the telephone because the person couldn’t see the list. The question would have to be rewritten. Another drawback with telephone surveys is that even though federal and state “do not call” laws generally don’t prohibit companies from gathering survey information over the phone, people often screen such calls using answering machines and caller ID.

Probably the biggest drawback of both surveys conducted face-to-face and administered over the phone by a person is that they are labor intensive and therefore costly. Mailing out questionnaires is costly, too, and the response rates can be rather low. Think about why that might be so: if you receive a questionnaire in the mail, it is easy to throw it in the trash; it’s harder to tell a market researcher who approaches you on the street that you don’t want to be interviewed.

By contrast, gathering survey data collected by a computer, either over the telephone or on the Internet, can be very cost-effective and in some cases free. SurveyMonkey and Zoomerang are two Web sites that will allow you to create online questionnaires, e-mail them to up to one hundred people for free, and view the responses in real time as they come in. For larger surveys, you have to pay a subscription price of a few hundred dollars. But that still can be extremely cost-effective. The two Web sites also have a host of other features such as online-survey templates you can use to create your questionnaire, a way to set up automatic reminders sent to people who haven’t yet completed their surveys, and tools you can use to create graphics to put in your final research report. To see how easy it is to put together a survey in SurveyMonkey, click on the following link: http://help.surveymonkey.com/app/tutorials/detail/a_id/423 .

Like a face-to-face survey, an Internet survey can enable you to show buyers different visuals such as ads, pictures, and videos of products and their packaging. Web surveys are also fast, which is a major plus. Whereas face-to-face and mailed surveys often take weeks to collect, you can conduct a Web survey in a matter of days or even hours. And, of course, because the information is electronically gathered it can be automatically tabulated. You can also potentially reach a broader geographic group than you could if you had to personally interview people. The Zoomerang Web site allows you to create surveys in forty different languages.

Another plus for Web and computer surveys (and electronic phone surveys) is that there is less room for human error because the surveys are administered electronically. For instance, there’s no risk that the interviewer will ask a question wrong or use a tone of voice that could mislead the respondents. Respondents are also likely to feel more comfortable inputting the information into a computer if a question is sensitive than they would divulging the information to another person face-to-face or over the phone. Given all of these advantages, it’s not surprising that the Internet is quickly becoming the top way to collect primary data. However, like mail surveys, surveys sent to people over the Internet are easy to ignore.

Lastly, before the data collection process begins, the surveyors and observers need to be trained to look for the same things, ask questions the same way, and so forth. If they are using rankings or rating scales, they need to be “on the same page,” so to speak, as to what constitutes a high ranking or a low ranking. As an analogy, you have probably had some teachers grade your college papers harder than others. The goal of training is to avoid a wide disparity between how different observers and interviewers record the data.

Figure 10.11

Training people so they know what constitutes different ratings when they are collecting data will improve the quality of the information gathered in a marketing research study.

Ricardo Rodriquez – Satisfaction survey – CC BY-NC-ND 2.0.

For example, if an observation form asks the observers to describe whether a shopper’s behavior is hurried, moderately hurried, or unhurried, they should be given an idea of what defines each rating. Does it depend on how much time the person spends in the store or in the individual aisles? How fast they walk? In other words, the criteria and ratings need to be spelled out.

Collecting International Marketing Research Data

Gathering marketing research data in foreign countries poses special challenges. However, that doesn’t stop firms from doing so. Marketing research companies are located all across the globe, in fact. Eight of the ten largest marketing research companies in the world are headquartered in the United States. However, five of these eight firms earn more of their revenues abroad than they do in the United States. There’s a reason for this: many U.S. markets were saturated, or tapped out, long ago in terms of the amount that they can grow. Coke is an example. As you learned earlier in the book, most of the Coca-Cola Company’s revenues are earned in markets abroad. To be sure, the United States is still a huge market when it comes to the revenues marketing research firms generate by conducting research in the country: in terms of their spending, American consumers fuel the world’s economic engine. Still, emerging countries with growing middle classes, such as China, India, and Brazil, are hot new markets companies want to tap.

What kind of challenges do firms face when trying to conduct marketing research abroad? As we explained, face-to-face surveys are commonly used in third world countries to collect information from people who cannot read or lack phones and computers. However, face-to-face surveys are also common in Europe, despite the fact that phones and computers are readily available. In-home surveys are also common in parts of Europe. By contrast, in some countries, including many Asian countries, it’s considered taboo or rude to try to gather information from strangers either face-to-face or over the phone. In many Muslim countries, women are forbidden to talk to strangers.

And how do you figure out whom to research in foreign countries? That in itself is a problem. In the United States, researchers often ask if they can talk to the heads of households to conduct marketing research. But in countries in which domestic servants or employees are common, the heads of households aren’t necessarily the principal shoppers; their domestic employees are (Malhotra).

Translating surveys is also an issue. Have you ever watched the TV comedians Jay Leno and David Letterman make fun of the English translations found on ethnic menus and products? Research tools such as surveys can suffer from the same problem. Hiring someone who is bilingual to translate a survey into another language can be a disaster if the person isn’t a native speaker of the language to which the survey is being translated.

One way companies try to deal with translation problems is by using back translation. When back translation is used, a native speaker translates the survey into the foreign language and then translates it back again to the original language to determine if there were gaps in meaning—that is, if anything was lost in translation. And it’s not just the language that’s an issue. If the research involves any visual images, they, too, could be a point of confusion. Certain colors, shapes, and symbols can have negative connotations in other countries. For example, the color white represents purity in many Western cultures, but in China, it is the color of death and mourning (Zouhali-Worrall, 2008). Also, look back at the cartoon-completion exercise in Figure 10.8 “Example of a Cartoon-Completion Projective Technique” . What would women in Muslim countries who aren’t allowed to converse with male sellers think of it? Chances are, the cartoon wouldn’t provide you with the information you’re seeking if Muslim women in some countries were asked to complete it.

One way marketing research companies are dealing with the complexities of global research is by merging with or acquiring marketing research companies abroad. The Nielsen Company is the largest marketing research company in the world. The firm operates in more than a hundred countries and employs more than forty thousand people. Many of its expansions have been the result of acquisitions and mergers.

Step 6: Analyze the Data

Step 6 involves analyzing the data to ensure it’s as accurate as possible. If the research is collected by hand using a pen and pencil, it’s entered into a computer. Or respondents might have already entered the information directly into a computer. For example, when Toyota goes to an event such as a car show, the automaker’s marketing personnel ask would-be buyers to complete questionnaires directly on computers. Companies are also beginning to experiment with software that can be used to collect data using mobile phones.

Once all the data is collected, the researchers begin the data cleaning , which is the process of removing data that have accidentally been duplicated (entered twice into the computer) or correcting data that have obviously been recorded wrong. A program such as Microsoft Excel or a statistical program such as Predictive Analytics Software (PASW, which was formerly known as SPSS) is then used to tabulate, or calculate, the basic results of the research, such as the total number of participants and how collectively they answered various questions. The programs can also be used to calculate averages, such as the average age of respondents, their average satisfaction, and so forth. The same can done for percentages, and other values you learned about, or will learn about, in a statistics course, such as the standard deviation, mean, and median for each question.

The information generated by the programs can be used to draw conclusions, such as what all customers might like or not like about an offering based on what the sample group liked or did not like. The information can also be used to spot differences among groups of people. For example, the research might show that people in one area of the country like the product better than people in another area. Trends to predict what might happen in the future can also be spotted.

If there are any open-ended questions respondents have elaborated upon—for example, “Explain why you like the current brand you use better than any other brand”—the answers to each are pasted together, one on top of another, so researchers can compare and summarize the information. As we have explained, qualitative information such as this can give you a fuller picture of the results of the research.

Part of analyzing the data is to see if it seems sound. Does the way in which the research was conducted seem sound? Was the sample size large enough? Are the conclusions that become apparent from it reasonable?

The two most commonly used criteria used to test the soundness of a study are (1) validity and (2) reliability. A study is valid if it actually tested what it was designed to test. For example, did the experiment you ran in Second Life test what it was designed to test? Did it reflect what could really happen in the real world? If not, the research isn’t valid. If you were to repeat the study, and get the same results (or nearly the same results), the research is said to be reliable . If you get a drastically different result if you repeat the study, it’s not reliable. The data collected, or at least some it, can also be compared to, or reconciled with, similar data from other sources either gathered by your firm or by another organization to see if the information seems on target.

Stage 7: Write the Research Report and Present Its Findings

If you end up becoming a marketing professional and conducting a research study after you graduate, hopefully you will do a great job putting the study together. You will have defined the problem correctly, chosen the right sample, collected the data accurately, analyzed it, and your findings will be sound. At that point, you will be required to write the research report and perhaps present it to an audience of decision makers. You will do so via a written report and, in some cases, a slide or PowerPoint presentation based on your written report.

The six basic elements of a research report are as follows.

• Title Page . The title page explains what the report is about, when it was conducted and by whom, and who requested it.
• Table of Contents . The table of contents outlines the major parts of the report, as well as any graphs and charts, and the page numbers on which they can be found.
• Executive Summary . The executive summary summarizes all the details in the report in a very quick way. Many people who receive the report—both executives and nonexecutives—won’t have time to read the entire report. Instead, they will rely on the executive summary to quickly get an idea of the study’s results and what to do about those results.

Methodology and Limitations . The methodology section of the report explains the technical details of how the research was designed and conducted. The section explains, for example, how the data was collected and by whom, the size of the sample, how it was chosen, and whom or what it consisted of (e.g., the number of women versus men or children versus adults). It also includes information about the statistical techniques used to analyze the data.

Every study has errors—sampling errors, interviewer errors, and so forth. The methodology section should explain these details, so decision makers can consider their overall impact. The margin of error is the overall tendency of the study to be off kilter—that is, how far it could have gone wrong in either direction. Remember how newscasters present the presidential polls before an election? They always say, “This candidate is ahead 48 to 44 percent, plus or minus 2 percent.” That “plus or minus” is the margin of error. The larger the margin of error is, the less likely the results of the study are accurate. The margin of error needs to be included in the methodology section.

• Findings . The findings section is a longer, fleshed-out version of the executive summary that goes into more detail about the statistics uncovered by the research that bolster the study’s findings. If you have related research or secondary data on hand that back up the findings, it can be included to help show the study did what it was designed to do.
• Recommendations . The recommendations section should outline the course of action you think should be taken based on the findings of the research and the purpose of the project. For example, if you conducted a global market research study to identify new locations for stores, make a recommendation for the locations (Mersdorf, 2009).

As we have said, these are the basic sections of a marketing research report. However, additional sections can be added as needed. For example, you might need to add a section on the competition and each firm’s market share. If you’re trying to decide on different supply chain options, you will need to include a section on that topic.

As you write the research report, keep your audience in mind. Don’t use technical jargon decision makers and other people reading the report won’t understand. If technical terms must be used, explain them. Also, proofread the document to ferret out any grammatical errors and typos, and ask a couple of other people to proofread behind you to catch any mistakes you might have missed. If your research report is riddled with errors, its credibility will be undermined, even if the findings and recommendations you make are extremely accurate.

Many research reports are presented via PowerPoint. If you’re asked to create a slideshow presentation from the report, don’t try to include every detail in the report on the slides. The information will be too long and tedious for people attending the presentation to read through. And if they do go to the trouble of reading all the information, they probably won’t be listening to the speaker who is making the presentation.

Instead of including all the information from the study in the slides, boil each section of the report down to key points and add some “talking points” only the presenter will see. After or during the presentation, you can give the attendees the longer, paper version of the report so they can read the details at a convenient time, if they choose to.

Key Takeaway

Step 1 in the marketing research process is to define the problem. Businesses take a look at what they believe are symptoms and try to drill down to the potential causes so as to precisely define the problem. The next task for the researcher is to put into writing the research objective, or goal, the research is supposed to accomplish. Step 2 in the process is to design the research. The research design is the “plan of attack.” It outlines what data you are going to gather, from whom, how, and when, and how you’re going to analyze it once it has been obtained. Step 3 is to design the data-collection forms, which need to be standardized so the information gathered on each is comparable. Surveys are a popular way to gather data because they can be easily administered to large numbers of people fairly quickly. However, to produce the best results, survey questionnaires need to be carefully designed and pretested before they are used. Step 4 is drawing the sample, or a subset of potential buyers who are representative of your entire target market. If the sample is not correctly selected, the research will be flawed. Step 5 is to actually collect the data, whether it’s collected by a person face-to-face, over the phone, or with the help of computers or the Internet. The data-collection process is often different in foreign countries. Step 6 is to analyze the data collected for any obvious errors, tabulate the data, and then draw conclusions from it based on the results. The last step in the process, Step 7, is writing the research report and presenting the findings to decision makers.

Review Questions

• Explain why it’s important to carefully define the problem or opportunity a marketing research study is designed to investigate.
• Describe the different types of problems that can occur when marketing research professionals develop questions for surveys.
• How does a probability sample differ from a nonprobability sample?
• What makes a marketing research study valid? What makes a marketing research study reliable?
• What sections should be included in a marketing research report? What is each section designed to do?

1 “Questionnaire Design,” QuickMBA , http://www.quickmba.com/marketing/research/qdesign (accessed December 14, 2009).

Barnes, B., “Disney Expert Uses Science to Draw Boy Viewers,” New York Times , April 15, 2009, http://www.nytimes.com/2009/04/14/arts/television/14boys.html?pagewanted=1&_r=1 (accessed December 14, 2009).

Burns A. and Ronald Bush, Marketing Research , 6th ed. (Upper Saddle River, NJ: Prentice Hall, 2010), 85.

Malhotra, N., Marketing Research: An Applied Approach , 6th ed. (Upper Saddle River, NJ: Prentice Hall), 764.

McDaniel, C. D. and Roger H. Gates, Marketing Research Essentials , 2nd ed. (Cincinnati: South-Western College Publishing, 1998), 61.

McWilliams, J., “A-B Puts Super-Low-Calorie Beer in Ring with Miller,” St. Louis Post-Dispatch , August 16, 2009, http://www.stltoday.com/business/next-matchup-light-weights-a-b-puts-super-low-calorie/article_47511bfe-18ca-5979-bdb9-0526c97d4edf.html (accessed April 13, 2012).

Mersdorf, S., “How to Organize Your Next Survey Report,” Cvent , August 24, 2009, http://survey.cvent.com/blog/cvent-survey/0/0/how-to-organize-your-next-survey-report (accessed December 14, 2009).

Rappeport A. and David Gelles, “Facebook to Form Alliance with Nielsen,” Financial Times , September 23, 2009, 16.

Spangler, T., “Disney Lab Tracks Feelings,” Multichannel News 30, no. 30 (August 3, 2009): 26.

Wagner, J., “Marketing in Second Life Doesn’t Work…Here Is Why!” GigaOM , April 4, 2007, http://gigaom.com/2007/04/04/3-reasons-why-marketing-in-second-life-doesnt-work (accessed December 14, 2009).

Wrenn, B., Robert E. Stevens, and David L. Loudon, Marketing Research: Text and Cases , 2nd ed. (Binghamton, NY: Haworth Press, 2007), 180.

Zouhali-Worrall, M., “Found in Translation: Avoiding Multilingual Gaffes,” CNNMoney.com , July 14, 2008, http://money.cnn.com/2008/07/07/smallbusiness/language_translation.fsb/index.htm (accessed December 14, 2009).

Principles of Marketing Copyright © 2015 by University of Minnesota is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

What is Marketing Research? Examples and Best Practices

12 min read

Marketing research is essentially a method utilized by companies to collect valuable information regarding their target market. Through the common practice of conducting market research, companies gather essential information that enables them to make informed decisions and develop products that resonate with consumers. It encompasses the gathering, analysis, and interpretation of data, which aids in identifying consumer demands, anticipating market trends, and staying ahead of the competition.

Exploratory research is one of the initial steps in the marketing research process. It helps businesses gain broad insights when specific information is unknown. If you are seeking insight into how marketing research can influence the trajectory of your SaaS, then you have come to the right place!

• Market research is a systematic and objective process crucial for understanding target markets, refining business strategies, and informing decisions, which includes collecting, analyzing, and interpreting data on customers, competitors, and the industry.
• Primary market research gathers specific data directly from the target audience using tools like surveys and focus groups, while secondary market research utilizes existing data from various sources to provide broader market insights.
• Effective market research combines both qualitative methods, which explore consumer motivations, and quantitative methods, which provide measurable statistics, to create comprehensive insights that guide business strategy and decision-making.

Try Userpilot and Take Your Product Marketing to the Next Level

• 14 Day Trial
• No Credit Card Required

Defining marketing research

Launching a product without knowing what your target audience wants is like walking in the dark. Market research lights the way, helping you collect, analyze, and understand information about your target market. This allows you to refine your business strategies and make decisions based on solid evidence.

Gone are the days when just intuition or subjective judgment was enough. Objective insights from market research help avoid costly mistakes and meet consumer needs by identifying trends and changes in the market. This is crucial for assessing a product’s potential success, optimizing marketing strategies, and preparing for market shifts.

Market research is a systematic approach that provides essential information, helping businesses navigate the complexities of the commercial world. Partnering with market research companies can offer additional benefits, leveraging their expertise in understanding market demands, trends, market size, economic indicators, location, market saturation, and pricing. Whether starting a new business, developing products, or updating marketing plans, understanding how to conduct effective market research is key to success.

To conduct market research effectively, businesses must determine study goals, identify target consumers, collect and analyze data, and use the findings to make informed decisions. This process is vital for evaluating past performance, measuring changes over time, and addressing specific business needs. It guides businesses in product development, marketing strategies, and overall decision-making, ensuring a better ROI and providing an eye-opening view of the market through various research methods, whether conducted in-house or outsourced.

The purpose of marketing research

Conducting marketing research is more than just gathering data; it’s about turning that data into actionable insights to refine your business strategies. This process helps you understand what motivates your customers, enabling you to tailor your products and services to minimize risks from the start. Importantly, market research plays a pivotal role in measuring and enhancing customer satisfaction and loyalty, which are critical for understanding key demographics, improving user experience, designing better products, and driving customer retention. Customer satisfaction is measured as a key outcome, directly linked to the success of marketing strategies and business activities.

For SaaS product managers, market research, including competitive analysis, is crucial. It evaluates past strategies and gauges the potential success of new offerings. This research provides essential insights into brand strength, consumer behavior, and market position, which are vital for teams focused on sales, marketing, and product development.

A key aspect of market research is analyzing customer attitudes and usage. This analysis offers detailed insights into what customers want, the choices they make, and the challenges they face. It helps identify opportunities in the market and aids in formulating effective strategies for market entry.

Overall, market research equips SaaS entrepreneurs with the knowledge to meet their target audience’s needs effectively, guiding product adjustments and innovations based on informed decisions.

Key components of market research

Conducting market research is analogous to preparing a cake, requiring precise ingredients in specific quantities to achieve the intended outcome. Within this realm, necessary components consist of primary and secondary data gathering, thorough analysis, and insightful interpretation.

Primary research techniques such as exploratory studies, product evolution inquiries, estimations of market dimensions and shares, and consumer behavior examinations play a crucial role in collecting targeted information that can be directly applied. These methods afford a deeper understanding of your target demographic, allowing for customized strategy development.

In contrast, secondary research enriches the specificity of primary findings by adding wider context. It taps into external resources encompassing works from other investigators, sector-specific reports, and demographics data, which provide an expansive yet less particularized landscape view of the marketplace.

The subsequent phase involves meticulous analysis of collated data offering unbiased perspectives critical for identifying deficiencies while recognizing emerging patterns. Technological progress now facilitates examination efforts on both structured and unstructured datasets effectively addressing large-scale analytical complexities.

Ultimately, it’s through expert-led interpretation that value transcends raw figures, yielding strategies grounded in deep comprehension. Akin to decoding recipes using selected ingredients—this interpretative step enables crafting optimal business maneuvers just as one would bake their ideal confectionery creation utilizing proper culinary guidance.

Types of market research: primary and secondary

Now that you know the importance of clear research objectives, let’s explore the different types of market research and the techniques available to achieve these goals. Market research methods can be divided into two main categories: primary research and secondary research . The choice between these depends on factors like your budget, time constraints, and whether you need exploratory data or definitive answers.

Primary research involves collecting new data directly from sources. This process is like mining for precious metals, as it requires using various methods to gather fresh insights.

• Surveys (here – in-app survey templates from Userpilot ).

• Interviews.

• Focus groups.
• Product trials.

This approach gives you first-hand insight into your target audience.

Conversely, secondary research uses already established datasets of primary data – which can add depth and reinforcement to your firsthand findings.

Conducting your own market research using primary research tools can be a cost-effective strategy, allowing businesses to gather valuable insights directly and tailor their research to specific needs.

Let’s look a bit deeper into them now.

What is primary market research?

Market research uses primary market research as an essential tool. This involves collecting new data directly from your target audience using various methods, such as surveys , focus groups, and interviews.

Each method has its benefits. For example, observational studies allow you to see how consumers interact with your product.

There are many ways to conduct primary research.

Focus Groups : Hold discussions with small groups of 5 to 10 people from your target audience. These discussions can provide valuable feedback on products, perceptions of your company’s brand name, or opinions on competitors. Additionally, these discussions can help understand the characteristics, challenges, and buying habits of target customers, optimizing brand strategy.

Interviews : Have one-on-one conversations to gather detailed information from individuals in your target audience.

Surveys : These are a common tool in primary market research and can be used instead of focus groups to understand consumer attitudes. Surveys use structured questions and can reach a broad audience efficiently.

Navigating secondary market research

While marketing research using primary methods is like discovering precious metals, secondary market research technique is like using a treasure map. This approach uses data collected by others from various sources, providing a broad industry view. These sources include market analyses from agencies like Statista, historical data such as census records, and academic studies.

Secondary research provides the basic knowledge necessary for conducting primary market research goals but may lack detail on specific business questions and could also be accessible to competitors.

To make the most of secondary market research, it’s important to analyze summarized data to identify trends, rely on reputable sources for accurate data, and remain unbiased in data collection methods.

The effectiveness of secondary research depends significantly on how well the data is interpreted, ensuring that this information complements the insights from primary research.

Qualitative vs quantitative research

Market research employs both qualitative and quantitative methods, offering distinct insights that complement each other. Qualitative research aims to understand consumer behaviors and motivations through detailed analysis, while quantitative research collects measurable data for statistical analysis.

The selection of qualitative or quantitative methods should align with your research goals. If you need to uncover initial insights or explore deep consumer motivations, qualitative techniques like surveys or interviews are ideal.

On the other hand, if you need data that can be measured and analyzed for reliability, quantitative methods are more suitable.

However, these approaches don’t have to be used separately. Combining qualitative and quantitative methods in mixed-method studies allows you to capture both detailed exploratory responses and concrete numerical data. This integration offers a comprehensive view of the market, leveraging the strengths of both approaches to provide a fuller understanding of market conditions.

Implementing market research tools: Userpilot’s role

Similar to how a compass is essential for navigation at sea, businesses need appropriate instruments to carry out effective market research. Userpilot’s suite of product analytics and in-app engagement tools are critical components for this purpose.

Acting as a Buyer Persona Research instrument, Userpilot’s product analytics provide key quantitative research capabilities. This helps clearly define and comprehend the attributes and behaviors of potential customers, providing you with insights into your ICP (Ideal Customer Persona), user preferences, and product-market fit.

Beyond product analytics, Userpilot offers robust in-app engagement features such as modals and surveys that support real time collection of market research information. These interactive features work synergistically with the analytical tools to enable companies to gather detailed data and feedback crucial for informed business decision-making.

Marketing research process: Step-by-step guide

Marketing research conists of several critical stages:

• Defining precise goals.
• Delving into the knowledge of your target demographic.
• Collecting and scrutinizing data.
• Revealing insights that can be translated into tangible actions.

Following these steps allows you to gather critical information that guides business decisions.

An effective research strategy is crucial and involves:

• Properly allocating funds.
• Formulating testable hypotheses.
• Choosing appropriate methods for the study.
• Determining the number of study participants.
• Considering external variables.

A well-planned strategy ensures that your market research is focused, efficient, and produces useful outcomes.

After collecting data, the next step is to analyze it. This involves comparing the data to your initial questions to draw conclusions relevant to your business strategies.

Userpilot makes your data analysis easier by providing handy analytics dashboards for key user metrics such as activation, engagement, core feature adoption, and retention out of the box:

Finally, you report the findings and the process, providing recommendations based on the evidence. This is like solving a puzzle: each piece helps to complete the overall picture.

Challenges and best practices in market research

Delving into market research comes with its own set of hurdles. Those conducting the research must deliver more profound insights within increasingly shorter timespans, and they need to cultivate strategic, continuous research methods to stay abreast of an ever-changing business landscape.

Ensuring high-quality data can be demanding due to issues such as disjointed tools or insufficient analytical expertise. New solutions like Userpilot are surfacing that make these obstacles less daunting by offering accessible and user-friendly options. Maintaining clear lines of communication with your market research team is crucial for achieving both punctuality and quality in outcomes.

The advantages of engaging in marketing research cannot be overstated.

Real-life examples of successful market research

Real-life examples of market research in the SaaS industry often showcase innovative approaches to understanding customer needs and product-market fit.

For instance, Slack, the communication platform, utilized extensive market research to identify gaps in communication tools and understand the workflows of teams. This led to the development of features that seamlessly integrated with other tools and catered to the needs of various team sizes and structures.

Another example is HubSpot, which conducted market research to understand the pain points of small to medium-sized businesses in managing customer relationships. The insights gained helped shape their all-in-one inbound marketing, sales, and service platform, which has become integral to their users’ daily operations. These examples demonstrate how SaaS companies can employ market research to inform product development, improve user experience, and strategically position themselves in a competitive market.

Choosing the right market research tools

For B2B SaaS product managers aiming to do market research, having the right set of tools can make a significant difference. Here’s a list of valuable SaaS tools that can be leveraged for effective market research:

• Userpilot : A comprehensive Product Growth Platform offering in-depth product analytics, a code-free in-app experience builder, bespoke in-app survey capabilities, and robust integration options with platforms like Salesforce and Hubspot. This tool is particularly useful for understanding user behavior, enhancing user engagement, and gathering targeted feedback.
• Qualtrics : Known for its powerful survey tools, Qualtrics helps businesses gather and analyze customer feedback effectively. Its advanced analytics features are ideal for testing market hypotheses and understanding customer sentiments.
• SurveyMonkey : A versatile tool that enables product managers to create, send, and analyze surveys quickly and easily. SurveyMonkey is suitable for gauging customer satisfaction and collecting feedback on potential new features.
• Mixpanel : Specializes in user behavior analytics, offering detailed insights into how users interact with your product. This is essential for identifying patterns and optimizing product features.
• Hotjar : Combines analytics and feedback tools to give teams insights into user behavior and preferences. Hotjar’s heatmaps and session recordings are invaluable for understanding the user experience on a deeper level.
• Tableau : A leading platform for business intelligence and data visualization, Tableau allows product managers to create comprehensive visual reports that can inform strategic decisions based on user data analysis.

Each of these tools provides unique functionalities that can assist SaaS product managers in conducting thorough market research, thereby ensuring that their products are perfectly aligned with user needs and market demands.

Measuring the impact of market research

The pivotal challenge for market research lies in demonstrating its return on investment (ROI) and overall influence on corporate success sufficiently enough to justify regular financial commitment from company leaders. The worth attributed to a market research firm hinges not only on their ability to deliver relevant and high-caliber information, but also on their pricing structures and their contribution towards propelling organizational growth.

To gauge how effectively business choices made based on market research findings succeed, various metrics and key performance indicators (KPIs) are utilized. These numerical tools act as navigational aids directing enterprises toward achieving objectives while simultaneously verifying that efforts invested in conducting market analysis are yielding fruitful guidance.

Throughout our look at market research, we’ve seen its importance and impact. Our discussion covered the basics of market research, its key components, and different types, including both qualitative and quantitative methods, and the role of Userpilot’s tools. We’ve examined the details of the market research process, tackled challenges, identified best practices, and shared success stories. We also provided advice on choosing the right market research partner and how to measure the effectiveness of your market research.

In today’s data-driven world, comprehensive market research is crucial for companies that want to succeed. It acts like a guide, helping businesses navigate the complex market landscape. Start your own detailed research today, supported by insightful analytics to help you succeed.

Frequently asked questions

What is market research and why is it important.

Understanding your target market, honing business strategies, and making informed decisions are all essential components that depend heavily on effective market research. It offers objective insights to help avoid expensive errors and foresees the needs of customers .

What is the difference between primary and secondary market research?

Primary market research is characterized by the direct gathering of data, in contrast to secondary market research which leverages existing information from alternative sources for addressing research inquiries.

Such a distinction can guide you in selecting an approach that aligns with your precise needs for conducting specific research.

What are some examples of successful market research?

Examples of successful market research are evident in the operations of well-known companies such as Starbucks, Apple, and McDonald’s. They have harnessed this tool to fine-tune their business strategies and make decisions based on solid information.

By employing market research, these businesses have managed to gain insight into their customers’ desires and needs, which has contributed significantly to their success.

How can I choose the right market research partner?

Selecting an ideal market research ally involves identifying a firm that resonates with your project requirements, financial plan, and corporate goals while also verifying their track record of dependability and consistency via reviews from previous clients.

Best wishes on your endeavor!

How is the impact of market research measured?

The effectiveness of market research hinges on the precision, representativeness, and pertinence of its data, along with how successful business decisions are when they’re based on the findings from this research. These elements define the impact of the research conducted.

Leave a comment Cancel reply

Save my name, email, and website in this browser for the next time I comment.

Get The Insights!

The fastest way to learn about Product Growth,Management & Trends.

The coolest way to learn about Product Growth, Management & Trends. Delivered fresh to your inbox, weekly.

The fastest way to learn about Product Growth, Management & Trends.

You might also be interested in ...

Product-led vs sales-led: what’s the difference, and which one is right for you.

Userpilot Content Team

What Are Marketing Funnels? A Comprehensive Guide to Streamlining Your User Journey

Best marketing analytics software for saas.

Module 6: Marketing Information and Research

The marketing research process, learning objectives.

• Identify the steps of conducting a marketing research project

A Standard Approach to Research Inquiries

Marketing research is a useful and necessary tool for helping marketers and an organization’s executive leadership make wise decisions. Carrying out marketing research can involve highly specialized skills that go deeper than the information outlined in this module. However, it is important for any marketer to be familiar with the basic procedures and techniques of marketing research.

It is very likely that at some point a marketing professional will need to supervise an internal marketing research activity or to work with an outside marketing research firm to conduct a research project. Managers who understand the research function can do a better job of framing the problem and critically appraising the proposals made by research specialists. They are also in a better position to evaluate their findings and recommendations.

Periodically marketers themselves need to find solutions to marketing problems without the assistance of marketing research specialists inside or outside the company. If you are familiar with the basic procedures of marketing research, you can supervise and even conduct a reasonably satisfactory search for the information needed.

Step 1: Identify the Problem

The first step for any marketing research activity is to clearly identify and define the problem you are trying to solve. You start by stating the marketing or business problem you need to address and for which you need additional information to figure out a solution. Next, articulate the objectives for the research: What do you want to understand by the time the research project is completed? What specific information, guidance, or recommendations need to come out of the research in order to make it a worthwhile investment of the organization’s time and money?

It’s important to share the problem definition and research objectives with other team members to get their input and further refine your understanding of the problem and what is needed to solve it. At times, the problem you really need to solve is not the same problem that appears on the surface. Collaborating with other stakeholders helps refine your understanding of the problem, focus your thinking, and prioritize what you hope to learn from the research. Prioritizing your objectives is particularly helpful if you don’t have the time or resources to investigate everything you want.

To flesh out your understanding of the problem, it’s useful to begin brainstorming actual research questions you want to explore. What are the questions you need to answer in order to get to the research outcomes? What is the missing information that marketing research will help you find? The goal at this stage is to generate a set of preliminary, big-picture questions that will frame your research inquiry. You will revisit these research questions later in the process, but when you’re getting started, this exercise helps clarify the scope of the project, whom you need to talk to, what information may already be available, and where to look for the information you don’t yet have.

Applied Example: Marketing Research for Bookends

To illustrate the marketing research process, let’s return to Uncle Dan and his ailing bookstore, Bookends. You need a lot of information if you’re going to help Dan turn things around, so marketing research is a good idea. You begin by identifying the problem and then work to set down your research objectives and initial research questions:

Step 2: Develop a Research Plan

Once you have a problem definition, research objectives, and a preliminary set of research questions, the next step is to develop a research plan. Essential to this plan is identifying precisely what information you need to answer your questions and achieve your objectives. Do you need to understand customer opinions about something? Are you looking for a clearer picture of customer needs and related behaviors? Do you need sales, spending, or revenue data? Do you need information about competitors’ products, or insight about what will make prospective customers notice you? When do need the information, and what’s the time frame for getting it? What budget and resources are available?

Once you have clarified what kind of information you need and the timing and budget for your project, you can develop the research design. This details how you plan to collect and analyze the information you’re after. Some types of information are readily available through  secondary research and secondary data sources. Secondary research analyzes information that has already been collected for another purpose by a third party, such as a government agency, an industry association, or another company. Other types of information need to from talking directly to customers about your research questions. This is known as primary research , which collects primary data captured expressly for your research inquiry.   Marketing research projects may include secondary research, primary research, or both.

Depending on your objectives and budget, sometimes a small-scale project will be enough to get the insight and direction you need. At other times, in order to reach the level of certainty or detail required, you may need larger-scale research involving participation from hundreds or even thousands of individual consumers. The research plan lays out the information your project will capture—both primary and secondary data—and describes what you will do with it to get the answers you need. (Note: You’ll learn more about data collection methods and when to use them later in this module.)

Your data collection plan goes hand in hand with your analysis plan. Different types of analysis yield different types of results. The analysis plan should match the type of data you are collecting, as well as the outcomes your project is seeking and the resources at your disposal. Simpler research designs tend to require simpler analysis techniques. More complex research designs can yield powerful results, such as understanding causality and trade-offs in customer perceptions. However, these more sophisticated designs can require more time and money to execute effectively, both in terms of data collection and analytical expertise.

The research plan also specifies who will conduct the research activities, including data collection, analysis, interpretation, and reporting on results. At times a singlehanded marketing manager or research specialist runs the entire research project. At other times, a company may contract with a marketing research analyst or consulting firm to conduct the research. In this situation, the marketing manager provides supervisory oversight to ensure the research delivers on expectations.

Finally, the research plan indicates who will interpret the research findings and how the findings will be reported. This part of the research plan should consider the internal audience(s) for the research and what reporting format will be most helpful. Often, senior executives are primary stakeholders, and they’re anxious for marketing research to inform and validate their choices. When this is the case, getting their buy-in on the research plan is recommended to make sure that they are comfortable with the approach and receptive to the potential findings.

Applied Example: A Bookends Research Plan

You talk over the results of your problem identification work with Dan. He thinks you’re on the right track and wants to know what’s next. You explain that the next step is to put together a detailed plan for getting answers to the research questions.

Dan is enthusiastic, but he’s also short on money. You realize that such a financial constraint will limit what’s possible, but with Dan’s help you can do something worthwhile. Below is the research plan you sketch out:

Step 3: Conduct the Research

Conducting research can be a fun and exciting part of the marketing research process. After struggling with the gaps in your knowledge of market dynamics—which led you to embark on a marketing research project in the first place—now things are about to change. Conducting research begins to generate information that helps answer your urgent marketing questions.

Typically data collection begins by reviewing any existing research and data that provide some information or insight about the problem. As a rule, this is secondary research. Prior research projects, internal data analyses, industry reports, customer-satisfaction survey results, and other information sources may be worthwhile to review. Even though these resources may not answer your research questions fully, they may further illuminate the problem you are trying to solve. Secondary research and data sources are nearly always cheaper than capturing new information on your own. Your marketing research project should benefit from prior work wherever possible.

After getting everything you can from secondary research, it’s time to shift attention to primary research, if this is part of your research plan. Primary research involves asking questions and then listening to and/or observing the behavior of the target audience you are studying. In order to generate reliable, accurate results, it is important to use proper scientific methods for primary research data collection and analysis. This includes identifying the right individuals and number of people to talk to, using carefully worded surveys or interview scripts, and capturing data accurately.

Without proper techniques, you may inadvertently get bad data or discover bias in the responses that distorts the results and points you in the wrong direction. The module on Marketing Research Techniques discusses these issues in further detail, since the procedures for getting reliable data vary by research method.

Applied Example: Getting the Data on Bookends

Dan is on board with the research plan, and he’s excited to dig into the project. You start with secondary data, getting a dump of Dan’s sales data from the past two years, along with related information: customer name, zip code, frequency of purchase, gender, date of purchase, and discounts/promotions (if any).

You visit the U.S. Census Bureau Web site to download demographic data about your metro area. The data show all zip codes in the area, along with population size, gender breakdown, age ranges, income, and education levels.

The next part of the project is customer-survey data. You work with Dan to put together a short survey about customer attitudes toward Bookends, how often and why they come, where else they spend money on books and entertainment, and why they go other places besides Bookends. Dan comes up with the great idea of offering a 5 percent discount coupon to anyone who completes the survey. Although it eats into his profits, this scheme gets more people to complete the survey and buy books, so it’s worth it.

For a couple of days, you and Dan take turns doing “man on the street” interviews (you interview the guy in the red hat, for instance). You find people who say they’ve never been to Bookends and ask them a few questions about why they haven’t visited the store, where else they buy books and other entertainment, and what might get them interested in visiting Bookends sometime. This is all a lot of work, but for a zero-budget project, it’s coming together pretty well.

Step 4: Analyze and Report Findings

Analyzing the data obtained in a market survey involves transforming the primary and/or secondary data into useful information and insights that answer the research questions. This information is condensed into a format to be used by managers—usually a presentation or detailed report.

Analysis starts with formatting, cleaning, and editing the data to make sure that it’s suitable for whatever analytical techniques are being used. Next, data are tabulated to show what’s happening: What do customers actually think? What’s happening with purchasing or other behaviors? How do revenue figures actually add up? Whatever the research questions, the analysis takes source data and applies analytical techniques to provide a clearer picture of what’s going on. This process may involve simple or sophisticated techniques, depending on the research outcomes required. Common analytical techniques include regression analysis to determine correlations between factors; conjoint analysis to determine trade-offs and priorities; predictive modeling to anticipate patterns and causality; and analysis of unstructured data such as Internet search terms or social media posts to provide context and meaning around what people say and do.

Good analysis is important because the interpretation of research data—the “so what?” factor—depends on it. The analysis combs through data to paint a picture of what’s going on. The interpretation goes further to explain what the research data mean and make recommendations about what managers need to know and do based on the research results. For example, what is the short list of key findings and takeaways that managers should remember from the research? What are the market segments you’ve identified, and which ones should you target?  What are the primary reasons your customers choose your competitor’s product over yours, and what does this mean for future improvements to your product?

Individuals with a good working knowledge of the business should be involved in interpreting the data because they are in the best position to identify significant insights and make recommendations from the research findings. Marketing research reports incorporate both analysis and interpretation of data to address the project objectives.

The final report for a marketing research project may be in written form or slide-presentation format, depending on organizational culture and management preferences. Often a slide presentation is the preferred format for initially sharing research results with internal stakeholders. Particularly for large, complex projects, a written report may be a better format for discussing detailed findings and nuances in the data, which managers can study and reference in the future.

Applied Example: Analysis and Insights for Bookends

Getting the data was a bit of a hassle, but now you’ve got it, and you’re excited to see what it reveals. Your statistician cousin, Marina, turns out to be a whiz with both the sales data and the census data. She identified several demographic profiles in the metro area that looked a lot like lifestyle segments. Then she mapped Bookends’ sales data into those segments to show who is and isn’t visiting Bookends. After matching customer-survey data to the sales data, she broke down the segments further based on their spending levels and reasons they visit Bookends.

Gradually a clearer picture of Bookends’ customers is beginning to emerge: who they are, why they come, why they don’t come, and what role Bookends plays in their lives. Right away, a couple of higher-priority segments—based on their spending levels, proximity, and loyalty to Bookends—stand out. You and your uncle are definitely seeing some possibilities for making the bookstore a more prominent part of their lives. You capture these insights as “recommendations to be considered” while you evaluate the right marketing mix for each of the new segments you’d like to focus on.

Step 5: Take Action

Once the report is complete, the presentation is delivered, and the recommendations are made, the marketing research project is over, right? Wrong.

What comes next is arguably the most important step of all: taking action based on your research results.

If your project has done a good job interpreting the findings and translating them into recommendations for the marketing team and other areas of the business, this step may seem relatively straightforward. When the research results validate a path the organization is already on, the “take action” step can galvanize the team to move further and faster in that same direction.

Things are not so simple when the research results indicate a new direction or a significant shift is advisable. In these cases, it’s worthwhile to spend time helping managers understand the research, explain why it is wise to shift course, and explain how the business will benefit from the new path. As with any important business decision, managers must think deeply about the new approach and carefully map strategies, tactics, and available resources to plan effectively. By making the results available and accessible to managers and their execution teams, the marketing research project can serve as an ongoing guide and touchstone to help the organization plan, execute, and adjust course as it works toward desired goals and outcomes.

It is worth mentioning that many marketing research projects are never translated into management action. Sometimes this is because the report is too technical and difficult to understand. In other cases, the research conclusions fail to provide useful insights or solutions to the problem, or the report writer fails to offer specific suggestions for translating the research findings into management strategy. These pitfalls can be avoided by paying due attention to the research objectives throughout the project and allocating sufficient time and resources to do a good job interpreting research results for those who will need to act on them.

Applied Example: Bookends’ New Customer Campaign

Your research findings and recommendations identified three segments for Bookends to focus on. Based on the demographics, lifestyle, and spending patterns found during your marketing research, you’re able to name them: 1) Bored Empty-Nesters, 2) Busy Families, and 3) Hipster Wannabes. Dan has a decent-sized clientele across all three groups, and they are pretty good spenders when they come in. But until now he hasn’t done much to purposely attract any of them.

With newly identified segments in focus, you and Dan begin brainstorming about a marketing mix to target each group. What types of books and other products would appeal to each one? What activities or events would bring them into the store? Are there promotions or particular messages that would induce them to buy at Bookends instead of Amazon or another bookseller? How will Dan reach and communicate with each group? And what can you do to bring more new customers into the store within these target groups?

Even though Bookends is a real-life project with serious consequences for your uncle Dan, it’s also a fun laboratory where you can test out some of the principles you’re learning in your marketing class. You’re figuring out quickly what it’s like to be a marketer.

Well done, rookie!

Check Your Understanding

Answer the question(s) below to see how well you understand the topics covered in this outcome. This short quiz does  not  count toward your grade in the class, and you can retake it an unlimited number of times.

Use this quiz to check your understanding and decide whether to (1) study the previous section further or (2) move on to the next section.

• Revision and Adaptation. Authored by : Lumen Learning. License : CC BY: Attribution
• Chapter 3: Marketing Research: An Aid to Decision Making, from Introducing Marketing. Authored by : John Burnett. Provided by : Global Text. Located at : http://solr.bccampus.ca:8001/bcc/file/ddbe3343-9796-4801-a0cb-7af7b02e3191/1/Core%20Concepts%20of%20Marketing.pdf . License : CC BY: Attribution
• Urban life (Version 2.0). Authored by : Ian D. Keating. Located at : https://www.flickr.com/photos/ian-arlett/19313315520/ . License : CC BY: Attribution

Internal Validity in Research: Definition, Threats, Examples

Appinio Research · 19.02.2024 · 37min read

Ever wondered how researchers ensure that their findings accurately reflect cause-and-effect relationships? Understanding internal validity is key. Internal validity answers the question: "Are we measuring what we think we're measuring?" In this guide, we'll explore the fundamentals of internal validity, its importance across various industries, and strategies for enhancing it in research studies. Whether you're a researcher, a professional, or simply curious about the reliability of research findings, this guide will provide you with valuable insights into everything internal validity.

What is Internal Validity?

Internal validity refers to the degree to which the results of a research study accurately reflect the causal relationship between the independent variable(s) and the dependent variable without the influence of confounding variables or biases . In essence, it assesses the extent to which the observed effects can be attributed to the manipulation of the independent variable(s) rather than to other factors.

Importance of Internal Validity

Ensuring internal validity is crucial for the credibility and reliability of research findings across various disciplines and industries.

Here are several reasons why internal validity is important:

• Accurate Causal Inferences:  Internal validity allows researchers to draw accurate conclusions about the causal relationship between variables. By controlling for extraneous variables and biases, researchers can confidently attribute observed effects to the manipulated independent variable(s).
• Validity of Research Findings:  High internal validity enhances the validity of research findings, increasing their trustworthiness and applicability. Valid research findings serve as a foundation for theory development, evidence-based practice, and informed decision-making in academia, healthcare , policy, business, and other fields.
• Effective Decision-Making:  Reliable research findings with high internal validity provide stakeholders with actionable insights and evidence to guide decision-making processes. Whether it's designing effective interventions, formulating policies, optimizing marketing strategies, or developing innovative products , internal validity ensures that decisions are based on accurate information.
• Ethical Considerations:  Maintaining internal validity is essential for upholding ethical standards in research. By minimizing the influence of confounding variables and biases, researchers ensure the integrity and transparency of their research, safeguarding the rights and well-being of participants and the integrity of the scientific process.
• Resource Allocation:  Conducting research with high internal validity optimizes the allocation of resources by focusing efforts on interventions, strategies, or treatments that have been demonstrated to be effective. Stakeholders can allocate resources more efficiently and maximize impact by avoiding investments in ineffective or misleading approaches.
• Building Cumulative Knowledge:  Research with high internal validity contributes to the accumulation of knowledge within a particular field or discipline. Valid findings serve as building blocks for future research, facilitating the advancement of theories, developing best practices, and refining methodologies over time.
• Enhanced Reproducibility:  Internal validity is closely linked to the reproducibility of research findings. Studies with high internal validity are more likely to be replicable as they accurately capture the effects of the manipulated variables under controlled conditions. Reproducible research fosters confidence in scientific discoveries and promotes scientific progress.

Internal validity is essential for generating credible and reliable research findings that advance knowledge, inform decision-making, and address real-world challenges. By prioritizing internal validity in research design, implementation, and analysis, researchers can produce high-quality evidence that withstands scrutiny and contributes to meaningful outcomes across diverse domains.

Internal vs External Validity

Understanding the distinction between internal and external validity is crucial for effectively designing and interpreting research studies.

• Internal Validity: Internal validity refers to the degree to which the results of a study can be attributed to the manipulation of the independent variable rather than confounding variables. High internal validity indicates that the observed effects are likely due to the experimental manipulation and not other factors. Internal validity is influenced by factors such as research design , methodology, and control over extraneous variables.
• External Validity: External validity refers to the generalizability of research findings beyond the specific conditions of the study. It assesses whether the results can be applied to other populations, settings, or contexts. High external validity indicates that the findings are likely to hold true in other situations, increasing the generalizability and practical relevance of the research.

Key Differences

• Internal validity focuses on the accuracy and reliability of the causal inferences drawn from the study, while external validity focuses on the applicability and generalizability of the findings.
• Internal validity is primarily concerned with controlling for threats to the study's validity within the research setting, whereas external validity considers the extent to which the findings can be extrapolated to real-world situations.
• Enhancing internal validity involves controlling for potential confounding variables and sources of bias within the study, while enhancing external validity involves ensuring the representativeness and diversity of the study sample and conditions.

Considerations

• Researchers should strive to achieve a balance between internal and external validity, recognizing that increasing one may sometimes compromise the other.
• While internal validity is essential for establishing causal relationships within the study, external validity is necessary for ensuring the practical relevance and utility of the findings in real-world settings.
• Researchers should carefully consider the trade-offs between internal and external validity when designing their studies and interpreting the implications of their findings.

Key Concepts and Terminology

In research, understanding key concepts and terminology is essential for navigating the complexities of internal validity. Let's explore some fundamental concepts that will help you grasp the nuances of internal validity.

Causality lies at the heart of scientific inquiry, as researchers seek to understand the relationships between variables and determine whether changes in one variable cause changes in another. Establishing causality requires more than just observing a relationship; it necessitates demonstrating that changes in the independent variable lead to changes in the dependent variable while ruling out alternative explanations.

To establish causality, researchers often employ experimental designs to manipulate the independent variable and observe its effects on the dependent variable. Random assignment helps minimize the influence of confounding variables, enhancing the validity of causal inferences.

Confounding Variables

Confounding variables are extraneous factors that systematically vary with the independent variable and may influence the dependent variable. Failing to account for confounding variables can lead to erroneous conclusions about the relationship between the variables of interest.

Suppose a researcher is investigating the effects of a new teaching method on student performance. If the students in the experimental group have higher motivation levels than those in the control group, motivation could act as a confounding variable, influencing the observed differences in performance.

Control Groups

Control groups serve as a baseline for comparison in experimental research . They receive either no treatment or a standard treatment, allowing researchers to isolate the effects of the independent variable. By comparing the outcomes of the experimental group to those of the control group, researchers can assess the impact of the treatment more accurately.

Control groups are particularly crucial for establishing causality and ruling out alternative explanations for observed effects. Without a control group, it becomes challenging to determine whether changes in the dependent variable are truly attributable to the manipulation of the independent variable.

Randomization

Randomization involves assigning participants to different experimental conditions or groups randomly. By randomly allocating participants, researchers ensure that individual differences are distributed evenly across groups, reducing the l ikelihood of bias and increasing the internal validity of the study.

Randomization helps minimize the influence of confounding variables, as any differences between groups are more likely to be due to chance rather than systematic factors. Random assignment is a hallmark of experimental research designs and is essential for making causal inferences.

Bias refers to systematic errors or distortions in research findings that arise from flaws in the study design, data collection, or analysis process. Common types of bias include selection bias, measurement bias, and experimenter bias.

Selection bias occurs when the sample selected for the study does not represent the population of interest, leading to skewed results. Measurement bias arises when the measurement instrument does not accurately assess the construct of interest, resulting in invalid or unreliable data. Experimenter bias occurs when the researcher's expectations or beliefs influence participant responses or the interpretation of results, leading to biased conclusions.

Reliability vs. Validity

Reliability and validity are essential concepts in research methodology, often used to assess the quality of measurement instruments and study designs.

• Reliability  refers to the consistency and stability of measurements over time and across different conditions. A reliable measurement instrument yields consistent results when administered repeatedly, indicating that it is free from random error.
• Validity , on the other hand, refers to the accuracy and appropriateness of a measurement instrument in assessing the construct of interest. A valid measurement instrument accurately captures the intended construct, providing meaningful and interpretable data.

While reliability is necessary for validity, a measurement instrument can be reliable without being valid. However, a valid measurement instrument must also be reliable to produce meaningful results. Therefore, researchers strive to ensure both reliability and validity in their studies to obtain accurate and trustworthy findings.

Threats to Internal Validity

Ensuring the internal validity of your research findings involves identifying and mitigating various threats that could compromise the integrity of your study. Let's explore some common threats to internal validity and how they can impact the validity of your research outcomes.

History Threats

History threats occur when external events or circumstances influence the outcomes of your study. These events could range from societal changes to environmental factors that occur during the course of your research. History threats are particularly relevant in longitudinal studies or studies with extended durations, where external factors may affect participants differently over time.

Suppose you're conducting a study on consumer behavior , and midway through your study, there's a significant economic recession. The economic downturn could influence participants' purchasing decisions, thereby confounding your results and threatening the internal validity of your study.

Maturation Threats

Maturation threats arise when participants naturally change or mature over the course of the study in ways that affect the outcome variable. This is especially pertinent in developmental research or studies involving populations undergoing significant life changes.

For instance, if you're studying the effectiveness of an intervention program for elderly adults over several months, participants may naturally experience physical or cognitive changes due to aging. These maturation effects could influence the outcomes of your study, making it challenging to attribute changes solely to the intervention.

Testing Threats

Testing threats occur when the act of measuring or assessing participants influences their subsequent responses. This phenomenon can lead to artificial inflation or deflation of scores on subsequent measures, thereby compromising the internal validity of your study.

For example, suppose participants become more familiar with the measurement instrument after repeated administrations. In that case, they may change their responses based on their prior experiences, rather than the actual intervention or treatment being studied.

Instrumentation Threats

Instrumentation threats arise when changes occur in the measurement instruments or procedures during the study. These changes can lead to inconsistencies in data collection, making it difficult to accurately assess the impact of the independent variable on the dependent variable.

For instance, if you're using different observers to assess participant behavior in a longitudinal study, differences in observer ratings or interpretations could introduce bias and threaten the internal validity of your findings.

Statistical Regression

Statistical regression, also known as regression toward the mean, occurs when extreme scores on a measure tend to move closer to the average upon retesting. This phenomenon can lead to misinterpretation of treatment effects, particularly if participants with extreme scores are selectively included in the study.

For example, if you're studying the effects of a tutoring program on student performance and only include students with exceptionally low grades at the outset, their subsequent improvement may be partly attributed to statistical regression rather than the effectiveness of the tutoring program.

Selection Bias

Selection bias occurs when there are systematic differences between the characteristics of participants in different groups, leading to non-equivalent groups. This can occur due to self-selection, non-random assignment, or attrition/mortality of participants during the study.

For example, if participants who volunteer for a study on weight loss are more motivated or health-conscious than those who decline to participate, the results may not be generalizable to the broader population, compromising the internal validity of the study.

Attrition/Mortality

Attrition or mortality refers to the loss of participants from your study over time. If the attrition rate is non-random and related to the variables being studied, it can introduce bias and threaten the internal validity of your findings.

For instance, if participants drop out of a longitudinal study on the effects of a fitness program due to injury or lack of motivation, the remaining sample may no longer be representative of the initial population, leading to biased conclusions about the program's effectiveness.

Experimenter Bias

Experimenter bias occurs when the researcher's expectations, beliefs, or behavior inadvertently influence the outcomes of the study. This can manifest in subtle cues or differential treatment of participants across experimental conditions, leading to biased results.

For example, if researchers administering a psychological intervention unconsciously provide more encouragement or support to participants in the treatment group compared to the control group, it could inflate the observed effects of the intervention, compromising internal validity.

Novelty Effects

Novelty effects occur when participants' responses are influenced by the novelty or unfamiliarity of the experimental procedure rather than the actual treatment or intervention being studied. This can lead to temporary changes in behavior that are not representative of participants' typical responses in real-world settings.

For example, suppose participants in a memory experiment perform better on a recall task simply because it is the first time they've encountered such a task. In that case, their performance may not accurately reflect their true memory abilities, threatening the internal validity of the study.

Maintaining internal validity is paramount to yield credible and reliable outcomes. However, navigating the intricacies of research can be daunting. That's where innovative platforms like Appinio step in, revolutionizing the way companies gather real-time consumer insights.

With Appinio , you're not just conducting research; you're embarking on a journey of discovery, empowered by fast, intuitive market research solutions. By seamlessly integrating real-time consumer feedback into your decision-making process, Appinio ensures that your strategies are grounded in accurate data, enhancing the internal validity of your research outcomes. Experience the power of data-driven decision-making with Appinio, and unlock a world of possibilities for your business. Ready to take the leap?

Book a Demo

How to Increase Internal Validity?

Enhancing internal validity requires careful planning and implementation of methodological strategies to minimize the influence of extraneous variables and ensure the accuracy of your research findings. Let's explore a variety of strategies that researchers employ to enhance internal validity in their studies.

• Purposeful Allocation:  Randomly assign participants to experimental conditions or groups to minimize bias.
• Random Sampling:  Use random sampling techniques to select participants from the population, increasing the generalizability of the findings.
• Randomization Checks:  Verify randomization procedures to ensure they were executed correctly and transparently.
• No-Treatment Control:  Compare the experimental group receiving the treatment to a group that receives no treatment.
• Placebo Control:  Implement a control group that receives a placebo treatment to control for the placebo effect.
• Active Control:  Include a control group that receives an alternative treatment to compare the effectiveness of different interventions.

Counterbalancing

Counterbalancing involves systematically varying the order of experimental conditions or treatments across participants to control for order effects, such as practice or fatigue effects. By counterbalancing the order of conditions, researchers can ensure that any observed differences are not due to the sequence in which conditions are presented.

• Complete Counterbalancing:  Present all possible orders of conditions to different participants to control for order effects.
• Latin Square Design:  Systematically vary the order of conditions across participants to control for order effects while ensuring balance.
• Randomization of Order:  Randomly assign the order of conditions to participants to prevent order effects from influencing the results.

Standardization

Standardization ensures consistency in procedures, measurement instruments, and data collection protocols across participants and conditions. By standardizing methods, researchers minimize variability and increase the reliability and internal validity of their study.

• Protocol Development:  Develop standardized protocols for data collection, intervention implementation, and participant instructions.
• Training Procedures:  Train research staff to follow standardized procedures consistently to minimize variability in data collection.
• Measurement Instrument Validation:  Validate measurement instruments to ensure they accurately and reliably measure the constructs of interest.

Pilot Testing

Pilot testing involves conducting a preliminary version of the study with a small sample of participants to identify and address potential issues before conducting the main study. Pilot testing helps researchers refine their study procedures, identify unanticipated challenges, and ensure the feasibility and validity of the study design.

• Small-Scale Trial:  Conduct a trial run of the study with a small sample size to identify logistical challenges and refine procedures.
• Feedback Collection:   Gather feedback from participants and research staff to identify areas for improvement and refinement.
• Protocol Adjustment:  Modify study protocols, procedures, or measurement instruments based on feedback and observations from the pilot test.

Blind/Double-Blind Procedures

Blind and double-blind procedures involve withholding information about the experimental condition from participants and researchers to prevent bias and ensure the integrity of the study. Blinding reduces the risk of experimenter bias and participant expectancy effects, thereby enhancing internal validity.

• Single-Blind Procedure:  Participants are unaware of their assigned condition, while researchers are aware.
• Double-Blind Procedure:  Both participants and researchers are unaware of the assigned condition until after data collection.
• Blinding Verification:  Verify the success of blinding procedures through debriefing or manipulation checks.

Matching involves pairing participants in different groups based on specific characteristics to ensure equivalence between groups. Matching helps control for potential confounding variables and increases the comparability of groups, thereby enhancing internal validity.

• Criteria Selection:  Identify matching criteria based on variables likely influencing the outcome variable.
• Pairing Procedure:  Pair participants in different groups based on matching criteria to create comparable groups.
• Validity Check:  Verify the effectiveness of matching procedures by comparing demographic and other relevant characteristics between groups.

Statistical Controls

Statistical controls involve using statistical techniques to account for potential confounding variables or sources of variation in the data analysis process. By controlling for covariates statistically, researchers can isolate the effects of the independent variable and enhance the internal validity of their study.

• Analysis of Covariance (ANCOVA):  Control for pre-existing differences between groups on a continuous outcome variable to reduce the influence of confounding variables.
• Propensity Score Matching:  Estimate the assignment probability to a particular condition based on observed covariates and match participants across conditions with similar propensity scores.
• Multivariate Analysis:  Use multivariate statistical techniques to control for multiple variables simultaneously and assess their combined effects on the outcome variable.

Research Design Considerations

Selecting an appropriate research design is critical for ensuring the internal validity of your study. Let's explore various design considerations, including experimental and non-experimental designs, and their implications for research.

Experimental vs. Non-experimental Designs

Experimental designs involve manipulating the independent variable to observe its effects on the dependent variable. These designs offer greater control over extraneous variables and are ideal for establishing causality. Non-experimental designs, on the other hand, do not involve the manipulation of variables and are better suited for exploratory or descriptive research.

• Experimental Designs:  Include randomized controlled trials (RCTs), quasi-experimental designs, and factorial designs.
• Non-experimental Designs:  Include correlational studies, case-control studies, and observational studies.
• Considerations:  Select the design that best aligns with your research question, objectives, and available resources.

Single-Group Designs

Single-group designs involve measuring the dependent variable in a single group of participants without a control group for comparison. While simple in design, single-group designs are susceptible to various threats to internal validity, such as history and maturation effects.

• Design Features:  Participants are measured on the dependent variable before and after an intervention or treatment.
• Limitations:  The lack of a control group makes it difficult to rule out alternative explanations for observed effects.
• Applications:  Commonly used in pilot studies, feasibility studies, and interventions with limited resources.

Pretest-Posttest Designs

Pretest-posttest designs involve measuring the dependent variable both before and after the administration of the treatment. While useful for assessing change over time, pretest-posttest designs may be susceptible to testing effects and instrumentation threats.

• Design Features:  Participants are measured on the dependent variable before and after receiving the treatment.
• Advantages:  Allow researchers to assess changes in the dependent variable over time and evaluate the effectiveness of interventions.
• Considerations:  Control for testing effects and instrumentation threats by using control groups or counterbalancing techniques.

Solomon Four-Group Designs

The Solomon four-group design combines elements of pretest-posttest and posttest-only designs to control for testing effects and assess the impact of pretesting on the outcomes of interest. By including both pretest and posttest measures in both experimental and control groups, researchers can strengthen the internal validity of their study.

• Design Features:  Includes two experimental groups (with and without pretest) and two control groups (with and without pretest).
• Advantages:  Controls for testing effects and allows for the assessment of the independent and interactive effects of pretesting.
• Applications:  Ideal for studies where pretesting may influence participants' responses or when testing effects need to be controlled systematically.

Factorial Designs

Factorial designs involve manipulating two or more independent variables simultaneously to assess their main effects and interactions on the dependent variable. By varying multiple factors, researchers can examine complex relationships and identify potential moderators or mediators of effects.

• Design Features:  Manipulate two or more independent variables in a systematic manner.
• Advantages:  Allow researchers to examine main effects, interaction effects, and potential moderators or mediators of effects.
• Considerations:  Ensure adequate sample size and statistical power to detect significant effects, especially in designs with multiple factors.

Quasi-Experimental Designs

Quasi-experimental designs lack random assignment of participants to experimental conditions, making it challenging to establish causality definitively. However, these designs are valuable when randomization is not feasible or ethical, allowing researchers to explore naturally occurring phenomena in real-world settings.

• Design Features:  Lack random assignment of participants to experimental conditions.
• Advantages:  Suitable for studying phenomena that cannot be manipulated experimentally, such as the effects of natural disasters or policy changes.
• Considerations:  Control for potential confounding variables through matching, statistical controls, or careful selection of comparison groups.

Observational Studies

Observational studies involve observing and documenting behavior or phenomena in their natural environment without intervention or manipulation by the researcher. These studies provide valuable insights into real-world behavior but may be susceptible to observer bias and lack of control over extraneous variables.

• Design Features:  Observing and documenting behavior or phenomena without intervention.
• Advantages:  Provide rich, qualitative data and insights into naturalistic behavior and phenomena.
• Considerations:  Control for observer bias and extraneous variables through rigorous data collection protocols and analysis techniques.

Longitudinal Studies

Longitudinal studies involve collecting data from the same participants over an extended period to assess changes or development over time. These studies are valuable for studying developmental trajectories, longitudinal trends, and the long-term effects of interventions or treatments.

• Design Features:  Collect data from the same participants at multiple time points over an extended period.
• Advantages:  Allow researchers to assess changes or developments over time and examine causal relationships longitudinally.
• Considerations:  Address attrition, maturation, and testing effects through careful study design and data analysis techniques.

Cross-sectional Studies

Cross-sectional studies involve collecting data from different individuals or groups at a single point in time to explore relationships between variables. While efficient and cost-effective, cross-sectional studies cannot establish causality definitively and may be susceptible to cohort effects and bias.

• Design Features:  Collect data from different individuals or groups at a single point in time.
• Advantages:  Provide a snapshot of relationships between variables at a specific point in time and allow for comparisons across different groups.
• Considerations:  Interpret findings cautiously due to the inability to establish causality and control for cohort effects and bias through careful sampling and analysis techniques.

Choosing the appropriate research design requires careful consideration of your research question, objectives, and the nature of the phenomenon under investigation. By selecting a design that aligns with your goals and addresses potential threats to internal validity, you can enhance the credibility and reliability of your research findings.

Internal Validity Examples

Internal validity is a critical concept across various industries and use cases, ensuring that research findings accurately reflect the effects of the manipulated variables. Let's explore several examples of internal validity in different sectors:

Healthcare and Clinical Research

In healthcare and clinical research, internal validity is paramount for evaluating the efficacy and safety of medical interventions and treatments. For example:

• Randomized Controlled Trials (RCTs):  Pharmaceutical companies conduct RCTs to test the effectiveness of new medications. By randomly assigning participants to treatment and control groups, researchers ensure that any observed differences in outcomes are due to the medication and not other factors.
• Double-Blind Studies: Clinical trials often employ double-blind procedures, where neither the participants nor the researchers know who is receiving the treatment and who is receiving the placebo. This minimizes bias and ensures that expectations or biases do not influence any improvements observed in the treatment group.

Marketing and Consumer Behavior

In marketing and consumer behavior research, internal validity is crucial for understanding the effects of marketing strategies and consumer preferences. For example:

• A/B Testing :  Digital marketers often use A/B testing to evaluate the effectiveness of different advertising campaigns or website designs. By randomly assigning users to different versions of an ad or webpage, marketers can determine which version leads to higher conversion rates, ensuring internal validity.
• Quasi-Experimental Designs :  In retail settings, researchers may use quasi-experimental designs to assess the impact of a promotional sale on consumer purchasing behavior. Researchers can infer causality by comparing sales data before and during the promotion while controlling for external factors such as seasonality.

Environmental Science and Policy

In environmental science and policy research, internal validity is essential for evaluating the effectiveness of environmental interventions and policy interventions. For instance:

• Longitudinal Studies :  Environmental scientists may conduct longitudinal studies to assess the long-term impact of conservation efforts on biodiversity. By monitoring ecological variables over time, researchers can determine whether changes in biodiversity are due to conservation efforts or other factors.
• Regression Analysis :  Policy analysts may use regression analysis to examine the relationship between environmental policies (e.g., carbon pricing) and greenhouse gas emissions. By controlling for confounding variables such as economic growth and technological advancements, analysts can estimate the causal effect of the policy on emissions.

Technology and Product Development

In technology and product development, internal validity is critical for evaluating the effectiveness and usability of new technologies and products. For example:

• Usability Testing :  User experience (UX) researchers conduct usability tests to assess the ease of use and effectiveness of software interfaces or mobile apps. Researchers can identify usability issues and iterate on the design by observing how users interact with the product and measuring task completion rates.
• Field Experiments:  Technology companies may conduct field experiments to evaluate the impact of new features or innovations on user behavior. By randomly exposing users to different versions of the product, companies can measure changes in user engagement or satisfaction, ensuring internal validity.

Internal validity is a fundamental concept that transcends various industries and use cases, ensuring that research findings accurately reflect the effects of manipulated variables. By employing rigorous research designs, controlling for potential confounding variables, and implementing appropriate data analysis techniques, practitioners across different sectors can enhance internal validity and make informed decisions based on reliable evidence.

How to Assess Internal Validity?

Assessing internal validity is crucial for determining the reliability and credibility of research findings. Let's delve into various methods and techniques used to evaluate internal validity and ensure the robustness of research outcomes.

Internal Validity Threat Checklist

The internal validity threat checklist is a systematic tool researchers use to identify potential threats to internal validity in their studies. By systematically reviewing various aspects of the research design, data collection, and analysis process, researchers can pinpoint potential sources of bias and take appropriate steps to mitigate them.

• History Threats:  Assess whether external events or circumstances may have influenced the study's outcomes.
• Maturation Threats:  Consider whether participants naturally changed or matured throughout the study in ways that could affect the outcome variable.
• Testing Threats:  Evaluate whether the act of measuring or assessing participants influenced their subsequent responses.
• Instrumentation Threats:  Examine whether changes occurred in the measurement instruments or procedures during the study.
• Statistical Regression:  Assess whether extreme scores on a measure tended to move closer to the average upon retesting, leading to misinterpretation of treatment effects.
• Selection Bias:  Consider whether there are systematic differences between the characteristics of participants in different groups.
• Attrition/Mortality:  Evaluate whether the study lost participants over time and whether it was related to the variables being studied.
• Experimenter Bias:  Assess whether the researcher's expectations, beliefs, or behavior influenced the outcomes of the study.
• Novelty Effects:  Consider whether the novelty or unfamiliarity of the experimental procedure influenced participants' responses.

Statistical Techniques for Assessing Validity

Statistical techniques play a crucial role in assessing the validity of research findings and determining the extent to which the observed effects are attributable to the independent variable rather than chance or confounding variables.

• Analysis of Variance (ANOVA):  Assess whether there are significant differences between groups on the dependent variable after controlling for potential confounding variables.
• Regression Analysis :  Determine the strength and direction of the relationship between the independent and dependent variables while controlling for other variables.
• Mediation Analysis:  Explore the underlying mechanisms or pathways through which the independent variable influences the dependent variable.
• Moderation Analysis:  Examine whether the relationship between the independent and dependent variables varies depending on the level of a third variable.
• Structural Equation Modeling (SEM):  Evaluate complex relationships between multiple variables and test theoretical causality models.

Triangulation

Triangulation involves using multiple methods, data sources, or researchers to corroborate findings and enhance the validity and reliability of research outcomes. By triangulating data from different sources or perspectives, researchers can overcome the limitations of individual methods and provide a more comprehensive understanding of the phenomenon under investigation.

• Methodological Triangulation:  Combine qualitative and quantitative methods to gain a more nuanced understanding of complex phenomena.
• Data Triangulation:  Collect data from multiple sources or informants to verify findings and reduce the risk of bias or misinterpretation.
• Researcher Triangulation:  Involve multiple researchers in data collection, analysis, or interpretation to enhance the credibility and trustworthiness of research findings.

Peer Review and Replication

Peer review and replication are essential components of the scientific process that help ensure the validity and reliability of research findings. Peer review involves subjecting research manuscripts to evaluation by experts in the field, who assess the research's quality, rigor, and validity before publication.

• Peer Review:  Provide constructive feedback, identify methodological flaws or limitations, and assess the validity and reliability of research findings.
• Open Science Practices:  Promote transparency, reproducibility, and openness in research by sharing data, materials, and analysis code with the scientific community.
• Replication Studies:  Conduct independent replications of research findings to verify their reliability and generalizability.
• Meta-Analysis :  Synthesize findings from multiple studies to estimate the overall effect size and assess the robustness of research conclusions.

Assessing internal validity requires a comprehensive understanding of potential threats and biases inherent in the research process. By employing systematic checklists, statistical techniques, triangulation methods, and engaging in peer review and replication efforts, researchers can ensure the validity and credibility of their research findings, contributing to the advancement of knowledge in their respective fields.

Conclusion for Internal Validity

Internal validity serves as the cornerstone of credible and reliable research. By ensuring that research findings accurately reflect the effects of manipulated variables, internal validity enhances the trustworthiness and applicability of research across diverse fields and industries. From healthcare to education, marketing to environmental science, the principles of internal validity guide researchers in making informed decisions, advancing knowledge, and addressing real-world challenges. By understanding the importance of internal validity and implementing strategies to enhance it, researchers can generate high-quality evidence that withstands scrutiny and contributes to meaningful outcomes. Whether it's designing experiments with rigorous controls, conducting thorough statistical analyses, or engaging in peer review and replication efforts, prioritizing internal validity is essential for producing research that informs practice, policy, and innovation. Ultimately, internal validity empowers you to confidently draw conclusions, make informed decisions, and drive positive change.

How to Ensure Research Validity?

Introducing Appinio , your real-time market research platform revolutionizing the way companies harness consumer insights. With Appinio, conducting your own market research becomes a breeze, ensuring the highest levels of internal validity for your decision-making process. Experience the thrill of fast, reliable market research backed by dedicated research consultants and powerful interactive reports.

Here's why you should join the excitement:

• Rapid Insights:  From questions to insights in minutes, Appinio empowers you to make data-driven decisions swiftly without delay.
• User-Friendly Interface:  No need for a PhD in research – our intuitive platform makes market research accessible to everyone, regardless of expertise.
• Global Reach:  With access to over 90 countries and 1200+ characteristics, defining your target group and gathering responses has never been easier.

Get free access to the platform!

Join the loop 💌

Be the first to hear about new updates, product news, and data insights. We'll send it all straight to your inbox.

Get the latest market research news straight to your inbox! 💌

Wait, there's more

30.05.2024 | 29min read

Pareto Analysis: Definition, Pareto Chart, Examples

28.05.2024 | 32min read

What is Systematic Sampling? Definition, Types, Examples

16.05.2024 | 30min read

Time Series Analysis: Definition, Types, Techniques, Examples

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Family Med Prim Care
• v.4(3); Jul-Sep 2015

Validity, reliability, and generalizability in qualitative research

Lawrence leung.

1 Department of Family Medicine, Queen's University, Kingston, Ontario, Canada

2 Centre of Studies in Primary Care, Queen's University, Kingston, Ontario, Canada

In general practice, qualitative research contributes as significantly as quantitative research, in particular regarding psycho-social aspects of patient-care, health services provision, policy setting, and health administrations. In contrast to quantitative research, qualitative research as a whole has been constantly critiqued, if not disparaged, by the lack of consensus for assessing its quality and robustness. This article illustrates with five published studies how qualitative research can impact and reshape the discipline of primary care, spiraling out from clinic-based health screening to community-based disease monitoring, evaluation of out-of-hours triage services to provincial psychiatric care pathways model and finally, national legislation of core measures for children's healthcare insurance. Fundamental concepts of validity, reliability, and generalizability as applicable to qualitative research are then addressed with an update on the current views and controversies.

Nature of Qualitative Research versus Quantitative Research

The essence of qualitative research is to make sense of and recognize patterns among words in order to build up a meaningful picture without compromising its richness and dimensionality. Like quantitative research, the qualitative research aims to seek answers for questions of “how, where, when who and why” with a perspective to build a theory or refute an existing theory. Unlike quantitative research which deals primarily with numerical data and their statistical interpretations under a reductionist, logical and strictly objective paradigm, qualitative research handles nonnumerical information and their phenomenological interpretation, which inextricably tie in with human senses and subjectivity. While human emotions and perspectives from both subjects and researchers are considered undesirable biases confounding results in quantitative research, the same elements are considered essential and inevitable, if not treasurable, in qualitative research as they invariable add extra dimensions and colors to enrich the corpus of findings. However, the issue of subjectivity and contextual ramifications has fueled incessant controversies regarding yardsticks for quality and trustworthiness of qualitative research results for healthcare.

Impact of Qualitative Research upon Primary Care

In many ways, qualitative research contributes significantly, if not more so than quantitative research, to the field of primary care at various levels. Five qualitative studies are chosen to illustrate how various methodologies of qualitative research helped in advancing primary healthcare, from novel monitoring of chronic obstructive pulmonary disease (COPD) via mobile-health technology,[ 1 ] informed decision for colorectal cancer screening,[ 2 ] triaging out-of-hours GP services,[ 3 ] evaluating care pathways for community psychiatry[ 4 ] and finally prioritization of healthcare initiatives for legislation purposes at national levels.[ 5 ] With the recent advances of information technology and mobile connecting device, self-monitoring and management of chronic diseases via tele-health technology may seem beneficial to both the patient and healthcare provider. Recruiting COPD patients who were given tele-health devices that monitored lung functions, Williams et al. [ 1 ] conducted phone interviews and analyzed their transcripts via a grounded theory approach, identified themes which enabled them to conclude that such mobile-health setup and application helped to engage patients with better adherence to treatment and overall improvement in mood. Such positive findings were in contrast to previous studies, which opined that elderly patients were often challenged by operating computer tablets,[ 6 ] or, conversing with the tele-health software.[ 7 ] To explore the content of recommendations for colorectal cancer screening given out by family physicians, Wackerbarth, et al. [ 2 ] conducted semi-structure interviews with subsequent content analysis and found that most physicians delivered information to enrich patient knowledge with little regard to patients’ true understanding, ideas, and preferences in the matter. These findings suggested room for improvement for family physicians to better engage their patients in recommending preventative care. Faced with various models of out-of-hours triage services for GP consultations, Egbunike et al. [ 3 ] conducted thematic analysis on semi-structured telephone interviews with patients and doctors in various urban, rural and mixed settings. They found that the efficiency of triage services remained a prime concern from both users and providers, among issues of access to doctors and unfulfilled/mismatched expectations from users, which could arouse dissatisfaction and legal implications. In UK, a care pathways model for community psychiatry had been introduced but its benefits were unclear. Khandaker et al. [ 4 ] hence conducted a qualitative study using semi-structure interviews with medical staff and other stakeholders; adopting a grounded-theory approach, major themes emerged which included improved equality of access, more focused logistics, increased work throughput and better accountability for community psychiatry provided under the care pathway model. Finally, at the US national level, Mangione-Smith et al. [ 5 ] employed a modified Delphi method to gather consensus from a panel of nominators which were recognized experts and stakeholders in their disciplines, and identified a core set of quality measures for children's healthcare under the Medicaid and Children's Health Insurance Program. These core measures were made transparent for public opinion and later passed on for full legislation, hence illustrating the impact of qualitative research upon social welfare and policy improvement.

Overall Criteria for Quality in Qualitative Research

Given the diverse genera and forms of qualitative research, there is no consensus for assessing any piece of qualitative research work. Various approaches have been suggested, the two leading schools of thoughts being the school of Dixon-Woods et al. [ 8 ] which emphasizes on methodology, and that of Lincoln et al. [ 9 ] which stresses the rigor of interpretation of results. By identifying commonalities of qualitative research, Dixon-Woods produced a checklist of questions for assessing clarity and appropriateness of the research question; the description and appropriateness for sampling, data collection and data analysis; levels of support and evidence for claims; coherence between data, interpretation and conclusions, and finally level of contribution of the paper. These criteria foster the 10 questions for the Critical Appraisal Skills Program checklist for qualitative studies.[ 10 ] However, these methodology-weighted criteria may not do justice to qualitative studies that differ in epistemological and philosophical paradigms,[ 11 , 12 ] one classic example will be positivistic versus interpretivistic.[ 13 ] Equally, without a robust methodological layout, rigorous interpretation of results advocated by Lincoln et al. [ 9 ] will not be good either. Meyrick[ 14 ] argued from a different angle and proposed fulfillment of the dual core criteria of “transparency” and “systematicity” for good quality qualitative research. In brief, every step of the research logistics (from theory formation, design of study, sampling, data acquisition and analysis to results and conclusions) has to be validated if it is transparent or systematic enough. In this manner, both the research process and results can be assured of high rigor and robustness.[ 14 ] Finally, Kitto et al. [ 15 ] epitomized six criteria for assessing overall quality of qualitative research: (i) Clarification and justification, (ii) procedural rigor, (iii) sample representativeness, (iv) interpretative rigor, (v) reflexive and evaluative rigor and (vi) transferability/generalizability, which also double as evaluative landmarks for manuscript review to the Medical Journal of Australia. Same for quantitative research, quality for qualitative research can be assessed in terms of validity, reliability, and generalizability.

Validity in qualitative research means “appropriateness” of the tools, processes, and data. Whether the research question is valid for the desired outcome, the choice of methodology is appropriate for answering the research question, the design is valid for the methodology, the sampling and data analysis is appropriate, and finally the results and conclusions are valid for the sample and context. In assessing validity of qualitative research, the challenge can start from the ontology and epistemology of the issue being studied, e.g. the concept of “individual” is seen differently between humanistic and positive psychologists due to differing philosophical perspectives:[ 16 ] Where humanistic psychologists believe “individual” is a product of existential awareness and social interaction, positive psychologists think the “individual” exists side-by-side with formation of any human being. Set off in different pathways, qualitative research regarding the individual's wellbeing will be concluded with varying validity. Choice of methodology must enable detection of findings/phenomena in the appropriate context for it to be valid, with due regard to culturally and contextually variable. For sampling, procedures and methods must be appropriate for the research paradigm and be distinctive between systematic,[ 17 ] purposeful[ 18 ] or theoretical (adaptive) sampling[ 19 , 20 ] where the systematic sampling has no a priori theory, purposeful sampling often has a certain aim or framework and theoretical sampling is molded by the ongoing process of data collection and theory in evolution. For data extraction and analysis, several methods were adopted to enhance validity, including 1 st tier triangulation (of researchers) and 2 nd tier triangulation (of resources and theories),[ 17 , 21 ] well-documented audit trail of materials and processes,[ 22 , 23 , 24 ] multidimensional analysis as concept- or case-orientated[ 25 , 26 ] and respondent verification.[ 21 , 27 ]

Reliability

In quantitative research, reliability refers to exact replicability of the processes and the results. In qualitative research with diverse paradigms, such definition of reliability is challenging and epistemologically counter-intuitive. Hence, the essence of reliability for qualitative research lies with consistency.[ 24 , 28 ] A margin of variability for results is tolerated in qualitative research provided the methodology and epistemological logistics consistently yield data that are ontologically similar but may differ in richness and ambience within similar dimensions. Silverman[ 29 ] proposed five approaches in enhancing the reliability of process and results: Refutational analysis, constant data comparison, comprehensive data use, inclusive of the deviant case and use of tables. As data were extracted from the original sources, researchers must verify their accuracy in terms of form and context with constant comparison,[ 27 ] either alone or with peers (a form of triangulation).[ 30 ] The scope and analysis of data included should be as comprehensive and inclusive with reference to quantitative aspects if possible.[ 30 ] Adopting the Popperian dictum of falsifiability as essence of truth and science, attempted to refute the qualitative data and analytes should be performed to assess reliability.[ 31 ]

Generalizability

Most qualitative research studies, if not all, are meant to study a specific issue or phenomenon in a certain population or ethnic group, of a focused locality in a particular context, hence generalizability of qualitative research findings is usually not an expected attribute. However, with rising trend of knowledge synthesis from qualitative research via meta-synthesis, meta-narrative or meta-ethnography, evaluation of generalizability becomes pertinent. A pragmatic approach to assessing generalizability for qualitative studies is to adopt same criteria for validity: That is, use of systematic sampling, triangulation and constant comparison, proper audit and documentation, and multi-dimensional theory.[ 17 ] However, some researchers espouse the approach of analytical generalization[ 32 ] where one judges the extent to which the findings in one study can be generalized to another under similar theoretical, and the proximal similarity model, where generalizability of one study to another is judged by similarities between the time, place, people and other social contexts.[ 33 ] Thus said, Zimmer[ 34 ] questioned the suitability of meta-synthesis in view of the basic tenets of grounded theory,[ 35 ] phenomenology[ 36 ] and ethnography.[ 37 ] He concluded that any valid meta-synthesis must retain the other two goals of theory development and higher-level abstraction while in search of generalizability, and must be executed as a third level interpretation using Gadamer's concepts of the hermeneutic circle,[ 38 , 39 ] dialogic process[ 38 ] and fusion of horizons.[ 39 ] Finally, Toye et al. [ 40 ] reported the practicality of using “conceptual clarity” and “interpretative rigor” as intuitive criteria for assessing quality in meta-ethnography, which somehow echoed Rolfe's controversial aesthetic theory of research reports.[ 41 ]

Food for Thought

Despite various measures to enhance or ensure quality of qualitative studies, some researchers opined from a purist ontological and epistemological angle that qualitative research is not a unified, but ipso facto diverse field,[ 8 ] hence any attempt to synthesize or appraise different studies under one system is impossible and conceptually wrong. Barbour argued from a philosophical angle that these special measures or “technical fixes” (like purposive sampling, multiple-coding, triangulation, and respondent validation) can never confer the rigor as conceived.[ 11 ] In extremis, Rolfe et al. opined from the field of nursing research, that any set of formal criteria used to judge the quality of qualitative research are futile and without validity, and suggested that any qualitative report should be judged by the form it is written (aesthetic) and not by the contents (epistemic).[ 41 ] Rolfe's novel view is rebutted by Porter,[ 42 ] who argued via logical premises that two of Rolfe's fundamental statements were flawed: (i) “The content of research report is determined by their forms” may not be a fact, and (ii) that research appraisal being “subject to individual judgment based on insight and experience” will mean those without sufficient experience of performing research will be unable to judge adequately – hence an elitist's principle. From a realism standpoint, Porter then proposes multiple and open approaches for validity in qualitative research that incorporate parallel perspectives[ 43 , 44 ] and diversification of meanings.[ 44 ] Any work of qualitative research, when read by the readers, is always a two-way interactive process, such that validity and quality has to be judged by the receiving end too and not by the researcher end alone.

In summary, the three gold criteria of validity, reliability and generalizability apply in principle to assess quality for both quantitative and qualitative research, what differs will be the nature and type of processes that ontologically and epistemologically distinguish between the two.

Source of Support: Nil.

Conflict of Interest: None declared.

6.3 Steps in a Successful Marketing Research Plan

Learning outcomes.

By the end of this section, you will be able to:

• 1 Identify and describe the steps in a marketing research plan.
• 2 Discuss the different types of data research.
• 3 Explain how data is analyzed.
• 4 Discuss the importance of effective research reports.

Define the Problem

There are seven steps to a successful marketing research project (see Figure 6.3 ). Each step will be explained as we investigate how a marketing research project is conducted.

The first step, defining the problem, is often a realization that more information is needed in order to make a data-driven decision. Problem definition is the realization that there is an issue that needs to be addressed. An entrepreneur may be interested in opening a small business but must first define the problem that is to be investigated. A marketing research problem in this example is to discover the needs of the community and also to identify a potentially successful business venture.

Many times, researchers define a research question or objectives in this first step. Objectives of this research study could include: identify a new business that would be successful in the community in question, determine the size and composition of a target market for the business venture, and collect any relevant primary and secondary data that would support such a venture. At this point, the definition of the problem may be “Why are cat owners not buying our new cat toy subscription service?”

Additionally, during this first step we would want to investigate our target population for research. This is similar to a target market, as it is the group that comprises the population of interest for the study. In order to have a successful research outcome, the researcher should start with an understanding of the problem in the current situational environment.

Develop the Research Plan

Step two is to develop the research plan. What type of research is necessary to meet the established objectives of the first step? How will this data be collected? Additionally, what is the time frame of the research and budget to consider? If you must have information in the next week, a different plan would be implemented than in a situation where several months were allowed. These are issues that a researcher should address in order to meet the needs identified.

Research is often classified as coming from one of two types of data: primary and secondary. Primary data is unique information that is collected by the specific researcher with the current project in mind. This type of research doesn’t currently exist until it is pulled together for the project. Examples of primary data collection include survey, observation, experiment, or focus group data that is gathered for the current project.

Secondary data is any research that was completed for another purpose but can be used to help inform the research process. Secondary data comes in many forms and includes census data, journal articles, previously collected survey or focus group data of related topics, and compiled company data. Secondary data may be internal, such as the company’s sales records for a previous quarter, or external, such as an industry report of all related product sales. Syndicated data , a type of external secondary data, is available through subscription services and is utilized by many marketers. As you can see in Table 6.1 , primary and secondary data features are often opposite—the positive aspects of primary data are the negative side of secondary data.

There are four research types that can be used: exploratory, descriptive, experimental, and ethnographic research designs (see Figure 6.4 ). Each type has specific formats of data that can be collected. Qualitative research can be shared through words, descriptions, and open-ended comments. Qualitative data gives context but cannot be reduced to a statistic. Qualitative data examples are categorical and include case studies, diary accounts, interviews, focus groups, and open-ended surveys. By comparison, quantitative data is data that can be reduced to number of responses. The number of responses to each answer on a multiple-choice question is quantitative data. Quantitative data is numerical and includes things like age, income, group size, and height.

Exploratory research is usually used when additional general information in desired about a topic. When in the initial steps of a new project, understanding the landscape is essential, so exploratory research helps the researcher to learn more about the general nature of the industry. Exploratory research can be collected through focus groups, interviews, and review of secondary data. When examining an exploratory research design, the best use is when your company hopes to collect data that is generally qualitative in nature. 7

For instance, if a company is considering a new service for registered users but is not quite sure how well the new service will be received or wants to gain clarity of exactly how customers may use a future service, the company can host a focus group. Focus groups and interviews will be examined later in the chapter. The insights collected during the focus group can assist the company when designing the service, help to inform promotional campaign options, and verify that the service is going to be a viable option for the company.

Descriptive research design takes a bigger step into collection of data through primary research complemented by secondary data. Descriptive research helps explain the market situation and define an “opinion, attitude, or behavior” of a group of consumers, employees, or other interested groups. 8 The most common method of deploying a descriptive research design is through the use of a survey. Several types of surveys will be defined later in this chapter. Descriptive data is quantitative in nature, meaning the data can be distilled into a statistic, such as in a table or chart.

Again, descriptive data is helpful in explaining the current situation. In the opening example of LEGO , the company wanted to describe the situation regarding children’s use of its product. In order to gather a large group of opinions, a survey was created. The data that was collected through this survey allowed the company to measure the existing perceptions of parents so that alterations could be made to future plans for the company.

Experimental research , also known as causal research , helps to define a cause-and-effect relationship between two or more factors. This type of research goes beyond a correlation to determine which feature caused the reaction. Researchers generally use some type of experimental design to determine a causal relationship. An example is A/B testing, a situation where one group of research participants, group A, is exposed to one treatment and then compared to the group B participants, who experience a different situation. An example might be showing two different television commercials to a panel of consumers and then measuring the difference in perception of the product. Another example would be to have two separate packaging options available in different markets. This research would answer the question “Does one design sell better than the other?” Comparing that to the sales in each market would be part of a causal research study. 9

The final method of collecting data is through an ethnographic design. Ethnographic research is conducted in the field by watching people interact in their natural environment. For marketing research, ethnographic designs help to identify how a product is used, what actions are included in a selection, or how the consumer interacts with the product. 10

Examples of ethnographic research would be to observe how a consumer uses a particular product, such as baking soda. Although many people buy baking soda, its uses are vast. So are they using it as a refrigerator deodorizer, a toothpaste, to polish a belt buckle, or to use in baking a cake?

Select the Data Collection Method

Data collection is the systematic gathering of information that addresses the identified problem. What is the best method to do that? Picking the right method of collecting data requires that the researcher understand the target population and the design picked in the previous step. There is no perfect method; each method has both advantages and disadvantages, so it’s essential that the researcher understand the target population of the research and the research objectives in order to pick the best option.

Sometimes the data desired is best collected by watching the actions of consumers. For instance, how many cars pass a specific billboard in a day? What website led a potential customer to the company’s website? When are consumers most likely to use the snack vending machines at work? What time of day has the highest traffic on a social media post? What is the most streamed television program this week? Observational research is the collecting of data based on actions taken by those observed. Many data observations do not require the researched individuals to participate in the data collection effort to be highly valuable. Some observation requires an individual to watch and record the activities of the target population through personal observations .

Unobtrusive observation happens when those being observed aren’t aware that they are being watched. An example of an unobtrusive observation would be to watch how shoppers interact with a new stuffed animal display by using a one-way mirror. Marketers can identify which products were handled more often while also determining which were ignored.

Other methods can use technology to collect the data instead. Instances of mechanical observation include the use of vehicle recorders, which count the number of vehicles that pass a specific location. Computers can also assess the number of shoppers who enter a store, the most popular entry point for train station commuters, or the peak time for cars to park in a parking garage.

When you want to get a more in-depth response from research participants, one method is to complete a one-on-one interview . One-on-one interviews allow the researcher to ask specific questions that match the respondent’s unique perspective as well as follow-up questions that piggyback on responses already completed. An interview allows the researcher to have a deeper understanding of the needs of the respondent, which is another strength of this type of data collection. The downside of personal interviews it that a discussion can be very time-consuming and results in only one respondent’s answers. Therefore, in order to get a large sample of respondents, the interview method may not be the most efficient method.

Taking the benefits of an interview and applying them to a small group of people is the design of a focus group . A focus group is a small number of people, usually 8 to 12, who meet the sample requirements. These individuals together are asked a series of questions where they are encouraged to build upon each other’s responses, either by agreeing or disagreeing with the other group members. Focus groups are similar to interviews in that they allow the researcher, through a moderator, to get more detailed information from a small group of potential customers (see Figure 6.5 ).

Link to Learning

Focus groups.

Focus groups are a common method for gathering insights into consumer thinking and habits. Companies will use this information to develop or shift their initiatives. The best way to understand a focus group is to watch a few examples or explanations. TED-Ed has this video that explains how focus groups work.

You might be asking when it is best to use a focus group or a survey. Learn the differences, the pros and cons of each, and the specific types of questions you ask in both situations in this article .

Preparing for a focus group is critical to success. It requires knowing the material and questions while also managing the group of people. Watch this video to learn more about how to prepare for a focus group and the types of things to be aware of.

One of the benefits of a focus group over individual interviews is that synergy can be generated when a participant builds on another’s ideas. Additionally, for the same amount of time, a researcher can hear from multiple respondents instead of just one. 11 Of course, as with every method of data collection, there are downsides to a focus group as well. Focus groups have the potential to be overwhelmed by one or two aggressive personalities, and the format can discourage more reserved individuals from speaking up. Finally, like interviews, the responses in a focus group are qualitative in nature and are difficult to distill into an easy statistic or two.

Combining a variety of questions on one instrument is called a survey or questionnaire . Collecting primary data is commonly done through surveys due to their versatility. A survey allows the researcher to ask the same set of questions of a large group of respondents. Response rates of surveys are calculated by dividing the number of surveys completed by the total number attempted. Surveys are flexible and can collect a variety of quantitative and qualitative data. Questions can include simplified yes or no questions, select all that apply, questions that are on a scale, or a variety of open-ended types of questions. There are four types of surveys (see Table 6.2 ) we will cover, each with strengths and weaknesses defined.

Let’s start off with mailed surveys —surveys that are sent to potential respondents through a mail service. Mailed surveys used to be more commonly used due to the ability to reach every household. In some instances, a mailed survey is still the best way to collect data. For example, every 10 years the United States conducts a census of its population (see Figure 6.6 ). The first step in that data collection is to send every household a survey through the US Postal Service (USPS). The benefit is that respondents can complete and return the survey at their convenience. The downside of mailed surveys are expense and timeliness of responses. A mailed survey requires postage, both when it is sent to the recipient and when it is returned. That, along with the cost of printing, paper, and both sending and return envelopes, adds up quickly. Additionally, physically mailing surveys takes time. One method of reducing cost is to send with bulk-rate postage, but that slows down the delivery of the survey. Also, because of the convenience to the respondent, completed surveys may be returned several weeks after being sent. Finally, some mailed survey data must be manually entered into the analysis software, which can cause delays or issues due to entry errors.

Phone surveys are completed during a phone conversation with the respondent. Although the traditional phone survey requires a data collector to talk with the participant, current technology allows for computer-assisted voice surveys or surveys to be completed by asking the respondent to push a specific button for each potential answer. Phone surveys are time intensive but allow the respondent to ask questions and the surveyor to request additional information or clarification on a question if warranted. Phone surveys require the respondent to complete the survey simultaneously with the collector, which is a limitation as there are restrictions for when phone calls are allowed. According to Telephone Consumer Protection Act , approved by Congress in 1991, no calls can be made prior to 8:00 a.m. or after 9:00 p.m. in the recipient’s time zone. 12 Many restrictions are outlined in this original legislation and have been added to since due to ever-changing technology.

In-person surveys are when the respondent and data collector are physically in the same location. In-person surveys allow the respondent to share specific information, ask questions of the surveyor, and follow up on previous answers. Surveys collected through this method can take place in a variety of ways: through door-to-door collection, in a public location, or at a person’s workplace. Although in-person surveys are time intensive and require more labor to collect data than some other methods, in some cases it’s the best way to collect the required data. In-person surveys conducted through a door-to-door method is the follow-up used for the census if respondents do not complete the mailed survey. One of the downsides of in-person surveys is the reluctance of potential respondents to stop their current activity and answer questions. Furthermore, people may not feel comfortable sharing private or personal information during a face-to-face conversation.

Electronic surveys are sent or collected through digital means and is an opportunity that can be added to any of the above methods as well as some new delivery options. Surveys can be sent through email, and respondents can either reply to the email or open a hyperlink to an online survey (see Figure 6.7 ). Additionally, a letter can be mailed that asks members of the survey sample to log in to a website rather than to return a mailed response. Many marketers now use links, QR codes, or electronic devices to easily connect to a survey. Digitally collected data has the benefit of being less time intensive and is often a more economical way to gather and input responses than more manual methods. A survey that could take months to collect through the mail can be completed within a week through digital means.

Design the Sample

Although you might want to include every possible person who matches your target market in your research, it’s often not a feasible option, nor is it of value. If you did decide to include everyone, you would be completing a census of the population. Getting everyone to participate would be time-consuming and highly expensive, so instead marketers use a sample , whereby a portion of the whole is included in the research. It’s similar to the samples you might receive at the grocery store or ice cream shop; it isn’t a full serving, but it does give you a good taste of what the whole would be like.

So how do you know who should be included in the sample? Researchers identify parameters for their studies, called sample frames . A sample frame for one study may be college students who live on campus; for another study, it may be retired people in Dallas, Texas, or small-business owners who have fewer than 10 employees. The individual entities within the sampling frame would be considered a sampling unit . A sampling unit is each individual respondent that would be considered as matching the sample frame established by the research. If a researcher wants businesses to participate in a study, then businesses would be the sampling unit in that case.

The number of sampling units included in the research is the sample size . Many calculations can be conducted to indicate what the correct size of the sample should be. Issues to consider are the size of the population, the confidence level that the data represents the entire population, the ease of accessing the units in the frame, and the budget allocated for the research.

There are two main categories of samples: probability and nonprobability (see Figure 6.8 ). Probability samples are those in which every member of the sample has an identified likelihood of being selected. Several probability sample methods can be utilized. One probability sampling technique is called a simple random sample , where not only does every person have an identified likelihood of being selected to be in the sample, but every person also has an equal chance of exclusion. An example of a simple random sample would be to put the names of all members of a group into a hat and simply draw out a specific number to be included. You could say a raffle would be a good example of a simple random sample.

Another probability sample type is a stratified random sample , where the population is divided into groups by category and then a random sample of each category is selected to participate. For instance, if you were conducting a study of college students from your school and wanted to make sure you had all grade levels included, you might take the names of all students and split them into different groups by grade level—freshman, sophomore, junior, and senior. Then, from those categories, you would draw names out of each of the pools, or strata.

A nonprobability sample is a situation in which each potential member of the sample has an unknown likelihood of being selected in the sample. Research findings that are from a nonprobability sample cannot be applied beyond the sample. Several examples of nonprobability sampling are available to researchers and include two that we will look at more closely: convenience sampling and judgment sampling.

The first nonprobability sampling technique is a convenience sample . Just like it sounds, a convenience sample is when the researcher finds a group through a nonscientific method by picking potential research participants in a convenient manner. An example might be to ask other students in a class you are taking to complete a survey that you are doing for a class assignment or passing out surveys at a basketball game or theater performance.

A judgment sample is a type of nonprobability sample that allows the researcher to determine if they believe the individual meets the criteria set for the sample frame to complete the research. For instance, you may be interested in researching mothers, so you sit outside a toy store and ask an individual who is carrying a baby to participate.

Collect the Data

Now that all the plans have been established, the instrument has been created, and the group of participants has been identified, it is time to start collecting data. As explained earlier in this chapter, data collection is the process of gathering information from a variety of sources that will satisfy the research objectives defined in step one. Data collection can be as simple as sending out an email with a survey link enclosed or as complex as an experiment with hundreds of consumers. The method of collection directly influences the length of this process. Conducting personal interviews or completing an experiment, as previously mentioned, can add weeks or months to the research process, whereas sending out an electronic survey may allow a researcher to collect the necessary data in a few days. 13

Analyze and Interpret the Data

Once the data has been collected, the process of analyzing it may begin. Data analysis is the distillation of the information into a more understandable and actionable format. The analysis itself can take many forms, from the use of basic statistics to a more comprehensive data visualization process. First, let’s discuss some basic statistics that can be used to represent data.

The first is the mean of quantitative data. A mean is often defined as the arithmetic average of values. The formula is:

A common use of the mean calculation is with exam scores. Say, for example, you have earned the following scores on your marketing exams: 72, 85, 68, and 77. To find the mean, you would add up the four scores for a total of 302. Then, in order to generate a mean, that number needs to be divided by the number of exam scores included, which is 4. The mean would be 302 divided by 4, for a mean test score of 75.5. Understanding the mean can help to determine, with one number, the weight of a particular value.

Another commonly used statistic is median. The median is often referred to as the middle number. To generate a median, all the numeric answers are placed in order, and the middle number is the median. Median is a common statistic when identifying the income level of a specific geographic region. 14 For instance, the median household income for Albuquerque, New Mexico, between 2015 and 2019 was $52,911. 15 In this case, there are just as many people with an income above the amount as there are below.

Mode is another statistic that is used to represent data of all types, as it can be used with quantitative or qualitative data and represents the most frequent answer. Eye color, hair color, and vehicle color can all be presented with a mode statistic. Additionally, some researchers expand on the concept of mode and present the frequency of all responses, not just identifying the most common response. Data such as this can easily be presented in a frequency graph, 16 such as the one in Figure 6.9 .

Additionally, researchers use other analyses to represent the data rather than to present the entirety of each response. For example, maybe the relationship between two values is important to understand. In this case, the researcher may share the data as a cross tabulation (see Figure 6.10 ). Below is the same data as above regarding social media use cross tabulated with gender—as you can see, the data is more descriptive when you can distinguish between the gender identifiers and how much time is spent per day on social media.

Not all data can be presented in a graphical format due to the nature of the information. Sometimes with qualitative methods of data collection, the responses cannot be distilled into a simple statistic or graph. In that case, the use of quotations, otherwise known as verbatims , can be used. These are direct statements presented by the respondents. Often you will see a verbatim statement when reading a movie or book review. The critic’s statements are used in part or in whole to represent their feelings about the newly released item.

Infographics

As they say, a picture is worth a thousand words. For this reason, research results are often shown in a graphical format in which data can be taken in quickly, called an infographic .

Check out this infographic on what components make for a good infographic. As you can see, a good infographic needs four components: data, design, a story, and the ability to share it with others. Without all four pieces, it is not as valuable a resource as it could be. The ultimate infographic is represented as the intersection of all four.

Infographics are particularly advantageous online. Refer to this infographic on why they are beneficial to use online .

Prepare the Research Report

The marketing research process concludes by sharing the generated data and makes recommendations for future actions. What starts as simple data must be interpreted into an analysis. All information gathered should be conveyed in order to make decisions for future marketing actions. One item that is often part of the final step is to discuss areas that may have been missed with the current project or any area of further study identified while completing it. Without the final step of the marketing research project, the first six steps are without value. It is only after the information is shared, through a formal presentation or report, that those recommendations can be implemented and improvements made. The first six steps are used to generate information, while the last is to initiate action. During this last step is also when an evaluation of the process is conducted. If this research were to be completed again, how would we do it differently? Did the right questions get answered with the survey questions posed to the respondents? Follow-up on some of these key questions can lead to additional research, a different study, or further analysis of data collected.

Methods of Quantifying Marketing Research

One of the ways of sharing information gained through marketing research is to quantify the research . Quantifying the research means to take a variety of data and compile into a quantity that is more easily understood. This is a simple process if you want to know how many people attended a basketball game, but if you want to quantify the number of students who made a positive comment on a questionnaire, it can be a little more complicated. Researchers have a variety of methods to collect and then share these different scores. Below are some of the most common types used in business.

Is a customer aware of a product, brand, or company? What is meant by awareness? Awareness in the context of marketing research is when a consumer is familiar with the product, brand, or company. It does not assume that the consumer has tried the product or has purchased it. Consumers are just aware. That is a measure that many businesses find valuable. There are several ways to measure awareness. For instance, the first type of awareness is unaided awareness . This type of awareness is when no prompts for a product, brand, or company are given. If you were collecting information on fast-food restaurants, you might ask a respondent to list all the fast-food restaurants that serve a chicken sandwich. Aided awareness would be providing a list of products, brands, or companies and the respondent selects from the list. For instance, if you give a respondent a list of fast-food restaurants and ask them to mark all the locations with a chicken sandwich, you are collecting data through an aided method. Collecting these answers helps a company determine how the business location compares to those of its competitors. 17

Customer Satisfaction (CSAT)

Have you ever been asked to complete a survey at the end of a purchase? Many businesses complete research on buying, returning, or other customer service processes. A customer satisfaction score , also known as CSAT, is a measure of how satisfied customers are with the product, brand, or service. A CSAT score is usually on a scale of 0 to 100 percent. 18 But what constitutes a “good” CSAT score? Although what is identified as good can vary by industry, normally anything in the range from 75 to 85 would be considered good. Of course, a number higher than 85 would be considered exceptional. 19

Customer Acquisition Cost (CAC) and Customer Effort Score (CES)

Other metrics often used are a customer acquisition cost (CAC) and customer effort score (CES). How much does it cost a company to gain customers? That’s the purpose of calculating the customer acquisition cost. To calculate the customer acquisition cost , a company would need to total all expenses that were accrued to gain new customers. This would include any advertising, public relations, social media postings, etc. When a total cost is determined, it is divided by the number of new customers gained through this campaign.

The final score to discuss is the customer effort score , also known as a CES. The CES is a “survey used to measure the ease of service experience with an organization.” 20 Companies that are easy to work with have a better CES than a company that is notorious for being difficult. An example would be to ask a consumer about the ease of making a purchase online by incorporating a one-question survey after a purchase is confirmed. If a number of responses come back negative or slightly negative, the company will realize that it needs to investigate and develop a more user-friendly process.

Knowledge Check

It’s time to check your knowledge on the concepts presented in this section. Refer to the Answer Key at the end of the book for feedback.

• Defining the problem
• Developing the research plan
• Selecting a data collection method
• Designing the sample
• you are able to send it to all households in an area
• it is inexpensive
• responses are automatically loaded into the software
• the data comes in quickly
• Primary data
• Secondary data
• Secondary and primary data
• Professional data
• It shows how respondents answered two variables in relation to each other and can help determine patterns by different groups of respondents.
• By presenting the data in the form of a picture, the information is easier for the reader to understand.
• It is an easy way to see how often one answer is selected by the respondents.
• This analysis can used to present interview or focus group data.

As an Amazon Associate we earn from qualifying purchases.

This book may not be used in the training of large language models or otherwise be ingested into large language models or generative AI offerings without OpenStax's permission.

Want to cite, share, or modify this book? This book uses the Creative Commons Attribution License and you must attribute OpenStax.

Access for free at https://openstax.org/books/principles-marketing/pages/1-unit-introduction

• Authors: Dr. Maria Gomez Albrecht, Dr. Mark Green, Linda Hoffman
• Publisher/website: OpenStax
• Book title: Principles of Marketing
• Publication date: Jan 25, 2023
• Location: Houston, Texas
• Book URL: https://openstax.org/books/principles-marketing/pages/1-unit-introduction
• Section URL: https://openstax.org/books/principles-marketing/pages/6-3-steps-in-a-successful-marketing-research-plan

© Jan 9, 2024 OpenStax. Textbook content produced by OpenStax is licensed under a Creative Commons Attribution License . The OpenStax name, OpenStax logo, OpenStax book covers, OpenStax CNX name, and OpenStax CNX logo are not subject to the Creative Commons license and may not be reproduced without the prior and express written consent of Rice University.

Social Media Automation: 8 Time-Saving Tools for 2024

Social media automation uses tools or software to manage repetitive tasks, like posting content or producing reports, without manual labor.

What Are Social Media KPIs? 25 Examples + How to Track Them

Setting smart social media KPIs will help your team focus on the metrics that really matter and track performance over time.

Experiment: How Many Instagram Stories Should You Post?

Which earns more reach: a single Instagram Story per day or many Stories per day? Is there even a “right” number of daily Stories to post?

Instagram Automation: What You Should Automate + Tools [2024]

Learn about legitimate Instagram automation strategies and save time automating routine daily tasks, without resorting to fake engagement and bots.

71 Social Media Post Templates to Speed Up Your Process

The right social media post template can speed up content creation, saving time while ensuring posts are on-brand and effective.

Facebook Automation: What You Can Automate + Tools [2024]

Facebook automations handle the tedious maintenance of your brand’s Facebook page more efficiently, so you can focus on the big picture stuff.

The Unbiased Guide to Meta Blueprint Certification in 2024

Meta Blueprint (f.k.a. Facebook Blueprint) online courses help you quickly learn new skills, but it’s not your only marketing education option.

Instagram Subscriptions: How To Monetize Your Account in 2024

Instagram subscriptions can be a great way for creators to earn money from their Instagram accounts. We’ll teach you how to use this feature.

How to Get Verified on Instagram in 2024

Want to know how to get verified on Instagram? You can either follow the traditional application process or pay for Meta Verified.

Become a better social marketer.

Get expert social media advice delivered straight to your inbox.

Experiments

Experiment: Where Should Your Instagram Hashtags Go?

Do Instagram hashtags belong in the caption or comments? Does hashtag location even matter? Time to put on our lab coats and start testing.

Experiment: Do Long-Form X Posts Give You More Reach?

What gives you better reach on X: a good old-fashioned tweet thread or one long-form post? Here’s what we found.

Experiment: Do Longer LinkedIn Comments Drive More Connection Requests?

Does being original on LinkedIn generate faster growth? Here’s a thrilling social experiment that answers the question.

Experiment: What Happens When AI Plans Your Entire Social Strategy?

Is AI really coming for your job? We decided to find out.

Hootsuite HQ

Hootsuite CEO Reveals How Long She Spends on Social Media Every Morning

Irina Novoselsky, Hootsuite CEO, shares how she has found her niche as a social CEO at Collision 2023—and much more.

The Story Behind Hootsuite’s ‘Proud Mamas’ Mother’s Day Tribute

Hootsuite social media manager Trish Riswick’s tribute to her mom for supporting her career (even when she doesn’t always understand it).

Creative Block Is Inevitable—Here’s How Our Social Team Beats It

All out of social media ideas? We know the feeling. Discover how to overcome creative block with these 9 tips from our in-house experts.

Meet Irina Novoselsky, Hootsuite’s new CEO

Irina Novoselsky shares her thoughts on why she’s excited to join the Hootsuite team as Chief Executive Officer.