parts of chapter 1 in quantitative research

• Introduction to Problem Statements, Purpose Statements, and Research Questions

Worksheets and Guides

Chapter 1 playlist.

• Student Experience Feedback Buttons
• Narrowing Your Topic
• Problem Statement
• Purpose Statement
• Conceptual Framework
• Theoretical Framework
• Quantitative Research Questions This link opens in a new window
• Qualitative Research Questions This link opens in a new window
• Qualitative & Quantitative Research Support with the ASC This link opens in a new window
• Library Research Consultations This link opens in a new window

Jump to DSE Guide

Need help ask us.

Chapter 1 introduces the research problem and the evidence supporting the existence of the problem. It outlines an initial review of the literature on the study topic and articulates the purpose of the study. The definitions of any technical terms necessary for the reader to understand are essential. Chapter 1 also presents the research questions and theoretical foundation (Ph.D.) or conceptual framework (Applied Doctorate) and provides an overview of the research methods (qualitative or quantitative) being used in the study.  

• Research Feasibility Checklist Use this checklist to make sure your study will be feasible, reasonable, justifiable, and necessary.
• Alignment Worksheet Use this worksheet to make sure your problem statement, purpose, and research questions are aligned. Alignment indicates the degree to which the purpose of the study follows logically from the problem statement; and the degree to which the research questions help address the study’s purpose. Alignment is important because it helps ensure that the research study is well-designed and based on logical arguments.
• SOBE Research Design and Chapter 1 Checklist If you are in the School of Business and Economics (SOBE), use this checklist one week before the Communication and Research Design Checkpoint. Work with your Chair to determine if you need to complete this.

Was this resource helpful?

• Next: Narrowing Your Topic >>
• Last Updated: Apr 24, 2023 1:37 PM
• URL: https://resources.nu.edu/c.php?g=1006886

• Chapter Four: Quantitative Methods (Part 1)

Once you have chosen a topic to investigate, you need to decide which type of method is best to study it. This is one of the most important choices you will make on your research journey. Understanding the value of each of the methods described in this textbook to answer different questions allows you to be able to plan your own studies with more confidence, critique the studies others have done, and provide advice to your colleagues and friends on what type of research they should do to answer questions they have. After briefly reviewing quantitative research assumptions, this chapter is organized in three parts or sections. These parts can also be used as a checklist when working through the steps of your study. Specifically, part 1 focuses on planning a quantitative study (collecting data), part two explains the steps involved in doing a quantitative study, and part three discusses how to make sense of your results (organizing and analyzing data).

• Chapter One: Introduction
• Chapter Two: Understanding the distinctions among research methods
• Chapter Three: Ethical research, writing, and creative work
• Chapter Four: Quantitative Methods (Part 2 - Doing Your Study)
• Chapter Four: Quantitative Methods (Part 3 - Making Sense of Your Study)
• Chapter Five: Qualitative Methods (Part 1)
• Chapter Five: Qualitative Data (Part 2)
• Chapter Six: Critical / Rhetorical Methods (Part 1)
• Chapter Six: Critical / Rhetorical Methods (Part 2)
• Chapter Seven: Presenting Your Results

Quantitative Worldview Assumptions: A Review

In chapter 2, you were introduced to the unique assumptions quantitative research holds about knowledge and how it is created, or what the authors referred to in chapter one as "epistemology." Understanding these assumptions can help you better determine whether you need to use quantitative methods for a particular research study in which you are interested.

Quantitative researchers believe there is an objective reality, which can be measured. "Objective" here means that the researcher is not relying on their own perceptions of an event. S/he is attempting to gather "facts" which may be separate from people's feeling or perceptions about the facts. These facts are often conceptualized as "causes" and "effects." When you ask research questions or pose hypotheses with words in them such as "cause," "effect," "difference between," and "predicts," you are operating under assumptions consistent with quantitative methods. The overall goal of quantitative research is to develop generalizations that enable the researcher to better predict, explain, and understand some phenomenon.

Because of trying to prove cause-effect relationships that can be generalized to the population at large, the research process and related procedures are very important for quantitative methods. Research should be consistently and objectively conducted, without bias or error, in order to be considered to be valid (accurate) and reliable (consistent). Perhaps this emphasis on accurate and standardized methods is because the roots of quantitative research are in the natural and physical sciences, both of which have at their base the need to prove hypotheses and theories in order to better understand the world in which we live. When a person goes to a doctor and is prescribed some medicine to treat an illness, that person is glad such research has been done to know what the effects of taking this medicine is on others' bodies, so s/he can trust the doctor's judgment and take the medicines.

As covered in chapters 1 and 2, the questions you are asking should lead you to a certain research method choice. Students sometimes want to avoid doing quantitative research because of fear of math/statistics, but if their questions call for that type of research, they should forge ahead and use it anyway. If a student really wants to understand what the causes or effects are for a particular phenomenon, they need to do quantitative research. If a student is interested in what sorts of things might predict a person's behavior, they need to do quantitative research. If they want to confirm the finding of another researcher, most likely they will need to do quantitative research. If a student wishes to generalize beyond their participant sample to a larger population, they need to be conducting quantitative research.

So, ultimately, your choice of methods really depends on what your research goal is. What do you really want to find out? Do you want to compare two or more groups, look for relationships between certain variables, predict how someone will act or react, or confirm some findings from another study? If so, you want to use quantitative methods.

A topic such as self-esteem can be studied in many ways. Listed below are some example RQs about self-esteem. Which of the following research questions should be answered with quantitative methods?

• Is there a difference between men's and women's level of self- esteem?
• How do college-aged women describe their ups and downs with self-esteem?
• How has "self-esteem" been constructed in popular self-help books over time?
• Is there a relationship between self-esteem levels and communication apprehension?

What are the advantages of approaching a topic like self-esteem using quantitative methods? What are the disadvantages?

For more information, see the following website: Analyse This!!! Learning to analyse quantitative data

Answers:  1 & 4

Quantitative Methods Part One: Planning Your Study

Planning your study is one of the most important steps in the research process when doing quantitative research. As seen in the diagram below, it involves choosing a topic, writing research questions/hypotheses, and designing your study. Each of these topics will be covered in detail in this section of the chapter.

Topic Choice

Decide on topic.

How do you go about choosing a topic for a research project? One of the best ways to do this is to research something about which you would like to know more. Your communication professors will probably also want you to select something that is related to communication and things you are learning about in other communication classes.

When the authors of this textbook select research topics to study, they choose things that pique their interest for a variety of reasons, sometimes personal and sometimes because they see a need for more research in a particular area. For example, April Chatham-Carpenter studies adoption return trips to China because she has two adopted daughters from China and because there is very little research on this topic for Chinese adoptees and their families; she studied home vs. public schooling because her sister home schools, and at the time she started the study very few researchers had considered the social network implications for home schoolers (cf.  http://www.uni.edu/chatham/homeschool.html ).

When you are asked in this class and other classes to select a topic to research, think about topics that you have wondered about, that affect you personally, or that know have gaps in the research. Then start writing down questions you would like to know about this topic. These questions will help you decide whether the goal of your study is to understand something better, explain causes and effects of something, gather the perspectives of others on a topic, or look at how language constructs a certain view of reality.

Review Previous Research

In quantitative research, you do not rely on your conclusions to emerge from the data you collect. Rather, you start out looking for certain things based on what the past research has found. This is consistent with what was called in chapter 2 as a deductive approach (Keyton, 2011), which also leads a quantitative researcher to develop a research question or research problem from reviewing a body of literature, with the previous research framing the study that is being done. So, reviewing previous research done on your topic is an important part of the planning of your study. As seen in chapter 3 and the Appendix, to do an adequate literature review, you need to identify portions of your topic that could have been researched in the past. To do that, you select key terms of concepts related to your topic.

Some people use concept maps to help them identify useful search terms for a literature review. For example, see the following website: Concept Mapping: How to Start Your Term Paper Research .

Narrow Topic to Researchable Area

Once you have selected your topic area and reviewed relevant literature related to your topic, you need to narrow your topic to something that can be researched practically and that will take the research on this topic further. You don't want your research topic to be so broad or large that you are unable to research it. Plus, you want to explain some phenomenon better than has been done before, adding to the literature and theory on a topic. You may want to test out what someone else has found, replicating their study, and therefore building to the body of knowledge already created.

To see how a literature review can be helpful in narrowing your topic, see the following sources.  Narrowing or Broadening Your Research Topic  and  How to Conduct a Literature Review in Social Science

Research Questions & Hypotheses

Write Your Research Questions (RQs) and/or Hypotheses (Hs)

Once you have narrowed your topic based on what you learned from doing your review of literature, you need to formalize your topic area into one or more research questions or hypotheses. If the area you are researching is a relatively new area, and no existing literature or theory can lead you to predict what you might find, then you should write a research question. Take a topic related to social media, for example, which is a relatively new area of study. You might write a research question that asks:

"Is there a difference between how 1st year and 4th year college students use Facebook to communicate with their friends?"

If, however, you are testing out something you think you might find based on the findings of a large amount of previous literature or a well-developed theory, you can write a hypothesis. Researchers often distinguish between  null  and  alternative  hypotheses. The alternative hypothesis is what you are trying to test or prove is true, while the null hypothesis assumes that the alternative hypothesis is not true. For example, if the use of Facebook had been studied a great deal, and there were theories that had been developed on the use of it, then you might develop an alternative hypothesis, such as: "First-year students spend more time on using Facebook to communicate with their friends than fourth-year students do." Your null hypothesis, on the other hand, would be: "First-year students do  not  spend any more time using Facebook to communication with their friends than fourth-year students do." Researchers, however, only state the alternative hypothesis in their studies, and actually call it "hypothesis" rather than "alternative hypothesis."

Process of Writing a Research Question/Hypothesis.

Once you have decided to write a research question (RQ) or hypothesis (H) for your topic, you should go through the following steps to create your RQ or H.

Name the concepts from your overall research topic that you are interested in studying.

RQs and Hs have variables, or concepts that you are interested in studying. Variables can take on different values. For example, in the RQ above, there are at least two variables – year in college and use of Facebook (FB) to communicate. Both of them have a variety of levels within them.

When you look at the concepts you identified, are there any concepts which seem to be related to each other? For example, in our RQ, we are interested in knowing if there is a difference between first-year students and fourth-year students in their use of FB, meaning that we believe there is some connection between our two variables.

• Decide what type of a relationship you would like to study between the variables. Do you think one causes the other? Does a difference in one create a difference in the other? As the value of one changes, does the value of the other change?

Identify which one of these concepts is the independent (or predictor) variable, or the concept that is perceived to be the cause of change in the other variable? Which one is the dependent (criterion) variable, or the one that is affected by changes in the independent variable? In the above example RQ, year in school is the independent variable, and amount of time spent on Facebook communicating with friends is the dependent variable. The amount of time spent on Facebook depends on a person's year in school.

If you're still confused about independent and dependent variables, check out the following site: Independent & Dependent Variables .

Express the relationship between the concepts as a single sentence – in either a hypothesis or a research question.

For example, "is there a difference between international and American students on their perceptions of the basic communication course," where cultural background and perceptions of the course are your two variables. Cultural background would be the independent variable, and perceptions of the course would be your dependent variable. More examples of RQs and Hs are provided in the next section.

APPLICATION: Try the above steps with your topic now. Check with your instructor to see if s/he would like you to send your topic and RQ/H to him/her via e-mail.

Types of Research Questions/Hypotheses

Once you have written your RQ/H, you need to determine what type of research question or hypothesis it is. This will help you later decide what types of statistics you will need to run to answer your question or test your hypothesis. There are three possible types of questions you might ask, and two possible types of hypotheses. The first type of question cannot be written as a hypothesis, but the second and third types can.

Descriptive Question.

The first type of question is a descriptive question. If you have only one variable or concept you are studying, OR if you are not interested in how the variables you are studying are connected or related to each other, then your question is most likely a descriptive question.

This type of question is the closest to looking like a qualitative question, and often starts with a "what" or "how" or "why" or "to what extent" type of wording. What makes it different from a qualitative research question is that the question will be answered using numbers rather than qualitative analysis. Some examples of a descriptive question, using the topic of social media, include the following.

"To what extent are college-aged students using Facebook to communicate with their friends?"

"Why do college-aged students use Facebook to communicate with their friends?"

Notice that neither of these questions has a clear independent or dependent variable, as there is no clear cause or effect being assumed by the question. The question is merely descriptive in nature. It can be answered by summarizing the numbers obtained for each category, such as by providing percentages, averages, or just the raw totals for each type of strategy or organization. This is true also of the following research questions found in a study of online public relations strategies:

"What online public relations strategies are organizations implementing to combat phishing" (Baker, Baker, & Tedesco, 2007, p. 330), and

"Which organizations are doing most and least, according to recommendations from anti- phishing advocacy recommendations, to combat phishing" (Baker, Baker, & Tedesco, 2007, p. 330)

The researchers in this study reported statistics in their results or findings section, making it clearly a quantitative study, but without an independent or dependent variable; therefore, these research questions illustrate the first type of RQ, the descriptive question.

Difference Question/Hypothesis.

The second type of question is a question/hypothesis of difference, and will often have the word "difference" as part of the question. The very first research question in this section, asking if there is a difference between 1st year and 4th year college students' use of Facebook, is an example of this type of question. In this type of question, the independent variable is some type of grouping or categories, such as age. Another example of a question of difference is one April asked in her research on home schooling: "Is there a difference between home vs. public schoolers on the size of their social networks?" In this example, the independent variable is home vs. public schooling (a group being compared), and the dependent variable is size of social networks. Hypotheses can also be difference hypotheses, as the following example on the same topic illustrates: "Public schoolers have a larger social network than home schoolers do."

Relationship/Association Question/Hypothesis.

The third type of question is a relationship/association question or hypothesis, and will often have the word "relate" or "relationship" in it, as the following example does: "There is a relationship between number of television ads for a political candidate and how successful that political candidate is in getting elected." Here the independent (or predictor) variable is number of TV ads, and the dependent (or criterion) variable is the success at getting elected. In this type of question, there is no grouping being compared, but rather the independent variable is continuous (ranges from zero to a certain number) in nature. This type of question can be worded as either a hypothesis or as a research question, as stated earlier.

Test out your knowledge of the above information, by answering the following questions about the RQ/H listed below. (Remember, for a descriptive question there are no clear independent & dependent variables.)

• What is the independent variable (IV)?
• What is the dependent variable (DV)?
• What type of research question/hypothesis is it? (descriptive, difference, relationship/association)
• "Is there a difference on relational satisfaction between those who met their current partner through online dating and those who met their current partner face-to-face?"
• "How do Fortune 500 firms use focus groups to market new products?"
• "There is a relationship between age and amount of time spent online using social media."

Answers: RQ1  is a difference question, with type of dating being the IV and relational satisfaction being the DV. RQ2  is a descriptive question with no IV or DV. RQ3  is a relationship hypothesis with age as the IV and amount of time spent online as the DV.

Design Your Study

The third step in planning your research project, after you have decided on your topic/goal and written your research questions/hypotheses, is to design your study which means to decide how to proceed in gathering data to answer your research question or to test your hypothesis. This step includes six things to do. [NOTE: The terms used in this section will be defined as they are used.]

• Decide type of study design: Experimental, quasi-experimental, non-experimental.
• Decide kind of data to collect: Survey/interview, observation, already existing data.
• Operationalize variables into measurable concepts.
• Determine type of sample: Probability or non-probability.
• Decide how you will collect your data: face-to-face, via e-mail, an online survey, library research, etc.
• Pilot test your methods.

Types of Study Designs

With quantitative research being rooted in the scientific method, traditional research is structured in an experimental fashion. This is especially true in the natural sciences, where they try to prove causes and effects on topics such as successful treatments for cancer. For example, the University of Iowa Hospitals and Clinics regularly conduct clinical trials to test for the effectiveness of certain treatments for medical conditions ( University of Iowa Hospitals & Clinics: Clinical Trials ). They use human participants to conduct such research, regularly recruiting volunteers. However, in communication, true experiments with treatments the researcher controls are less necessary and thus less common. It is important for the researcher to understand which type of study s/he wishes to do, in order to accurately communicate his/her methods to the public when describing the study.

There are three possible types of studies you may choose to do, when embarking on quantitative research: (a) True experiments, (b) quasi-experiments, and (c) non-experiments.

For more information to read on these types of designs, take a look at the following website and related links in it: Types of Designs .

The following flowchart should help you distinguish between the three types of study designs described below.

True Experiments.

The first two types of study designs use difference questions/hypotheses, as the independent variable for true and quasi-experiments is  nominal  or categorical (based on categories or groupings), as you have groups that are being compared. As seen in the flowchart above, what distinguishes a true experiment from the other two designs is a concept called "random assignment." Random assignment means that the researcher controls to which group the participants are assigned. April's study of home vs. public schooling was NOT a true experiment, because she could not control which participants were home schooled and which ones were public schooled, and instead relied on already existing groups.

An example of a true experiment reported in a communication journal is a study investigating the effects of using interest-based contemporary examples in a lecture on the history of public relations, in which the researchers had the following two hypotheses: "Lectures utilizing interest- based examples should result in more interested participants" and "Lectures utilizing interest- based examples should result in participants with higher scores on subsequent tests of cognitive recall" (Weber, Corrigan, Fornash, & Neupauer, 2003, p. 118). In this study, the 122 college student participants were randomly assigned by the researchers to one of two lecture video viewing groups: a video lecture with traditional examples and a video with contemporary examples. (To see the results of the study, look it up using your school's library databases).

A second example of a true experiment in communication is a study of the effects of viewing either a dramatic narrative television show vs. a nonnarrative television show about the consequences of an unexpected teen pregnancy. The researchers randomly assigned their 367 undergraduate participants to view one of the two types of shows.

Moyer-Gusé, E., & Nabi, R. L. (2010). Explaining the effects of narrative in an entertainment television program: Overcoming resistance to persuasion.  Human Communication Research, 36 , 26-52.

A third example of a true experiment done in the field of communication can be found in the following study.

Jensen, J. D. (2008). Scientific uncertainty in news coverage of cancer research: Effects of hedging on scientists' and journalists' credibility.  Human Communication Research, 34,  347-369.

In this study, Jakob Jensen had three independent variables. He randomly assigned his 601 participants to 1 of 20 possible conditions, between his three independent variables, which were (a) a hedged vs. not hedged message, (b) the source of the hedging message (research attributed to primary vs. unaffiliated scientists), and (c) specific news story employed (of which he had five randomly selected news stories about cancer research to choose from). Although this study was pretty complex, it does illustrate the true experiment in our field since the participants were randomly assigned to read a particular news story, with certain characteristics.

Quasi-Experiments.

If the researcher is not able to randomly assign participants to one of the treatment groups (or independent variable), but the participants already belong to one of them (e.g., age; home vs. public schooling), then the design is called a quasi-experiment. Here you still have an independent variable with groups, but the participants already belong to a group before the study starts, and the researcher has no control over which group they belong to.

An example of a hypothesis found in a communication study is the following: "Individuals high in trait aggression will enjoy violent content more than nonviolent content, whereas those low in trait aggression will enjoy violent content less than nonviolent content" (Weaver & Wilson, 2009, p. 448). In this study, the researchers could not assign the participants to a high or low trait aggression group since this is a personality characteristic, so this is a quasi-experiment. It does not have any random assignment of participants to the independent variable groups. Read their study, if you would like to, at the following location.

Weaver, A. J., & Wilson, B. J. (2009). The role of graphic and sanitized violence in the enjoyment of television dramas.  Human Communication Research, 35  (3), 442-463.

Benoit and Hansen (2004) did not choose to randomly assign participants to groups either, in their study of a national presidential election survey, in which they were looking at differences between debate and non-debate viewers, in terms of several dependent variables, such as which candidate viewers supported. If you are interested in discovering the results of this study, take a look at the following article.

Benoit, W. L., & Hansen, G. J. (2004). Presidential debate watching, issue knowledge, character evaluation, and vote choice.  Human Communication Research, 30  (1), 121-144.

Non-Experiments.

The third type of design is the non-experiment. Non-experiments are sometimes called survey designs, because their primary way of collecting data is through surveys. This is not enough to distinguish them from true experiments and quasi-experiments, however, as both of those types of designs may use surveys as well.

What makes a study a non-experiment is that the independent variable is not a grouping or categorical variable. Researchers observe or survey participants in order to describe them as they naturally exist without any experimental intervention. Researchers do not give treatments or observe the effects of a potential natural grouping variable such as age. Descriptive and relationship/association questions are most often used in non-experiments.

Some examples of this type of commonly used design for communication researchers include the following studies.

• Serota, Levine, and Boster (2010) used a national survey of 1,000 adults to determine the prevalence of lying in America (see  Human Communication Research, 36 , pp. 2-25).
• Nabi (2009) surveyed 170 young adults on their perceptions of reality television on cosmetic surgery effects, looking at several things: for example, does viewing cosmetic surgery makeover programs relate to body satisfaction (p. 6), finding no significant relationship between those two variables (see  Human Communication Research, 35 , pp. 1-27).
• Derlega, Winstead, Mathews, and Braitman (2008) collected stories from 238 college students on reasons why they would disclose or not disclose personal information within close relationships (see  Communication Research Reports, 25 , pp. 115-130). They coded the participants' answers into categories so they could count how often specific reasons were mentioned, using a method called  content analysis , to answer the following research questions:

RQ1: What are research participants' attributions for the disclosure and nondisclosure of highly personal information?

RQ2: Do attributions reflect concerns about rewards and costs of disclosure or the tension between openness with another and privacy?

RQ3: How often are particular attributions for disclosure/nondisclosure used in various types of relationships? (p. 117)

All of these non-experimental studies have in common no researcher manipulation of an independent variable or even having an independent variable that has natural groups that are being compared.

Identify which design discussed above should be used for each of the following research questions.

• Is there a difference between generations on how much they use MySpace?
• Is there a relationship between age when a person first started using Facebook and the amount of time they currently spend on Facebook daily?
• Is there a difference between potential customers' perceptions of an organization who are shown an organization's Facebook page and those who are not shown an organization's Facebook page?

[HINT: Try to identify the independent and dependent variable in each question above first, before determining what type of design you would use. Also, try to determine what type of question it is – descriptive, difference, or relationship/association.]

Answers: 1. Quasi-experiment 2. Non-experiment 3. True Experiment

Data Collection Methods

Once you decide the type of quantitative research design you will be using, you will need to determine which of the following types of data you will collect: (a) survey data, (b) observational data, and/or (c) already existing data, as in library research.

Using the survey data collection method means you will talk to people or survey them about their behaviors, attitudes, perceptions, and demographic characteristics (e.g., biological sex, socio-economic status, race). This type of data usually consists of a series of questions related to the concepts you want to study (i.e., your independent and dependent variables). Both of April's studies on home schooling and on taking adopted children on a return trip back to China used survey data.

On a survey, you can have both closed-ended and open-ended questions. Closed-ended questions, can be written in a variety of forms. Some of the most common response options include the following.

Likert responses – for example: for the following statement, ______ do you strongly agree agree neutral disagree strongly disagree

Semantic differential – for example: does the following ______ make you Happy ..................................... Sad

Yes-no answers for example: I use social media daily. Yes / No.

One site to check out for possible response options is  http://www.360degreefeedback.net/media/ResponseScales.pdf .

Researchers often follow up some of their closed-ended questions with an "other" category, in which they ask their participants to "please specify," their response if none of the ones provided are applicable. They may also ask open-ended questions on "why" a participant chose a particular answer or ask participants for more information about a particular topic. If the researcher wants to use the open-ended question responses as part of his/her quantitative study, the answers are usually coded into categories and counted, in terms of the frequency of a certain answer, using a method called  content analysis , which will be discussed when we talk about already-existing artifacts as a source of data.

Surveys can be done face-to-face, by telephone, mail, or online. Each of these methods has its own advantages and disadvantages, primarily in the form of the cost in time and money to do the survey. For example, if you want to survey many people, then online survey tools such as surveygizmo.com and surveymonkey.com are very efficient, but not everyone has access to taking a survey on the computer, so you may not get an adequate sample of the population by doing so. Plus you have to decide how you will recruit people to take your online survey, which can be challenging. There are trade-offs with every method.

For more information on things to consider when selecting your survey method, check out the following website:

Selecting the Survey Method .

There are also many good sources for developing a good survey, such as the following websites. Constructing the Survey Survey Methods Designing Surveys

Observation.

A second type of data collection method is  observation . In this data collection method, you make observations of the phenomenon you are studying and then code your observations, so that you can count what you are studying. This type of data collection method is often called interaction analysis, if you collect data by observing people's behavior. For example, if you want to study the phenomenon of mall-walking, you could go to a mall and count characteristics of mall-walkers. A researcher in the area of health communication could study the occurrence of humor in an operating room, for example, by coding and counting the use of humor in such a setting.

One extended research study using observational data collection methods, which is cited often in interpersonal communication classes, is John Gottman's research, which started out in what is now called "The Love Lab." In this lab, researchers observe interactions between couples, including physiological symptoms, using coders who look for certain items found to predict relationship problems and success.

Take a look at the YouTube video about "The Love Lab" at the following site to learn more about the potential of using observation in collecting data for a research study:  The "Love" Lab .

Already-Existing Artifacts.

The third method of quantitative data collection is the use of  already-existing artifacts . With this method, you choose certain artifacts (e.g., newspaper or magazine articles; television programs; webpages) and code their content, resulting in a count of whatever you are studying. With this data collection method, researchers most often use what is called quantitative  content analysis . Basically, the researcher counts frequencies of something that occurs in an artifact of study, such as the frequency of times something is mentioned on a webpage. Content analysis can also be used in qualitative research, where a researcher identifies and creates text-based themes but does not do a count of the occurrences of these themes. Content analysis can also be used to take open-ended questions from a survey method, and identify countable themes within the questions.

Content analysis is a very common method used in media studies, given researchers are interested in studying already-existing media artifacts. There are many good sources to illustrate how to do content analysis such as are seen in the box below.

See the following sources for more information on content analysis. Writing Guide: Content Analysis A Flowchart for the Typical Process of Content Analysis Research What is Content Analysis?

With content analysis and any method that you use to code something into categories, one key concept you need to remember is  inter-coder or inter-rater reliability , in which there are multiple coders (at least two) trained to code the observations into categories. This check on coding is important because you need to check to make sure that the way you are coding your observations on the open-ended answers is the same way that others would code a particular item. To establish this kind of inter-coder or inter-rater reliability, researchers prepare codebooks (to train their coders on how to code the materials) and coding forms for their coders to use.

To see some examples of actual codebooks used in research, see the following website:  Human Coding--Sample Materials .

There are also online inter-coder reliability calculators some researchers use, such as the following:  ReCal: reliability calculation for the masses .

Regardless of which method of data collection you choose, you need to decide even more specifically how you will measure the variables in your study, which leads us to the next planning step in the design of a study.

Operationalization of Variables into Measurable Concepts

When you look at your research question/s and/or hypotheses, you should know already what your independent and dependent variables are. Both of these need to be measured in some way. We call that way of measuring  operationalizing  a variable. One way to think of it is writing a step by step recipe for how you plan to obtain data on this topic. How you choose to operationalize your variable (or write the recipe) is one all-important decision you have to make, which will make or break your study. In quantitative research, you have to measure your variables in a valid (accurate) and reliable (consistent) manner, which we discuss in this section. You also need to determine the level of measurement you will use for your variables, which will help you later decide what statistical tests you need to run to answer your research question/s or test your hypotheses. We will start with the last topic first.

Level of Measurement

Level of measurement has to do with whether you measure your variables using categories or groupings OR whether you measure your variables using a continuous level of measurement (range of numbers). The level of measurement that is considered to be categorical in nature is called nominal, while the levels of measurement considered to be continuous in nature are ordinal, interval, and ratio. The only ones you really need to know are nominal, ordinal, and interval/ratio.

Nominal  variables are categories that do not have meaningful numbers attached to them but are broader categories, such as male and female, home schooled and public schooled, Caucasian and African-American.  Ordinal  variables do have numbers attached to them, in that the numbers are in a certain order, but there are not equal intervals between the numbers (e.g., such as when you rank a group of 5 items from most to least preferred, where 3 might be highly preferred, and 2 hated).  Interval/ratio  variables have equal intervals between the numbers (e.g., weight, age).

For more information about these levels of measurement, check out one of the following websites. Levels of Measurement Measurement Scales in Social Science Research What is the difference between ordinal, interval and ratio variables? Why should I care?

Validity and Reliability

When developing a scale/measure or survey, you need to be concerned about validity and reliability. Readers of quantitative research expect to see researchers justify their research measures using these two terms in the methods section of an article or paper.

Validity.   Validity  is the extent to which your scale/measure or survey adequately reflects the full meaning of the concept you are measuring. Does it measure what you say it measures? For example, if researchers wanted to develop a scale to measure "servant leadership," the researchers would have to determine what dimensions of servant leadership they wanted to measure, and then create items which would be valid or accurate measures of these dimensions. If they included items related to a different type of leadership, those items would not be a valid measure of servant leadership. When doing so, the researchers are trying to prove their measure has internal validity. Researchers may also be interested in external validity, but that has to do with how generalizable their study is to a larger population (a topic related to sampling, which we will consider in the next section), and has less to do with the validity of the instrument itself.

There are several types of validity you may read about, including face validity, content validity, criterion-related validity, and construct validity. To learn more about these types of validity, read the information at the following link: Validity .

To improve the validity of an instrument, researchers need to fully understand the concept they are trying to measure. This means they know the academic literature surrounding that concept well and write several survey questions on each dimension measured, to make sure the full idea of the concept is being measured. For example, Page and Wong (n.d.) identified four dimensions of servant leadership: character, people-orientation, task-orientation, and process-orientation ( A Conceptual Framework for Measuring Servant-Leadership ). All of these dimensions (and any others identified by other researchers) would need multiple survey items developed if a researcher wanted to create a new scale on servant leadership.

Before you create a new survey, it can be useful to see if one already exists with established validity and reliability. Such measures can be found by seeing what other respected studies have used to measure a concept and then doing a library search to find the scale/measure itself (sometimes found in the reference area of a library in books like those listed below).

Reliability .  Reliability  is the second criterion you will need to address if you choose to develop your own scale or measure. Reliability is concerned with whether a measurement is consistent and reproducible. If you have ever wondered why, when taking a survey, that a question is asked more than once or very similar questions are asked multiple times, it is because the researchers one concerned with proving their study has reliability. Are you, for example, answering all of the similar questions similarly? If so, the measure/scale may have good reliability or consistency over time.

Researchers can use a variety of ways to show their measure/scale is reliable. See the following websites for explanations of some of these ways, which include methods such as the test-retest method, the split-half method, and inter-coder/rater reliability. Types of Reliability Reliability

To understand the relationship between validity and reliability, a nice visual provided below is explained at the following website (Trochim, 2006, para. 2). Reliability & Validity

Self-Quiz/Discussion:

Take a look at one of the surveys found at the following poll reporting sites on a topic which interests you. Critique one of these surveys, using what you have learned about creating surveys so far.

http://www.pewinternet.org/ http://pewresearch.org/ http://www.gallup.com/Home.aspx http://www.kff.org/

One of the things you might have critiqued in the previous self-quiz/discussion may have had less to do with the actual survey itself, but rather with how the researchers got their participants or sample. How participants are recruited is just as important to doing a good study as how valid and reliable a survey is.

Imagine that in the article you chose for the last "self-quiz/discussion" you read the following quote from the Pew Research Center's Internet and American Life Project: "One in three teens sends more than 100 text messages a day, or 3000 texts a month" (Lenhart, 2010, para.5). How would you know whether you could trust this finding to be true? Would you compare it to what you know about texting from your own and your friends' experiences? Would you want to know what types of questions people were asked to determine this statistic, or whether the survey the statistic is based on is valid and reliable? Would you want to know what type of people were surveyed for the study? As a critical consumer of research, you should ask all of these types of questions, rather than just accepting such a statement as undisputable fact. For example, if only people shopping at an Apple Store were surveyed, the results might be skewed high.

In particular, related to the topic of this section, you should ask about the sampling method the researchers did. Often, the researchers will provide information related to the sample, stating how many participants were surveyed (in this case 800 teens, aged 12-17, who were a nationally representative sample of the population) and how much the "margin of error" is (in this case +/- 3.8%). Why do they state such things? It is because they know the importance of a sample in making the case for their findings being legitimate and credible.  Margin of error  is how much we are confident that our findings represent the population at large. The larger the margin of error, the less likely it is that the poll or survey is accurate. Margin of error assumes a 95% confidence level that what we found from our study represents the population at large.

For more information on margin of error, see one of the following websites. Answers.com Margin of Error Stats.org Margin of Error Americanresearchgroup.com Margin of Error [this last site is a margin of error calculator, which shows that margin of error is directly tied to the size of your sample, in relationship to the size of the population, two concepts we will talk about in the next few paragraphs]

In particular, this section focused on sampling will talk about the following topics: (a) the difference between a population vs. a sample; (b) concepts of error and bias, or "it's all about significance"; (c) probability vs. non-probability sampling; and (d) sample size issues.

Population vs. Sample

When doing quantitative studies, such as the study of cell phone usage among teens, you are never able to survey the entire population of teenagers, so you survey a portion of the population. If you study every member of a population, then you are conducting a census such as the United States Government does every 10 years. When, however, this is not possible (because you do not have the money the U.S. government has!), you attempt to get as good a sample as possible.

Characteristics of a population are summarized in numerical form, and technically these numbers are called  parameters . However, numbers which summarize the characteristics of a sample are called  statistics .

Error and Bias

If a sample is not done well, then you may not have confidence in how the study's results can be generalized to the population from which the sample was taken. Your confidence level is often stated as the  margin of error  of the survey. As noted earlier, a study's margin of error refers to the degree to which a sample differs from the total population you are studying. In the Pew survey, they had a margin of error of +/- 3.8%. So, for example, when the Pew survey said 33% of teens send more than 100 texts a day, the margin of error means they were 95% sure that 29.2% - 36.8% of teens send this many texts a day.

Margin of error is tied to  sampling error , which is how much difference there is between your sample's results and what would have been obtained if you had surveyed the whole population. Sample error is linked to a very important concept for quantitative researchers, which is the notion of  significance . Here, significance does not refer to whether some finding is morally or practically significant, it refers to whether a finding is statistically significant, meaning the findings are not due to chance but actually represent something that is found in the population.  Statistical significance  is about how much you, as the researcher, are willing to risk saying you found something important and be wrong.

For the difference between statistical significance and practical significance, see the following YouTube video:  Statistical and Practical Significance .

Scientists set certain arbitrary standards based on the probability they could be wrong in reporting their findings. These are called  significance levels  and are commonly reported in the literature as  p <.05  or  p <.01  or some other probability (or  p ) level.

If an article says a statistical test reported that  p < .05 , it simply means that they are most likely correct in what they are saying, but there is a 5% chance they could be wrong and not find the same results in the population. If p < .01, then there would be only a 1% chance they were wrong and would not find the same results in the population. The lower the probability level, the more certain the results.

When researchers are wrong, or make that kind of decision error, it often implies that either (a) their sample was biased and was not representative of the true population in some way, or (b) that something they did in collecting the data biased the results. There are actually two kinds of sampling error talked about in quantitative research: Type I and Type II error.  Type 1 error  is what happens when you think you found something statistically significant and claim there is a significant difference or relationship, when there really is not in the actual population. So there is something about your sample that made you find something that is not in the actual population. (Type I error is the same as the probability level, or .05, if using the traditional p-level accepted by most researchers.)  Type II error  happens when you don't find a statistically significant difference or relationship, yet there actually is one in the population at large, so once again, your sample is not representative of the population.

For more information on these two types of error, check out the following websites. Hypothesis Testing: Type I Error, Type II Error Type I and Type II Errors - Making Mistakes in the Justice System

Researchers want to select a sample that is representative of the population in order to reduce the likelihood of having a sample that is biased. There are two types of bias particularly troublesome for researchers, in terms of sampling error. The first type is  selection bias , in which each person in the population does not have an equal chance to be chosen for the sample, which happens frequently in communication studies, because we often rely on convenience samples (whoever we can get to complete our surveys). The second type of bias is  response bias , in which those who volunteer for a study have different characteristics than those who did not volunteer for the study, another common challenge for communication researchers. Volunteers for a study may very well be different from persons who choose not to volunteer for a study, so that you have a biased sample by relying just on volunteers, which is not representative of the population from which you are trying to sample.

Probability vs. Non-Probability Sampling

One of the best ways to lower your sampling error and reduce the possibility of bias is to do probability or random sampling. This means that every person in the population has an equal chance of being selected to be in your sample. Another way of looking at this is to attempt to get a  representative  sample, so that the characteristics of your sample closely approximate those of the population. A sample needs to contain essentially the same variations that exist in the population, if possible, especially on the variables or elements that are most important to you (e.g., age, biological sex, race, level of education, socio-economic class).

There are many different ways to draw a probability/random sample from the population. Some of the most common are a  simple random sample , where you use a random numbers table or random number generator to select your sample from the population.

There are several examples of random number generators available online. See the following example of an online random number generator:  http://www.randomizer.org/ .

A  systematic random sample  takes every n-th number from the population, depending on how many people you would like to have in your sample. A  stratified random sample  does random sampling within groups, and a  multi-stage  or  cluster sample  is used when there are multiple groups within a large area and a large population, and the researcher does random sampling in stages.

If you are interested in understanding more about these types of probability/random samples, take a look at the following website: Probability Sampling .

However, many times communication researchers use whoever they can find to participate in their study, such as college students in their classes since these people are easily accessible. Many of the studies in interpersonal communication and relationship development, for example, used this type of sample. This is called a convenience sample. In doing so, they are using a non- probability or non-random sample. In these types of samples, each member of the population does not have an equal opportunity to be selected. For example, if you decide to ask your facebook friends to participate in an online survey you created about how college students in the U.S. use cell phones to text, you are using a non-random type of sample. You are unable to randomly sample the whole population in the U.S. of college students who text, so you attempt to find participants more conveniently. Some common non-random or non-probability samples are:

• accidental/convenience samples, such as the facebook example illustrates
• quota samples, in which you do convenience samples within subgroups of the population, such as biological sex, looking for a certain number of participants in each group being compared
• snowball or network sampling, where you ask current participants to send your survey onto their friends.

For more information on non-probability sampling, see the following website: Nonprobability Sampling .

Researchers, such as communication scholars, often use these types of samples because of the nature of their research. Most research designs used in communication are not true experiments, such as would be required in the medical field where they are trying to prove some cause-effect relationship to cure or alleviate symptoms of a disease. Most communication scholars recognize that human behavior in communication situations is much less predictable, so they do not adhere to the strictest possible worldview related to quantitative methods and are less concerned with having to use probability sampling.

They do recognize, however, that with either probability or non-probability sampling, there is still the possibility of bias and error, although much less with probability sampling. That is why all quantitative researchers, regardless of field, will report statistical significance levels if they are interested in generalizing from their sample to the population at large, to let the readers of their work know how confident they are in their results.

Size of Sample

The larger the sample, the more likely the sample is going to be representative of the population. If there is a lot of variability in the population (e.g., lots of different ethnic groups in the population), a researcher will need a larger sample. If you are interested in detecting small possible differences (e.g., in a close political race), you need a larger sample. However, the bigger your population, the less you have to increase the size of your sample in order to have an adequate sample, as is illustrated by an example sample size calculator such as can be found at  http://www.raosoft.com/samplesize.html .

Using the example sample size calculator, see how you might determine how large of a sample you might need in order to study how college students in the U.S. use texting on their cell phones. You would have to first determine approximately how many college students are in the U.S. According to ANEKI, there are a little over 14,000,000 college students in the U.S. ( Countries with the Most University Students ). When inputting that figure into the sample size calculator below (using no commas for the population size), you would need a sample size of approximately 385 students. If the population size was 20,000, you would need a sample of 377 students. If the population was only 2,000, you would need a sample of 323. For a population of 500, you would need a sample of 218.

It is not enough, however, to just have an adequate or large sample. If there is bias in the sampling, you can have a very bad large sample, one that also does not represent the population at large. So, having an unbiased sample is even more important than having a large sample.

So, what do you do, if you cannot reasonably conduct a probability or random sample? You run statistics which report significance levels, and you report the limitations of your sample in the discussion section of your paper/article.

Pilot Testing Methods

Now that we have talked about the different elements of your study design, you should try out your methods by doing a pilot test of some kind. This means that you try out your procedures with someone to try to catch any mistakes in your design before you start collecting data from actual participants in your study. This will save you time and money in the long run, along with unneeded angst over mistakes you made in your design during data collection. There are several ways you might do this.

You might ask an expert who knows about this topic (such as a faculty member) to try out your experiment or survey and provide feedback on what they think of your design. You might ask some participants who are like your potential sample to take your survey or be a part of your pilot test; then you could ask them which parts were confusing or needed revising. You might have potential participants explain to you what they think your questions mean, to see if they are interpreting them like you intended, or if you need to make some questions clearer.

The main thing is that you do not just assume your methods will work or are the best type of methods to use until you try them out with someone. As you write up your study, in your methods section of your paper, you can then talk about what you did to change your study based on the pilot study you did.

Institutional Review Board (IRB) Approval

The last step of your planning takes place when you take the necessary steps to get your study approved by your institution's review board. As you read in chapter 3, this step is important if you are planning on using the data or results from your study beyond just the requirements for your class project. See chapter 3 for more information on the procedures involved in this step.

Conclusion: Study Design Planning

Once you have decided what topic you want to study, you plan your study. Part 1 of this chapter has covered the following steps you need to follow in this planning process:

• decide what type of study you will do (i.e., experimental, quasi-experimental, non- experimental);
• decide on what data collection method you will use (i.e., survey, observation, or already existing data);
• operationalize your variables into measureable concepts;
• determine what type of sample you will use (probability or non-probability);
• pilot test your methods; and
• get IRB approval.

At that point, you are ready to commence collecting your data, which is the topic of the next section in this chapter.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is Quantitative Research? | Definition, Uses & Methods

What Is Quantitative Research? | Definition, Uses & Methods

Published on June 12, 2020 by Pritha Bhandari . Revised on June 22, 2023.

Quantitative research is the process of collecting and analyzing numerical data. It can be used to find patterns and averages, make predictions, test causal relationships, and generalize results to wider populations.

Quantitative research is the opposite of qualitative research , which involves collecting and analyzing non-numerical data (e.g., text, video, or audio).

Quantitative research is widely used in the natural and social sciences: biology, chemistry, psychology, economics, sociology, marketing, etc.

• What is the demographic makeup of Singapore in 2020?
• How has the average temperature changed globally over the last century?
• Does environmental pollution affect the prevalence of honey bees?
• Does working from home increase productivity for people with long commutes?

Table of contents

Quantitative research methods, quantitative data analysis, advantages of quantitative research, disadvantages of quantitative research, other interesting articles, frequently asked questions about quantitative research.

You can use quantitative research methods for descriptive, correlational or experimental research.

• In descriptive research , you simply seek an overall summary of your study variables.
• In correlational research , you investigate relationships between your study variables.
• In experimental research , you systematically examine whether there is a cause-and-effect relationship between variables.

Correlational and experimental research can both be used to formally test hypotheses , or predictions, using statistics. The results may be generalized to broader populations based on the sampling method used.

To collect quantitative data, you will often need to use operational definitions that translate abstract concepts (e.g., mood) into observable and quantifiable measures (e.g., self-ratings of feelings and energy levels).

Note that quantitative research is at risk for certain research biases , including information bias , omitted variable bias , sampling bias , or selection bias . Be sure that you’re aware of potential biases as you collect and analyze your data to prevent them from impacting your work too much.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Once data is collected, you may need to process it before it can be analyzed. For example, survey and test data may need to be transformed from words to numbers. Then, you can use statistical analysis to answer your research questions .

Descriptive statistics will give you a summary of your data and include measures of averages and variability. You can also use graphs, scatter plots and frequency tables to visualize your data and check for any trends or outliers.

Using inferential statistics , you can make predictions or generalizations based on your data. You can test your hypothesis or use your sample data to estimate the population parameter .

First, you use descriptive statistics to get a summary of the data. You find the mean (average) and the mode (most frequent rating) of procrastination of the two groups, and plot the data to see if there are any outliers.

You can also assess the reliability and validity of your data collection methods to indicate how consistently and accurately your methods actually measured what you wanted them to.

Quantitative research is often used to standardize data collection and generalize findings . Strengths of this approach include:

• Replication

Repeating the study is possible because of standardized data collection protocols and tangible definitions of abstract concepts.

• Direct comparisons of results

The study can be reproduced in other cultural settings, times or with different groups of participants. Results can be compared statistically.

• Large samples

Data from large samples can be processed and analyzed using reliable and consistent procedures through quantitative data analysis.

• Hypothesis testing

Using formalized and established hypothesis testing procedures means that you have to carefully consider and report your research variables, predictions, data collection and testing methods before coming to a conclusion.

Despite the benefits of quantitative research, it is sometimes inadequate in explaining complex research topics. Its limitations include:

• Superficiality

Using precise and restrictive operational definitions may inadequately represent complex concepts. For example, the concept of mood may be represented with just a number in quantitative research, but explained with elaboration in qualitative research.

• Narrow focus

Predetermined variables and measurement procedures can mean that you ignore other relevant observations.

• Structural bias

Despite standardized procedures, structural biases can still affect quantitative research. Missing data , imprecise measurements or inappropriate sampling methods are biases that can lead to the wrong conclusions.

• Lack of context

Quantitative research often uses unnatural settings like laboratories or fails to consider historical and cultural contexts that may affect data collection and results.

Prevent plagiarism. Run a free check.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Chi square goodness of fit test
• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Mixed methods research
• Non-probability sampling
• Inclusion and exclusion criteria

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to systematically measure variables and test hypotheses . Qualitative methods allow you to explore concepts and experiences in more detail.

In mixed methods research , you use both qualitative and quantitative data collection and analysis methods to answer your research question .

Data collection is the systematic process by which observations or measurements are gathered in research. It is used in many different contexts by academics, governments, businesses, and other organizations.

Operationalization means turning abstract conceptual ideas into measurable observations.

For example, the concept of social anxiety isn’t directly observable, but it can be operationally defined in terms of self-rating scores, behavioral avoidance of crowded places, or physical anxiety symptoms in social situations.

Before collecting data , it’s important to consider how you will operationalize the variables that you want to measure.

Reliability and validity are both about how well a method measures something:

• Reliability refers to the  consistency of a measure (whether the results can be reproduced under the same conditions).
• Validity   refers to the  accuracy of a measure (whether the results really do represent what they are supposed to measure).

If you are doing experimental research, you also have to consider the internal and external validity of your experiment.

Hypothesis testing is a formal procedure for investigating our ideas about the world using statistics. It is used by scientists to test specific predictions, called hypotheses , by calculating how likely it is that a pattern or relationship between variables could have arisen by chance.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Bhandari, P. (2023, June 22). What Is Quantitative Research? | Definition, Uses & Methods. Scribbr. Retrieved April 15, 2024, from https://www.scribbr.com/methodology/quantitative-research/

Is this article helpful?

Pritha Bhandari

Other students also liked, descriptive statistics | definitions, types, examples, inferential statistics | an easy introduction & examples, what is your plagiarism score.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Korean Med Sci
• v.37(16); 2022 Apr 25

A Practical Guide to Writing Quantitative and Qualitative Research Questions and Hypotheses in Scholarly Articles

Edward barroga.

1 Department of General Education, Graduate School of Nursing Science, St. Luke’s International University, Tokyo, Japan.

Glafera Janet Matanguihan

2 Department of Biological Sciences, Messiah University, Mechanicsburg, PA, USA.

The development of research questions and the subsequent hypotheses are prerequisites to defining the main research purpose and specific objectives of a study. Consequently, these objectives determine the study design and research outcome. The development of research questions is a process based on knowledge of current trends, cutting-edge studies, and technological advances in the research field. Excellent research questions are focused and require a comprehensive literature search and in-depth understanding of the problem being investigated. Initially, research questions may be written as descriptive questions which could be developed into inferential questions. These questions must be specific and concise to provide a clear foundation for developing hypotheses. Hypotheses are more formal predictions about the research outcomes. These specify the possible results that may or may not be expected regarding the relationship between groups. Thus, research questions and hypotheses clarify the main purpose and specific objectives of the study, which in turn dictate the design of the study, its direction, and outcome. Studies developed from good research questions and hypotheses will have trustworthy outcomes with wide-ranging social and health implications.

INTRODUCTION

Scientific research is usually initiated by posing evidenced-based research questions which are then explicitly restated as hypotheses. 1 , 2 The hypotheses provide directions to guide the study, solutions, explanations, and expected results. 3 , 4 Both research questions and hypotheses are essentially formulated based on conventional theories and real-world processes, which allow the inception of novel studies and the ethical testing of ideas. 5 , 6

It is crucial to have knowledge of both quantitative and qualitative research 2 as both types of research involve writing research questions and hypotheses. 7 However, these crucial elements of research are sometimes overlooked; if not overlooked, then framed without the forethought and meticulous attention it needs. Planning and careful consideration are needed when developing quantitative or qualitative research, particularly when conceptualizing research questions and hypotheses. 4

There is a continuing need to support researchers in the creation of innovative research questions and hypotheses, as well as for journal articles that carefully review these elements. 1 When research questions and hypotheses are not carefully thought of, unethical studies and poor outcomes usually ensue. Carefully formulated research questions and hypotheses define well-founded objectives, which in turn determine the appropriate design, course, and outcome of the study. This article then aims to discuss in detail the various aspects of crafting research questions and hypotheses, with the goal of guiding researchers as they develop their own. Examples from the authors and peer-reviewed scientific articles in the healthcare field are provided to illustrate key points.

DEFINITIONS AND RELATIONSHIP OF RESEARCH QUESTIONS AND HYPOTHESES

A research question is what a study aims to answer after data analysis and interpretation. The answer is written in length in the discussion section of the paper. Thus, the research question gives a preview of the different parts and variables of the study meant to address the problem posed in the research question. 1 An excellent research question clarifies the research writing while facilitating understanding of the research topic, objective, scope, and limitations of the study. 5

On the other hand, a research hypothesis is an educated statement of an expected outcome. This statement is based on background research and current knowledge. 8 , 9 The research hypothesis makes a specific prediction about a new phenomenon 10 or a formal statement on the expected relationship between an independent variable and a dependent variable. 3 , 11 It provides a tentative answer to the research question to be tested or explored. 4

Hypotheses employ reasoning to predict a theory-based outcome. 10 These can also be developed from theories by focusing on components of theories that have not yet been observed. 10 The validity of hypotheses is often based on the testability of the prediction made in a reproducible experiment. 8

Conversely, hypotheses can also be rephrased as research questions. Several hypotheses based on existing theories and knowledge may be needed to answer a research question. Developing ethical research questions and hypotheses creates a research design that has logical relationships among variables. These relationships serve as a solid foundation for the conduct of the study. 4 , 11 Haphazardly constructed research questions can result in poorly formulated hypotheses and improper study designs, leading to unreliable results. Thus, the formulations of relevant research questions and verifiable hypotheses are crucial when beginning research. 12

CHARACTERISTICS OF GOOD RESEARCH QUESTIONS AND HYPOTHESES

Excellent research questions are specific and focused. These integrate collective data and observations to confirm or refute the subsequent hypotheses. Well-constructed hypotheses are based on previous reports and verify the research context. These are realistic, in-depth, sufficiently complex, and reproducible. More importantly, these hypotheses can be addressed and tested. 13

There are several characteristics of well-developed hypotheses. Good hypotheses are 1) empirically testable 7 , 10 , 11 , 13 ; 2) backed by preliminary evidence 9 ; 3) testable by ethical research 7 , 9 ; 4) based on original ideas 9 ; 5) have evidenced-based logical reasoning 10 ; and 6) can be predicted. 11 Good hypotheses can infer ethical and positive implications, indicating the presence of a relationship or effect relevant to the research theme. 7 , 11 These are initially developed from a general theory and branch into specific hypotheses by deductive reasoning. In the absence of a theory to base the hypotheses, inductive reasoning based on specific observations or findings form more general hypotheses. 10

TYPES OF RESEARCH QUESTIONS AND HYPOTHESES

Research questions and hypotheses are developed according to the type of research, which can be broadly classified into quantitative and qualitative research. We provide a summary of the types of research questions and hypotheses under quantitative and qualitative research categories in Table 1 .

Research questions in quantitative research

In quantitative research, research questions inquire about the relationships among variables being investigated and are usually framed at the start of the study. These are precise and typically linked to the subject population, dependent and independent variables, and research design. 1 Research questions may also attempt to describe the behavior of a population in relation to one or more variables, or describe the characteristics of variables to be measured ( descriptive research questions ). 1 , 5 , 14 These questions may also aim to discover differences between groups within the context of an outcome variable ( comparative research questions ), 1 , 5 , 14 or elucidate trends and interactions among variables ( relationship research questions ). 1 , 5 We provide examples of descriptive, comparative, and relationship research questions in quantitative research in Table 2 .

Hypotheses in quantitative research

In quantitative research, hypotheses predict the expected relationships among variables. 15 Relationships among variables that can be predicted include 1) between a single dependent variable and a single independent variable ( simple hypothesis ) or 2) between two or more independent and dependent variables ( complex hypothesis ). 4 , 11 Hypotheses may also specify the expected direction to be followed and imply an intellectual commitment to a particular outcome ( directional hypothesis ) 4 . On the other hand, hypotheses may not predict the exact direction and are used in the absence of a theory, or when findings contradict previous studies ( non-directional hypothesis ). 4 In addition, hypotheses can 1) define interdependency between variables ( associative hypothesis ), 4 2) propose an effect on the dependent variable from manipulation of the independent variable ( causal hypothesis ), 4 3) state a negative relationship between two variables ( null hypothesis ), 4 , 11 , 15 4) replace the working hypothesis if rejected ( alternative hypothesis ), 15 explain the relationship of phenomena to possibly generate a theory ( working hypothesis ), 11 5) involve quantifiable variables that can be tested statistically ( statistical hypothesis ), 11 6) or express a relationship whose interlinks can be verified logically ( logical hypothesis ). 11 We provide examples of simple, complex, directional, non-directional, associative, causal, null, alternative, working, statistical, and logical hypotheses in quantitative research, as well as the definition of quantitative hypothesis-testing research in Table 3 .

Research questions in qualitative research

Unlike research questions in quantitative research, research questions in qualitative research are usually continuously reviewed and reformulated. The central question and associated subquestions are stated more than the hypotheses. 15 The central question broadly explores a complex set of factors surrounding the central phenomenon, aiming to present the varied perspectives of participants. 15

There are varied goals for which qualitative research questions are developed. These questions can function in several ways, such as to 1) identify and describe existing conditions ( contextual research question s); 2) describe a phenomenon ( descriptive research questions ); 3) assess the effectiveness of existing methods, protocols, theories, or procedures ( evaluation research questions ); 4) examine a phenomenon or analyze the reasons or relationships between subjects or phenomena ( explanatory research questions ); or 5) focus on unknown aspects of a particular topic ( exploratory research questions ). 5 In addition, some qualitative research questions provide new ideas for the development of theories and actions ( generative research questions ) or advance specific ideologies of a position ( ideological research questions ). 1 Other qualitative research questions may build on a body of existing literature and become working guidelines ( ethnographic research questions ). Research questions may also be broadly stated without specific reference to the existing literature or a typology of questions ( phenomenological research questions ), may be directed towards generating a theory of some process ( grounded theory questions ), or may address a description of the case and the emerging themes ( qualitative case study questions ). 15 We provide examples of contextual, descriptive, evaluation, explanatory, exploratory, generative, ideological, ethnographic, phenomenological, grounded theory, and qualitative case study research questions in qualitative research in Table 4 , and the definition of qualitative hypothesis-generating research in Table 5 .

Qualitative studies usually pose at least one central research question and several subquestions starting with How or What . These research questions use exploratory verbs such as explore or describe . These also focus on one central phenomenon of interest, and may mention the participants and research site. 15

Hypotheses in qualitative research

Hypotheses in qualitative research are stated in the form of a clear statement concerning the problem to be investigated. Unlike in quantitative research where hypotheses are usually developed to be tested, qualitative research can lead to both hypothesis-testing and hypothesis-generating outcomes. 2 When studies require both quantitative and qualitative research questions, this suggests an integrative process between both research methods wherein a single mixed-methods research question can be developed. 1

FRAMEWORKS FOR DEVELOPING RESEARCH QUESTIONS AND HYPOTHESES

Research questions followed by hypotheses should be developed before the start of the study. 1 , 12 , 14 It is crucial to develop feasible research questions on a topic that is interesting to both the researcher and the scientific community. This can be achieved by a meticulous review of previous and current studies to establish a novel topic. Specific areas are subsequently focused on to generate ethical research questions. The relevance of the research questions is evaluated in terms of clarity of the resulting data, specificity of the methodology, objectivity of the outcome, depth of the research, and impact of the study. 1 , 5 These aspects constitute the FINER criteria (i.e., Feasible, Interesting, Novel, Ethical, and Relevant). 1 Clarity and effectiveness are achieved if research questions meet the FINER criteria. In addition to the FINER criteria, Ratan et al. described focus, complexity, novelty, feasibility, and measurability for evaluating the effectiveness of research questions. 14

The PICOT and PEO frameworks are also used when developing research questions. 1 The following elements are addressed in these frameworks, PICOT: P-population/patients/problem, I-intervention or indicator being studied, C-comparison group, O-outcome of interest, and T-timeframe of the study; PEO: P-population being studied, E-exposure to preexisting conditions, and O-outcome of interest. 1 Research questions are also considered good if these meet the “FINERMAPS” framework: Feasible, Interesting, Novel, Ethical, Relevant, Manageable, Appropriate, Potential value/publishable, and Systematic. 14

As we indicated earlier, research questions and hypotheses that are not carefully formulated result in unethical studies or poor outcomes. To illustrate this, we provide some examples of ambiguous research question and hypotheses that result in unclear and weak research objectives in quantitative research ( Table 6 ) 16 and qualitative research ( Table 7 ) 17 , and how to transform these ambiguous research question(s) and hypothesis(es) into clear and good statements.

a These statements were composed for comparison and illustrative purposes only.

b These statements are direct quotes from Higashihara and Horiuchi. 16

a This statement is a direct quote from Shimoda et al. 17

The other statements were composed for comparison and illustrative purposes only.

CONSTRUCTING RESEARCH QUESTIONS AND HYPOTHESES

To construct effective research questions and hypotheses, it is very important to 1) clarify the background and 2) identify the research problem at the outset of the research, within a specific timeframe. 9 Then, 3) review or conduct preliminary research to collect all available knowledge about the possible research questions by studying theories and previous studies. 18 Afterwards, 4) construct research questions to investigate the research problem. Identify variables to be accessed from the research questions 4 and make operational definitions of constructs from the research problem and questions. Thereafter, 5) construct specific deductive or inductive predictions in the form of hypotheses. 4 Finally, 6) state the study aims . This general flow for constructing effective research questions and hypotheses prior to conducting research is shown in Fig. 1 .

Research questions are used more frequently in qualitative research than objectives or hypotheses. 3 These questions seek to discover, understand, explore or describe experiences by asking “What” or “How.” The questions are open-ended to elicit a description rather than to relate variables or compare groups. The questions are continually reviewed, reformulated, and changed during the qualitative study. 3 Research questions are also used more frequently in survey projects than hypotheses in experiments in quantitative research to compare variables and their relationships.

Hypotheses are constructed based on the variables identified and as an if-then statement, following the template, ‘If a specific action is taken, then a certain outcome is expected.’ At this stage, some ideas regarding expectations from the research to be conducted must be drawn. 18 Then, the variables to be manipulated (independent) and influenced (dependent) are defined. 4 Thereafter, the hypothesis is stated and refined, and reproducible data tailored to the hypothesis are identified, collected, and analyzed. 4 The hypotheses must be testable and specific, 18 and should describe the variables and their relationships, the specific group being studied, and the predicted research outcome. 18 Hypotheses construction involves a testable proposition to be deduced from theory, and independent and dependent variables to be separated and measured separately. 3 Therefore, good hypotheses must be based on good research questions constructed at the start of a study or trial. 12

In summary, research questions are constructed after establishing the background of the study. Hypotheses are then developed based on the research questions. Thus, it is crucial to have excellent research questions to generate superior hypotheses. In turn, these would determine the research objectives and the design of the study, and ultimately, the outcome of the research. 12 Algorithms for building research questions and hypotheses are shown in Fig. 2 for quantitative research and in Fig. 3 for qualitative research.

EXAMPLES OF RESEARCH QUESTIONS FROM PUBLISHED ARTICLES

• EXAMPLE 1. Descriptive research question (quantitative research)
• - Presents research variables to be assessed (distinct phenotypes and subphenotypes)
• “BACKGROUND: Since COVID-19 was identified, its clinical and biological heterogeneity has been recognized. Identifying COVID-19 phenotypes might help guide basic, clinical, and translational research efforts.
• RESEARCH QUESTION: Does the clinical spectrum of patients with COVID-19 contain distinct phenotypes and subphenotypes? ” 19
• EXAMPLE 2. Relationship research question (quantitative research)
• - Shows interactions between dependent variable (static postural control) and independent variable (peripheral visual field loss)
• “Background: Integration of visual, vestibular, and proprioceptive sensations contributes to postural control. People with peripheral visual field loss have serious postural instability. However, the directional specificity of postural stability and sensory reweighting caused by gradual peripheral visual field loss remain unclear.
• Research question: What are the effects of peripheral visual field loss on static postural control ?” 20
• EXAMPLE 3. Comparative research question (quantitative research)
• - Clarifies the difference among groups with an outcome variable (patients enrolled in COMPERA with moderate PH or severe PH in COPD) and another group without the outcome variable (patients with idiopathic pulmonary arterial hypertension (IPAH))
• “BACKGROUND: Pulmonary hypertension (PH) in COPD is a poorly investigated clinical condition.
• RESEARCH QUESTION: Which factors determine the outcome of PH in COPD?
• STUDY DESIGN AND METHODS: We analyzed the characteristics and outcome of patients enrolled in the Comparative, Prospective Registry of Newly Initiated Therapies for Pulmonary Hypertension (COMPERA) with moderate or severe PH in COPD as defined during the 6th PH World Symposium who received medical therapy for PH and compared them with patients with idiopathic pulmonary arterial hypertension (IPAH) .” 21
• EXAMPLE 4. Exploratory research question (qualitative research)
• - Explores areas that have not been fully investigated (perspectives of families and children who receive care in clinic-based child obesity treatment) to have a deeper understanding of the research problem
• “Problem: Interventions for children with obesity lead to only modest improvements in BMI and long-term outcomes, and data are limited on the perspectives of families of children with obesity in clinic-based treatment. This scoping review seeks to answer the question: What is known about the perspectives of families and children who receive care in clinic-based child obesity treatment? This review aims to explore the scope of perspectives reported by families of children with obesity who have received individualized outpatient clinic-based obesity treatment.” 22
• EXAMPLE 5. Relationship research question (quantitative research)
• - Defines interactions between dependent variable (use of ankle strategies) and independent variable (changes in muscle tone)
• “Background: To maintain an upright standing posture against external disturbances, the human body mainly employs two types of postural control strategies: “ankle strategy” and “hip strategy.” While it has been reported that the magnitude of the disturbance alters the use of postural control strategies, it has not been elucidated how the level of muscle tone, one of the crucial parameters of bodily function, determines the use of each strategy. We have previously confirmed using forward dynamics simulations of human musculoskeletal models that an increased muscle tone promotes the use of ankle strategies. The objective of the present study was to experimentally evaluate a hypothesis: an increased muscle tone promotes the use of ankle strategies. Research question: Do changes in the muscle tone affect the use of ankle strategies ?” 23

EXAMPLES OF HYPOTHESES IN PUBLISHED ARTICLES

• EXAMPLE 1. Working hypothesis (quantitative research)
• - A hypothesis that is initially accepted for further research to produce a feasible theory
• “As fever may have benefit in shortening the duration of viral illness, it is plausible to hypothesize that the antipyretic efficacy of ibuprofen may be hindering the benefits of a fever response when taken during the early stages of COVID-19 illness .” 24
• “In conclusion, it is plausible to hypothesize that the antipyretic efficacy of ibuprofen may be hindering the benefits of a fever response . The difference in perceived safety of these agents in COVID-19 illness could be related to the more potent efficacy to reduce fever with ibuprofen compared to acetaminophen. Compelling data on the benefit of fever warrant further research and review to determine when to treat or withhold ibuprofen for early stage fever for COVID-19 and other related viral illnesses .” 24
• EXAMPLE 2. Exploratory hypothesis (qualitative research)
• - Explores particular areas deeper to clarify subjective experience and develop a formal hypothesis potentially testable in a future quantitative approach
• “We hypothesized that when thinking about a past experience of help-seeking, a self distancing prompt would cause increased help-seeking intentions and more favorable help-seeking outcome expectations .” 25
• “Conclusion
• Although a priori hypotheses were not supported, further research is warranted as results indicate the potential for using self-distancing approaches to increasing help-seeking among some people with depressive symptomatology.” 25
• EXAMPLE 3. Hypothesis-generating research to establish a framework for hypothesis testing (qualitative research)
• “We hypothesize that compassionate care is beneficial for patients (better outcomes), healthcare systems and payers (lower costs), and healthcare providers (lower burnout). ” 26
• Compassionomics is the branch of knowledge and scientific study of the effects of compassionate healthcare. Our main hypotheses are that compassionate healthcare is beneficial for (1) patients, by improving clinical outcomes, (2) healthcare systems and payers, by supporting financial sustainability, and (3) HCPs, by lowering burnout and promoting resilience and well-being. The purpose of this paper is to establish a scientific framework for testing the hypotheses above . If these hypotheses are confirmed through rigorous research, compassionomics will belong in the science of evidence-based medicine, with major implications for all healthcare domains.” 26
• EXAMPLE 4. Statistical hypothesis (quantitative research)
• - An assumption is made about the relationship among several population characteristics ( gender differences in sociodemographic and clinical characteristics of adults with ADHD ). Validity is tested by statistical experiment or analysis ( chi-square test, Students t-test, and logistic regression analysis)
• “Our research investigated gender differences in sociodemographic and clinical characteristics of adults with ADHD in a Japanese clinical sample. Due to unique Japanese cultural ideals and expectations of women's behavior that are in opposition to ADHD symptoms, we hypothesized that women with ADHD experience more difficulties and present more dysfunctions than men . We tested the following hypotheses: first, women with ADHD have more comorbidities than men with ADHD; second, women with ADHD experience more social hardships than men, such as having less full-time employment and being more likely to be divorced.” 27
• “Statistical Analysis
• ( text omitted ) Between-gender comparisons were made using the chi-squared test for categorical variables and Students t-test for continuous variables…( text omitted ). A logistic regression analysis was performed for employment status, marital status, and comorbidity to evaluate the independent effects of gender on these dependent variables.” 27

EXAMPLES OF HYPOTHESIS AS WRITTEN IN PUBLISHED ARTICLES IN RELATION TO OTHER PARTS

• EXAMPLE 1. Background, hypotheses, and aims are provided
• “Pregnant women need skilled care during pregnancy and childbirth, but that skilled care is often delayed in some countries …( text omitted ). The focused antenatal care (FANC) model of WHO recommends that nurses provide information or counseling to all pregnant women …( text omitted ). Job aids are visual support materials that provide the right kind of information using graphics and words in a simple and yet effective manner. When nurses are not highly trained or have many work details to attend to, these job aids can serve as a content reminder for the nurses and can be used for educating their patients (Jennings, Yebadokpo, Affo, & Agbogbe, 2010) ( text omitted ). Importantly, additional evidence is needed to confirm how job aids can further improve the quality of ANC counseling by health workers in maternal care …( text omitted )” 28
• “ This has led us to hypothesize that the quality of ANC counseling would be better if supported by job aids. Consequently, a better quality of ANC counseling is expected to produce higher levels of awareness concerning the danger signs of pregnancy and a more favorable impression of the caring behavior of nurses .” 28
• “This study aimed to examine the differences in the responses of pregnant women to a job aid-supported intervention during ANC visit in terms of 1) their understanding of the danger signs of pregnancy and 2) their impression of the caring behaviors of nurses to pregnant women in rural Tanzania.” 28
• EXAMPLE 2. Background, hypotheses, and aims are provided
• “We conducted a two-arm randomized controlled trial (RCT) to evaluate and compare changes in salivary cortisol and oxytocin levels of first-time pregnant women between experimental and control groups. The women in the experimental group touched and held an infant for 30 min (experimental intervention protocol), whereas those in the control group watched a DVD movie of an infant (control intervention protocol). The primary outcome was salivary cortisol level and the secondary outcome was salivary oxytocin level.” 29
• “ We hypothesize that at 30 min after touching and holding an infant, the salivary cortisol level will significantly decrease and the salivary oxytocin level will increase in the experimental group compared with the control group .” 29
• EXAMPLE 3. Background, aim, and hypothesis are provided
• “In countries where the maternal mortality ratio remains high, antenatal education to increase Birth Preparedness and Complication Readiness (BPCR) is considered one of the top priorities [1]. BPCR includes birth plans during the antenatal period, such as the birthplace, birth attendant, transportation, health facility for complications, expenses, and birth materials, as well as family coordination to achieve such birth plans. In Tanzania, although increasing, only about half of all pregnant women attend an antenatal clinic more than four times [4]. Moreover, the information provided during antenatal care (ANC) is insufficient. In the resource-poor settings, antenatal group education is a potential approach because of the limited time for individual counseling at antenatal clinics.” 30
• “This study aimed to evaluate an antenatal group education program among pregnant women and their families with respect to birth-preparedness and maternal and infant outcomes in rural villages of Tanzania.” 30
• “ The study hypothesis was if Tanzanian pregnant women and their families received a family-oriented antenatal group education, they would (1) have a higher level of BPCR, (2) attend antenatal clinic four or more times, (3) give birth in a health facility, (4) have less complications of women at birth, and (5) have less complications and deaths of infants than those who did not receive the education .” 30

Research questions and hypotheses are crucial components to any type of research, whether quantitative or qualitative. These questions should be developed at the very beginning of the study. Excellent research questions lead to superior hypotheses, which, like a compass, set the direction of research, and can often determine the successful conduct of the study. Many research studies have floundered because the development of research questions and subsequent hypotheses was not given the thought and meticulous attention needed. The development of research questions and hypotheses is an iterative process based on extensive knowledge of the literature and insightful grasp of the knowledge gap. Focused, concise, and specific research questions provide a strong foundation for constructing hypotheses which serve as formal predictions about the research outcomes. Research questions and hypotheses are crucial elements of research that should not be overlooked. They should be carefully thought of and constructed when planning research. This avoids unethical studies and poor outcomes by defining well-founded objectives that determine the design, course, and outcome of the study.

Disclosure: The authors have no potential conflicts of interest to disclose.

Author Contributions:

• Conceptualization: Barroga E, Matanguihan GJ.
• Methodology: Barroga E, Matanguihan GJ.
• Writing - original draft: Barroga E, Matanguihan GJ.
• Writing - review & editing: Barroga E, Matanguihan GJ.

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

3.1 What is Quantitative Research?

Quantitative research is a research method that uses numerical data and statistical analysis to study phenomena. 1 Quantitative research plays an important role in scientific inquiry by providing a rigorous, objective, systematic process using numerical data to test relationships and examine cause-and-effect associations between variables. 1, 2 The goal is to make generalisations about a population (extrapolate findings from the sample to the general population). 2 The data and variables are predetermined and measured as consistently and accurately as possible, and statistical analysis is used to evaluate the outcomes. 2 Quantitative research is based on the scientific method, wherein deductive reductionist reasoning is used to formulate hypotheses about a particular phenomenon.

An Introduction to Research Methods for Undergraduate Health Profession Students Copyright © 2023 by Faith Alele and Bunmi Malau-Aduli is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License , except where otherwise noted.

Key Concepts in Quantitative Research

In this module, we are going to explore the nuances of quantitative research, including the main types of quantitative research, more exploration into variables (including confounding and extraneous variables), and causation.

Content includes:

• Flaws, “Proof”, and Rigor
• The Steps of Quantitative Methodology
• Major Classes of Quantitative Research
• Experimental versus Non-Experimental Research
• Types of Experimental Research
• Types of Non-Experimental Research
• Research Variables
• Confounding/Extraneous Variables
• Causation versus correlation/association

Objectives:

• Discuss the flaws, proof, and rigor in research.
• Describe the differences between independent variables and dependent variables.
• Describe the steps in quantitative research methodology.
• Describe experimental, quasi-experimental, and non-experimental research studies
• Describe confounding and extraneous variables.
• Differentiate cause-and-effect (causality) versus association/correlation

Flaws, Proof, and Rigor in Research

One of the biggest hurdles that students and seasoned researchers alike struggle to grasp, is that research cannot “ prove ” nor “ disprove ”. Research can only support a hypothesis with reasonable, statistically significant evidence.

Indeed. You’ve heard it incorrectly your entire life. You will hear professors, scientists, radio ads, podcasts, and even researchers comment something to the effect of, “It has been proven that…” or “Research proves that…” or “Finally! There is proof that…”

We have been duped. Consider the “ prove ” word a very bad word in this course. The forbidden “P” word. Do not say it, write it, allude to it, or repeat it. And, for the love of avocados and all things fluffy, do not include the “P” word on your EBP poster. You will be deducted some major points.

We can only conclude with reasonable certainty through statistical analyses that there is a high probability that something did not happen by chance but instead happened due to the intervention that the researcher tested. Got that? We will come back to that concept but for now know that it is called “statistical significance”.

All research has flaws. We might not know what those flaws are, but we will be learning about confounding and extraneous variables later on in this module to help explain how flaws can happen.

Remember this: Sometimes, the researcher might not even know that there was a flaw that occurred. No research project is perfect. There is no 100% awesome. This is a major reason why it is so important to be able to duplicate a research project and obtain similar results. The more we can duplicate research with the same exact methodology and protocols, the more certainty we have in the results and we can start accounting for flaws that may have sneaked in.

Finally, not all research is equal. Some research is done very sloppily, and other research has a very high standard of rigor. How do we know which is which when reading an article? Well, within this module, we will start learning about some things to look for in a published research article to help determine rigor. We do not want lazy research to determine our actions as nurses, right? We want the strongest, most reliable, most valid, most rigorous research evidence possible so that we can take those results and embed them into patient care. Who wants shoddy evidence determining the actions we take with your grandmother’s heart surgery?

Independent Variables and Dependent Variables

As we were already introduced to, there are measures called “variables” in research. This will be a bit of a review but it is important to bring up again, as it is a hallmark of quantitative research. In quantitative studies, the concepts being measured are called variables (AKA: something that varies). Variables are something that can change – either by manipulation or from something causing a change. In the article snapshots that we have looked at, researchers are trying to find causes for phenomena. Does a nursing intervention cause an improvement in patient outcomes? Does the cholesterol medication cause a decrease in cholesterol level? Does smoking cause  cancer?

The presumed cause is called the independent variable. The presumed effect is called the dependent variable. The dependent variable is “dependent” on something causing it to change. The dependent variable is the outcome that a researcher is trying to understand, explain, or predict.

Think back to our PICO questions. You can think of the intervention (I) as the independent variable and the outcome (O) as the dependent variable.

The independent variable is manipulated by the researcher or can be variants of influence. Whereas the dependent variable is never manipulated.

Variables do not always measure cause-and-effect. They can also measure a direction of influence.

Here is an example of that: If we compared levels of depression among men and women diagnosed with pancreatic cancer and found men to be more depressed, we cannot conclude that depression was caused by gender. However, we can note that the direction of influence   clearly runs from gender to depression. It makes no sense to suggest the depression influenced their gender.

In the above example, what is the independent variable (IV) and what is the dependent variable (DV)? If you guessed gender as the IV and depression as the DV, you are correct! Important to note in this case that the researcher did not manipulate the IV, but the IV is manipulated on its own (male or female).

Researchers do not always have just one IV. In some cases, more than one IV may be measured. Take, for instance, a study that wants to measure the factors that influence one’s study habits. Independent variables of gender, sleep habits, and hours of work may be considered. Likewise, multiple DVs can be measured. For example, perhaps we want to measure weight and abdominal girth on a plant-based diet (IV).

Now, some studies do not have an intervention. We will come back to that when we talk about non-experimental research.

The point of variables is so that researchers have a very specific measurement that they seek to study.

Let’s look at a couple of examples:

Now you try! Identify the IVs and DVs:

IV and DV Case Studies (Leibold, 2020)

Case Three:   Independent variable: Healthy Lifestyle education with a focus on physical activity; Dependent variable: Physical activity rate before and after education intervention, Heart rate before and after education intervention, Blood pressures before and after education intervention.

Case Four:   Independent variable: Playing classical music; Dependent variable:  Grade point averages post classical music, compared to pre-classical music.

Case Five: Independent variable: No independent variable as there is no intervention.  Dependent variable: The themes that emerge from the qualitative data.

The Steps in Quantitative Research Methodology

Now, as we learned in the last module, quantitative research is completely objective. There is no subjectivity to it. Why is this? Well, as we have learned, the purpose of quantitative research is to make an inference about the results in order to generalize these results to the population.

In quantitative studies, there is a very systematic approach that moves from the beginning point of the study (writing a research question) to the end point (obtaining an answer). This is a very linear and purposeful flow across the study, and all quantitative research should follow the same sequence.

• Identifying a problem and formulating a research question . Quantitative research begins with a theory . As in, “something is wrong and we want to fix it or improve it”.  Think back to when we discussed research problems and formulating a research question. Here we are! That is the first step in formulating a quantitative research plan.
• Formulate a hypothesis . This step is key. Researchers need to know exactly what they are testing so that testing the hypothesis can be achieved through specific statistical analyses.
• A thorough literature review .  At this step, researchers strive to understand what is already known about a topic and what evidence already exists.
• Identifying a framework .  When an appropriate framework is identified, the findings of a study may have broader significance and utility (Polit & Beck, 2021).
• Choosing a study design . The research design will determine exactly how the researcher will obtain the answers to the research question(s). The entire design needs to be structured and controlled, with the overarching goal of minimizing bias and errors. The design determines what data will be collected and how, how often data will be collected, what types of comparisons will be made. You can think of the study design as the architectural backbone of the entire study.
• Sampling . The researcher needs to determine a subset of the population that is to be studied. We will come back to the sampling concept in the next module. However, the goal of sampling is to choose a subset of the population that adequate reflects the population of interest.
• I nstruments to be used to collect data (with reliability and validity as a priority). Researchers must find a way to measure the research variables (intervention and outcome) accurately. The task of measuring is complex and challenging, as data needs to be collected reliably (measuring consistently each time) and valid. Reliability and validity are both about how well a method measures something. The next module will cover this in detail.
• Obtaining approval for ethical/legal human rights procedures . As we will learn in an upcoming module, there needs to be methods in place to safeguard human rights.
• Data collection . The fun part! Finally, after everything has been organized and planned, the researcher(s) begin to collect data. The pre-established plan (methodology) determines when data collection begins, how to accomplish it, how data collection staff will be trained, and how data will be recorded.
• Data analysis . Here comes the statistical analyses. The next module will dive into this.
• Discussion . After all the analyses have been complete, the researcher then needs to interpret the results and examine the implications. Researchers attempt to explain the findings in light of the theoretical framework, prior evidence, theory, clinical experience, and any limitations in the study now that it has been completed. Often, the researcher discusses not just the statistical significance, but also the clinical significance, as it is common to have one without the other.
• Summary/references . Part of the final steps of any research project is to disseminate (AKA: share) the findings. This may be in a published article, conference, poster session, etc. The point of this step is to communicate to others the information found through the study.  All references are collected so that the researchers can give credit to others.
• Budget and funding . As a last mention in the overall steps, budget and funding for research is a consideration. Research can be expensive. Often, researchers can obtain a grant or other funding to help offset the costs.

Edit: Steps in Quantitative Research video. Step 12 should say “Dissemination” (sharing the results).

Experimental, Quasi-Experimental, and Non-Experimental Studies

To start this section, please watch this wonderful video by Jenny Barrow, MSN, RN, CNE, that explains experimental versus nonexperimental research.

(Jenny Barrow, 2019)

Now that you have that overview, continue reading this module.

Experimental Research : In experimental research, the researcher is seeking to draw a conclusion between an independent variable and a dependent variable. This design attempts to establish cause-effect relationships among the variables. You could think of experimental research as experimenting with “something” to see if it caused “something else”.

A true experiment is called a Randomized Controlled Trial (or RCT). An RCT is at the top of the echelon as far as quantitative experimental research. It’s the gold standard of scientific research. An RCT, a true experimental design, must have 3 features:

• An intervention : The experiment does something to the participants by the option of manipulating the independent variable.
• Control : Some participants in the study receive either the standard care, or no intervention at all. This is also called the counterfactual – meaning, it shows what would happen if no intervention was introduced.
• Randomization : Randomization happens when the researcher makes sure that it is completely random who receives the intervention and who receives the control. The purpose is to make the groups equal regarding all other factors except receipt of the intervention.

Note: There is a lot of confusion with students (and even some researchers!) when they refer to “ random assignment ” versus “ random sampling ”. Random assignment  is a signature of a true experiment. This means that if participants are not truly randomly assigned to intervention groups, then it is not a true experiment. We will talk more about random sampling in the next module.

One very common method for RCT’s is called a pretest-posttest design .  This is when the researcher measures the outcome before and after the intervention. For example, if the researcher had an IV (intervention/treatment) of a pain medication, the DV (pain) would be measured before the intervention is given and after it is given. The control group may just receive a placebo. This design permits the researcher to see if the change in pain was caused by the pain medication because only some people received it (Polit & Beck, 2021).

Another experimental design is called a crossover design . This type of design involves exposing participants to more than one treatment. For example, subject 1 first receives treatment A, then treatment B, then treatment C. Subject 2 might first receive treatment B, then treatment A, and then treatment C. In this type of study, the three conditions for an experiment are met: Intervention, randomization, and control – with the subjects serving as their own control group.

Control group conditions can be done in 4 ways:

• No intervention is used; control group gets no treatment at all
• “Usual care” or standard of care or normal procedures used
• An alternative intervention is uses (e.g. auditory versus visual stimulation)
• A placebo or pseudo-intervention, presumed to have no therapeutic value, is used

Quasi-Experimental Research : Quasi-experiments involve an experiment just like true experimental research. However, they lack randomization and some even lack a control group.  Therefore, there is implementation and testing of an intervention, but there is an absence of randomization.

For example, perhaps we wanted to measure the effect of yoga for nursing students. The IV (intervention of yoga) is being offered to all nursing students and therefore randomization is not possible. For comparison, we could measure quality of life data on nursing students at a different university. Data is collected from both groups at baseline and then again after the yoga classes. Note, that in quasi-experiments, the phrase “comparison group” is sometimes used instead of “control group” against which outcome measures are collected.

Sometimes there is no comparison group either. This would be called a one-group pretest-posttest design .

Non-Experimental Research : Sometimes, cause-problem research questions cannot be answered with an experimental or quasi-experimental design because the IV cannot be manipulated. For example, if we want to measure what impact prerequisite grades have on student success in nursing programs, we obviously cannot manipulate the prerequisite grades. In another example, if we wanted to investigate how low birth weight impacts developmental progression in children, we cannot manipulate the birth weight. Often, you will see the word “observational” in lieu of non-experimental researcher. This does not mean the researcher is just standing and watching people, but instead it refers to the method of observing data that has already been established without manipulation.

There are various types of non-experimental research:

Correlational research : A correlational research design investigates relationships between two variables (or more) without the researcher controlling or manipulating any of them. In the example of prerequisites and nursing program success, that is a correlational design. Consider hypothetically, a researcher is studying a correlation between cancer and marriage. In this study, there are two variables: disease and marriage. Let us say marriage has a negative association with cancer. This means that married people are less likely to develop cancer.

Cohort design (also called a prospective design) : In a cohort study, the participants do not have the outcome of interest to begin with. They are selected based on the exposure status of the individual. They are then followed over time to evaluate for the occurrence of the outcome of interest. Cohorts may be divided into exposure categories once baseline measurements of a defined population are made. For example, the Framingham Cardiovascular Disease Study (CVD) used baseline measurements to divide the population into categories of CVD risk factors. Another example:  An example of a cohort study is comparing the test scores of one group of people who underwent extensive tutoring and a special curriculum and those who did not receive any extra help. The group could be studied for years to assess whether their scores improve over time and at what rate.

Retrospective design : In retrospective studies, the outcome of interest has already occurred (or not occurred – e.g., in controls) in each individual by the time s/he is enrolled, and the data are collected either from records or by asking participants to recall exposures. There is no follow-up of participants. For example, a researcher might examine the medical histories of 1000 elderly women to identify the causes of health problems.

Case-control design : A study that compares two groups of people: those with the disease or condition under study (cases) and a very similar group of people who do not have the condition. For example, investigators conducted a case-control study to determine if there is an association between colon cancer and a high fat diet. Cases were all confirmed colon cancer cases in North Carolina in 2010. Controls were a sample of North Carolina residents without colon cancer.

Descriptive research : Descriptive research design is a type of research design that aims to obtain information to systematically describe a phenomenon, situation, or population. More specifically, it helps answer the what, when, where, and how questions regarding the research problem, rather than the why. For example, the researcher might wish to discover the percentage of motorists who tailgate – the prevalence  of a certain behavior.

There are two other designs to mention, which are both on a time continuum basis.

Cross-sectional design : All data are collected at a single point in time. Retrospective studies are usually cross-sectional. The IV usually concerns events or behaviors occurring in the past. One cross-sectional study example in medicine is a data collection of smoking habits and lung cancer incidence in a given population. A cross-sectional study like this cannot solely determine that smoking habits cause lung cancer, but it can suggest a relationship that merits further investigation. Cross-sectional studies serve many purposes, and the cross-sectional design is the most relevant design when assessing the prevalence of disease, attitudes and knowledge among patients and health personnel, in validation studies comparing, for example, different measurement instruments, and in reliability studies.

Longitudinal design : Data are collected two or more times over an extended period. Longitudinal designs are better at showing patterns of change and at clarifying whether a cause occurred before an effect (outcome). A challenge in longitudinal studies is attrition or the loss of participants over time. In a longitudinal study subjects are followed over time with continuous or repeated monitoring of risk factors or health outcomes, or both. Such investigations vary enormously in their size and complexity. At one extreme a large population may be studied over decades. An example of a longitudinal design is a multiyear comparative study of the same children in an urban and a suburban school to record their cognitive development in depth.

Confounding and Extraneous Variables

Confounding variables  are a type of extraneous variable that occur which interfere with or influence the relationship between the independent and dependent variables. In research that investigates a potential cause-and-effect relationship, a confounding variable is an unmeasured third variable that influences both the supposed cause and the supposed effect.

It’s important to consider potential confounding variables and account for them in research designs to ensure results are valid. You can imagine that if something sneaks in to influence the measured variables, it can really muck up the study!

Here is an example:

You collect data on sunburns and ice cream consumption. You find that higher ice cream consumption is associated with a higher probability of sunburn. Does that mean ice cream consumption causes sunburn?

Here, the confounding variable is temperature: hot temperatures cause people to both eat more ice cream and spend more time outdoors under the sun, resulting in more sunburns.

To ensure the internal validity of research, the researcher must account for confounding variables. If he/she fails to do so, the results may not reflect the actual relationship between the variables that they are interested in.

For instance, they may find a cause-and-effect relationship that does not actually exist, because the effect they measure is caused by the confounding variable (and not by the independent variable).

Here is another example:

The researcher finds that babies born to mothers who smoked during their pregnancies weigh significantly less than those born to non-smoking mothers. However, if the researcher does not account for the fact that smokers are more likely to engage in other unhealthy behaviors, such as drinking or eating less healthy foods, then he/she might overestimate the relationship between smoking and low birth weight.

Extraneous variables are any variables that the researcher is not investigating that can potentially affect the outcomes of the research study. If left uncontrolled, extraneous variables can lead to inaccurate conclusions about the relationship between IVs and DVs.

Extraneous variables can threaten the internal validity of a study by providing alternative explanations for the results. In an experiment, the researcher manipulates an independent variable to study its effects on a dependent variable.

In a study on mental performance, the researcher tests whether wearing a white lab coat, the independent variable (IV), improves scientific reasoning, the dependent variable (DV).

Students from a university are recruited to participate in the study. The researcher manipulates the independent variable by splitting participants into two groups:

• Participants in the experimental   group are asked to wear a lab coat during the study.
• Participants in the control group are asked to wear a casual coat during the study.

All participants are given a scientific knowledge quiz, and their scores are compared between groups.

When extraneous variables are uncontrolled, it’s hard to determine the exact effects of the independent variable on the dependent variable, because the effects of extraneous variables may mask them.

Uncontrolled extraneous variables can also make it seem as though there is a true effect of the independent variable in an experiment when there’s actually none.

In the above experiment example, these extraneous variables can affect the science knowledge scores:

• Participant’s major (e.g., STEM or humanities)
• Participant’s interest in science
• Demographic variables such as gender or educational background
• Time of day of testing
• Experiment environment or setting

If these variables systematically differ between the groups, you can’t be sure whether your results come from your independent variable manipulation or from the extraneous variables.

In summary, an extraneous variable is anything that could influence the dependent variable. A confounding variable influences the dependent variable, and also correlates with or causally affects the independent variable.

Cause-and-Effect (Causality) Versus Association/Correlation  

A very important concept to understand is cause-and-effect, also known as causality, versus correlation. Let’s look at these two concepts in very simplified statements. Causation means that one thing caused  another thing to happen. Correlation means there is some association between the two thing we are measuring.

It would be nice if it were as simple as that. These two concepts can indeed by confused by many. Let’s dive deeper.

Two or more variables are considered to be related or associated, in a statistical context, if their values change so that as the value of one variable increases or decreases so does the value of the other variable (or the opposite direction).

For example, for the two variables of “hours worked” and “income earned”, there is a relationship between the two if the increase in hours is associated with an increase in income earned.

However, correlation is a statistical measure that describes the size and direction of a relationship between two or more variables. A correlation does not automatically mean that the change in one variable caused the change in value in the other variable.

Theoretically, the difference between the two types of relationships is easy to identify — an action or occurrence can cause another (e.g. smoking causes an increase in the risk of developing lung cancer), or it can correlate with another (e.g. smoking is correlated with alcoholism, but it does not cause alcoholism). In practice, however, it remains difficult to clearly establish cause and effect, compared with establishing correlation.

Simplified in this image, we can say that hot and sunny weather causes an increase in ice cream consumption. Similarly, we can demise that hot and sunny weather increases the incidence of sunburns. However, we cannot say that ice cream caused a sunburn (or that a sunburn increases consumption of ice cream). It is purely coincidental. In this example, it is pretty easy to anecdotally surmise correlation versus causation. However, in research, we have statistical tests that help researchers differentiate via specialized analyses.

Here is a great Khan Academy video of about 5 minutes that shows a worked example of correlation versus causation with regard to sledding accidents and frostbite cases:

https://www.khanacademy.org/test-prep/praxis-math/praxis-math-lessons/gtp–praxis-math–lessons–statistics-and-probability/v/gtp–praxis-math–video–correlation-and-causation

References & Attribution

“ Light bulb doodle ” by rawpixel licensed CC0 .

“ Magnifying glass ” by rawpixel licensed CC0

“ Orange flame ” by rawpixel licensed CC0 .

Jenny Barrow. (2019). Experimental versus nonexperimental research. https://www.youtube.com/watch?v=FJo8xyXHAlE

Leibold, N. (2020). Research variables. Measures and Concepts Commonly Encountered in EBP. Creative Commons License: BY NC

Polit, D. & Beck, C. (2021).  Lippincott CoursePoint Enhanced for Polit’s Essentials of Nursing Research  (10th ed.). Wolters Kluwer Health.

Evidence-Based Practice & Research Methodologies Copyright © by Tracy Fawns is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Chapter 1: Introduction to Research Methods

Learning Objectives

At the end of this chapter, you will be able to:

• Define the term “research methods”.
• List the nine steps in undertaking a research project.
• Differentiate between applied and basic research.
• Explain where research ideas come from.
• Define ontology and epistemology and explain the difference between the two.
• Identify and describe five key research paradigms in social sciences.
• Differentiate between inductive and deductive approaches to research.

Welcome to Introduction to Research Methods. In this textbook, you will learn why research is done and, more importantly, about the methods researchers use to conduct research. Research comes in many forms and, although you may feel that it has no relevance to you and/ or that you know nothing about it, you are exposed to research multiple times a day. You also undertake research yourself, perhaps without even realizing it. This course will help you to understand the research you are exposed to on a daily basis, and how to be more critical of the research you read and use in your own life and career.

This text is intended as an introduction. A plethora of resources exists related to more detailed aspects of conducting research; it is not our intention to replace any of these more comprehensive resources. Keep notes and build your own reading list of articles as you go through the course. Feedback helps to improve this open-source textbook, and is appreciated in the development of the resource.

Research Methods, Data Collection and Ethics Copyright © 2020 by Valerie Sheppard is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

The Structure of Quantitative Studies

• First Online: 10 February 2022

Cite this chapter

• Charles P. Friedman 4 ,
• Jeremy C. Wyatt 5 &
• Joan S. Ash 6  

Part of the book series: Health Informatics ((HI))

1395 Accesses

This chapter introduces the methods of quantitative studies. It describes the process of measurement, which is fundamental to all quantitative methods, and then offers an important distinction between measurement and demonstration studies. The chapter concludes with a description of three types of demonstration studies: descriptive, interventional, and correlational.

Download chapter PDF

• Measurement studies
• Demonstration studies
• Descriptive studies
• Interventional studies
• Correlational studies

The text, examples, and self-tests in this chapter will enable the reader to:

For a given measurement process, identify the attribute being measured, the object class, the instruments being employed, and what constitutes an independent observation.

For a given measurement process, identify of the level of measurement (nominal, ordinal, interval, ratio) of the attribute(s) included in the measurement process.

Explain why measurement is fundamental to the credibility of quantitative studies and why, according to the fundamental precepts of quantitative methods, even the most abstract constructs can be measured objectively.

Given a study design, classify it as primarily a measurement or demonstration study.

Given a demonstration study design, identify the categories of subjects/participants as well as the independent and dependent variables.

Given a demonstration study design, classify the study as prospective or retrospective and also as descriptive, interventional, or correlational.

1 Introduction

What the famed epidemiologist Alvin Feinstein wrote in 1987 still rings true today and sets the tone for this chapter.

Important human and clinical phenomena are regularly omitted when patient care is . . . analyzed in statistical comparisons of therapy. The phenomena are omitted either because they lack formal expressions to identify them or because the available expressions are regarded as scientifically unacceptable. (Feinstein 1987 )

This chapter begins the exploration of quantitative studies in detail. Chapters 6 through 13 address the design of studies, along with how to develop measurement procedures to collect data and how subsequently to analyze the data collected. The methods introduced relate directly to the comparison-based, objectives-based, and decision-facilitation approaches to evaluation described in Chap. 2 . They are useful for addressing most of the purposes of evaluation in informatics, the specific questions that can be explored, and the types of studies that can be undertaken—all as introduced in Chap. 3 .

More specifically, this chapter introduces a conceptual framework to guide the design and conduct of quantitative studies. The framework employs terminology; and the terminology introduced in this chapter is used consistently in the chapters on quantitative studies that follow. Much of this terminology may already be familiar, but some of these familiar terms are likely used here in ways that are novel. Unfortunately, there is no single accepted terminology for describing the structure of quantitative studies. Epidemiologists, behavioral and social scientists, computer and information scientists, and statisticians have developed their own unique variations. The terminology introduced here is a composite that makes sense for the hybrid field of informatics. Importantly, the terms introduced here are more than just labels. They represent concepts that are central to understanding the structure of quantitative studies and, ultimately, to their design and conduct.

A major theme of this chapter, and indeed all eight chapters on quantitative studies, is the importance of measurement. Three of the chapters in this group focus explicitly on measurement because many of the major problems to be overcome in qualitative study design are, at their core, problems of measurement. Measurement issues are also stressed here because they are often overlooked in research methods courses based in other disciplines.

After introducing the concept of measurement along with some related terminology, this chapter formally establishes the distinction between:

measurement studies designed to explore with how well “quantities of interest” in informatics can actually be measured; and

demonstration studies , which apply these measurement procedures to address evaluation questions of substantive and practical concern.

This discussion will make clear that demonstration studies are what study teams ultimately want to do, but measurement studies sometimes are an initial step required before successful demonstration studies are possible.

The distinction between measurement and demonstration studies is more than academic. In the informatics literature, it appears that measurement issues usually are embedded in, and often confounded with, demonstration issues. For example, a recent review of quantitative studies of clinical decision support systems revealed that only 47 of 391 published studies paid explicit attention to issues of measurement (Scott et al. 2019 ). This matter is of substantial significance because, as will be seen in Chap. 8 (Section 8.5 ), deficiencies in measurement can profoundly affect the conclusions drawn from demonstration studies. The quotation that begins this chapter alerts us to the fact that ability to investigate is shaped by ability to measure. Unless study teams possess or can develop ways to measure what is important to know about information resources and the people who use them, their ability to conduct evaluation studies—at least those using quantitative approaches—is substantially limited.

This chapter continues with a discussion of demonstration studies, seeking to make clear how measurement and demonstration studies differ. This discussion introduces some differences in terminology that arise between the two types of studies. The discussion continues by introducing categories of demonstration studies that are important in matching study designs to the study questions that guide them.

2 Elements of a Measurement Process

This section introduces some general rules, definitions, and synonyms that relate to the process of measurement. As noted earlier, these definitions may use some familiar words in unfamiliar ways. The process of measurement and the interrelations of the terms to be defined are illustrated in Fig. 6.1 .

The process of measurement

2.1 Measurement

Measurement is the process of assigning a value corresponding to the presence, absence, or degree of presence, of a specific attribute in a specific object. The terms “attribute” and “object” are defined below. Measurement results in either: (1) assignment of a numerical score representing the extent to which the attribute of interest is present in the object, or (2) assignment of an object to a specific category. Taking the temperature (attribute) of a person (object) is an example of the process of measurement.

2.2 Attribute

An attribute is what is being measured. Speed (of an information resource), blood pressure (of a person), the correct diagnosis (of a diseased patient), the number of new patient admissions per day (in a hospital), the number of kilobases (in a strand of DNA), and computer literacy (of a person) are examples of attributes that are pertinent within informatics.

2.3 Object and Object Class

The object is the entity on which the measurement is made. In the examples of attributes in the section above, the object that pairs with an attribute is given in parentheses. It is useful to think of an attribute as a property of an object, and conversely, as an object as the “holder” or “carrier” of an attribute.

In some cases, it is possible to make measurements directly on the object itself; for example, when measuring the height of a person or the weight of a physical object. In other cases, the measurements are made on some representation of the object. For example, when measuring quality of performance (an attribute) of a procedure (the object), the actual measurement may be made via a recording of the procedure rather than a direct, real-time observation of it.

It is also useful to think of each object as a member of a class. Each act of measurement is performed on an individual object, which is a member of the class. Information resources, persons (who can be patients, care providers, students, and researchers), groups of persons, and organizations (healthcare and academic) are important examples of object classes in informatics on which measurements are frequently made.

2.4 Attribute–Object Class Pairs

Having defined an attribute as a property of a specific object that is a member of a class, measurement processes can be framed in terms of paired attributes and object classes. Table 6.1 illustrates this pairing of attributes and object classes for the examples discussed above. It is important to be able to analyze any given measurement process by identifying the pertinent attribute and object class. To do this, certain questions might be asked. To identify the attribute , the questions might be: What is being measured? What will the result of the measurement be called? To identify the object class , the question might be: On whom or on what is the measurement made? There is a worked example followed by Self-test 6.1 , later in this chapter, immediately after Sect. 6.2.7 . Together, these will expand your knowledge of these important concepts.

2.5 Attributes as “Constructs”

All attributes—that is, all things that are measured--are abstractions. This may be difficult to imagine because some of the attributes routinely measured in day-to-day life, especially properties of physical objects such as size and weight, are so intuitive that they become almost labels permanently attached to these objects. Seeing a tall person, we instinctively say, “That person must be 2 meters tall”. But in point of fact, the height (attribute) of that person (object) is not known until it is measured. Using imagination, it is possible to conceive of a society where height was unimportant, so it went unmeasured; and moreover, the concept of “height” might not even exist in the minds of the people of that society.

For this reason, attributes are sometimes referred to as “constructs” in part as a reminder that attributes of objects, that are measured, are not an indigenous part of the physical world but rather products of human ingenuity (Binning 2016 ). Over time, each scientific field develops a set of constructs, “attributes worth measuring,” that become part of the culture and discovery methods of that field. Study teams may tend to view the attributes that are part of their field’s research tradition as a persistent part of the landscape and fail to recognize that, at some earlier point in history, these concepts were unknown. Blood pressure, for example, had no meaning to humankind until circulation was understood. Computer literacy is a more recent construct stimulated by contemporary technological developments. Indeed, many of the most creative works of science propose completely new constructs, develop methods to measure them, and subsequently demonstrate their value in describing or predicting phenomena of interest.

Because informatics is a “people science”, many studies in informatics address human behavior and the abstract states of mind (knowledge, attitudes, beliefs) that are presumed to shape this behavior. To perform quantitative studies, these abstract states of mind must be formulated as constructs and measured “objectively” in accord with the philosophical precepts discussed in Sect. 2.5 . The behavioral, social, and decision sciences have contributed specific methods that enable measurement of such attributes and use of them in studies. A key idea for this chapter, that may be counterintuitive to those trained in the physical sciences, is that “states of mind” which cannot be directly observed with human senses, can still be measured objectively and employed with rigor in quantitative studies.

2.6 Measurement Instruments

An instrument is the technology used for measurement. The “instrument” encodes and embodies the procedures used to determine the presence, absence, or extent of an attribute in an object. The instrument used in a measurement process follows from the attribute being measured and the object on which the measurement is being made. A diverse array of instruments exists to support studies in informatics. These include questionnaires to measure attitudes of people, image acquisition devices to determine the presence or absence of disease, forms to record appraisals of the observed performance of individuals and groups, and software that automatically records aspects of information resource performance. Software can also function as instruments by computing the values of attributes, such as care quality measures, from EHRs and other health data repositories.

While many types of measurements in informatics can be made by electro-mechanical and computational instruments, other types of measurements require humans “in the loop” as an integral part of the measurement process. For example, performance of complex tasks by people using information resources is appraised by observers who are called “judges”. In such instances, a human “judge,” or perhaps a panel of judges, are an essential part of the instrumentation for a measurement process. Humans who are part of the instrumentation should not be confused with humans whose performance is being measured.

2.7 Independent Observations

An observation is one independent element of measurement data. “Independent” means that, at least in theory, the result of each observation is not influenced in any way by the results of other observations. In some measurement processes, the independence of observations is easily recognized. For example, if a person steps onto a scale (the instrument) on three successive occasions, each of the three measurements (the attribute is weight, the object is a person) is readily assumed to be independent. When quality of a task performance (the attribute) by a surgical team (the object) is rated by a panel of judges (the instrument), each judge’s rating can be considered an independent observation as long as the judges do not communicate before offering their assessments. When a questionnaire is the measurement instrument, we assume that a person’s response to each question (often called an “item”) on the questionnaire is a response to that question only and is uninfluenced by their responses to other questions.

As measurement is customarily carried out, multiple independent observations are employed to estimate the value of an attribute for an object. As seen in the next chapter, this is because multiple independent observations produce a better estimate of the “true” value of the attribute than any single observation. Use of multiple observations also allows for the determination of how much variability exists across observations, which is necessary to estimate the error inherent in the measurement.

An Example of a Measurement Process

Consider an information resource, such as Isabel , designed to improve medical diagnosis (Graber and Mathew 2008 ). Such a system would take patient signs, symptoms, and tests as input. The resource’s algorithms would then suggest diseases that are consistent with these findings. A study team is interested in how reasonable the diagnoses suggested by the resource are, even when these diagnoses are not exactly correct. They conduct a study where the “top-five” diagnoses generated by the resource for a sample of test cases are referred to a panel of five experienced physicians for review. The panelists independently review, in the context of the case, each diagnosis set and rate the set on a 1–5 scale of “reasonableness”.

The measurement aspects of this process can be divided into its component parts: the attribute being measured, the relevant object class, the instrument, and what constitutes the multiple independent observations.

To identify the attribute , one might ask: What is being measured and what should the result of the measurement be called? The result of the measurement is the “reasonableness” of the top five diagnoses, so that is the attribute. This is a highly abstract construct because “reasonableness” is a state of mind. Nothing like it exists in the physical world.

To identify the object class , one might ask: On whom or on what is the actual measurement made? The measurement is made on the diagnosis set generated by the resource for each case, so the “diagnosis set for each case” is the object class of measurement.

The instrument in this example is a human judge coupled to a form on which the judgements are recorded.

An independent observation is the appraisal of one diagnosis set for one case by one judge.

Self-test 6.1

To determine the performance of a computer-based reminder system, a sample of alerts generated by the system (and the patient record from which each alert was generated) is given to a panel of physicians. Each panelist rates each alert on a four-point scale from “highly appropriate to the clinical situation” to “completely inappropriate.” Focusing on the measurement aspects of this process, name the attribute being measured, the class of measurement objects, the instrument used, and what constitutes an independent observation.

Staff members of a large community hospital undergo training to use a new administrative information system. After the training, each staff member completes a “test,” which is comprised of 30 questions about the system, to help the developers understand how much knowledge about the system has been conveyed via the training. Name the attribute being measured, the class of measurement objects, and the instrument used. Describe how this measurement process employs multiple independent observations.

In a test of a prototype mobile health app, four testers rated “ease of use” for each of 10 tasks the app was designed to accomplish, using a 5-point scale for each rating. A rating of 5 means “extremely easy to use for this task” and at the other extreme, a rating of 1 means “nearly impossible to use for this task”. What are the attribute and object class for this measurement, and what are the independent observations?

3 Key Object Classes and Types of Observations for Measurement in Informatics

3.1 key object classes.

Turning now to the range of measurement issues encountered in the real world of informatics, there are four specific categories of object classes that are often of primary interest in informatics studies: (1) professionals who may be health care providers, researchers, or educators; (2) clients of these professionals, usually patients or students; (3) biomedical information resources themselves; and (4) work groups or organizations that conduct research, provide health care, or provide education.

Among the classes of objects, professionals are important in informatics because attributes of these individuals influence whether and how information resources are used. Attributes of professionals that are important to measure include their domain-specific biomedical knowledge, their attitudes toward information technology and their work environment, their experience with information technology, among many others.

Clients emerge as objects of interest for many reasons. When clients are patients receiving health care, their health problems are complex and the attributes of these problems, central to the conduct of evaluation studies of information resources designed to improve their care, are difficult to assess. Important attributes of patients that often require measurement are diagnosis, prognosis, appropriateness of actual or recommended management, the typicality of their disease presentation, as well as their own beliefs and attitudes about health and disease. As patients increasingly access health information and some health services directly from the Internet, many of the attributes of professionals, as listed above, assume increased importance for patients as well. When clients are students receiving training in the health professions or biomedical research, measured attributes about them can be important determinants of what they will learn and what they are capable of learning.

Information resources have many attributes (e.g., data quality, speed of task execution, ease of use, cost, reliability, and degradation at the limits of their domain) that are of vital interest to informatics study teams.

Finally, work groups and organizations have many attributes (e.g., mission, age, size, budget structure, complexity, and integration) that determine how rapidly they adopt new technology and, once they do, how they use it.

3.2 Key Categories of Observations

The four categories of observations of frequent interest for measurement in informatics are:

Tasks : In many studies, measurements are made by giving the members of an object class something to do or a problem to solve. Different information resources (objects) may be challenged to process sets of microarray data (tasks) to determine speed of processing or usefulness of results (measured attributes). Alternatively, health care professionals or students (objects) may be asked to review sets of clinical case summaries (tasks) to develop a diagnosis or treatment plan (measured attributes). Within these kinds of performance-based assessments, which occur often in informatics, the challenges or problems assigned to objects are generically referred to as tasks. The credibility of quantitative studies often hinges on the way the study team manages tasks in measurement and demonstration study design.

Judges : Many measurement processes in informatics employ judges—humans with particular expertise who provide their informed opinions, usually by completing a rating form, about behavior they observe directly or review, in retrospect, from some record of that behavior. Judges are necessary to measurement in informatics when the attribute being assessed is complex or where there is no clear standard against which performance may be measured.

Items : These are the individual elements of a form, questionnaire, or test that is used to record ratings, knowledge, attitudes, opinions, or perceptions. On a knowledge test or attitude questionnaire, for example, each individual question would be considered an item.

Logistical factors : Many measurement processes are strongly influenced by procedural, temporal, or geographic factors, such as the places where and times when observations take place.

4 Levels of Measurement

In the process of quantitative measurement, a value of an attribute is assigned to each object. Attributes differ according to how their values are naturally expressed or represented. Attributes such as height and weight are naturally expressed using continuous numerical values whereas attributes such as “marital status” are expressed using discrete values. An attribute’s level of measurement denotes how its values can be represented. As discussed in later chapters, an attribute’s level of measurement directs the design of measurement instruments and the statistical analyses that can be applied to the measurement results.

There are four such levels of measurement:

Nominal : Measurement on a nominal attribute results in the assignment of each object to a specific category. The categories themselves do not form a continuum or have a meaningful order. Examples of attributes measured at the nominal level are ethnicity, medical specialty, and the base-pairs comprising a nucleotide. To represent the results of a nominal measurement quantitatively, the results must be assigned arbitrary codes (e.g., 1 for “internists,” 2 for “surgeons,” 3 for “family practitioners”). The only aspect of importance for such codes is that they be employed consistently. Their actual numerical or alphanumerical values have no significance.

Ordinal : Measurement at the ordinal level also results in assignment of objects to categories, but the categories have some meaningful order or ranking. For example, physicians often use a “plus” system of recording clinical signs (“++ edema”), which represents an ordinal measurement. The staging of cancers is another clinical example of an ordinal measurement. The status ranking of universities is another example of ordinal measurement. When coding the results of ordinal measurements, a numerical code is typically assigned to each category, but no aspect of these codes except for their numerical order contains interpretable information.

Note that both nominal and ordinal measurements result in discrete or categorical values and are often referenced using those terms. However, use of the term “categorical” as an umbrella descriptor for nominal and ordinal measures conceals the important difference between them.

Interval : Results of measurements at the interval level take on continuous numerical values that have an arbitrarily chosen zero point. The classic examples are the Fahrenheit and Celsius scales of temperature. This level of measurement derives its name from the “equal interval” assumption, which all interval measures must satisfy. To satisfy this assumption, equal differences between two measurements must have the same meaning irrespective of where they occur on the scale of possible values. On the Fahrenheit scale, the difference between 50 and 40 degrees has the same meaning as the difference between 20 and 10 degrees. An “interval” of 10 degrees is interpreted identically all along the scale.

Ratio : Results of measurements at the ratio level have the additional property of a true zero point. The Kelvin scale of temperature, with a zero point that is not arbitrarily chosen, has the properties of ratio measurement. Most physiological attributes (such as blood pressure) and physical measures (such as length) have ratio properties. This level of measurement is so named because one can assign meaning to the ratio of two measurement results in addition to the difference between them.

Just as nominal and ordinal measures are often grouped under the heading of “categorical” or “discrete”, interval and ratio measures are often grouped under the heading of “continuous”.

In quantitative measurement, it is often desirable to collect data at the highest level of measurement possible for the attribute of interest, with ratio measurement being the highest of the levels. In other words, if the attribute allows measurement at the interval or ratio level, the measurement should be recorded at that level. Doing this ensures that the measured results contain the maximum amount of information. For example, in a survey of healthcare providers, a study team may want to know each respondent’s years of professional experience which is, naturally, a ratio measure. Frequently, however, such attributes are assessed using discrete response categories, each containing a range of years. Although this measurement strategy provides some convenience and possibly some sense of anonymity for the respondent (which may generate more complete data with fewer missing values), it reduces to ordinal status what is naturally a ratio variable, with inevitable loss of information. Even if the data are later going to be categorized in service to privacy by preventing re-identification, collecting the data initially at the highest level of measurement is the preferred strategy. Data can always be converted from higher to lower levels of measurement, but it is not possible to go in the other direction.

Self-test 6.2

Determine the level of measurement of each of the following:

A person’s serum potassium level.

A health sciences center’s national ranking in research grant funding.

The distance between the position of an atom in a protein, as predicted by a computer model, and its actual position in the protein.

The “stage” of a patient’s cancer diagnosis.

The hospital unit to which a patient is assigned following admission to the hospital.

A person’s marital status.

A person’s score on an intelligence test, such as an IQ test.

5 Importance of Measurement in Quantitative Studies

The major premises underlying the quantitative approaches to evaluation, originally introduced in Sect. 2.5 , highlight why measurement is so important. These premises are re-stated here in a somewhat revised form to take advantage of the new concepts and terminology introduced in this chapter.

In quantitative studies, the following are assumed:

Attributes are inherent in the object under study. Merit and worth are part of the object and can be measured unambiguously. A study team can measure these attributes without affecting the object’s structure or function.

All rational persons agree (or can be brought to consensus) on what attributes of an object are important to measure and what measurement results would be associated with high merit or worth.

Because measurement of attributes allows precise statistical portrayals and comparisons across groups and across time, numerical measurement is prima facie superior to verbal description.

Through comparisons of measured attributes across selected groups of objects, it is possible to rigorously address evaluation questions of importance to informatics.

These premises make clear that the proper execution of quantitative studies requires careful and specific attention to methods of measurement. Accurate and precise measurement cannot be an afterthought. Footnote 1 Measurement is of particular importance in informatics because, as a relatively young and still evolving field (Hasman et al. 2011 ), informatics does not have a well-established tradition of “things worth measuring” or proven methods for measuring them. By and large, those planning studies are faced with the task of first deciding what to measure and then developing their own measurement methods. For most study teams, this task proves more difficult and more time-consuming than initially anticipated. In some cases, informatics study teams can adapt the measures used by others, but they often need to apply these measures to a different setting, where prior experience may not apply.

The choice of what to measure, and how, is an area where there are few prescriptions and where sound judgment, experience, and knowledge of methods come into play. Decisions about what and, above all, how to measure require knowledge of the study questions, the intervention and setting, and the experience of others who have done similar work. A methodological expert in measurement is of assistance only when teamed with others who know the terrain of biomedical and health informatics.

6 Measurement and Demonstration Studies

This section establishes a formal distinction between studies undertaken to develop and refine methods for making measurements, which are called measurement studies , and the subsequent use of these methods to address questions of direct importance in informatics, which are called demonstration studies . Footnote 2

Measurement studies seek to determine with how much error an attribute of interest can be measured in a population of objects, often also indicating how this error can be reduced. In an ideal quantitative measurement, all independent observations yield identical results. Any disagreement is due to measurement error. For example, measuring the air pressure (attribute) in an automobile tire (object) done three consecutive times at 15 s intervals would be expected to yield identical results and any discrepancies would be attributed to “measurement error” and would call into question some aspect of the measurement process. The developers of a new tire pressure gauge (an instrument) would be well advised to conduct measurement studies, testing their new instrument by determining whether it generates reproducible results over a range of conditions such as different types of inflation valves.

Measurement procedures developed and vetted through measurement studies provide study teams with what they need to conduct demonstration studies . Once it is known with how much error an attribute can be measured using a particular procedure, the measured values of this attribute can be employed to answer questions of importance in the world. Continuing our example, if tire manufacturers have confidence in the new pressure gauge as a measurement instrument, they can use it with confidence to measure things of importance to them, such as how quickly a tire, when punctured, loses air.

Example Contrasting Measurement and Demonstration

Return to the example of measuring the “reasonableness” (attribute) of diagnostic hypothesis sets (object) generated by an AI system, where the observation type is human judges. In a perfect quantitative measurement, all judges would give identical ratings to the hypothesis set generated by the system for each case, but this is very unlikely to occur. A measurement study would explore the amount of disagreement that exists among the judges across a set of cases. This is a necessary step in evaluating the system, but not what the developers and potential users of the program are primarily interested in knowing. However, once the performance of the measurement process is established, and the amount of error is considered to be acceptable, then a wide range of demonstration studies becomes possible. Important demonstration studies might explore the “reasonableness” of the diagnoses the AI system generates, whether the hypotheses generated by the AI system are more “reasonable” than those generated by human clinicians without aid from the system, and whether exposure to the diagnoses generated by the system increases the “reasonableness” of the human clinicians’ diagnoses.

The attribute addressed here reflects a highly abstract construct, what may be seen as a “state of mind” of the judges. Even so, quantitative methods can be applied. The abstractness of the attribute does not require use of qualitative methods.

The bottom line is that study teams must know that their measurement methods are fit for purpose—“fitness for purpose” will be defined more rigorously in the next chapter— before collecting data for their demonstration studies. As shown in Fig. 6.2 , it is necessary to perform a measurement study to establish the adequacy of all measurement procedures unless the measurement methods to be used have an established “track record.”

Measurement and demonstration studies

A track record for a measurement procedure develops from measurement studies undertaken and published by others. Even if the measurement procedures of interest have a track record in a particular health care or research environment, they may not perform equally well in a different environment. In these cases, measurement studies may still be necessary even when apparently tried-and-true measurement approaches are being employed. Study teams should always ask themselves—“Are the measurement methods we want to use fit for our purpose?—whenever they are planning a study and before proceeding to the demonstration phase. The “box” below offers several examples of measurement studies from the literature.

Examples of Published Measurement Studies

The results of measurement studies can and should be published. When published, they establish a “track record” for a measurement process that enables study teams to employ it with confidence in demonstration studies. Finding existing measurement methods prior to embarking on a study is an example of “Evidence-based Health Informatics” as discussed in Chap. 5 . There is a published algorithm to aid study teams in identifying measurement studies and measurement instruments (Terwee et al. 2009 ).

The most numerous measurement studies explore the reliability and validity of questionnaires to assess such attributes as health ICT and mobile health app usability (Yen et al. 2014 ; Zhou et al. 2019 ); mobile device proficiency (Roque and Boot 2018 ) as well as more general attitudes toward health ICT (Dykes et al. 2007 ).

Other published measurement studies explore methods to measure the impact of clinical decision support (Ramnarayan et al. 2003 ).

The literature also includes measurement studies that address the automated computation of quality measures (Kern et al. 2013 ).

7 Goals and Structure of Measurement Studies

The overall goal of a measurement study is to estimate with how much error an attribute of interest can be measured for a class of objects, ultimately leading to a viable measurement process for later application to demonstration studies. Chapter 7 describes in more detail how measurement errors are estimated. Chapters 8 and 9 address the development of measurement methods in greater technical detail.

One specific goal of a measurement study is to determine how many independent observations are necessary to reduce error to a level acceptable for the demonstration studies to follow. In general, the greater the number of independent observations comprising a measurement process, the smaller the measurement error. (The following chapter will explain why.) This relationship suggests an important trade-off because each independent observation comes at a cost. Returning to the example of the AI diagnostic advisory system, the more judges who rate each hypothesis set, the “better” the measurement will be. However, the time of expert judges is both expensive and a scarce resource. Experts are busy people and may have limited time to devote to participating in this role. Without a measurement study conducted in advance, there is no way to quantify this trade-off and determine an optimal balance point between quality of measurement and availability of judges.

Another goal of measurement studies is to verify that measurement instruments are well designed and functioning as intended. Even a measurement process with an ample number of independent observations--judges, in our running example--will have a high error rate if there are fundamental flaws in the way the process is conducted. For example, if the judges do not discuss, in advance, what “plausibility of a hypothesis set” actually means, the results will reveal unacceptably high error. Fatigue may be a factor if the judges are asked to do too much, too fast. As another example, consider a computer program developed to compute patients’ medication costs (attribute) automatically from a computer-based patient record (object). If this program has a bug that causes it to fail to include certain classes of medications, it will not return accurate results. Only an appropriate measurement study can detect these kinds of problems.

Teams designing a measurement study also should try to build into the study features that challenge the measurement process in ways that might be expected to occur in the demonstration study to follow. Only in this way is it possible to determine if the results of the measurement study will apply when the measurement process is put to use. For example, in the diagnostic hypothesis rating task mentioned above, the judges should be challenged in the measurement study with a range of cases typical of those expected in the demonstration study. The program to compute medication costs should be tested with a representative sample of cases from a variety of clinical services in the hospital where the demonstration study will ultimately be conducted. Additionally, a measurement technique may perform well with individuals from one particular culture, but perform poorly when transferred to a different culture. For example, the same questionnaire administered to patients in two hospitals serving very different ethnic groups may yield different results because respondents are interpreting the questions differently. This issue, too, can be explored via an appropriately designed measurement study that includes the full range of sociocultural settings where questionnaire might be used.

Measurement studies are also important to ensure that measurement processes are not trivialized. A measurement process might seem straightforward until it is actually necessary to use it. As an example, consider a study of a new admission–discharge–transfer (ADT) system for hospitals. Measuring the attribute “time to process admission” for the object class of patients is central to the success of this study. Although, on the surface, this attribute might seem trivial to measure, many potential difficulties arise on closer scrutiny. For example, when did the admission process for a patient actually begin and end: when each patient entered a waiting room or when they began speaking to an admissions clerk? Were there interruptions, and when did each of these begin and end? Should interruptions be counted as part of processing time?

If human observers are going to be the measurement instrument for this process, a measurement study might include three or four such observers who simultaneously observe the same set of admissions. The discrepancies in the observers’ estimates of the time to process these admissions would determine how many observers (whose individual results are averaged) are necessary to obtain an acceptable error rate. The measurement study might reveal flaws in the form on which the observers are recording their observations, or it might reveal that the observers had not been provided with adequate instructions about how, for example, to deal with interruptions. The measurement study could be performed in the admissions suite of a hospital similar in many respects to the ones in which the demonstration study will later be performed. The demonstration study, once the measurement methods have been established, would then explore whether the hospital actually processes admissions faster with the new system than with its predecessor.

Self-test 6.3

Clarke and colleagues developed the TraumAID system (Clarke et al. 1994 ) to advise on initial treatment of patients with penetrating injuries to the chest and abdomen. Measurement studies of the utility of TraumAID’s advice required panels of judges to rate the adequacy of management of a set of “test cases”—as recommended by TraumAID and as carried out by care providers. To perform this study, case data were fed into TraumAID to generate a treatment plan for each case. The wording of TraumAID’s plans was edited carefully in hope of ensuring that judges performing subsequent ratings would not know whether the described care was performed by a human or recommended by a computer.

Two groups of judges were employed: one from the medical center where the resource was developed, the other a group of senior physicians from across the country. The purposes of this study were to determine the extent to which judges’ ratings of each plan were in agreement, whether judges could detect plans generated by computer merely by the way they are phrased, and whether the home institution of the judges affected the ratings. Is this a measurement or a demonstration study? Explain.

8 The Structure and Differing Terminologies of Demonstration Studies

Demonstration studies differ from measurement studies in several respects. First, they aim to say something meaningful about an information resource or address some other question of substantive interest in informatics. With measurement studies, the primary concern is the error inherent in assigning a value of an attribute to each individual object, whereas with demonstration studies, the concern is different. Demonstration studies are concerned with determining the actual magnitude of that attribute in a group of objects, determining if certain groups of objects differ in the magnitude of that attribute, or if there is a relationship between that attribute and other attributes of interest. For example, in a study of an information resource to support management of patients in the intensive care unit:

A measurement study would be concerned with how accurately and precisely the “optimal care” (attribute) of patients (object class) can be determined.

A subsequent demonstration study might explore whether care providers supported by the resource deliver care more closely approximating optimal care.

In a study of an “app” to support patients’ control their blood pressure, study teams might be interested in whether users are more satisfied with a new version of the app compared to an older version:

A measurement study may focus on how well satisfaction with the app (attribute) as perceived by its users (object class) can be measured with a short questionnaire.

A subsequent demonstration study would compare satisfaction among users of the new of old versions.

The terminology used to describe the structure of demonstration studies also differs from that used for measurement studies. Most notably:

When the object class of measurement in a measurement study is people, they are typically referred to as subjects or participants in a demonstration study.

An attribute in a measurement study is typically referred to as a variable in a demonstration study.

In describing the design of quantitative studies, variables can be of two types: dependent and independent. The dependent variables are a subset of the variables in the study that capture outcomes of interest to the study team. For this reason, dependent variables are also called “outcome variables.” The independent variables are those included in a study to explain the measured values of the dependent variables. Returning to some examples introduced earlier in this chapter, a demonstration study might contrast the time to admit new patients to a hospital before and after a new information resource designed to facilitate the process is introduced. In this example, “time to admit” would be the dependent variable and “before vs. after” the introduction of the new system would be the independent variable.

9 Demonstration Study Designs

There are many variations of demonstration study designs that suit many different purposes. Figure 6.3 illustrates these variations.

Demonstration study designs differentiated by time-orientation and intention

9.1 Prospective and Retrospective Studies

Demonstration studies vary according to the time orientation of data collection.

Prospective Studies : A prospective study is planned and designed in advance, in anticipation of data collection that will occur primarily in the future. The planning of the study precedes the collection of the most important data, and in this sense, prospective studies look forward in time.

Examples of prospective studies include: needs assessment surveys, usability studies of a prototype resource conducted in laboratory conditions, and field user-effect studies that randomize individuals to groups that do and do not employ a patient-facing app.

It is common for teams conducting prospective studies to include in their study some data that pre-existed the planning of the study. For example, a team conducting a field user effect study of a patient-facing app might, with consent, bring into the study dataset some information about each participant’s health behavior prior to any participants’ use of the app.

Retrospective Studies : In a retrospective study, the data exist prior to the planning and design of the study. Retrospective studies look backward upon data already collected. The data employed in retrospective studies are often collected routinely as a by-product of practice and stored in clinical, research or educational data repositories which makes the data available for a wide range of studies. In a retrospective study, the team selects which data to employ in their analyses out of a larger set of data that is available to them. Retrospective studies are also called “ex post facto” studies.

Examples of purely retrospective studies include needs assessments for a new patient portal that exclusively examines records of previous use of the existing portal, a lab function study to develop and test the accuracy of a machine learning model to predict sepsis based on data accumulated in a clinical data repository, and a problem impact study tracking research productivity following introduction of a research information management system 3 years in the past.

9.2 Descriptive, Interventional, and Correlational Studies

Demonstration studies also differ according to intention.

Descriptive Studies : Descriptive studies seek primarily to estimate the value of a dependent variable or set of dependent variables in a selected sample of subjects. Purely descriptive studies have no independent variables. If a group of nurses were given a rating form (previously verified through a measurement study to be an acceptable measurement tool) to ascertain the “ease of use” of a nursing information system, the mean value of this variable in a sample of nurses would be the key result of this descriptive study. If this value were found to be toward the low end of the scale, the study team might conclude from this descriptive study that the resource was in need of substantial revision. Although they seem deceptively simple, descriptive studies can be highly informative. Studies of the quality of health information on the Internet illustrate the importance of well-conducted descriptive studies (Daraz et al. 2019 ).

Referring back to the categorization of evaluation approaches introduced in Sect. 2.7 , descriptive studies also can be tied to the “objectives based” approach. When a study team seeks to determine whether a resource has met a predetermined set of performance objectives--for example, whether the response time meets predetermined criteria for being “fast enough”--the logic and design of the resulting demonstration study is descriptive. There is no independent variable. The dependent variable in this example is “speed of execution”, usually averaged over a range of tasks executed by the resource.

As illustrated in Fig. 6.3 , descriptive studies can be either prospective or retrospective. In the example above, testing the speed of a resource in executing a set of tasks is a prospective study. An example of a retrospective descriptive study would entail inspection of usage log files to determine the extent and nature of use of an information resource.

Interventional Studies : Here, the study team creates a contrasting set of conditions that enable the exploration of the questions of interest. For example, if the study question of primary interest is: “Does a particular patient-facing app promote weight loss?”, the study team would typically create one group of users and another group of non-users of the app. After identifying a sample of subjects for the study, the team then assigns each subject to either the “user” or “non-user” condition. In this example, “use” vs. “non-use” of the app is the independent variable. A dependent (outcome) variable of interest, in this case weight loss, is then measured for each subject, and the mean values of this variable for subjects in each condition are compared. Interventional studies align with the “comparison-based” approach to evaluation introduced in Sect. 2.7 . They are often referred to as experiments or quasi-experiments.

The classic field user-effect study of reminder systems by McDonald and colleagues (McDonald et al. 1984 ) is an example of an interventional study applied to informatics. In this study, groups of clinicians (subjects/participants) either received or did not receive computer-generated reminders to carry out recommended aspects of care. Receipt or non-receipt of the reminders were the conditions comprising the independent variable. The study team compared the extent to which clinical actions consistent with recommended care (the dependent variable) took place in each group.

When the study conditions are created prospectively by the study team and all other differences are eliminated by random assignment of subjects, it is possible to isolate and explore the effect due solely to the difference between the conditions. This allows the study team to assert that these effects are causal rather than merely coincidental.

As illustrated in Fig. 6.3 , all interventional studies are prospective.

Correlational Studies : Here, study teams explore the relationships among a set of variables, the values of which are routinely generated as a by-product of ongoing life experience, health care, research, educational, or public health practice. Correlational studies are increasingly called “real world evidence” studies and may also be called datamining or quality assessment studies. These studies are becoming increasingly common in both clinical and biological application domains.

Correlational studies may be either prospective or retrospective, as shown in Fig. 6.3 . The best-known example of a prospective correlational study is the Framingham Study of heart disease, where a cohort of participants was first identified and then followed over many decades (Dawber 2013 ). This is a prospective study because the effort was planned before any data were collected, but there were no interventions and all of the data originated from the life and health care experiences of the participants. An informatics example of a prospective correlational study is a field function study which “pseudo-deploys” a machine learning algorithm in a care environment and sends the predictions of the algorithm back to the study team to determine the predictive power of the model. An informatics example of a retrospective correlational study examines use of an online portal by a health system’s patients. Looking back several years following the introduction of the portal, a study team might examine the extent and nature of portal use in relation to demographic characteristics, disease experience and other characteristics (Palen et al. 2012 ).

In correlational studies, there will almost always be more data available than are needed to address a particular set of study questions. Study teams select the variables to include in their analysis from this larger set, and must choose which variables to label as “independent” and “dependent”, associating the independent variables with presumed predictors or causes and associating the dependent variables with presumed outcomes or effects. Correlational studies are linked most closely to the “decision facilitation” approach to evaluation discussed in Sect. 2.7.1 . Because there is no deliberate creation of experimental conditions or randomization in correlational studies, attributions of cause and effect among the variables selected for analysis require careful consideration, as will be discussed in Chap. 13 .

10 Demonstration Studies and Stages of Resource Development

This concluding section calls attention to some important relationships between the three demonstration study types, the lifecycle of resource development, and the settings in which demonstration studies can be conducted.

Descriptive studies are useful over the entire lifecycle and can be conducted in the lab or in the field. Descriptive studies are particularly valuable in early stages of resource development to determine if the resource is achieving pre-determined design specifications; but they may also be valuable to document the extent and nature of use of a resource after it is deployed.

Interventional studies require a prototype or a deployed version of a resource. Study teams can carry out interventional studies in laboratory settings—for example, to compare two interfaces of a resource under controlled conditions prior to deployment. Alternatively, and as discussed above, study teams can conduct an interventional field study to understand the effects of a deployed resource.

Correlational studies require a deployed resource and are carried out in field settings that generate real-world data.

Returning to the themes of earlier chapters, study teams can almost always choose among several options for demonstration studies. This chapter emphasizes that, irrespective of the selected demonstration study design, all quantitative studies rely on sound methods of measurement.

Self-test 6.4

Classify each of the following demonstration study designs as descriptive, interventional, or correlational. In each study, who or what are the subjects/participants? Identify the independent and dependent variables.

An information resource is developed to aid in identifying patients who are eligible for specific clinical protocols. A demonstration study of the resource is implemented to examine protocol enrollment rates at sites where the resource was and was not deployed.

A new clinical workstation is introduced into a network of medical offices. Logs of 1 week of resource use by nurses are studied to document to what extent and for what purposes this resource is being used.

A number of clinical decision support resources have been deployed to support care in a large health system. A study team examines which characteristics of patients are predictive of the extent of use of these resources.

A study team compiles a database of single nucleotide polymorphisms (SNPs), which are variations in an individual’s genomic sequences. The team then examines these SNPs in relation to diseases that these individuals develop, as reflected in a clinical data repository.

Students are given access to a database to help them solve problems in a biomedical domain. By random assignment, half of the students use a version of the database emphasizing hypertext browsing capabilities; half use a version emphasizing Boolean queries for information. The proficiency of these students at solving problems is assessed at the beginning and again at the end of the study period.

Terms such as “accuracy” and “precision” are used loosely in this chapter. They will be defined more rigorously in Chap. 7 .

The concept of the measurement study in informatics can be traced to the work of Michaelis et al. ( 1990 ).

Binning JF. Construct. In: Encyclopedia Britannica. 2016. https://www.britannica.com/science/construct . Accessed 30 Dec 2020.

Clarke JR, Webber BL, Gertner A, Rymon KJ. On-line decision support for emergency trauma management. Proc Symp Comput Applications Med Care. 1994;18:1028.

Google Scholar  

Daraz L, Morrow AS, Ponce OJ, Beuschel B, Farah MH, Katabi A, et al. Can patients trust online health information? A meta-narrative systematic review addressing the quality of health information on the internet. J Gen Intern Med. 2019;34:1884–91.

Article   Google Scholar  

Dawber TR. The Framingham study: the epidemiology of atherosclerotic disease. Harvard University Press; 2013.

Dykes PC, Hurley A, Cashen M, Bakken S, Duffy ME. Development and psychometric evaluation of the Impact of Health Information Technology (I-HIT) scale. J Am Med Inform Assoc. 2007;14:507–14.

Feinstein AR. Clinimetrics. New Haven: Yale University Press; 1987. p. viii.

Book   Google Scholar  

Graber ML, Mathew A. Performance of a web-based clinical diagnosis support system for internists. J Gen Intern Med. 2008;23:37–40.

Hasman A, Ammenwerth E, Dickhaus H, Knaup P, Lovis C, Mantas J, et al. Biomedical informatics–a confluence of disciplines? Methods Inf Med. 2011;50:508–24.

Article   CAS   Google Scholar  

Kern LM, Malhotra S, Barrón Y, Quaresimo J, Dhopeshwarkar R, Pichardo M, et al. Accuracy of electronically reported “meaningful use” clinical quality measures: a cross-sectional study. Ann Int Med. 2013;158:77–83.

McDonald CJ, Hui SL, Smith DM, et al. Reminders to physicians from an introspective computer medical record: a two-year randomized trial. Ann Intern Med. 1984;100:130–8.

Michaelis J, Wellek S, Willems JL. Reference standards for software evaluation. Methods Inf Med. 1990;29:289–97.

Palen TE, Ross C, Powers JD, Xu S. Association of online patient access to clinicians and medical records with use of clinical services. JAMA. 2012;308:2012–9.

Ramnarayan P, Kapoor RR, Coren M, Nanduri V, Tomlinson AL, Taylor PM, et al. Measuring the impact of diagnostic decision support on the quality of clinical decision making: development of a reliable and valid composite score. J Am Med Inform Assoc. 2003;10:563–72.

Roque NA, Boot WR. A new tool for assessing mobile device proficiency in older adults: the mobile device proficiency questionnaire. J Appl Gerontol. 2018;37:131–56.

Scott PJ, Brown AW, Adedeji T, Wyatt JC, Georgiou A, Eisenstein EL, et al. A review of measurement practice in studies of clinical decision support systems 1998–2017. J Am Med Inform Assoc. 2019;26:1120–8.

Terwee CB, Jansma EP, Riphagen II, de Vet HC. Development of a methodological PubMed search filter for finding studies on measurement properties of measurement instruments. Qual Life Res. 2009;18:1115–23.

Yen PY, Sousa KH, Bakken S. Examining construct and predictive validity of the Health-IT Usability Evaluation Scale: confirmatory factor analysis and structural equation modeling results. J Am Med Inform Assoc. 2014;21:e241–8.

Zhou L, Bao J, Setiawan IM, Saptono A, Parmanto B. The mHealth APP usability questionnaire (MAUQ): development and validation study. JMIR. 2019;7:e11500.

Download references

Author information

Authors and affiliations.

Department of Learning Health Sciences, University of Michigan Medical School, Ann Arbor, MI, USA

Charles P. Friedman

Department of Primary Care, Population Sciences and Medical Education, School of Medicine, University of Southampton, Southampton, UK

Jeremy C. Wyatt

Department of Medical Informatics and Clinical Epidemiology, School of Medicine, Oregon Health & Science University, Portland, OR, USA

Joan S. Ash

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Charles P. Friedman .

Answers to Self-tests

The attribute is “appropriateness” of each alert. “Alerts” comprise the object class. (Although the panelists need access to the cases to perform the ratings, cases are not the object class here because each alert is what is directly rated—the attribute of “appropriateness” is a characteristic of each alert—and because each case may have generated multiple alerts related to its different clinical aspects.) The instrumentation is the rating form as completed by a human judge. Each individual judge’s rating of the appropriateness of an alert constitutes an independent observation.

The attribute is “knowledge about the administrative information system.” Staff members are the object class. The instrument is the written test. Each question on the test constitutes an independent observation.

The attribute is “ease of use” of the app. Tasks are the object class. The independent observations are the ratings by each tester.

Interval (In IQ testing, the average score of 100 is completely arbitrary)

This is a measurement study. The stated purposes of the study have nothing to do with the actual quality of TraumAID’s advice. The purposes are exclusively concerned with how well this quality, whatever it turns out to be, can be measured. The quality of TraumAID’s advice would be the focus of a separate demonstration study.

It is an interventional study because the study team presumably had some control over where the resource was or was not deployed. The site is the “subject” for this study. (Note that this point is a bit ambiguous. Patients could possibly be seen as the subjects in the study; however, as the question is phrased, the enrollment rates at the sites are going to be the basis of comparison. Because the enrollment rate must be computed for a site , then site must be the “subject.”) It follows that the dependent variable is the protocol enrollment rate; the independent variable is the presence or absence of the resource.

It is a descriptive study. Nurses using the system are the subjects. There is no independent variable. Dependent variables are the extent of workstation use for each purpose.

It is a correlational study. Patients are the subjects. The independent variables are the characteristics of the patients; the dependent variable is the extent of use of information resources.

This is also a correlational study. Patients are the subjects. The independent variable is the genetic information; the dependent variable is the diseases they develop. There is, however, no manipulation or purposeful intervention.

Interventional study. Students are the subjects. Independent variable(s) are the version of the database and time of assessment. The dependent variable is the score on each problem-solving assessment.

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this chapter

Friedman, C.P., Wyatt, J.C., Ash, J.S. (2022). The Structure of Quantitative Studies. In: Evaluation Methods in Biomedical and Health Informatics. Health Informatics. Springer, Cham. https://doi.org/10.1007/978-3-030-86453-8_6

Download citation

DOI : https://doi.org/10.1007/978-3-030-86453-8_6

Published : 10 February 2022

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-86452-1

Online ISBN : 978-3-030-86453-8

eBook Packages : Medicine Medicine (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Module 3 Chapter 1: From Research Questions to Research Approaches

The approaches that social work investigators adopt in their research studies are directly related to the nature of the research questions being addressed.In Module 2 you learned about exploratory, descriptive, and explanatory research questions. Let’s consider different approaches to finding answers to each type of question.

In this chapter we build on what was learned in Module 2 about research questions, examining how investigators’ approaches to research are determined by the nature of those questions. The approaches we explore are all systematic, scientific approaches, and when properly conducted and reported, they all contribute empirical evidence to build knowledge.  In this chapter you will read about:

• qualitative research approaches for understanding diverse populations, social problems, and social phenomena,
• quantitative research approaches for understanding diverse populations, social problems, and social phenomena,
• mixed methods research approaches for understanding diverse populations, social problems, and social phenomena.

Overview of Qualitative Approaches

Questions of a descriptive or exploratory nature are often asked and addressed through  qualitative research . The specific aim in these studies is to understand diverse populations, social work problems, or social phenomena as they naturally occur, situated in their natural environments, providing rich, in-depth, participant-centered descriptions of the phenomena being studied. Qualitative research approaches have been described as “humanistic” in aiming to study the world from the perspective of those who are experiencing it themselves; this also contributes to a social justice commitment in that the approaches give “voice” to the individuals who are experiencing the phenomena of interest (Denzen & Lincoln, 2011).  As such, qualitative research approaches are also credited with being sensitive and responsive to diversity—embracing feminist, ethnic, class, critical race, queer, and ability/disability theory and lenses.

In qualitative research, the investigator is engaged as an observer and interpreter, being acutely aware of the subjectivity of the resulting observations and interpretations.

“At this level, qualitative research involves an interpretive, naturalistic approach to the world” (Denzin & Lincoln, 2011, p. 3)

Because the data are rich and deep, a lot of information is collected by involving relatively few participants; otherwise, the investigator would be overwhelmed by a tremendous volume of information to collect, sift through, process, interpret, and analyze. Thus, a single qualitative study has a relatively low level of generalizability  to the population as a whole because of its methodology, but that is not the aim or goal of this approach.

In addition, because the aim is to develop understanding of the participating individuals’ lived experiences, the investigator in a qualitative study seldom imposes structure with standardized measurement tools. The investigator may not even start with preconceived theory and hypotheses. Instead, the methodologies involve a great deal of open-ended triggers, questions, or stimuli to be interpreted by the persons providing insight:

“Qualitative research’s express purpose is to produce descriptive data in an individual’s own written or spoken words and/or observable behavior” (Holosko, 2006, p. 12).

Furthermore, investigators often become a part of the qualitative research process: they maintain awareness of their own influences on the data being collected and on the impact of their own experiences and processes in interpreting the data provided by participants. In some qualitative methodologies, the investigator actually enters into/becomes immersed in the events or phenomena being studied, to both live and observe the experiences first-hand.

Qualitative data and interpretations are recognized as being subjective in nature—that is the purpose—rather than assuming objectivity. Qualitative research is based on experientially derived data and is interpretive, meaning it is “concerned with understanding the meaning of human experience from the subject’s own frame of reference” (Holosko, 2006, p. 13). In this approach, conclusions about the nature of reality are specific to each individual study participant, following his or her own interpretation of that reality. These approaches are considered to flow from an inductive reasoning process where specific themes or patterns are derived from general data (Creswell & Poth, 2018).

Several purposes of qualitative approaches in social work include:

• describing and exploring the nature of phenomena, events, or relationships at any system level (individual to global)
• generating theory
• initially test ideas or assumptions (in theory or about practices)
• evaluate participants’ lived experiences with practices, programs, policies, or participation in a research study, particularly with diverse participants
• explore “fit” of quantitative research conclusions with participants’ lived experiences, particularly with diverse participants
• inform the development of clinical or research assessment/measurement tools, particularly with diverse participants.

Overview of Quantitative Approaches

Questions of the exploratory, descriptive, or explanatory type are often asked and addressed through quantitative research  approaches, particularly questions that have a numeric component. Exploratory and descriptive quantitative studies rely on objective measures for data collection which is a major difference from qualitative studies which are aimed at understanding subjective perspectives and experiences. Explanatory quantitative studies often begin with theory and hypotheses, and proceed to empirically test the hypotheses that investigators generated. By their quantitative (numeric) nature, statistical hypothesis testing is possible in many types of quantitative studies.

Quantitative research studies utilize methodologies that enhance generalizability of results to the greatest extent possible—individual differences are de-emphasized, similarities across individuals are emphasized. These studies can be quite large in terms of participant numbers, and the study samples need to be developed in such a manner as to support generalization to the larger populations of interest.

The process is generally described as following a deductive logical system where specific data points are combined to lead to developing a generalizable conclusion. The philosophical roots (epistemology) underlying quantitative approaches is positivism, involving the seeking of empirical “facts or causes of social phenomena based on experimentallyderived evidence and/or valid observations” (Holosko, 2006, p. 13). The empirical orientation is objective in that investigators attempt to be detached from the collection and interpretation of data in order to minimize their own influences and biases. Furthermore, investigators utilize objective measurement tools to the greatest extent possible in the process of collecting quantitative study data.

Several purposes of quantitative approaches in social work include:

• describing and exploring the dimensions of diverse populations, phenomena, events, or relationships at any system level (individual to global)—how much, how many, how large, how often, etc. (including epidemiology questions and methods)
• testing theory (including etiology questions)
• experimentally determining the existence of relationships between factors that might influence phenomena or relationships at any system level (including epidemiology and etiology questions)
• testing causal pathways between factors that might influence phenomena or relationships at any system level (including etiology questions)
• evaluate quantifiable outcomes of practices, programs, or policies
• assess the reliability and validity of clinical or research assessment/measurement tools.

Overview of Mixed-Method Approaches

Important dimensions distinguish between qualitative and quantitative approaches. First, qualitative approaches rely on “insider” perspectives, whereas quantitative approaches are directed by “outsiders” in the role of investigator (Padgett, 2008). Second, qualitative results are presented holistically, whereas quantitative approaches present results in terms of specific variables dissected from the whole for close examination; qualitative studies emphasize the context of individuals’ experiences, whereas quantitative studies tend to decontextualize the phenomena under study (Padgett, 2008). Third, quantitative research approaches tend to follow a positivist philosophy, seeking objectivity and representation of what actually exists; qualitative research approaches follow from a post-positivist philosophy, recognizing that observation is always shaped by the observer, therefore is always subjective in nature and this should be acknowledged and embraced. In post-positivist qualitative research traditions, realities are perceived as being socially constructed, whereas in positivist quantitative research, a single reality exists, waiting to be discovered or understood. The quantitative perspective on reality has a long tradition in the physical and natural sciences (physics, chemistry, anatomy, physiology, astronomy, and others). The social construction perspective has a strong hold in social science and understanding social phenomena. But what if an investigator’s questions are relevant to both qualitative and quantitative approaches?

Given the fundamental philosophical and practical differences, some scholars argue that there can be no mixing of the approaches, that the underlying paradigms are too different. However, mixed-methods research  has also been described as a new paradigm (since the 1980s) for social science:

“Like the mythology of the phoenix, mixed methods research has arisen out of the ashes of the paradigm wars to become the third methodological movement. The fields of applied social science and evaluation are among those which have shown the greatest popularity and uptake of mixed methods research designs” (Cameron & Miller, 2007, p. 3). 

Mixed-methods research approaches are used to address in a single study the acknowledged limitations of both quantitative and qualitative approaches. Mixed methods research combines elements of both qualitative and quantitative approaches for the purpose of achieving both depth and breadth of understanding, along with corroboration of results (Johnson, Onwuegbuzie, & Turner, 2007, p. 123). One mixed-methods strategy is related to the concept of  triangulation : understanding an event or phenomenon from the use of varied data sources and methods all applied to understanding the same phenomenon (Denzin & Lincoln, 2011; see Figure 1-1).

Figure 1-1. Depiction of triangulation as synthesis of different data sources

For example, in a survey research study of student debt load experienced by social work doctoral students, the investigators gathered quantitative data concerning demographics, dollar amounts of debt and resources, and other numeric data from students and programs (Begun & Carter, 2017). In addition, they collected qualitative data about the experience of incurring and managing debt load, how debt shaped students’ career path decisions, practices around mentoring doctoral students about student debt load, and ideas for addressing the problem. Triangulation came into play in two ways: first, collecting data from students and programs about the topics, and second, a sub-sample of the original surveyed participants engaged in qualitative interviews concerning the “fit” or validity of conclusions drawn from the prior qualitative and quantitative data.

Three different types of mixed methods approaches are used:

• Convergent designs involve the simultaneous collection of both qualitative and quantitative data, followed by analysis of both data sets, and merging the two sets of results in a comparative manner.
• Explanatory sequential designs use quantitative methods first, and then apply qualitative methods to help explain and further interpret the quantitative results.
• Exploratory sequential designs first explore a problem or phenomenon through qualitative methods, especially if the topic is previously unknown or the population is understudied and unfamiliar. These qualitative findings are then used to build the quantitative phase of a project (Creswell, 2014, p. 6).

Mixed methods approaches are useful in developing and testing new research or clinical measurement tools. For example, this is done in an exploratory sequential process whereby detail-rich qualitative data inform the creation of a quantitative instrument. The quantitative instrument is then tested in both quantitative and qualitative ways to confirm that it is adequate for its intended use. This iterative process is depicted in Figure 1-2.

Figure 1-2. Iterative qualitative and quantitative process of instrument development

One example of how this mixed-methods approach was utilized was in development of the Safe-At-Home instrument for assessing individuals’ subjective readiness to change their intimate partner violence behavior (Begun et al., 2003; 2008). The transtheoretical model of behavior change (TMBC) underlies the instrument’s development: identifying stages in readiness to change one’s behavior and matching these stages to the most appropriate type of intervention strategy (Begun et al., 2001). The first step in developing the intimate partner violence Safe-At-Home instrument for assessing readiness to change was to qualitatively generate a list of statements that could be used in a quantitative rating scale. Providers of treatment services to individuals arrested for domestic or relationship violence were engaged in mutual teaching/learning with the investigators concerning the TMBC as it might relate to the perpetration of intimate partner violence. They independently generated lists of the kinds of statements they heard from individuals in their treatment programs, statements they believed were demonstrative of what they understood as the different stages in the change process. The investigators then worked with them to reduce the amassed list of statements into stage-representative categories, eliminating duplicates and ambiguous statements, and retaining the original words and phrases they heard to the greatest extent possible. The second phase was both quantitative and qualitative in nature: testing the instrument with a small sample of men engaged in batters’ treatment programs and interviewing the men about the experience of using the instrument. Based on the results and their feedback, the instrument was revised. This process was followed through several iterations. The next phases were quantitative: determining the psychometric characteristics of the instrument and using it to quantitatively evaluate batterer treatment programs—the extent to which individuals were helped to move forward in stages of the change cycle.

Interactive Excel Workbook Activities

Complete the following Workbook Activity:

• SWK 3401.3-1.1 Getting Started: Understanding an Excel Data File

Chapter Summary

In this chapter you were introduced to three general approaches for moving from research question to research method. You were provided with a brief overview of the philosophical underpinnings and uses of qualitative, quantitative, and mixed-methods approaches. Next, you are provided with more detailed descriptions of qualitative and quantitative traditions and their associated methodologies.

Take a moment to complete the following activity.

Social Work 3401 Coursebook Copyright © by Dr. Audrey Begun is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License , except where otherwise noted.

Share This Book

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

Step'by-step guide to critiquing research. Part 1: quantitative research

Related Papers

Patricia Cronin

When caring for patients it is essential that nurses are using the current best practice. To determine what this is, nurses must be able to read research critically. But for many qualified and student nurses the terminology used in research can be difficult to understand thus making critical reading even more daunting. It is imperative in nursing that care has its foundations in sound research and it is essential that all nurses have the ability to critically appraise research to identify what is best practice. This article is a step-by step-approach to critiquing quantitative research to help nurses demystify the process and decode the terminology. Key words: Quantitative research methodologies Review process • Research]or many qualified nurses and nursing students research is research, and it is often quite difficult to grasp what others are referring to when they discuss the limitations and or strengths within a research study. Research texts and journals refer to critiquing the ...

Ahmad Albashaireh

Becky Ingham-Broomfield

Gina Taylor

British Journal of Nursing

Lena Törnkvist

Sarahjane Jones

Journal of Clinical Nursing

Professor Margaret Fry

RELATED PAPERS

Media Kesehatan Masyarakat Indonesia

Kasman Usman

Trends in Transplantation

Adrian Martínez

European Journal of Pharmacology: Molecular Pharmacology

jean paul Behr

redalyc.uaemex.mx

harold zooki Castro Diaz

Theoretical and Applied Climatology

Mamadou Sall

Daniel Hernandez

Sven Spieckermann

David Fishman

Mohd Alfa Shamsuri

IACR Transactions on Symmetric Cryptology

Mohamed Tolba

Janis Kreslins

2009 International Conference on Management of e-Commerce and e-Government, ICMeCG 2009

Trans/Form/Ação

Gregor Nosan

The Journal of Asian Finance, Economics and Business

Fábio Lúcio Santos

مجلة جامعة الشارقة للعلوم الانسانية والاجتماعية

عبدالله علي

Journal of Chemical Physics

Xavier Blase

Social Science Research Network

Edward Golosov

Abanti Sahoo

The Astrophysical Journal

Thomas H. Puzia

DergiPark (Istanbul University)

Gülşen AKGÜL

DevPolicy Blog

Anna Naupa , Jennifer Kalpokas-Doan

Medine Sivri

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024