conclusion and recommendation meaning in research

Get science-backed answers as you write with Paperpal's Research feature

How to Write a Conclusion for Research Papers (with Examples)

The conclusion of a research paper is a crucial section that plays a significant role in the overall impact and effectiveness of your research paper. However, this is also the section that typically receives less attention compared to the introduction and the body of the paper. The conclusion serves to provide a concise summary of the key findings, their significance, their implications, and a sense of closure to the study. Discussing how can the findings be applied in real-world scenarios or inform policy, practice, or decision-making is especially valuable to practitioners and policymakers. The research paper conclusion also provides researchers with clear insights and valuable information for their own work, which they can then build on and contribute to the advancement of knowledge in the field.

The research paper conclusion should explain the significance of your findings within the broader context of your field. It restates how your results contribute to the existing body of knowledge and whether they confirm or challenge existing theories or hypotheses. Also, by identifying unanswered questions or areas requiring further investigation, your awareness of the broader research landscape can be demonstrated.

Remember to tailor the research paper conclusion to the specific needs and interests of your intended audience, which may include researchers, practitioners, policymakers, or a combination of these.

Table of Contents

What is a conclusion in a research paper, summarizing conclusion, editorial conclusion, externalizing conclusion, importance of a good research paper conclusion, how to write a conclusion for your research paper, research paper conclusion examples.

• How to write a research paper conclusion with Paperpal? 

Frequently Asked Questions

A conclusion in a research paper is the final section where you summarize and wrap up your research, presenting the key findings and insights derived from your study. The research paper conclusion is not the place to introduce new information or data that was not discussed in the main body of the paper. When working on how to conclude a research paper, remember to stick to summarizing and interpreting existing content. The research paper conclusion serves the following purposes: 1

• Warn readers of the possible consequences of not attending to the problem.
• Recommend specific course(s) of action.
• Restate key ideas to drive home the ultimate point of your research paper.
• Provide a “take-home” message that you want the readers to remember about your study.

Types of conclusions for research papers

In research papers, the conclusion provides closure to the reader. The type of research paper conclusion you choose depends on the nature of your study, your goals, and your target audience. I provide you with three common types of conclusions:

A summarizing conclusion is the most common type of conclusion in research papers. It involves summarizing the main points, reiterating the research question, and restating the significance of the findings. This common type of research paper conclusion is used across different disciplines.

An editorial conclusion is less common but can be used in research papers that are focused on proposing or advocating for a particular viewpoint or policy. It involves presenting a strong editorial or opinion based on the research findings and offering recommendations or calls to action.

An externalizing conclusion is a type of conclusion that extends the research beyond the scope of the paper by suggesting potential future research directions or discussing the broader implications of the findings. This type of conclusion is often used in more theoretical or exploratory research papers.

Align your conclusion’s tone with the rest of your research paper. Start Writing with Paperpal Now!  

The conclusion in a research paper serves several important purposes:

• Offers Implications and Recommendations : Your research paper conclusion is an excellent place to discuss the broader implications of your research and suggest potential areas for further study. It’s also an opportunity to offer practical recommendations based on your findings.
• Provides Closure : A good research paper conclusion provides a sense of closure to your paper. It should leave the reader with a feeling that they have reached the end of a well-structured and thought-provoking research project.
• Leaves a Lasting Impression : Writing a well-crafted research paper conclusion leaves a lasting impression on your readers. It’s your final opportunity to leave them with a new idea, a call to action, or a memorable quote.

Writing a strong conclusion for your research paper is essential to leave a lasting impression on your readers. Here’s a step-by-step process to help you create and know what to put in the conclusion of a research paper: 2

• Research Statement : Begin your research paper conclusion by restating your research statement. This reminds the reader of the main point you’ve been trying to prove throughout your paper. Keep it concise and clear.
• Key Points : Summarize the main arguments and key points you’ve made in your paper. Avoid introducing new information in the research paper conclusion. Instead, provide a concise overview of what you’ve discussed in the body of your paper.
• Address the Research Questions : If your research paper is based on specific research questions or hypotheses, briefly address whether you’ve answered them or achieved your research goals. Discuss the significance of your findings in this context.
• Significance : Highlight the importance of your research and its relevance in the broader context. Explain why your findings matter and how they contribute to the existing knowledge in your field.
• Implications : Explore the practical or theoretical implications of your research. How might your findings impact future research, policy, or real-world applications? Consider the “so what?” question.
• Future Research : Offer suggestions for future research in your area. What questions or aspects remain unanswered or warrant further investigation? This shows that your work opens the door for future exploration.
• Closing Thought : Conclude your research paper conclusion with a thought-provoking or memorable statement. This can leave a lasting impression on your readers and wrap up your paper effectively. Avoid introducing new information or arguments here.
• Proofread and Revise : Carefully proofread your conclusion for grammar, spelling, and clarity. Ensure that your ideas flow smoothly and that your conclusion is coherent and well-structured.

Write your research paper conclusion 2x faster with Paperpal. Try it now!

Remember that a well-crafted research paper conclusion is a reflection of the strength of your research and your ability to communicate its significance effectively. It should leave a lasting impression on your readers and tie together all the threads of your paper. Now you know how to start the conclusion of a research paper and what elements to include to make it impactful, let’s look at a research paper conclusion sample.

How to write a research paper conclusion with Paperpal?

A research paper conclusion is not just a summary of your study, but a synthesis of the key findings that ties the research together and places it in a broader context. A research paper conclusion should be concise, typically around one paragraph in length. However, some complex topics may require a longer conclusion to ensure the reader is left with a clear understanding of the study’s significance. Paperpal, an AI writing assistant trusted by over 800,000 academics globally, can help you write a well-structured conclusion for your research paper. 

• Sign Up or Log In: Create a new Paperpal account or login with your details.  
• Navigate to Features : Once logged in, head over to the features’ side navigation pane. Click on Templates and you’ll find a suite of generative AI features to help you write better, faster.  
• Generate an outline: Under Templates, select ‘Outlines’. Choose ‘Research article’ as your document type.  
• Select your section: Since you’re focusing on the conclusion, select this section when prompted.  
• Choose your field of study: Identifying your field of study allows Paperpal to provide more targeted suggestions, ensuring the relevance of your conclusion to your specific area of research. 
• Provide a brief description of your study: Enter details about your research topic and findings. This information helps Paperpal generate a tailored outline that aligns with your paper’s content. 
• Generate the conclusion outline: After entering all necessary details, click on ‘generate’. Paperpal will then create a structured outline for your conclusion, to help you start writing and build upon the outline.  
• Write your conclusion: Use the generated outline to build your conclusion. The outline serves as a guide, ensuring you cover all critical aspects of a strong conclusion, from summarizing key findings to highlighting the research’s implications. 
• Refine and enhance: Paperpal’s ‘Make Academic’ feature can be particularly useful in the final stages. Select any paragraph of your conclusion and use this feature to elevate the academic tone, ensuring your writing is aligned to the academic journal standards. 

By following these steps, Paperpal not only simplifies the process of writing a research paper conclusion but also ensures it is impactful, concise, and aligned with academic standards. Sign up with Paperpal today and write your research paper conclusion 2x faster .  

The research paper conclusion is a crucial part of your paper as it provides the final opportunity to leave a strong impression on your readers. In the research paper conclusion, summarize the main points of your research paper by restating your research statement, highlighting the most important findings, addressing the research questions or objectives, explaining the broader context of the study, discussing the significance of your findings, providing recommendations if applicable, and emphasizing the takeaway message. The main purpose of the conclusion is to remind the reader of the main point or argument of your paper and to provide a clear and concise summary of the key findings and their implications. All these elements should feature on your list of what to put in the conclusion of a research paper to create a strong final statement for your work.

A strong conclusion is a critical component of a research paper, as it provides an opportunity to wrap up your arguments, reiterate your main points, and leave a lasting impression on your readers. Here are the key elements of a strong research paper conclusion: 1. Conciseness : A research paper conclusion should be concise and to the point. It should not introduce new information or ideas that were not discussed in the body of the paper. 2. Summarization : The research paper conclusion should be comprehensive enough to give the reader a clear understanding of the research’s main contributions. 3 . Relevance : Ensure that the information included in the research paper conclusion is directly relevant to the research paper’s main topic and objectives; avoid unnecessary details. 4 . Connection to the Introduction : A well-structured research paper conclusion often revisits the key points made in the introduction and shows how the research has addressed the initial questions or objectives. 5. Emphasis : Highlight the significance and implications of your research. Why is your study important? What are the broader implications or applications of your findings? 6 . Call to Action : Include a call to action or a recommendation for future research or action based on your findings.

The length of a research paper conclusion can vary depending on several factors, including the overall length of the paper, the complexity of the research, and the specific journal requirements. While there is no strict rule for the length of a conclusion, but it’s generally advisable to keep it relatively short. A typical research paper conclusion might be around 5-10% of the paper’s total length. For example, if your paper is 10 pages long, the conclusion might be roughly half a page to one page in length.

In general, you do not need to include citations in the research paper conclusion. Citations are typically reserved for the body of the paper to support your arguments and provide evidence for your claims. However, there may be some exceptions to this rule: 1. If you are drawing a direct quote or paraphrasing a specific source in your research paper conclusion, you should include a citation to give proper credit to the original author. 2. If your conclusion refers to or discusses specific research, data, or sources that are crucial to the overall argument, citations can be included to reinforce your conclusion’s validity.

The conclusion of a research paper serves several important purposes: 1. Summarize the Key Points 2. Reinforce the Main Argument 3. Provide Closure 4. Offer Insights or Implications 5. Engage the Reader. 6. Reflect on Limitations

Remember that the primary purpose of the research paper conclusion is to leave a lasting impression on the reader, reinforcing the key points and providing closure to your research. It’s often the last part of the paper that the reader will see, so it should be strong and well-crafted.

• Makar, G., Foltz, C., Lendner, M., & Vaccaro, A. R. (2018). How to write effective discussion and conclusion sections. Clinical spine surgery, 31(8), 345-346.
• Bunton, D. (2005). The structure of PhD conclusion chapters.  Journal of English for academic purposes ,  4 (3), 207-224.

Paperpal is a comprehensive AI writing toolkit that helps students and researchers achieve 2x the writing in half the time. It leverages 21+ years of STM experience and insights from millions of research articles to provide in-depth academic writing, language editing, and submission readiness support to help you write better, faster.  

Get accurate academic translations, rewriting support, grammar checks, vocabulary suggestions, and generative AI assistance that delivers human precision at machine speed. Try for free or upgrade to Paperpal Prime starting at US$19 a month to access premium features, including consistency, plagiarism, and 30+ submission readiness checks to help you succeed.  

Experience the future of academic writing – Sign up to Paperpal and start writing for free!  

Related Reads:

• 5 Reasons for Rejection After Peer Review
• Ethical Research Practices For Research with Human Subjects

7 Ways to Improve Your Academic Writing Process

• Paraphrasing in Academic Writing: Answering Top Author Queries

Preflight For Editorial Desk: The Perfect Hybrid (AI + Human) Assistance Against Compromised Manuscripts

You may also like, measuring academic success: definition & strategies for excellence, phd qualifying exam: tips for success , ai in education: it’s time to change the..., is it ethical to use ai-generated abstracts without..., what are journal guidelines on using generative ai..., quillbot review: features, pricing, and free alternatives, what is an academic paper types and elements , should you use ai tools like chatgpt for..., 9 steps to publish a research paper, what are the different types of research papers.

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 9. The Conclusion
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

The conclusion is intended to help the reader understand why your research should matter to them after they have finished reading the paper. A conclusion is not merely a summary of the main topics covered or a re-statement of your research problem, but a synthesis of key points derived from the findings of your study and, if applicable, where you recommend new areas for future research. For most college-level research papers, two or three well-developed paragraphs is sufficient for a conclusion, although in some cases, more paragraphs may be required in describing the key findings and their significance.

Conclusions. The Writing Center. University of North Carolina; Conclusions. The Writing Lab and The OWL. Purdue University.

Importance of a Good Conclusion

A well-written conclusion provides you with important opportunities to demonstrate to the reader your understanding of the research problem. These include:

• Presenting the last word on the issues you raised in your paper . Just as the introduction gives a first impression to your reader, the conclusion offers a chance to leave a lasting impression. Do this, for example, by highlighting key findings in your analysis that advance new understanding about the research problem, that are unusual or unexpected, or that have important implications applied to practice.
• Summarizing your thoughts and conveying the larger significance of your study . The conclusion is an opportunity to succinctly re-emphasize  your answer to the "So What?" question by placing the study within the context of how your research advances past research about the topic.
• Identifying how a gap in the literature has been addressed . The conclusion can be where you describe how a previously identified gap in the literature [first identified in your literature review section] has been addressed by your research and why this contribution is significant.
• Demonstrating the importance of your ideas . Don't be shy. The conclusion offers an opportunity to elaborate on the impact and significance of your findings. This is particularly important if your study approached examining the research problem from an unusual or innovative perspective.
• Introducing possible new or expanded ways of thinking about the research problem . This does not refer to introducing new information [which should be avoided], but to offer new insight and creative approaches for framing or contextualizing the research problem based on the results of your study.

Bunton, David. “The Structure of PhD Conclusion Chapters.” Journal of English for Academic Purposes 4 (July 2005): 207–224; Conclusions. The Writing Center. University of North Carolina; Kretchmer, Paul. Twelve Steps to Writing an Effective Conclusion. San Francisco Edit, 2003-2008; Conclusions. The Writing Lab and The OWL. Purdue University; Assan, Joseph. "Writing the Conclusion Chapter: The Good, the Bad and the Missing." Liverpool: Development Studies Association (2009): 1-8.

Structure and Writing Style

I.  General Rules

The general function of your paper's conclusion is to restate the main argument . It reminds the reader of the strengths of your main argument(s) and reiterates the most important evidence supporting those argument(s). Do this by clearly summarizing the context, background, and necessity of pursuing the research problem you investigated in relation to an issue, controversy, or a gap found in the literature. However, make sure that your conclusion is not simply a repetitive summary of the findings. This reduces the impact of the argument(s) you have developed in your paper.

When writing the conclusion to your paper, follow these general rules:

• Present your conclusions in clear, concise language. Re-state the purpose of your study, then describe how your findings differ or support those of other studies and why [i.e., what were the unique, new, or crucial contributions your study made to the overall research about your topic?].
• Do not simply reiterate your findings or the discussion of your results. Provide a synthesis of arguments presented in the paper to show how these converge to address the research problem and the overall objectives of your study.
• Indicate opportunities for future research if you haven't already done so in the discussion section of your paper. Highlighting the need for further research provides the reader with evidence that you have an in-depth awareness of the research problem but that further investigations should take place beyond the scope of your investigation.

Consider the following points to help ensure your conclusion is presented well:

• If the argument or purpose of your paper is complex, you may need to summarize the argument for your reader.
• If, prior to your conclusion, you have not yet explained the significance of your findings or if you are proceeding inductively, use the end of your paper to describe your main points and explain their significance.
• Move from a detailed to a general level of consideration that returns the topic to the context provided by the introduction or within a new context that emerges from the data [this is opposite of the introduction, which begins with general discussion of the context and ends with a detailed description of the research problem]. 

The conclusion also provides a place for you to persuasively and succinctly restate the research problem, given that the reader has now been presented with all the information about the topic . Depending on the discipline you are writing in, the concluding paragraph may contain your reflections on the evidence presented. However, the nature of being introspective about the research you have conducted will depend on the topic and whether your professor wants you to express your observations in this way. If asked to think introspectively about the topics, do not delve into idle speculation. Being introspective means looking within yourself as an author to try and understand an issue more deeply, not to guess at possible outcomes or make up scenarios not supported by the evidence.

II.  Developing a Compelling Conclusion

Although an effective conclusion needs to be clear and succinct, it does not need to be written passively or lack a compelling narrative. Strategies to help you move beyond merely summarizing the key points of your research paper may include any of the following:

• If your essay deals with a critical, contemporary problem, warn readers of the possible consequences of not attending to the problem proactively.
• Recommend a specific course or courses of action that, if adopted, could address a specific problem in practice or in the development of new knowledge leading to positive change.
• Cite a relevant quotation or expert opinion already noted in your paper in order to lend authority and support to the conclusion(s) you have reached [a good source would be from your literature review].
• Explain the consequences of your research in a way that elicits action or demonstrates urgency in seeking change.
• Restate a key statistic, fact, or visual image to emphasize the most important finding of your paper.
• If your discipline encourages personal reflection, illustrate your concluding point by drawing from your own life experiences.
• Return to an anecdote, an example, or a quotation that you presented in your introduction, but add further insight derived from the findings of your study; use your interpretation of results from your study to recast it in new or important ways.
• Provide a "take-home" message in the form of a succinct, declarative statement that you want the reader to remember about your study.

III. Problems to Avoid

Failure to be concise Your conclusion section should be concise and to the point. Conclusions that are too lengthy often have unnecessary information in them. The conclusion is not the place for details about your methodology or results. Although you should give a summary of what was learned from your research, this summary should be relatively brief, since the emphasis in the conclusion is on the implications, evaluations, insights, and other forms of analysis that you make. Strategies for writing concisely can be found here .

Failure to comment on larger, more significant issues In the introduction, your task was to move from the general [the field of study] to the specific [the research problem]. However, in the conclusion, your task is to move from a specific discussion [your research problem] back to a general discussion framed around the implications and significance of your findings [i.e., how your research contributes new understanding or fills an important gap in the literature]. In short, the conclusion is where you should place your research within a larger context [visualize your paper as an hourglass--start with a broad introduction and review of the literature, move to the specific analysis and discussion, conclude with a broad summary of the study's implications and significance].

Failure to reveal problems and negative results Negative aspects of the research process should never be ignored. These are problems, deficiencies, or challenges encountered during your study. They should be summarized as a way of qualifying your overall conclusions. If you encountered negative or unintended results [i.e., findings that are validated outside the research context in which they were generated], you must report them in the results section and discuss their implications in the discussion section of your paper. In the conclusion, use negative results as an opportunity to explain their possible significance and/or how they may form the basis for future research.

Failure to provide a clear summary of what was learned In order to be able to discuss how your research fits within your field of study [and possibly the world at large], you need to summarize briefly and succinctly how it contributes to new knowledge or a new understanding about the research problem. This element of your conclusion may be only a few sentences long.

Failure to match the objectives of your research Often research objectives in the social and behavioral sciences change while the research is being carried out. This is not a problem unless you forget to go back and refine the original objectives in your introduction. As these changes emerge they must be documented so that they accurately reflect what you were trying to accomplish in your research [not what you thought you might accomplish when you began].

Resist the urge to apologize If you've immersed yourself in studying the research problem, you presumably should know a good deal about it [perhaps even more than your professor!]. Nevertheless, by the time you have finished writing, you may be having some doubts about what you have produced. Repress those doubts! Don't undermine your authority as a researcher by saying something like, "This is just one approach to examining this problem; there may be other, much better approaches that...." The overall tone of your conclusion should convey confidence to the reader about the study's validity and realiability.

Assan, Joseph. "Writing the Conclusion Chapter: The Good, the Bad and the Missing." Liverpool: Development Studies Association (2009): 1-8; Concluding Paragraphs. College Writing Center at Meramec. St. Louis Community College; Conclusions. The Writing Center. University of North Carolina; Conclusions. The Writing Lab and The OWL. Purdue University; Freedman, Leora  and Jerry Plotnick. Introductions and Conclusions. The Lab Report. University College Writing Centre. University of Toronto; Leibensperger, Summer. Draft Your Conclusion. Academic Center, the University of Houston-Victoria, 2003; Make Your Last Words Count. The Writer’s Handbook. Writing Center. University of Wisconsin Madison; Miquel, Fuster-Marquez and Carmen Gregori-Signes. “Chapter Six: ‘Last but Not Least:’ Writing the Conclusion of Your Paper.” In Writing an Applied Linguistics Thesis or Dissertation: A Guide to Presenting Empirical Research . John Bitchener, editor. (Basingstoke,UK: Palgrave Macmillan, 2010), pp. 93-105; Tips for Writing a Good Conclusion. Writing@CSU. Colorado State University; Kretchmer, Paul. Twelve Steps to Writing an Effective Conclusion. San Francisco Edit, 2003-2008; Writing Conclusions. Writing Tutorial Services, Center for Innovative Teaching and Learning. Indiana University; Writing: Considering Structure and Organization. Institute for Writing Rhetoric. Dartmouth College.

Writing Tip

Don't Belabor the Obvious!

Avoid phrases like "in conclusion...," "in summary...," or "in closing...." These phrases can be useful, even welcome, in oral presentations. But readers can see by the tell-tale section heading and number of pages remaining that they are reaching the end of your paper. You'll irritate your readers if you belabor the obvious.

Assan, Joseph. "Writing the Conclusion Chapter: The Good, the Bad and the Missing." Liverpool: Development Studies Association (2009): 1-8.

Another Writing Tip

New Insight, Not New Information!

Don't surprise the reader with new information in your conclusion that was never referenced anywhere else in the paper. This why the conclusion rarely has citations to sources. If you have new information to present, add it to the discussion or other appropriate section of the paper. Note that, although no new information is introduced, the conclusion, along with the discussion section, is where you offer your most "original" contributions in the paper; the conclusion is where you describe the value of your research, demonstrate that you understand the material that you’ve presented, and position your findings within the larger context of scholarship on the topic, including describing how your research contributes new insights to that scholarship.

Assan, Joseph. "Writing the Conclusion Chapter: The Good, the Bad and the Missing." Liverpool: Development Studies Association (2009): 1-8; Conclusions. The Writing Center. University of North Carolina.

• << Previous: Limitations of the Study
• Next: Appendices >>
• Last Updated: Apr 29, 2024 1:49 PM
• URL: https://libguides.usc.edu/writingguide

• Privacy Policy

Home » Research Paper Conclusion – Writing Guide and Examples

Research Paper Conclusion – Writing Guide and Examples

Table of Contents

Research Paper Conclusion

Definition:

A research paper conclusion is the final section of a research paper that summarizes the key findings, significance, and implications of the research. It is the writer’s opportunity to synthesize the information presented in the paper, draw conclusions, and make recommendations for future research or actions.

The conclusion should provide a clear and concise summary of the research paper, reiterating the research question or problem, the main results, and the significance of the findings. It should also discuss the limitations of the study and suggest areas for further research.

Parts of Research Paper Conclusion

The parts of a research paper conclusion typically include:

Restatement of the Thesis

The conclusion should begin by restating the thesis statement from the introduction in a different way. This helps to remind the reader of the main argument or purpose of the research.

Summary of Key Findings

The conclusion should summarize the main findings of the research, highlighting the most important results and conclusions. This section should be brief and to the point.

Implications and Significance

In this section, the researcher should explain the implications and significance of the research findings. This may include discussing the potential impact on the field or industry, highlighting new insights or knowledge gained, or pointing out areas for future research.

Limitations and Recommendations

It is important to acknowledge any limitations or weaknesses of the research and to make recommendations for how these could be addressed in future studies. This shows that the researcher is aware of the potential limitations of their work and is committed to improving the quality of research in their field.

Concluding Statement

The conclusion should end with a strong concluding statement that leaves a lasting impression on the reader. This could be a call to action, a recommendation for further research, or a final thought on the topic.

How to Write Research Paper Conclusion

Here are some steps you can follow to write an effective research paper conclusion:

• Restate the research problem or question: Begin by restating the research problem or question that you aimed to answer in your research. This will remind the reader of the purpose of your study.
• Summarize the main points: Summarize the key findings and results of your research. This can be done by highlighting the most important aspects of your research and the evidence that supports them.
• Discuss the implications: Discuss the implications of your findings for the research area and any potential applications of your research. You should also mention any limitations of your research that may affect the interpretation of your findings.
• Provide a conclusion : Provide a concise conclusion that summarizes the main points of your paper and emphasizes the significance of your research. This should be a strong and clear statement that leaves a lasting impression on the reader.
• Offer suggestions for future research: Lastly, offer suggestions for future research that could build on your findings and contribute to further advancements in the field.

Remember that the conclusion should be brief and to the point, while still effectively summarizing the key findings and implications of your research.

Example of Research Paper Conclusion

Here’s an example of a research paper conclusion:

Conclusion :

In conclusion, our study aimed to investigate the relationship between social media use and mental health among college students. Our findings suggest that there is a significant association between social media use and increased levels of anxiety and depression among college students. This highlights the need for increased awareness and education about the potential negative effects of social media use on mental health, particularly among college students.

Despite the limitations of our study, such as the small sample size and self-reported data, our findings have important implications for future research and practice. Future studies should aim to replicate our findings in larger, more diverse samples, and investigate the potential mechanisms underlying the association between social media use and mental health. In addition, interventions should be developed to promote healthy social media use among college students, such as mindfulness-based approaches and social media detox programs.

Overall, our study contributes to the growing body of research on the impact of social media on mental health, and highlights the importance of addressing this issue in the context of higher education. By raising awareness and promoting healthy social media use among college students, we can help to reduce the negative impact of social media on mental health and improve the well-being of young adults.

Purpose of Research Paper Conclusion

The purpose of a research paper conclusion is to provide a summary and synthesis of the key findings, significance, and implications of the research presented in the paper. The conclusion serves as the final opportunity for the writer to convey their message and leave a lasting impression on the reader.

The conclusion should restate the research problem or question, summarize the main results of the research, and explain their significance. It should also acknowledge the limitations of the study and suggest areas for future research or action.

Overall, the purpose of the conclusion is to provide a sense of closure to the research paper and to emphasize the importance of the research and its potential impact. It should leave the reader with a clear understanding of the main findings and why they matter. The conclusion serves as the writer’s opportunity to showcase their contribution to the field and to inspire further research and action.

When to Write Research Paper Conclusion

The conclusion of a research paper should be written after the body of the paper has been completed. It should not be written until the writer has thoroughly analyzed and interpreted their findings and has written a complete and cohesive discussion of the research.

Before writing the conclusion, the writer should review their research paper and consider the key points that they want to convey to the reader. They should also review the research question, hypotheses, and methodology to ensure that they have addressed all of the necessary components of the research.

Once the writer has a clear understanding of the main findings and their significance, they can begin writing the conclusion. The conclusion should be written in a clear and concise manner, and should reiterate the main points of the research while also providing insights and recommendations for future research or action.

Characteristics of Research Paper Conclusion

The characteristics of a research paper conclusion include:

• Clear and concise: The conclusion should be written in a clear and concise manner, summarizing the key findings and their significance.
• Comprehensive: The conclusion should address all of the main points of the research paper, including the research question or problem, the methodology, the main results, and their implications.
• Future-oriented : The conclusion should provide insights and recommendations for future research or action, based on the findings of the research.
• Impressive : The conclusion should leave a lasting impression on the reader, emphasizing the importance of the research and its potential impact.
• Objective : The conclusion should be based on the evidence presented in the research paper, and should avoid personal biases or opinions.
• Unique : The conclusion should be unique to the research paper and should not simply repeat information from the introduction or body of the paper.

Advantages of Research Paper Conclusion

The advantages of a research paper conclusion include:

• Summarizing the key findings : The conclusion provides a summary of the main findings of the research, making it easier for the reader to understand the key points of the study.
• Emphasizing the significance of the research: The conclusion emphasizes the importance of the research and its potential impact, making it more likely that readers will take the research seriously and consider its implications.
• Providing recommendations for future research or action : The conclusion suggests practical recommendations for future research or action, based on the findings of the study.
• Providing closure to the research paper : The conclusion provides a sense of closure to the research paper, tying together the different sections of the paper and leaving a lasting impression on the reader.
• Demonstrating the writer’s contribution to the field : The conclusion provides the writer with an opportunity to showcase their contribution to the field and to inspire further research and action.

Limitations of Research Paper Conclusion

While the conclusion of a research paper has many advantages, it also has some limitations that should be considered, including:

• I nability to address all aspects of the research: Due to the limited space available in the conclusion, it may not be possible to address all aspects of the research in detail.
• Subjectivity : While the conclusion should be objective, it may be influenced by the writer’s personal biases or opinions.
• Lack of new information: The conclusion should not introduce new information that has not been discussed in the body of the research paper.
• Lack of generalizability: The conclusions drawn from the research may not be applicable to other contexts or populations, limiting the generalizability of the study.
• Misinterpretation by the reader: The reader may misinterpret the conclusions drawn from the research, leading to a misunderstanding of the findings.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

How to Cite Research Paper – All Formats and...

Data Collection – Methods Types and Examples

Delimitations in Research – Types, Examples and...

Research Paper Format – Types, Examples and...

Research Process – Steps, Examples and Tips

Research Design – Types, Methods and Examples

When you choose to publish with PLOS, your research makes an impact. Make your work accessible to all, without restrictions, and accelerate scientific discovery with options like preprints and published peer review that make your work more Open.

• PLOS Biology
• PLOS Climate
• PLOS Complex Systems
• PLOS Computational Biology
• PLOS Digital Health
• PLOS Genetics
• PLOS Global Public Health
• PLOS Medicine
• PLOS Mental Health
• PLOS Neglected Tropical Diseases
• PLOS Pathogens
• PLOS Sustainability and Transformation
• PLOS Collections
• How to Write Discussions and Conclusions

The discussion section contains the results and outcomes of a study. An effective discussion informs readers what can be learned from your experiment and provides context for the results.

What makes an effective discussion?

When you’re ready to write your discussion, you’ve already introduced the purpose of your study and provided an in-depth description of the methodology. The discussion informs readers about the larger implications of your study based on the results. Highlighting these implications while not overstating the findings can be challenging, especially when you’re submitting to a journal that selects articles based on novelty or potential impact. Regardless of what journal you are submitting to, the discussion section always serves the same purpose: concluding what your study results actually mean.

A successful discussion section puts your findings in context. It should include:

• the results of your research,
• a discussion of related research, and
• a comparison between your results and initial hypothesis.

Tip: Not all journals share the same naming conventions.

You can apply the advice in this article to the conclusion, results or discussion sections of your manuscript.

Our Early Career Researcher community tells us that the conclusion is often considered the most difficult aspect of a manuscript to write. To help, this guide provides questions to ask yourself, a basic structure to model your discussion off of and examples from published manuscripts. 

Questions to ask yourself:

• Was my hypothesis correct?
• If my hypothesis is partially correct or entirely different, what can be learned from the results? 
• How do the conclusions reshape or add onto the existing knowledge in the field? What does previous research say about the topic? 
• Why are the results important or relevant to your audience? Do they add further evidence to a scientific consensus or disprove prior studies? 
• How can future research build on these observations? What are the key experiments that must be done? 
• What is the “take-home” message you want your reader to leave with?

How to structure a discussion

Trying to fit a complete discussion into a single paragraph can add unnecessary stress to the writing process. If possible, you’ll want to give yourself two or three paragraphs to give the reader a comprehensive understanding of your study as a whole. Here’s one way to structure an effective discussion:

Writing Tips

While the above sections can help you brainstorm and structure your discussion, there are many common mistakes that writers revert to when having difficulties with their paper. Writing a discussion can be a delicate balance between summarizing your results, providing proper context for your research and avoiding introducing new information. Remember that your paper should be both confident and honest about the results! 

• Read the journal’s guidelines on the discussion and conclusion sections. If possible, learn about the guidelines before writing the discussion to ensure you’re writing to meet their expectations. 
• Begin with a clear statement of the principal findings. This will reinforce the main take-away for the reader and set up the rest of the discussion. 
• Explain why the outcomes of your study are important to the reader. Discuss the implications of your findings realistically based on previous literature, highlighting both the strengths and limitations of the research. 
• State whether the results prove or disprove your hypothesis. If your hypothesis was disproved, what might be the reasons? 
• Introduce new or expanded ways to think about the research question. Indicate what next steps can be taken to further pursue any unresolved questions. 
• If dealing with a contemporary or ongoing problem, such as climate change, discuss possible consequences if the problem is avoided. 
• Be concise. Adding unnecessary detail can distract from the main findings. 

Don’t

• Rewrite your abstract. Statements with “we investigated” or “we studied” generally do not belong in the discussion. 
• Include new arguments or evidence not previously discussed. Necessary information and evidence should be introduced in the main body of the paper. 
• Apologize. Even if your research contains significant limitations, don’t undermine your authority by including statements that doubt your methodology or execution. 
• Shy away from speaking on limitations or negative results. Including limitations and negative results will give readers a complete understanding of the presented research. Potential limitations include sources of potential bias, threats to internal or external validity, barriers to implementing an intervention and other issues inherent to the study design. 
• Overstate the importance of your findings. Making grand statements about how a study will fully resolve large questions can lead readers to doubt the success of the research. 

Snippets of Effective Discussions:

Consumer-based actions to reduce plastic pollution in rivers: A multi-criteria decision analysis approach

Identifying reliable indicators of fitness in polar bears

• How to Write a Great Title
• How to Write an Abstract
• How to Write Your Methods
• How to Report Statistics
• How to Edit Your Work

The contents of the Peer Review Center are also available as a live, interactive training session, complete with slides, talking points, and activities. …

The contents of the Writing Center are also available as a live, interactive training session, complete with slides, talking points, and activities. …

There’s a lot to consider when deciding where to submit your work. Learn how to choose a journal that will help your study reach its audience, while reflecting your values as a researcher…

Organizing Academic Research Papers: 9. The Conclusion

• Purpose of Guide
• Design Flaws to Avoid
• Glossary of Research Terms
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Choosing a Title
• Making an Outline
• Paragraph Development
• Executive Summary
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tertiary Sources
• What Is Scholarly vs. Popular?
• Qualitative Methods
• Quantitative Methods
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Annotated Bibliography
• Dealing with Nervousness
• Using Visual Aids
• Grading Someone Else's Paper
• How to Manage Group Projects
• Multiple Book Review Essay
• Reviewing Collected Essays
• About Informed Consent
• Writing Field Notes
• Writing a Policy Memo
• Writing a Research Proposal
• Acknowledgements

The conclusion is intended to help the reader understand why your research should matter to them after they have finished reading the paper. A conclusion is not merely a summary of your points or a re-statement of your research problem but a synthesis of key points. For most essays, one well-developed paragraph is sufficient for a conclusion, although in some cases, a two-or-three paragraph conclusion may be required.

Importance of a Good Conclusion

A well-written conclusion provides you with several important opportunities to demonstrate your overall understanding of the research problem to the reader. These include:

• Presenting the last word on the issues you raised in your paper . Just as the introduction gives a first impression to your reader, the conclusion offers a chance to leave a lasting impression. Do this, for example, by highlighting key points in your analysis or findings.
• Summarizing your thoughts and conveying the larger implications of your study . The conclusion is an opportunity to succinctly answer the "so what?" question by placing the study within the context of past research about the topic you've investigated.
• Demonstrating the importance of your ideas . Don't be shy. The conclusion offers you a chance to elaborate on the significance of your findings.
• Introducing possible new or expanded ways of thinking about the research problem . This does not refer to introducing new information [which should be avoided], but to offer new insight and creative approaches for framing/contextualizing the research problem based on the results of your study.

Conclusions . The Writing Center. University of North Carolina; Kretchmer, Paul. Twelve Steps to Writing an Effective Conclusion . San Francisco Edit, 2003-2008.

Structure and Writing Style

https://writing.wisc.edu/wp-content/uploads/sites/535/2018/07/conclusions_uwmadison_writingcenter_aug2012.pdf I.  General Rules

When writing the conclusion to your paper, follow these general rules:

• State your conclusions in clear, simple language.
• Do not simply reiterate your results or the discussion.
• Indicate opportunities for future research, as long as you haven't already done so in the discussion section of your paper.

The function of your paper's conclusion is to restate the main argument . It reminds the reader of the strengths of your main argument(s) and reiterates the most important evidence supporting those argument(s). Make sure, however, that your conclusion is not simply a repetitive summary of the findings because this reduces the impact of the argument(s) you have developed in your essay.

Consider the following points to help ensure your conclusion is appropriate:

• If the argument or point of your paper is complex, you may need to summarize the argument for your reader.
• If, prior to your conclusion, you have not yet explained the significance of your findings or if you are proceeding inductively, use the end of your paper to describe your main points and explain their significance.
• Move from a detailed to a general level of consideration that returns the topic to the context provided by the introduction or within a new context that emerges from the data.

The conclusion also provides a place for you to persuasively and succinctly restate your research problem, given that the reader has now been presented with all the information about the topic . Depending on the discipline you are writing in, the concluding paragraph may contain your reflections on the evidence presented, or on the essay's central research problem. However, the nature of being introspective about the research you have done will depend on the topic and whether your professor wants you to express your observations in this way.

NOTE : Don't delve into idle speculation. Being introspective means looking within yourself as an author to try and understand an issue more deeply not to guess at possible outcomes.

II.  Developing a Compelling Conclusion

Strategies to help you move beyond merely summarizing the key points of your research paper may include any of the following.

• If your essay deals with a contemporary problem, warn readers of the possible consequences of not attending to the problem.
• Recommend a specific course or courses of action.
• Cite a relevant quotation or expert opinion to lend authority to the conclusion you have reached [a good place to look is research from your literature review].
• Restate a key statistic, fact, or visual image to drive home the ultimate point of your paper.
• If your discipline encourages personal reflection, illustrate your concluding point with a relevant narrative drawn from your own life experiences.
• Return to an anecdote, an example, or a quotation that you introduced in your introduction, but add further insight that is derived from the findings of your study; use your interpretation of results to reframe it in new ways.
• Provide a "take-home" message in the form of a strong, succient statement that you want the reader to remember about your study.

III. Problems to Avoid Failure to be concise The conclusion section should be concise and to the point. Conclusions that are too long often have unnecessary detail. The conclusion section is not the place for details about your methodology or results. Although you should give a summary of what was learned from your research, this summary should be relatively brief, since the emphasis in the conclusion is on the implications, evaluations, insights, etc. that you make. Failure to comment on larger, more significant issues In the introduction, your task was to move from general [the field of study] to specific [your research problem]. However, in the conclusion, your task is to move from specific [your research problem] back to general [your field, i.e., how your research contributes new understanding or fills an important gap in the literature]. In other words, the conclusion is where you place your research within a larger context. Failure to reveal problems and negative results Negative aspects of the research process should never be ignored. Problems, drawbacks, and challenges encountered during your study should be included as a way of qualifying your overall conclusions. If you encountered negative results [findings that are validated outside the research context in which they were generated], you must report them in the results section of your paper. In the conclusion, use the negative results as an opportunity to explain how they provide information on which future research can be based. Failure to provide a clear summary of what was learned In order to be able to discuss how your research fits back into your field of study [and possibly the world at large], you need to summarize it briefly and directly. Often this element of your conclusion is only a few sentences long. Failure to match the objectives of your research Often research objectives change while the research is being carried out. This is not a problem unless you forget to go back and refine your original objectives in your introduction, as these changes emerge they must be documented so that they accurately reflect what you were trying to accomplish in your research [not what you thought you might accomplish when you began].

Resist the urge to apologize If you've immersed yourself in studying the research problem, you now know a good deal about it, perhaps even more than your professor! Nevertheless, by the time you have finished writing, you may be having some doubts about what you have produced. Repress those doubts!  Don't undermine your authority by saying something like, "This is just one approach to examining this problem; there may be other, much better approaches...."

Concluding Paragraphs. College Writing Center at Meramec. St. Louis Community College; Conclusions . The Writing Center. University of North Carolina; Conclusions . The Writing Lab and The OWL. Purdue University; Freedman, Leora  and Jerry Plotnick. Introductions and Conclusions . The Lab Report. University College Writing Centre. University of Toronto; Leibensperger, Summer. Draft Your Conclusion. Academic Center, the University of Houston-Victoria, 2003; Make Your Last Words Count . The Writer’s Handbook. Writing Center. University of Wisconsin, Madison; Tips for Writing a Good Conclusion . Writing@CSU. Colorado State University; Kretchmer, Paul. Twelve Steps to Writing an Effective Conclusion . San Francisco Edit, 2003-2008; Writing Conclusions . Writing Tutorial Services, Center for Innovative Teaching and Learning. Indiana University; Writing: Considering Structure and Organization . Institute for Writing Rhetoric. Dartmouth College.

Writing Tip

Don't Belabor the Obvious!

Avoid phrases like "in conclusion...," "in summary...," or "in closing...." These phrases can be useful, even welcome, in oral presentations. But readers can see by the tell-tale section heading and number of pages remaining to read, when an essay is about to end. You'll irritate your readers if you belabor the obvious.

Another Writing Tip

New Insight, Not New Information!

Don't surprise the reader with new information in your Conclusion that was never referenced anywhere else in the paper. If you have new information to present, add it to the Discussion or other appropriate section of the paper.  Note that, although no actual new information is introduced, the conclusion is where you offer your most "original" contributions in the paper; it's where you describe the value of your research, demonstrate your understanding of the material that you’ve presented, and locate your findings within the larger context of scholarship on the topic.

• << Previous: Limitations of the Study
• Next: Appendices >>
• Last Updated: Jul 18, 2023 11:58 AM
• URL: https://library.sacredheart.edu/c.php?g=29803
• QuickSearch
• Library Catalog
• Databases A-Z
• Publication Finder
• Course Reserves
• Citation Linker
• Digital Commons
• Our Website

Research Support

• Ask a Librarian
• Appointments
• Interlibrary Loan (ILL)
• Research Guides
• Databases by Subject
• Citation Help

Using the Library

• Reserve a Group Study Room
• Renew Books
• Honors Study Rooms
• Off-Campus Access
• Library Policies
• Library Technology

User Information

• Grad Students
• Online Students
• COVID-19 Updates
• Staff Directory
• News & Announcements
• Library Newsletter

My Accounts

• Interlibrary Loan
• Staff Site Login

FIND US ON  

Implications or Recommendations in Research: What's the Difference?

• Peer Review

High-quality research articles that get many citations contain both implications and recommendations. Implications are the impact your research makes, whereas recommendations are specific actions that can then be taken based on your findings, such as for more research or for policymaking.

Updated on August 23, 2022

That seems clear enough, but the two are commonly confused.

This confusion is especially true if you come from a so-called high-context culture in which information is often implied based on the situation, as in many Asian cultures. High-context cultures are different from low-context cultures where information is more direct and explicit (as in North America and many European cultures).

Let's set these two straight in a low-context way; i.e., we'll be specific and direct! This is the best way to be in English academic writing because you're writing for the world.

Implications and recommendations in a research article

The standard format of STEM research articles is what's called IMRaD:

• Introduction
• Discussion/conclusions

Some journals call for a separate conclusions section, while others have the conclusions as the last part of the discussion. You'll write these four (or five) sections in the same sequence, though, no matter the journal.

The discussion section is typically where you restate your results and how well they confirmed your hypotheses. Give readers the answer to the questions for which they're looking to you for an answer.

At this point, many researchers assume their paper is finished. After all, aren't the results the most important part? As you might have guessed, no, you're not quite done yet.

The discussion/conclusions section is where to say what happened and what should now happen

The discussion/conclusions section of every good scientific article should contain the implications and recommendations.

The implications, first of all, are the impact your results have on your specific field. A high-impact, highly cited article will also broaden the scope here and provide implications to other fields. This is what makes research cross-disciplinary.

Recommendations, however, are suggestions to improve your field based on your results.

These two aspects help the reader understand your broader content: How and why your work is important to the world. They also tell the reader what can be changed in the future based on your results.

These aspects are what editors are looking for when selecting papers for peer review.

Implications and recommendations are, thus, written at the end of the discussion section, and before the concluding paragraph. They help to “wrap up” your paper. Once your reader understands what you found, the next logical step is what those results mean and what should come next.

Then they can take the baton, in the form of your work, and run with it. That gets you cited and extends your impact!

The order of implications and recommendations also matters. Both are written after you've summarized your main findings in the discussion section. Then, those results are interpreted based on ongoing work in the field. After this, the implications are stated, followed by the recommendations.

Writing an academic research paper is a bit like running a race. Finish strong, with your most important conclusion (recommendation) at the end. Leave readers with an understanding of your work's importance. Avoid generic, obvious phrases like "more research is needed to fully address this issue." Be specific.

The main differences between implications and recommendations (table)

Now let's dig a bit deeper into actually how to write these parts.

What are implications?

Research implications tell us how and why your results are important for the field at large. They help answer the question of “what does it mean?” Implications tell us how your work contributes to your field and what it adds to it. They're used when you want to tell your peers why your research is important for ongoing theory, practice, policymaking, and for future research.

Crucially, your implications must be evidence-based. This means they must be derived from the results in the paper.

Implications are written after you've summarized your main findings in the discussion section. They come before the recommendations and before the concluding paragraph. There is no specific section dedicated to implications. They must be integrated into your discussion so that the reader understands why the results are meaningful and what they add to the field.

A good strategy is to separate your implications into types. Implications can be social, political, technological, related to policies, or others, depending on your topic. The most frequently used types are theoretical and practical. Theoretical implications relate to how your findings connect to other theories or ideas in your field, while practical implications are related to what we can do with the results.

Key features of implications

• State the impact your research makes
• Helps us understand why your results are important
• Must be evidence-based
• Written in the discussion, before recommendations
• Can be theoretical, practical, or other (social, political, etc.)

Examples of implications

Let's take a look at some examples of research results below with their implications.

The result : one study found that learning items over time improves memory more than cramming material in a bunch of information at once .

The implications : This result suggests memory is better when studying is spread out over time, which could be due to memory consolidation processes.

The result : an intervention study found that mindfulness helps improve mental health if you have anxiety.

The implications : This result has implications for the role of executive functions on anxiety.

The result : a study found that musical learning helps language learning in children .

The implications : these findings suggest that language and music may work together to aid development.

What are recommendations?

As noted above, explaining how your results contribute to the real world is an important part of a successful article.

Likewise, stating how your findings can be used to improve something in future research is equally important. This brings us to the recommendations.

Research recommendations are suggestions and solutions you give for certain situations based on your results. Once the reader understands what your results mean with the implications, the next question they need to know is "what's next?"

Recommendations are calls to action on ways certain things in the field can be improved in the future based on your results. Recommendations are used when you want to convey that something different should be done based on what your analyses revealed.

Similar to implications, recommendations are also evidence-based. This means that your recommendations to the field must be drawn directly from your results.

The goal of the recommendations is to make clear, specific, and realistic suggestions to future researchers before they conduct a similar experiment. No matter what area your research is in, there will always be further research to do. Try to think about what would be helpful for other researchers to know before starting their work.

Recommendations are also written in the discussion section. They come after the implications and before the concluding paragraphs. Similar to the implications, there is usually no specific section dedicated to the recommendations. However, depending on how many solutions you want to suggest to the field, they may be written as a subsection.

Key features of recommendations

• Statements about what can be done differently in the field based on your findings
• Must be realistic and specific
• Written in the discussion, after implications and before conclusions
• Related to both your field and, preferably, a wider context to the research

Examples of recommendations

Here are some research results and their recommendations.

A meta-analysis found that actively recalling material from your memory is better than simply re-reading it .

• The recommendation: Based on these findings, teachers and other educators should encourage students to practice active recall strategies.

A medical intervention found that daily exercise helps prevent cardiovascular disease .

• The recommendation: Based on these results, physicians are recommended to encourage patients to exercise and walk regularly. Also recommended is to encourage more walking through public health offices in communities.

A study found that many research articles do not contain the sample sizes needed to statistically confirm their findings .

The recommendation: To improve the current state of the field, researchers should consider doing power analysis based on their experiment's design.

What else is important about implications and recommendations?

When writing recommendations and implications, be careful not to overstate the impact of your results. It can be tempting for researchers to inflate the importance of their findings and make grandiose statements about what their work means.

Remember that implications and recommendations must be coming directly from your results. Therefore, they must be straightforward, realistic, and plausible.

Another good thing to remember is to make sure the implications and recommendations are stated clearly and separately. Do not attach them to the endings of other paragraphs just to add them in. Use similar example phrases as those listed in the table when starting your sentences to clearly indicate when it's an implication and when it's a recommendation.

When your peers, or brand-new readers, read your paper, they shouldn't have to hunt through your discussion to find the implications and recommendations. They should be clear, visible, and understandable on their own.

That'll get you cited more, and you'll make a greater contribution to your area of science while extending the life and impact of your work.

The AJE Team

See our "Privacy Policy"

Careers in Qual

Quick answers, conclusions and recommendations, related terms.

Conducting Biosocial Surveys: Collecting, Storing, Accessing, and Protecting Biospecimens and Biodata (2010)

Chapter: 5 findings, conclusions, and recommendations, 5 findings, conclusions, and recommendations.

A s the preceding chapters have made clear, incorporating biological specimens into social science surveys holds great scientific potential, but also adds a variety of complications to the tasks of both individual researchers and institutions. These complications arise in a number of areas, including collecting, storing, using, and distributing biospecimens; sharing data while protecting privacy; obtaining informed consent from participants; and engaging with Institutional Review Boards (IRBs). Any effort to make such research easier and more effective will need to address the issues in these areas.

In considering its recommendations, the panel found it useful to think of two categories: (1) recommendations that apply to individual investigators, and (2) recommendations that are addressed to the National Institute on Aging (NIA) or other institutions, particularly funding agencies. Researchers who wish to collect biological specimens with social science data will need to develop new skills in a variety of areas, such as the logistics of specimen storage and management, the development of more diverse informed consent forms, and ways of dealing with the disclosure risks associated with sharing biogenetic data. At the same time, NIA and other funding agencies must provide researchers the tools they need to succeed. These tools include such things as biorepositories for maintaining and distributing specimens, better guidance on informed consent policies, and better ways to share data without risking confidentiality.

TAKING ADVANTAGE OF EXISTING EXPERTISE

Although working with biological specimens will be new and unfamiliar to many social scientists, it is an area in which biomedical researchers have a great deal of expertise and experience. Many existing documents describe recommended procedures and laboratory practices for the handling of biospecimens. These documents provide an excellent starting point for any social scientist who is interested in adding biospecimens to survey research.

Recommendation 1: Social scientists who are planning to add biological specimens to their survey research should familiarize themselves with existing best practices for the collection, storage, use, and distribution of biospecimens. First and foremost, the design of the protocol for collec tion must ensure the safety of both participants and survey staff (data and specimen collectors and handlers).

Although existing best-practice documents were not developed with social science surveys in mind, their guidelines have been field-tested and approved by numerous IRBs and ethical oversight committees. The most useful best-practice documents are updated frequently to reflect growing knowledge and changing opinions about the best ways to collect, store, use, and distribute biological specimens. At the same time, however, many issues arising from the inclusion of biospecimens in social science surveys are not fully addressed in the best-practice documents intended for biomedical researchers. For guidance on these issues, it will be necessary to seek out information aimed more specifically at researchers at the intersection of social science and biomedicine.

COLLECTING, STORING, USING, AND DISTRIBUTING BIOSPECIMENS

As described in Chapter 2 , the collection, storage, use, and distribution of biospecimens and biodata are tasks that are likely to be unfamiliar to many social scientists and that raise a number of issues with which even specialists are still grappling. For example, which biospecimens in a repository should be shared, given that in most cases the amount of each specimen is limited? And given that the available technology for cost-efficient analysis of biospecimens, particularly genetic analysis, is rapidly improving, how much of any specimen should be used for immediate research and analysis, and how much should be stored for analysis at a later date? Collecting, storing, using, and distributing biological specimens also present significant practical and financial challenges for social scientists. Many of the questions they must address, such as exactly what should be held, where it should be held, and what should be shared or distributed, have not yet been resolved.

Developing Data Sharing Plans

An important decision concerns who has access to any leftover biospecimens. This is a problem more for biospecimens than for biodata because in most cases, biospecimens can be exhausted. Should access be determined according to the principle of first funded, first served? Should there be a formal application process for reviewing the scientific merits of a particular investigation? For studies that involve international collaboration, should foreign investigators have access? And how exactly should these decisions be made? Recognizing that some proposed analyses may lie beyond the competence of the original investigators, as well as the possibility that principal investigators may have a conflict of interest in deciding how to use any remaining biospecimens, one option is for a principal investigator to assemble a small scientific committee to judge the merits of each application, including the relevance of the proposed study to the parent study and the capacities of the investigators. Such committees should publish their review criteria to help prospective applicants. A potential problem with such an approach, however, is that many projects may not have adequate funding to carry out such tasks.

Recommendation 2: Early in the planning process, principal investigators who will be collecting biospecimens as part of a social science survey should develop a complete data sharing plan.

This plan should spell out the criteria for allowing other researchers to use (and therefore deplete) the available stock of biospecimens, as well as to gain access to any data derived therefrom. To avoid any appearance of self-interest, a project might empower an external advisory board to make decisions about access to its data. The data sharing plan should also include provisions for the storage and retrieval of biospecimens and clarify how the succession of responsibility for and control of the biospecimens will be handled at the conclusion of the project.

Recommendation 3: NIA (or preferably the National Institutes of Health [NIH]) should publish guidelines for principal investigators containing a list of points that need to be considered for an acceptable data sharing plan. In addition to staff review, Scientific Review Panels should read and comment on all proposed data sharing plans. In much the same way as an unacceptable human subjects plan, an inadequate data sharing plan should hold up an otherwise acceptable proposal.

Supporting Social Scientists in the Storage of Biospecimens

The panel believes that many social scientists who decide to add the collection of biospecimens to their surveys may be ill equipped to provide for the storage and distribution of the specimens.

Conclusion: The issues related to the storage and distribution of biospecimens are too complex and involve too many hidden costs to assume that social scientists without suitable knowledge, experience, and resources can handle them without assistance.

Investigators should therefore have the option of delegating the storage and distribution of biospecimens collected as part of social science surveys to a centralized biorepository. Depending on the circumstances, a project might choose to utilize such a facility for immediate use, long-term or archival storage, or not at all.

Recommendation 4: NIA and other relevant funding agencies should support at least one central facility for the storage and distribution of biospecimens collected as part of the research they support.

PROTECTING PRIVACY AND CONFIDENTIALITY: SHARING DIGITAL REPRESENTATIONS OF BIOLOGICAL AND SOCIAL DATA

Several different types of data must be kept confidential: survey data, data derived from biospecimens, and all administrative and operational data. In the discussion of protecting confidentiality and privacy, this report has focused on biodata, but the panel believes it is important to protect all the data collected from survey participants. For many participants, for example, data on wealth, earnings, or sexual behavior can be as or more sensitive than genetic data.

Conclusion: Although biodata tend to receive more attention in discussions of privacy and confidentiality, social science and operational data can be sensitive in their own right and deserve similar attention in such discussions.

Protecting the participants in a social science survey that collects biospecimens requires securing the data, but data are most valuable when they are made available to researchers as widely as possible. Thus there is an inherent tension between the desire to protect the privacy of the participants and the desire to derive as much scientific value from the data as possible, particularly since the costs of data collection and analysis are so high. The following recommendations regarding confidentiality are made in the spirit of balancing these equally important needs.

Genomic data present a particular challenge. Several researchers have demonstrated that it is possible to identify individuals with even modest amounts of such data. When combined with social science data, genomic data may pose an even greater risk to confidentiality. It is difficult to know how much or which genomic data, when combined with social science data, could become critical identifiers in the future. Although the problem is most significant with genomic data, similar challenges can arise with other kinds of data derived from biospecimens.

Conclusion: Unrestricted distribution of genetic and other biodata risks violating promises of confidentiality made to research participants.

There are two basic approaches to protecting confidentiality: restricting data and restricting access. Restricting data—for example, by stripping individual and spatial identifiers and modifying the data to make it difficult or impossible to trace them back to their source—usually makes it possible to release social science data widely. In the case of biodata, however, there is no answer to how little data is required to make a participant uniquely identifiable. Consequently, any release of biodata must be carefully managed to protect confidentiality.

Recommendation 5: No individual-level data containing uniquely identify ing variables, such as genomic data, should be publicly released without explicit informed consent.

Recommendation 6: Genomic data and other individual-level data con taining uniquely identifying variables that are stored or in active use by investigators on their institutional or personal computers should be encrypted at all times.

Even if specific identifying variables, such as names and addresses, are stripped from data, it is still often possible to identify the individuals associated with the data by other means, such as using the variables that remain (age, sex, marital status, family income, etc.) to zero in on possible candidates. In the case of biodata that do not uniquely identify individuals and can change with time, such as blood pressure and physical measurements, it may be possible to share the data with no more protection than stripping identifying variables. Even these data, however, if known to intruders, can increase identification disclosure risk when combined with enough other data. With sufficient characteristics to match, intruders can uniquely identify individuals in shared data if given access to another data source that contains the same information plus identifiers.

Conclusion: Even nonunique biodata, if combined with social science data, may pose a serious risk of reidentification.

In the case of high-dimensional genomic data, standard disclosure limitation techniques, such as data perturbation, are not effective with respect to preserving the utility of the data because they involve such extreme alterations that they would severely distort analyses aimed at determining gene–gene and gene–environment interactions. Standard disclosure limitation methods could be used to generate public-use data sets that would enable low-dimensional analyses involving genes, for example, one gene at a time. However, with several such public releases, it may be possible for a key match to be used to construct a data set with higher-dimensional genomic data.

Conclusion: At present, no data restriction strategy has been demonstrated to protect confidentiality while preserving the usefulness of the data for drawing inferences involving high-dimensional interactions among genomic and social science variables, which are increasingly the target of research. Providing public-use genomic data requires such intense data masking to protect confidentiality that it would distort the high-dimensional analyses that could result in ground-breaking research progress.

Recommendation 7: Both rich genomic data acquired for research and sensitive and potentially identifiable social science data that do not change (or change very little) with time should be shared only under restricted circumstances, such as licensing and (actual or virtual) data enclaves.

As discussed in Chapter 3 , the four basic ways to restrict access to data are licensing, remote execution centers, data enclaves, and virtual data enclaves. Each has its advantages and disadvantages. 1 Licensing, for example, is the least restrictive for a researcher in terms of access to the data, but the licensing process itself can be lengthy and burdensome. Thus it would be useful if the licensing process could be facilitated.

Recommendation 8: NIA (or preferably NIH) should develop new stan dards and procedures for licensing confidential data in ways that will maximize timely access while maintaining security and that can be used by data repositories and by projects that distribute data.

Ways to improve the other approaches to restricted access are needed as well. For example, improving the convenience and availability of virtual data enclaves could increase the use of combined social science and biodata without

a significant increase in risk to confidentiality. The panel notes that much of the discussion of the confidentiality risk posed by the various approaches is theoretical; no one has a clear idea of just what disclosure risks are associated with the various ways of sharing data. It is important to learn more about these disclosure risks for a variety of reasons—determining how to minimize the risks, for instance, or knowing which approaches to sharing data pose the least risk. It would also be useful to be able to describe disclosure risks more accurately to survey participants.

Recommendation 9: NIA and other funding agencies should assess the strength of confidentiality protections through periodic expert audits of confidentiality and computer security. Willingness to participate in such audits should be a condition for receipt of NIA support. Beyond enforce ment, the purpose of such audits would be to identify challenges and solutions.

Evaluating risks and applying protection methods, whether they involve restricted access or restricted data, is a complex process requiring expertise in disclosure protection methods that exceeds what individual principal investigators and their institutions usually possess. Currently, not enough is known to be able to represent these risks either fully or accurately. The NIH requirement for data sharing necessitates a large investment of resources to anticipate which variables are potentially available to intruders and to alter data in ways that reduce disclosure risks while maintaining the utility of the data. Such resources are better spent by principal investigators on collecting and analyzing the data.

Recommendation 10: NIH should consider funding Centers of Excellence to explore new ways of protecting digital representations of data and to assist principal investigators wishing to share data with others. NIH should also support research on disclosure risks and limitations.

Principal investigators could send digital data to these centers, which would organize and manage any restricted access or restricted data policies or provide advisory services to investigators. NIH would maintain the authority to penalize those who violated any confidentiality agreements, for example, by denying them or their home institution NIH funding. Models for these centers include the Inter-university Consortium for Political and Social Research (ICPSR) and its projects supported by NIH and the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD) and the UK data sharing archive. The centers would alleviate the burden of data sharing as mandated of principal investigators by NIH and place it in expert hands. However, excellence in the design of data access and control systems

is likely to require intimate knowledge of each specific data resource, so data producers should be involved in the systems’ development.

INFORMED CONSENT

As described in Chapter 4 , informed consent is a complex subject involving many issues that are still being debated; the growing power of genetic analysis techniques and bioinformatics has only added to this complexity. Given the rapid pace of advances in scientific knowledge and in the technology used to analyze biological materials, it is impossible to predict what information might be gleaned from biological specimens just a few years hence; accordingly, it is impossible, even in theory, to talk about perfectly informed consent. The best one can hope for is relatively well-informed consent from a study’s participants, but knowing precisely what that means is difficult. Determining the scope of informed consent adds another layer of complexity. Will new analyses be covered under the existing consent, for example? There are no clear guidelines on such questions, yet specific details on the scope of consent will likely affect an IRB’s reaction to a study proposal.

What Individual Researchers Need to Know and Do Regarding Informed Consent

To be sure, there is a wide range of views about the practicality of providing adequate protection to participants while proceeding with the scientific enterprise, from assertions that it is simply not possible to provide adequate protection to offers of numerous procedural safeguards but no iron-clad guarantees. This report takes the latter position—that investigators should do their best to communicate adequately and accurately with participants, to provide procedural safeguards to the extent possible, and not to promise what is not possible. 2 Social science researchers need to know that adding the collection of biospecimens to social science surveys changes the nature of informed consent. Informed consent for a traditional social science survey may entail little more than reading a short script over the phone and asking whether the participant is willing to continue; obtaining informed consent for the collection and use of biospecimens and biodata is generally a much more involved process.

Conclusion: Social scientists should be made aware that the process of obtaining informed consent for the use of biospecimens and biodata typically differs from social science norms.

If participants are to provide truly informed consent to taking part in any study, they must be given a certain minimum amount of information. They should be told, for example, what the purpose of the study is, how it is to be carried out, and what participants’ roles are. In addition, because of the unique risks associated with providing biospecimens, participants in a social science survey that involves the collection of such specimens should be provided with other types of information as well. In particular, they should be given detail on the storage and use of the specimens that relates to those risks and can assist them in determining whether to take part in the study.

Recommendation 11: In designing a consent form for the collection of biospecimens, in addition to those elements that are common to social science and biomedical research, investigators should ensure that certain other information is provided to participants:

how long researchers intend to retain their biospecimens and the genomic and other biodata that may be derived from them;

both the risks associated with genomic data and the limits of what they can reveal;

which other researchers will have access to their specimens, to the data derived therefrom, and to information collected in a survey questionnaire;

the limits on researchers’ ability to maintain confidentiality;

any potential limits on participants’ ability to withdraw their speci mens or data from the research;

the penalties 3 that may be imposed on researchers for various types of breaches of confidentiality; and

what plans have been put in place to return to them any medically relevant findings.

Researchers who fail to properly plan for and handle all of these issues before proceeding with a study are in essence compromising assurances under informed consent. The literature on informed consent emphasizes the importance of ensuring that participants understand reasonably well what they are consenting to. This understanding cannot be taken for granted, particularly as it pertains to the use of biological specimens and the data derived therefrom.

While it is not possible to guarantee that participants have a complete understanding of the scientific uses of their specimens or all the possible risks of their participation, they should be able to make a relatively well-informed decision about whether to take part in the study. Thus the ability of various participants to understand the research and the informed consent process must be considered. Even impaired individuals may be able to participate in research if their interests are protected and they can do so only through proxy consent. 4

Recommendation 12: NIA should locate and publicize positive examples of the documentation of consent processes for the collection of biospeci mens. In particular, these examples should take into account the special needs of certain individuals, such as those with sensory problems and the cognitively impaired.

Participants in a biosocial survey are likely to have different levels of comfort concerning how their biospecimens and data will be used. Some may be willing to provide only answers to questions, for example, while others may both answer questions and provide specimens. Among those who provide specimens, some may be willing for the specimens to be used only for the current study, while others may consent to their use in future studies. One effective way to deal with these different comfort levels is to offer a tiered approach to consent that allows participants to determine just how their specimens and data will be used. Tiers might include participating in the survey, providing specimens for genetic and/or nongenetic analysis in a particular study, and allowing the specimens and data to be stored for future uses (genetic and/or nongenetic). For those participants who are willing to have their specimens and data used in future studies, researchers should tell them what sort of approval will be obtained for such use. For example, an IRB may demand reconsent, in which case participants may have to be contacted again before their specimens and data can be used. Ideally, researchers should design their consent forms to avoid the possibility that an IRB will demand a costly or infeasible reconsent process.

Recommendation 13: Researchers should consider adopting a tiered approach to obtaining consent. Participants who are willing to have their specimens and data used in future studies should be informed about the process that will be used to obtain approval for such uses.

What Institutions Should Do Regarding Informed Consent

Because the details of informed consent vary from study to study, individual investigators must bear ultimate responsibility for determining the details of informed consent for any particular study. Thus researchers must understand the various issues and concerns surrounding informed consent and be prepared to make decisions about the appropriate approach for their research in consultation with staff of survey organizations. These decisions should be addressed in the training of survey interviewers. As noted above, however, the issues surrounding informed consent are complex and not completely resolved, and researchers have few options for learning about informed consent as it applies to social science studies that collect biospecimens. Thus it makes sense for agencies funding this research, the Office for Human Research Protection (OHRP), or other appropriate organizations (for example, Public Responsibility in Medicine and Research [PRIM&R]) to provide opportunities for such learning, taking into account the fact that the issues arising in biosocial research do not arise in the standard informed consent situations encountered in social science research. It should also be made clear that the researchers’ institution is usually deemed (e.g., in the courts) to bear much of the responsibility for informed consent.

Recommendation 14: NIA, OHRP, and other appropriate organizations should sponsor training programs, create training modules, and hold informational workshops on informed consent for investigators, staff of survey organizations, including field staff, administrators, and mem bers of IRBs who oversee surveys that collect social science data and biospecimens.

The Return of Medically Relevant Information

An issue related to informed consent is how much information to provide to survey participants once their biological specimens have been analyzed and in particular, how to deal with medically relevant information that may arise from the analysis. What, for example, should a researcher do if a survey participant is found to have a genetic disease that does not appear until later in life? Should the participant be notified? Should participants be asked as part of the initial interview whether they wish to be notified about such a discovery? At this time, there are no generally agreed-upon answers to such questions, but researchers should expect to have to deal with these issues as they analyze the data derived from biological specimens.

Recommendation 15: NIH should direct investigators to formulate a plan in advance concerning the return of any medically relevant findings to

survey participants and to implement that plan in the design and conduct of their informed consent procedures.

INSTITUTIONAL REVIEW BOARDS

Investigators seeking IRB approval for biosocial research face a number of challenges. Few IRBs are familiar with both social and biological science; thus, investigators may find themselves trying to justify standard social science protocols to a biologically oriented IRB or explaining standard biological protocols to an IRB that is used to dealing with social science—or sometimes both. Researchers can expect these obstacles, which arise from the interdisciplinary nature of their work, to be exacerbated by a number of other factors that are characteristic of IRBs in general (see Chapter 4 ).

Recommendation 16: In institutions that have separate biomedical and social science IRBs, mechanisms should be created for sharing expertise during the review of biosocial protocols. 5

What Individual Researchers Need to Do Regarding IRBs

Because the collection of biospecimens as part of social science surveys is still relatively unfamiliar to many IRBs, researchers planning such a study can expect their interactions with the IRB overseeing the research to involve a certain learning curve. The IRB may need extra time to become familiar and comfortable with the proposed practices of the survey, and conversely, the researchers will need time to learn what the IRB will require. Thus it will be advantageous if researchers conducting such studies plan from the beginning to devote additional time to working with their IRBs.

Recommendation 17: Investigators considering collecting biospecimens as part of a social science survey should consult with their IRBs early and often.

What Research Agencies Should Do Regarding IRBs

One way to improve the IRB process would be to give members of IRBs an opportunity to learn more about biosocial research and the risks it entails.

This could be done by individual institutions, but it would be more effective if a national funding agency took the lead (see Recommendation 14).

It is the panel’s hope that its recommendations will support the incorporation of social science and biological data into empirical models, allowing researchers to better document the linkages among social, behavioral, and biological processes that affect health and other measures of well-being while avoiding or minimizing many of the challenges that may arise. Implementing these recommendations will require the combined efforts of both individual investigators and the agencies that support them.

This page intentionally left blank.

Recent years have seen a growing tendency for social scientists to collect biological specimens such as blood, urine, and saliva as part of large-scale household surveys. By combining biological and social data, scientists are opening up new fields of inquiry and are able for the first time to address many new questions and connections. But including biospecimens in social surveys also adds a great deal of complexity and cost to the investigator's task. Along with the usual concerns about informed consent, privacy issues, and the best ways to collect, store, and share data, researchers now face a variety of issues that are much less familiar or that appear in a new light.

In particular, collecting and storing human biological materials for use in social science research raises additional legal, ethical, and social issues, as well as practical issues related to the storage, retrieval, and sharing of data. For example, acquiring biological data and linking them to social science databases requires a more complex informed consent process, the development of a biorepository, the establishment of data sharing policies, and the creation of a process for deciding how the data are going to be shared and used for secondary analysis--all of which add cost to a survey and require additional time and attention from the investigators. These issues also are likely to be unfamiliar to social scientists who have not worked with biological specimens in the past. Adding to the attraction of collecting biospecimens but also to the complexity of sharing and protecting the data is the fact that this is an era of incredibly rapid gains in our understanding of complex biological and physiological phenomena. Thus the tradeoffs between the risks and opportunities of expanding access to research data are constantly changing.

Conducting Biosocial Surveys offers findings and recommendations concerning the best approaches to the collection, storage, use, and sharing of biospecimens gathered in social science surveys and the digital representations of biological data derived therefrom. It is aimed at researchers interested in carrying out such surveys, their institutions, and their funding agencies.

READ FREE ONLINE

Welcome to OpenBook!

You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

Do you want to take a quick tour of the OpenBook's features?

Show this book's table of contents , where you can jump to any chapter by name.

...or use these buttons to go back to the previous chapter or skip to the next one.

Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

Switch between the Original Pages , where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

To search the entire text of this book, type in your search term here and press Enter .

Share a link to this book page on your preferred social network or via email.

View our suggested citation for this chapter.

Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

Get Email Updates

Do you enjoy reading reports from the Academies online for free ? Sign up for email notifications and we'll let you know about new publications in your areas of interest when they're released.

• FREE PROJECT TOPICS
• NCE PROJECT TOPICS
• OND PROJECT TOPICS
• HND PROJECT TOPICS
• BSC PROJECT TOPICS
• PGDE PROJECT TOPICS
• MBA PROJECT TOPICS
• MSC PROJECT TOPICS
• HIRE A WRITER FOR NCE
• HIRE A WRITER FOR OND
• HIRE A WRITER FOR HND
• HIRE A WRITER FOR BSC
• HIRE A WRITER FOR PGD
• HIRE A WRITER FOR MSC
• HIRE A WRITER FOR MBA

How to Write Summary, Conclusion and Recommendations in a Research

The summary, conclusion and recommendations are the last part of the research work . It is essential to have this in all research work, both qualitative and quantitative . When perusing a research work (thesis or research articles ), the readers go directly to the summary, conclusion, and recommendation and see if they could point out what the dissertation is saying because it is common in this chapter.

The gap in the literature identified in the introduction signifies what the scholars decided to look at, what they finally assert when their study is done. What did it inform them, and what they are teaching us about the matter. Did they get the expected outcomes? If so, why or why not? The thesis is likely to be a very logical and provisional assertion instead of a broad statement.

Almost every research study concludes by encouraging other academics to continue the work by stating that more investigation is necessary on the topic. Nevertheless, please do not confuse this ruling with the thesis; it is merely a tradition. Frequently, the researchers offer detailed information about possible future research that might or can be undertaken in an attempt to make sense of the findings of their research. The following steps will guide in write a good chapter five:

Step 1: Summary

It is now time to go through each section and highlight the critical statements. What information does the reader have to fully comprehend the article’s central argument or inference? Remember that a summary does not necessitate rephrasing every single line of the article. The idea is to identify the main elements while excluding any background knowledge or optional information. A summary of findings reveals and summarises the most critical factors and outcomes of a study, including the best theoretical boundaries and the finality of the substantiation for each result. It tells the reader what has been done, how it has been done, and the study results. An engaging summary of findings allows the readers to see as many or more minor findings and just about relevant data about each result, see effect estimates presented in various ways, and view clarifications of the evidence supplied.

Step 2: Conclusions

After analysing the literature, the conclusion should aid in understanding why the study is essential to them. A conclusion is a synthesis of critical elements, not just a description of the points or a re-statement of the problem statement. For most research studies, one well-developed paragraph suffices as a conclusion. However, a two- or three paragraph conclusion may be considered necessary in some situations. It is vital to include a conclusion in a thesis, journal article or dissertation to inform the readers of the strength and effect of the assertions in the study. Concluding statements in a thesis can also aid in refocusing the reader’s attention on the quality statements and verifiable details of the research. Conclusions can also form a foundation for further research, generate new ideas to address an issue raised in the thesis or propose novel approaches to a problem. Consider the steps below to help you get started when writing the conclusion of your study:

• Restate the research topic.
• Reiterate the thesis (objective of the study).
• Make a summary of the main points.
• Mention the relevance or outcomes.
• Wrap up your thoughts.

Avoidable Issues

• Inability to be concise.
• Inability to make a statement on more significant, more important issues.
• Failure to expose problems leads to adverse outcomes.
• Inability to provide a brief overview of what was observed.
• Failure to align the research aim and objectives.
• Refrain from apologizing.

Step 3: Recommendations

You may have already created suggestions for future studies in the discussion, but the recommendation is a great place to explain, taking into account the potential ramifications of your research results for practice and theory. The recommendations should be premised on the conclusions of the study.

Specific instances

• Predicated on these conclusions, managers should consider… Additional researches could address…
• Further research is necessary to confirm the causative factors of/effects of/relationship between…

Avoid overstating the practicability of the study. If you’re making policy, business, or other policy implications, it’s best to structure them as suggestions instead of instructions. Academic research aims to educate, demonstrate, and explore rather than to instruct.

Make sure not to undermine the research carried out when making recommendations for additional research. Academic research aims to educate, demonstrate, and explore rather than to instruct.

Make sure not to undermine the research carried out when making recommendations for additional research.

Step 4: Recommendations for further study

Future studies may confirm, build on, or supplement your findings, but they should not be considered necessary to accomplish them. Highlight the contributions. Make sure the reader understands how the study has contributed to knowledge in the field in focus.

The suggestions for further study should address other areas that your study did not cover. That is, suggestions for further study should expand on the limitations and scope of your study.

Conclusions and Recommendations for Future Research

• First Online: 08 April 2021

Cite this chapter

• Gert Janssenswillen   ORCID: orcid.org/0000-0002-7474-2088 7  

Part of the book series: Lecture Notes in Business Information Processing ((LNBIP,volume 412))

218 Accesses

At the start of this thesis, we set out on a quest for process realism: viewing and representing processes as they really are, as distinguished from the speculative.

Puzzles are sort of like life because you can mess up and rebuild later, and you’re likely smarter the next time around. Adam Silvera

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Author information

Authors and affiliations.

Research Group Business Informatics, Hasselt University, Diepenbeek, Belgium

Gert Janssenswillen

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Gert Janssenswillen .

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this chapter

Janssenswillen, G. (2021). Conclusions and Recommendations for Future Research. In: Unearthing the Real Process Behind the Event Data. Lecture Notes in Business Information Processing, vol 412. Springer, Cham. https://doi.org/10.1007/978-3-030-70733-0_10

Download citation

DOI : https://doi.org/10.1007/978-3-030-70733-0_10

Published : 08 April 2021

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-70732-3

Online ISBN : 978-3-030-70733-0

eBook Packages : Computer Science Computer Science (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

• The Importance of Conclusions and Recommendations in the Monitoring and Evaluation (M&E) Process
• Learning Center

Drawing conclusions and making recommendations are other important aspects of the monitoring and evaluation process . The conclusions and recommendations that are drawn from the evaluation findings can help to inform decision-making, improve program effectiveness, and promote learning and accountability.

Here are some key considerations when drawing conclusions and making recommendations in the monitoring and evaluation process.

Table of Contents

What are the conclusions?

Examples of conclusions in the monitoring and evaluation (m&e), what are the recommendations, examples of recommendations in the monitoring and evaluation (m&e), example of a conclusion and recommendation section from a monitoring and evaluation report, review the evaluation findings, consider the context, identify strengths and weaknesses, make actionable recommendations, communicate findings and recommendations.

Conclusions in the Monitoring and Evaluation (M&E) process refer to the key findings or results of the evaluation that provide insights into the effectiveness and efficiency of the program or intervention being evaluated. They are based on the analysis of data collected during the evaluation process and are used to determine the extent to which the program or intervention has achieved its intended objectives.

Conclusions in M&E typically include a summary of the evaluation results, highlighting what worked well and what did not work. They also provide an assessment of the program or intervention’s impact, sustainability, and scalability. Conclusions are an essential component of the M&E process as they provide decision-makers with evidence-based information that can be used to improve future programs or interventions.

Catch HR’s eye instantly?

• Resume Review
• Resume Writing
• Resume Optimization

Premier global development resume service since 2012

Stand Out with a Pro Resume

Here are some examples of conclusions in the Monitoring and Evaluation (M&E) process:

• Program effectiveness: The evaluation may conclude that the program was effective in achieving its intended objectives, based on the analysis of data collected during the evaluation process. For example, if a health education program was intended to increase knowledge of healthy eating habits and the evaluation found that the program achieved this goal, the conclusion would be that the program was effective.
• Program efficiency: The evaluation may conclude that the program was efficient in achieving its objectives, based on the resources used to implement the program. For example, if a job training program was implemented at a lower cost than similar programs and achieved similar outcomes, the conclusion would be that the program was efficient.
• Program impact: The evaluation may conclude that the program had a positive impact on the target population, based on the analysis of data collected during the evaluation process. For example, if a youth development program was intended to reduce delinquency rates and the evaluation found a significant reduction in delinquency rates among program participants, the conclusion would be that the program had a positive impact.
• Program scalability: The evaluation may conclude that the program can be scaled up to reach a larger population, based on the program’s success in achieving its intended objectives. For example, if a literacy program was implemented in one community and was successful in improving reading levels, the conclusion would be that the program could be scaled up to reach other communities.
• Program sustainability: The evaluation may conclude that the program is sustainable, based on the program’s ability to continue achieving its intended objectives over time. For example, if a community-based environmental program has been successful in reducing pollution levels and has secured long-term funding, the conclusion would be that the program is sustainable.

These are just a few examples of the types of conclusions that may be drawn from the M&E process. The specific conclusions will depend on the evaluation results and the program’s goals and objectives.

Recommendations in the Monitoring and Evaluation (M&E) process refer to suggestions or proposals for actions that should be taken based on the evaluation results. They are based on the analysis of data collected during the evaluation process and are used to improve the program or intervention being evaluated.

Recommendations in M&E typically include specific actions that should be taken to address the program’s weaknesses or to build on its strengths. They may also include suggestions for improving program design, implementation, monitoring, and evaluation. Recommendations may be directed towards program managers, stakeholders, funders, or policymakers, depending on the intended audience.

Recommendations are an essential component of the M&E process as they provide decision-makers with evidence-based information that can be used to improve the program’s effectiveness and efficiency. They help to ensure that future programs or interventions are designed and implemented in a way that maximizes their impact and achieves their intended objectives.

Related: Recommendations in Evaluation

Here are some examples of recommendations in the Monitoring and Evaluation (M&E) process:

• Improve program design: If the evaluation finds that the program design is not effective, the recommendation may be to redesign the program and its activities to ensure they are more relevant and appropriate to the target population’s needs.
• Enhance implementation: If the evaluation reveals implementation challenges, the recommendation may be to implement strategies to address these challenges, such as providing additional training to program staff, improving program management, or increasing community engagement.
• Strengthen monitoring and evaluation: If the evaluation finds that the monitoring and evaluation system is weak, the recommendation may be to improve the quality of data collected, increase the frequency of data collection, or develop a better system for data analysis and reporting.
• Expand program coverage: If the evaluation finds that the program has had a positive impact, the recommendation may be to expand the program’s coverage to reach more people in the target population.
• Increase program sustainability: If the evaluation reveals that the program’s sustainability is at risk, the recommendation may be to develop a sustainability plan that outlines strategies for long-term program funding and support.

These are just a few examples of the types of recommendations that may be made as a result of the M&E process. The specific recommendations will depend on the evaluation results and the program’s goals and objectives.

Related: Eval Rec’s – EVAL CAFE

Here is an example of a conclusion and recommendation section from a monitoring and evaluation report on a water supply project in rural Ethiopia:

The evaluation found that the water supply project has achieved its objectives of increasing access to safe and reliable water sources for 15,000 people in five rural communities in Ethiopia. The project has also contributed to improved health, hygiene, education, and livelihood outcomes for the beneficiaries, as well as enhanced social cohesion and gender equality. The project has demonstrated good practices in community participation, capacity building, sustainability, and innovation. However, the project also faced some challenges and limitations, such as delays in construction, technical issues with some water points, lack of adequate monitoring data, and insufficient coordination with other actors. The evaluation identified some areas for improvement and learning for future similar projects.

Recommendations

Based on the evaluation findings and conclusions, the following recommendations are proposed:

• To the project team: Conduct regular maintenance and repair of the water points to ensure their functionality and durability. Strengthen the monitoring system to collect more accurate and comprehensive data on the project outputs, outcomes, and impacts. Enhance the communication and collaboration with other stakeholders, such as local authorities, NGOs, and donors, to avoid duplication of efforts and to leverage synergies.
• To the community water committees: Continue to mobilize and educate the community members on the importance of water conservation, hygiene practices, and payment of water fees. Ensure that the water fees are collected transparently and used efficiently for the operation and maintenance of the water points. Promote the inclusion and empowerment of women and marginalized groups in the decision-making and management of water resources.
• To the donor: Provide continued financial and technical support to the project team and the community water committees to ensure the sustainability and scalability of the project. Share the evaluation findings and lessons learned with other relevant actors to disseminate good practices and to inform future policy and programming.

Related: How to write a good M&E report – guidelines & best practices – TolaData

Reviewing the evaluation findings is a crucial step in drawing conclusions and making recommendations in the monitoring and evaluation process. This involves a thorough examination and analys is of the data collected during the evaluation, including both quantitativ e and qualitative data.

Quantitative data may include numerical data such as statistics, percentages, and figures, while qualitative data may include non-numerical data such as narratives, descriptions, and observations. Both types of data need to be analyzed carefully to identify patterns, trends, and themes that can inform the evaluation conclusions and recommendations.

The analysis of the evaluation findings should also consider the limitations of the data, such as sample size, response rate, and potential biases, and how these limitations may affect the reliability and validity of the conclusions and recommendations.

In summary, reviewing the evaluation findings is a critical step in drawing conclusions and making recommendations, as it provides the evidence base for the evaluation and ensures that the conclusions and recommendations are grounded in the data.

Ready to optimize your resume for a better career? Try Our FREE Resume Scanner!

Optimize your resume for ATS with formatting, keywords, and quantified experience.

• Compare job description keywords with your resume.
• Tailor your resume to match as many keywords as possible.
• Grab the recruiter’s attention with a standout resume.

Context refers to the broader circumstances or environment in which the program operates, and it plays a crucial role in shaping the program’s outcomes and effectiveness.

To draw accurate conclusions and make relevant recommendations, it is essential to consider various contextual factors, including political, economic, social, and cultural dimensions. Political factors such as government policies, regulations, and political stability can significantly impact the success of a program. Economic factors such as funding, market trends, and economic growth can also influence the program’s outcomes.

Similarly, social and cultural factors such as social norms, cultural beliefs, and community attitudes can shape how the program is perceived and implemented. Therefore, it is critical to consider these contextual factors while evaluating the program’s goals and objectives to determine whether they align with the local context and to make recommendations accordingly.

Overall, taking into account the program’s goals and objectives in light of its contextual factors will help in drawing accurate conclusions and making informed recommendations that are relevant and effective in achieving the program’s intended outcomes.

Strengths and weaknesses analysis is a crucial component of program evaluation, and the statement rightly highlights its importance in drawing conclusions and informing future program planning and implementation.

Strengths analysis allows program evaluators to identify the program’s positive aspects, including its achievements, successes, and benefits. These strengths can help inform future program planning by highlighting successful strategies and practices that can be replicated or expanded to improve program effectiveness. Moreover, identifying strengths can also help to build on the program’s positive aspects, improving its overall impact.

On the other hand, weaknesses analysis helps identify areas of the program that require improvement or restructuring. These weaknesses can include issues related to program design, implementation, or outcomes. Identifying weaknesses is essential to inform future program planning and improve program effectiveness. Moreover, weaknesses analysis can also provide opportunities to learn from past mistakes, and it can help to avoid repeating them in the future.

Overall, identifying the strengths and weaknesses of a program is essential to draw accurate conclusions and inform future program planning and implementation. By recognizing the program’s positive and negative aspects, program evaluators can develop evidence-based recommendations to improve program effectiveness, ensuring that the program achieves its intended goals and objectives.

Making actionable recommendations ensures that the recommendations are relevant, feasible, and likely to lead to improvements in program performance.

To make actionable recommendations, it is crucial to base them on the evidence generated through program evaluation. The recommendations should address specific issues or challenges identified during the evaluation, and they should be specific, measurable, achievable, relevant, and time-bound ( SMART ).

• For example, a program evaluation may identify that the program’s target population is not adequately engaged, and this is affecting program outcomes. Based on this finding, the recommendation could be to enhance community outreach efforts to increase engagement among the target population. This recommendation is actionable, and specific, and provides a clear path forward for program managers to improve the program’s performance.

Moreover, recommendations may also include changes to program design or implementation, such as modifying the program’s goals or objectives, revising the program’s theory of change, or enhancing the program’s monitoring and evaluation framework. These recommendations should also be actionable, feasible, and backed by evidence to ensure that they are likely to result in program improvements.

Finally, recommendations may include suggestions for future research to address gaps in knowledge or evaluate the effectiveness of specific program components. These recommendations should also be actionable, specific, and feasible, providing a clear path forward for researchers to conduct further studies.

Overall, making actionable recommendations is essential to ensure that program evaluation leads to tangible improvements in program performance, and that the recommendations are feasible, relevant, and likely to lead to positive change.

Effective communication is essential to ensure that stakeholde rs understand the evaluation results, the rationale for the recommendations, and their implications for program planning and implementation.

• To communicate evaluation findings and recommendations effectively, it is essential to tailor the communication approach to the audience. For example, program managers may require a more detailed presentation of the evaluation results, including the methodology, data analysis, and limitations. In contrast, funders may require a concise summary of the evaluation results, including the key findings and recommendations.

In addition to tailoring the communication approach, it is also essential to use clear, concise, and jargon-free language to ensure that stakeholders understand the evaluation results and recommendations. Effective communication should also provide opportunities for stakeholders to ask questions, clarify doubts, and provide feedback on the evaluation results and recommendations.

Furthermore, effective communication should also highlight the benefits of the evaluation , such as identifying successful strategies, opportunities for program improvement, and lessons learned. This can help to build support for future evaluations and ensure that stakeholders understand the value of the monitoring and evaluation process .

Overall, effective communication of evaluation findings and recommendations is essential to ensure that stakeholders understand the evaluation results and can use them to inform future program planning and implementation. Effective communication can help build support for the evaluation process and ensure that stakeholders are engaged and invested in program improvement.

The importance of conclusions and recommendations in monitoring and evaluation practice cannot be overstated. Conclusions are the results of an evaluation study, which provide an analysis of the findings and an interpretation of their meaning. Recommendations, on the other hand, are suggestions for action that are based on the conclusions.

Effective monitoring and evaluation requires that conclusions and recommendations are well-thought-out and communicated clearly. It is important that the conclusions and recommendations are evidence-based, comprehensive, and practical. The conclusions and recommendations should be tailored to the needs of the stakeholders and communicated in a way that is easily understandable.

In addition, it is important to consider the limitations and constraints of the evaluation study when formulating conclusions and recommendations. The evaluation team should be transparent about the methods used, the data collected, and the conclusions drawn.

Overall, the conclusions and recommendations are critical components of any monitoring and evaluation exercise. They provide valuable insights into the effectiveness of programs, policies, and interventions and can guide decision-makers in making informed decisions about future actions.

Patrick Kapuot

Very informative, interesting and clearly said.

I suggest in future to include a sample report that have key traits in it that were applied in the report.

Leave a Comment Cancel Reply

Your email address will not be published.

How strong is my Resume?

Only 2% of resumes land interviews.

Land a better, higher-paying career

Recommended Jobs

Monitoring & evaluation technical specialist (indicators & data).

• Washington, DC, USA
• United States Department of Treasury, Office of Technical Assistance

Project Assistant

Collaboration, learning, adapting (cla) specialist, ymelp ii – finance and administration consultant (stta), usaid/haiti/pcps program design advisor, digital mel manager – digital strategy implementation mechanism (dsim) washington, dc, program quality & meal manager.

• Honiara, Solomon Islands
• Save the Children Australia

Justice Sector Innovation Specialist

Court administration specialist, kosovo expert, justice sector specialist, kosovo expert, justice sector specialist, international expert, evaluation team leader/evaluation specialist, manager ii, institutional support program implementation, usaid guatemala- program analyst (information coordinator), gender experts – sahel regional gender assessment sahel, services you might be interested in, useful guides ....

How to Create a Strong Resume

Monitoring And Evaluation Specialist Resume

Resume Length for the International Development Sector

Types of Evaluation

Monitoring, Evaluation, Accountability, and Learning (MEAL)

LAND A JOB REFERRAL IN 2 WEEKS (NO ONLINE APPS!)

Sign Up & To Get My Free Referral Toolkit Now:

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Institute of Medicine (US) Committee on Serious and Complex Medical Conditions; Chrvala CA, Sharfstein S, editors. Definition of Serious and Complex Medical Conditions. Washington (DC): National Academies Press (US); 1999.

Definition of Serious and Complex Medical Conditions.

• Hardcopy Version at National Academies Press

3 Conclusions and Recommendations

• Introduction

Throughout the course of this study, the committee has been extremely aware of the fact that the topic addressed by this report concerns one of the most critical issues confronting HCFA, health care plans and providers, and patients today. The Medicare+Choice regulations focus on the most vulnerable populations in need of medical care and other services—those with serious or complex medical conditions. Caring for these highly vulnerable populations poses a number of challenges.

In the absence of a fully developed research and clinical knowledge base, combined with the time and resource constraints of this study, the committee chose what it believes is the most prudent and thoughtful approach to its charge. Specifically, the committee has chosen to provide guidance to HCFA and health care plans to support initial steps to address the major issues confronting health care plans and their beneficiaries with serious and complex medical conditions. This guidance is supported by presentation of possible strategies that might be used to screen enrollees for the presence of serious and complex medical conditions. Health care plans are encouraged to be innovative in their initial efforts to develop case-screening methodologies. As plans develop experience with different screening modalities, information can be shared nationally. It is the committee's intent that this will lead to the identification of best practices and subsequent standardized methods for ongoing screening of beneficiaries for serious and complex conditions. The committee also recommends strategies to ensure that the care offered to persons with serious and complex conditions is responsive to the diverse, multidisciplinary needs for services to address a full spectrum of medical, social, and mental health issues. Finally, the committee feels strongly that evaluation and research efforts must occur simultaneously with the implementation of the guidance and recommendations concerning screening and care of patients with serious and complex medical conditions. As HCFA and health care plans develop the necessary expertise and resources to provide comprehensive, multidisciplinary care to patients with serious and complex conditions, this knowledge can be generalized to address the needs of patients with serious but not complex and complex but not serious medical conditions. The following conclusions and recommendations are based on consideration of these diverse issues.

Recommendation 1: The establishment of an administrative definition for serious and complex medical conditions by the Health Care Financing Administration would be premature at this time.

It is the opinion of the committee that the current state of clinical and research literature does not support development of an operational definition for serious and complex medical conditions. There are a large number of potential ways to screen and identify patients with serious and complex medical conditions, although each has its own set of limitations. Disease classification systems such as diagnosis-related groups, diagnostic cost groups, and the International Classification of Diseases—Ninth Edition lack the sensitivity and specificity required to effectively distinguish between patients with acute care needs and those who need more extensive treatment and service arrays. The validity of patient classifications systems (e.g., diagnosis-related groups) and functional status definitions (e.g., Karnofsky and SF-36 scales) has not yet been determined. Efforts to develop case-mix-adjusted reimbursement systems to ensure cost-effectiveness and high-quality interventions for patients with serious and complex conditions are still in the early stages or are limited to only a single disease category. Similarly, efforts to develop standardized protocols for care coordination, continuity of care, care management strategies, and service reimbursement plans have met with varying degrees of success.

Clinical research relevant to defining patient populations as having serious and complex conditions and developing the most effective care protocols to respond to patient needs is evolving rapidly, but only preliminary results are available. For example, research in correlating patient volume with patient outcomes, differentiating access to services as a function of health plan benefits, and addressing variations in quality of care has just begun to yield meaningful results about particular groups of patients. The committee anticipates that this research will continue to evolve and will provide more definitive conclusions to guide the efforts of health plans to identify patients with serious and complex medical conditions and ensure appropriate levels of treatment and care management. As the science base is strengthened, so too will be the ability of HCFA and health plans to correctly identify patients with serious and complex medical conditions who would benefit from broad strategies of care management.

Recognizing these limitations, the committee recommends that plans first focus their attentions on patient populations with conditions that are serious AND complex. These conditions are described as persistent, substantially disabling or life threatening that require treatments and interventions across a broad scope of medical, social, and mental health services. Plans will require time to develop expertise and resources to develop methods of case finding and provision of an integrated care management approach for these patients. As such expertise is developed and resources are allocated, plans can then expand their efforts to include patients with serious but not complex conditions and those with complex but not serious conditions.

Recommendation 2: The committee recommends that the Health Care Financing Administration should provide guidance to health plans to assist their efforts to identify patients with serious and complex medical conditions. Specifically, the committee recommends the following language be used to facilitate efforts of plans to identify their enrollees with ''serious and complex conditions": "A serious and complex condition is one that is persistent and substantially disabling or life threatening that requires treatments and services across a variety of domains of care to ensure the best possible outcomes for each unique patient of member."

The committee recognizes the importance at this time of providing health plans with guidance to facilitate operationalization of their own definitions of serious and complex medical conditions. This guidance includes a description of a serious and complex medical condition as one that is persistent and substantially disabling or life threatening that requires treatments and services across a variety of domains of care. The committee reviewed a number of possible categories of conditions that might be considered serious and complex. These are discussed in Chapter 1 of this report and include, but are not limited to, life-threatening conditions, conditions that cause serious disability without necessarily being life threatening, conditions associated with severe consequences, conditions affecting multiple organ systems, conditions requiring coordination of management by multiple specialties, and conditions requiring treatments that carry a risk of serious complications.

Clinical research to identify patient populations with serious and complex medical conditions and develop the most effective care management strategies to respond to patient needs is evolving rapidly, with only preliminary results available. The current state of clinical and research literature does not adequately address all of the challenges and issues relevant to the identification and care of these patients. These issues include, but are not limited to, identification of methods and criteria that can be applied to screen enrollees for the presence of serious and complex conditions; development of strategies for proper care and management of the complex array of medical, social, and mental health problems confronting these patients; development of strategies to evaluate the effectiveness, efficacy, and efficiency of different strategies of care management; and ongoing research to ensure continuous improvements in the care of patients. Health care service research is only beginning to emerge with initial findings relevant to these issues.

The committee anticipates that this research will continue to evolve and provide more definitive conclusions to guide the efforts of health plans to identify patients with serious and complex medical conditions and help plans to provide appropriate levels of treatment and care management. The committee urges health care plans, in collaboration with federal agencies and research institutions, to support this ongoing research and facilitate a synthesis of emerging results. As the science base is strengthened, so too will be the ability of HCFA and health plans to correctly identify patients who would benefit from a broad care management strategy including those with serious but not complex or complex but not serious conditions.

Recommendation 3: The committee recommends that health plans develop a broad strategy for care management to enable patients and providers to achieve the best possible outcomes for each unique patient or member with a serious or complex medical condition.

This care management strategy should include, but not necessarily be limited to, case finding; screening and selection; problem assessment and identification of strengths; development of treatment or care plans; implementation of care plans with an emphasis on proactive interventions; and monitoring of care plan implementation and outcomes.

The treatments for individuals with serious and complex medical conditions will inevitably vary due to many factors, including characteristics of the individual patient and the patient's social situation, characteristics of individual providers, provider organizations, health plans, resources in the community, health plan benefits, and so forth. Although these factors will vary for all patients, they have less impact on preventive care (e.g., immunizations) and acute care (e.g., setting a broken arm) interventions. Successful outcomes for persons with serious and complex medical conditions will rely on a diverse mix of medical, social, functional, financial, and psychological factors, as well as on how these factors mesh with a community's health and social services resources. There is no single guideline that can prescribe the care or treatments for persons with serious and complex medical conditions.

The committee's recommendations concerning elements of the care management strategy have been derived, in part, from existing publications (Case Management Society of America, 1996; HMO Workgroup on Care Management, 1999). A brief explanation for each of the recommended steps in the care process is provided below.

Case finding refers to methods for identifying persons who may have a serious and complex medical condition. Utilization of the committee's recommended description for serious and complex conditions can facilitate efforts by health plans and providers to identify possible categories of patients for whom case finding should be conducted. Depending on the organization of a health plan and its provider group(s), case finding methods may include electronic searching of administrative databases; phone screening of new or existing members; referral by patient or family member; referral by a clinician or other professional; or notification by an emergency room, hospital, nursing home, or other provider organization.

Case screening and selection outline a process for deciding exactly which patients meet a plan's criteria for serious and complex medical conditions. These criteria should be established by each health plan with review by HCFA. Examples of selection criteria might include factors such as diagnostic codes, patient age, presence of comorbidities, severity of illness or disability, duration of a disability, and so forth. It is expected that patients meeting these criteria would benefit from coordinated care management.

The next step in the care process should involve an assessment of the patient's problems and strengths. This is a tool for gathering information about the patient's medical, social, functional, and financial status that will be used to further determine his or her need for care management and the resources available to access the necessary services.

Treatment or care planning is also recommended by the committee as an essential element of a care management strategy for patients with serious and complex medical conditions. Such planning supports the development of individualized, goal-focused care plans (based on assessment findings and drawing on evidence-based treatments when available) to provide a guide for delineating patient, provider, and plan responsibilities for accessing and coordinating needed services.

The care management strategy also demands that steps be taken to ensure the implementation of care plans. Execution of the care management plan includes strategies and tactics for providing or purchasing needed services, or for linking patients to covered services, services available in the community, or those for which the patient will have to pay. It also includes advocating for the provision of informal services by family, friends, and neighbors.

The final step of the care process is monitoring implementation and outcomes of the care management strategy. This can be based upon periodic reviews of a patient's status and treatments to determine whether goals are being achieved, whether interventions and treatments continue to be appropriate, whether there is a medical need for continued services, or whether proactive care is required to address potential or emerging new problems. This type of monitoring can also include measurements to assess the impact of care management and treatment on clinical, functional, cost, and satisfaction outcomes.

Throughout the entire care management strategy for persons with serious and complex medical conditions, three principles should be evident. First, the care management strategy should reflect a commitment to continuity and coordination of care, as described among the requirements for National Committee for Quality Assurance accreditation. This entails monitoring continuity and coordination activities, analyzing data to identify opportunities for improvement, and taking actions to bring about improvements, as indicated. Second, the care process should include multidisciplinary perspectives and treatments, as appropriate. The care of persons with serious and complex medical conditions may require the assessment and treatment expertise of primary care providers; medical and surgical specialists; nurses and nurse specialists; social workers; pharmacists; occupational, speech, and physical therapists; behavioral and mental health professionals; and community-based service providers and resources. Access to expertise from these various disciplines should be available as needed. Third, and perhaps most important, patients and their family members should be involved at every step so that the care process incorporates the patient's expectations and preferences and documents the patient's role in achieving treatment goals.

The committee's recommendations concerning the care management process for patients with serious and complex conditions assumes patient willingness to collaborate with a multidisciplinary team of providers in the development and implementation of that plan. In those instances where a patient or family member declines such participation, appropriate documentation should be entered into the patient record. The presence of such documentation should be considered by HCFA as evidence that a good faith effort has been made by the plan to involve the patient and family in the development of the case management plan.

Recommendation 4: The committee recommends that health plans design and implement strategies for routine screening and selection to identify those beneficiaries with serious and complex medical conditions. These strategies should be consistent with the guidance outlined in Recommendation 2 to determine which patients meet a plan's threshold for serious and complex medical conditions and would benefit from a coordinated care management strategy.

The committee recommends that health plans identify specific categories of patients or health conditions for which screening for the presence of serious and complex conditions should occur on a routine basis. Examples of such categories are provided in Chapter 1 . The committee is of the opinion that screening a health care plan's entire population of enrollees is not a feasible, cost-effective, or efficient method of identifying subpopulations with serious and complex medical conditions. The committee feels strongly, however, that early efforts to develop case-screening methodologies should not be prescriptive; rather, innovation should be encouraged. Documentation of locally derived methods by plans will result in extensive nationwide experience that can be assessed and can lead to the identification of best practices and subsequent standardized methods for ongoing routine screening of patient populations.

The appropriate application of resources by a health plan to patients with serious and complex medical conditions requires the identification of those individuals whose health outcomes would benefit by such designation. For the purposes of this report, the committee has not included patients suffering from acute conditions that might require coordinated delivery of sophisticated medical services to treat complications of acute decompensation. On the other hand, many patients who have endured prolonged hospitalization for an acute medical condition may subsequently need coordinated long-term care to maximize their rehabilitation potential; to treat persistent sequelae of the disease processes that resulted from the acute, severe illness; and to ensure that they receive proactive care to prevent further exacerbation of the illness or disability.

Conceptually, various methodologies could be applied to administrative databases that could produce relevant lists of patients with serious and complex medical conditions who would be likely to benefit from written care management plans. High-cost patients, for example, might merit designation as having serious and complex conditions. Some of these individuals, however, may have undergone a single event such as bypass surgery that is self-limited and not appropriate for such designation. In contrast, others will have conditions, such as organ transplants, disabling traumatic injury, and HIV/AIDS, that clearly require ongoing medical attention and resources to sustain or improve their health status. Aside from costs, certain diagnostic entities could be identified for designation as serious and complex medical conditions. Examples of such entities could include progressive, degenerative disorders, active collagen vascular disease, and metastatic cancers. The committee does not believe it is feasible at this time to create an all-encompassing list of diagnoses appropriate for designation as serious and complex conditions. Indeed, within any diagnostic label, there exists a continuum of disease and disability that requires varying degrees of medical and other health-related support.

Beneficiary survey tools could supplement the use of administrative data sets to identify appropriate patients. Questionnaires that quantify functional status could assist plans in capturing individuals who need special attention in the outpatient setting to improve or sustain functional capacity or physiological stability. Similarly, home health agencies or medical professionals could identify frail elderly patients who require care management to sustain homeostasis. These agents could refer patients' names to health plans for development of a formal management program for serious and complex medical conditions. Such patients could include those of advanced age (over 85) with several comorbid medical conditions such as congestive heart failure, chronic renal failure, and diabetes. These persistent conditions in concert with reduced cognitive functioning or physical capacity to sustain daily living activities produce a constellation of physiological deficits that require supportive health care services for the patient to maintain independent living in a community setting.

There are several reasons why it is essential to the care of persons with serious and complex medical conditions that screening and selection of a plan's designated patient categories or conditions occur on a routine basis. First, periodic screening allows the multidisciplinary care team to adjust the care management approach as patient needs change over time. Second, routine screening is necessary to identify new patients with serious and complex medical conditions. Third, it ensures that persons who are no longer considered to have a serious and complex condition will be identified and services adjusted accordingly, which could result in a cost benefit.

The committee believes that over time, the health care system will identify reliable, accurate screening techniques for populations of patients with serious and complex medical conditions. In many cases, timely identification will permit appropriate treatment at the level of the primary care provider. Nevertheless, even with robust methodologies in place, there will be patients and families who believe that they require additional resources or attention to adequately manage a persistent health condition. In instances in which a treatment plan is not mutually agreeable, individuals should be able to appeal to the medical director of the provider group or health plan. The review process should be timely and should allow for both internal and independent external review.

Recommendation 5: The committee recommends that health care plans develop a care management strategy that integrates the participation of all those involved in the care of the patient, including primary care physicians; medical and surgical specialists; nurses and nurse specialists; behavioral and mental health specialists; physical, occupational, and speech therapists; social workers; allied health professionals; and community-based service providers.

The development of a collaborative care management strategy involving all those who participate in providing care to the individual patient is critical to the treatment of patients with serious and complex conditions. The treatment plan serves several important functions within the care management strategy. First, it provides a road map and a guide to indicate direction of and demarcate landmarks in care management, making it easier to ensure that all providers involved in the patient's care share the same goals, and that these goals are accessible to the patient and the patient's family. Second, it provides a way to measure progress and the achievement of care goals, and it serves as a powerful communication and coordination tool among the participants involved in the care of the patient, avoiding overlapping or redundant efforts as well as gaps and inattention to components of care assumed to be the responsibility of others on the care team. Third, a treatment plan offers an opportunity to delineate which provider is taking responsibility for specific aspects of care.

Although there is currently no evidence to support the identification of a single format or model as superior to all others, the committee feels confident in recommending certain key components that should be part of any treatment plan for patients with serious and complex medical conditions. Specifically, the plan should involve the collaborative participation of all those taking part in the care of the patient, including primary care physicians; medical and surgical specialists; nurses and nurse specialists; physical, occupational, and speech therapists; rehabilitation specialists; behavioral and mental health specialists; social workers; allied health professionals; and community-based service providers and resources.

The perspectives, health beliefs, and values of the patient and the patient's family should be taken into consideration, and the care management plan should be constructed in a culturally appropriate manner. Elements of the care management plan should be appropriate both for the age or life phase of each patient as well as the stage or severity of the health condition. This will ensure selection of treatment options that are responsive to the unique needs of patients of different age groups and with conditions of varying severity. The patient should be involved in the development of the plan and should have access to the plan.

The plan should be linked to relevant clinical practice guidelines and should reflect the process of adapting population-based clinical practice guidelines to the unique care and service needs of the individual. The treatment plan should cross areas of expertise and should address the need for coordination of care across the continuum of care delivery settings. The plan should be longitudinal, with accommodations for proactive interventions to prevent potential problems from arising, and it should also enable longitudinal assessment of outcomes. The treatment plan should be a living document that is periodically reviewed, updated, and tested for continuing relevance to the patient's circumstances (e.g., the life phase) and clinical condition (e.g., the disease stage).

Recommendation 6: The committee recommends that health care plans have programs in place to monitor care management plans for both process and outcomes for patients with serious and complex conditions at the level of population, patient, provider, and best practices of care .

The Health Care Financing Administration should establish a formal mechanism to evaluate a Medicare+Choice provider organization's program to monitor the outcomes of care for patients with serious and complex medical conditions. The committee recognizes that the monitoring and evaluation programs will vary between health plans but recommends that all programs include population-based monitoring of the outcomes of care for patients with serious or complex conditions, as well as individual case monitoring, assessment of provider interventions, and evaluation of best practices of care.

Assessment of population-based measures may include, but not be limited to, review of plan-wide policies and procedures; application of working definitions of serious and complex conditions; adoption of evidence-based clinical guidelines; review of health outcomes survey, patient satisfaction and complaint data; and review of clinical outcomes data for patients with serious or complex medical conditions. Individual case monitoring may include, but not be limited to, analysis of complaint data, patient satisfaction, internal and external appeals of the care plan, and overturned denials. Evaluation and monitoring at the level of providers will include efforts to assess clinical quality of care, timeliness of care, and effectiveness of specific interventions. Best practices of care will be identified by data that describe clinical, functional, and patient satisfaction outcomes.

Recommendation 7: The committee recommends that the Health Care Financing Administration collaborate with federal, private, and voluntary agencies, organizations, and consumers to establish an ongoing process to address policy issues to ensure continuous improvements in the care of populations with serious and complex medical conditions .

Access to providers with the appropriate training and expertise is important, but other issues must be addressed to ensure that beneficiaries with serious and complex conditions achieve the best possible outcomes. The committee recognizes that a number of these issues supersede activities that can be accomplished at the level of individual health plans. Issues that must be addressed include, but are not limited to, the following:

• Payment.

The current Medicare capitated payment mechanisms for health plans are not risk adjusted and are derived from experience in the fee-for-service sector. These payment mechanisms may not provide adequate compensation to some health plans. Furthermore, health plans use a variety of methods to compensate individual providers and hospitals. Greater attention should be focused on the improved alignment of financial incentives at all levels to ensure appropriate payment for serious and complex cases through adequate funding and risk-adjusted payment systems.

• Benefits.

Improving the care of persons with serious and complex medical conditions can be supported by ensuring that Medicare beneficiaries can choose among different health plans. When beneficiaries have multiple affordable options, they can select based upon their own evaluations of quality. Where benefits are provided through multiple payment programs (i.e., Medicare and Medicaid dual eligibles), coordination of benefit packages and eligibility requirements is critical. The movement of patients with serious and complex conditions into managed care holds the promise of greater health for these patients and therefore reduced expenditures for Medicaid programs due to reduced nursing home use. Prescription coverage for these patients would result in a direct reduction in state Medicaid prescription coverage expenses. Medicare requirements for management of patients with serious and complex conditions may also serve as the future basis for Medicaid managed care contracts.

• Medicaid and Medicare Fee-for-Service Programs.

The committee was unable to address the third point of its charge with any degree of thoroughness. This is primarily due to the fact that the committee lacked sufficient information and time to conduct informed deliberations about the implications of alternative definitions for serious and complex medical conditions for Medicaid and commercial programs, especially when applied to Medicare beneficiaries who are dual eligible. The committee strongly recommends that HCFA, in collaboration with Medicare and Medicaid provider plans, establish this as a research priority. Resources and expertise should be allocated to conduct a study to define the relevant issues and generate possible approaches to address this question.

In the meantime, the committee suggests that the basic guidance provided in this report concerning the description of serious and complex medical conditions and recommendations concerning strategies for care management for this patient population should be extended to include Medicaid beneficiaries and those that are dual eligible. In addition, this guidance should apply to patients enrolled in fee-for-service Medicare programs. This should be a priority area for collaborative research on policy issues to ensure continuous improvements in the care of populations with serious and complex medical conditions covered by Medicaid and fee-for-service Medicare plans.

• Consumer Education and Participation.

Many patients, especially those with persistent and disabling conditions, are ready and able to become more active participants in the care delivery process. Attempts should be made to increase patient participation in clinical decision-making and caregiving processes consistent with improving clinical outcomes and satisfaction.

• Professional Training.

To serve persons with serious and complex medical conditions, an aggressive strategy must be developed to educate and train health professionals to provide appropriate care for this population. There is currently a great undersupply of persons trained in care management processes, multidisciplinary team care, geriatrics, patient education and self-management, and cultural competence.

• Information Technology.

More sophisticated information technology is required to provide high-quality, coordinated care to those with complex or serious medical conditions. More sophisticated information systems are needed to support efforts to monitor, analyze, and improve patient care. Continued development of electronic medical records will be an important factor in ensuring that all health care organizations and providers involved in the care of a patient have timely access to complete information on the patient's clinical conditions, diagnostic and therapeutic test results, and services received. Confidentiality protections must be respected throughout the process of technology development.

• Regulatory Framework.

Developing a regulatory approach for the Medicare+Choice program that affords adequate protections for beneficiaries with serious and complex medical conditions while encouraging innovation in a rapidly changing health care environment will be an ongoing challenge. Periodic evaluation and modification will be essential.

• Applied Research.

Research is needed to further inform the policy process and to identify best practices in the care of patients with serious and complex conditions. Attention should be focused on the following areas: effects of benefits coverage and payment methods on care delivery; determination and dissemination of best practices in the identification of patients with serious and complex conditions; efficacy, effectiveness, and efficiency of multidisciplinary teams; use of information technology to support care coordination and delivery; and development of innovative methods of designing, implementing, and coordinating medical, social, and other services.

These and related issues will require HCFA to actively collaborate with all types of payors, health care plans, providers, research organizations, and consumer and disability groups to develop strategies and solutions that will maximize the quality of health care and other services for patients with serious and complex medical conditions.

• Conclusions

The needs of patients with serious and complex medical conditions far exceed the boundaries of a single issue such as access to specialists for care. For health care delivery systems to provide high-quality, effective care for these patients, the systems must be responsive to a diverse array of clinical, social, mental, and functional health issues. The development of a health care delivery system that will provide quality and effective care for these patient populations must be responsive to a diverse group of clinical, social, mental, and functional health issues. The health care delivery system must also be capable of working effectively with other community and social support systems.

The questions raised by HCFA concerning implementation and regulations for the Medicare+Choice program apply broadly to include all health care delivery and payment systems, including fee-for-service care. This study is one of the first to address these complex and extremely important topics. Thus, the committee urges HCFA, health care plans, and consumer groups to view this report, and especially its conclusions and recommendations, as first steps in the effort to address the unique and complicated needs of persons with serious and complex medical conditions and to ensure that they receive the most cost-effective and best-quality care possible. The work of this committee is intended to support and provide direction for future work to be conducted by HCFA, in collaboration with all types of health care plans and consumer organizations, to design innovative strategies to ensure adequate reimbursement, access to broad care management, and the highest possible quality of treatment services required by patients with serious and complex medical conditions.

• Cite this Page Institute of Medicine (US) Committee on Serious and Complex Medical Conditions; Chrvala CA, Sharfstein S, editors. Definition of Serious and Complex Medical Conditions. Washington (DC): National Academies Press (US); 1999. 3, Conclusions and Recommendations.
• PDF version of this title (730K)

In this Page

Recent activity.

• Conclusions and Recommendations - Definition of Serious and Complex Medical Cond... Conclusions and Recommendations - Definition of Serious and Complex Medical Conditions

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

This paper is in the following e-collection/theme issue:

Published on 30.4.2024 in Vol 26 (2024)

Digital Food Frequency Questionnaire Assessing Adherence to the Norwegian Food–Based Dietary Guidelines and Other National Lifestyle Recommendations: Instrument Validation Study

Authors of this article:

Original Paper

• Hege Berg Henriksen 1 , PhD   ; 
• Markus Dines Knudsen 1, 2, 3 , PhD   ; 
• Anette Hjartåker 1 , PhD   ; 
• Rune Blomhoff 1, 4 , PhD   ; 
• Monica Hauger Carlsen 1 , PhD  

1 Department of Nutrition, Institute of Basic Medical Sciences, University of Oslo, Oslo, Norway

2 Section for Colorectal Cancer Screening, Cancer Registry of Norway, Oslo, Norway

3 Department of Transplantation Medicine, Division of Surgery, Inflammatory Diseases and Transplantation, Oslo University Hospital, Oslo, Norway

4 Department of Clinical Service, Division of Cancer Medicine, Oslo University Hospital, Oslo, Norway

Corresponding Author:

Hege Berg Henriksen, PhD

Department of Nutrition

Institute of Basic Medical Sciences

University of Oslo

Sognsvannsveien 9

PO Box 1046, Blindern

Phone: 47 99459673

Email: [email protected]

Background: Valid assessment tools are needed when investigating adherence to national dietary and lifestyle guidelines.

Objective: The relative validity of the new digital food frequency questionnaire, the DIGIKOST-FFQ, against 7-day weighed food records and activity sensors was investigated.

Methods: In total, 77 participants were included in the validation study and completed the DIGIKOST-FFQ and the weighed food record, and of these, 56 (73%) also used the activity sensors. The DIGIKOST-FFQ estimates the intake of foods according to the Norwegian food–based dietary guidelines (FBDGs) in addition to lifestyle factors.

Results: At the group level, the DIGIKOST-FFQ showed good validity in estimating intakes according to the Norwegian FBDG. The median differences were small and well below portion sizes for all foods except “water” (median difference 230 g/day). The DIGIKOST-FFQ was able to rank individual intakes for all foods ( r =0.2-0.7). However, ranking estimates of vegetable intakes should be interpreted with caution. Between 69% and 88% of the participants were classified into the same or adjacent quartile for foods and between 71% and 82% for different activity intensities. The Bland-Altman plots showed acceptable agreements between DIGIKOST-FFQ and the reference methods. The absolute amount of time in “moderate to vigorous intensity” was underestimated with the DIGIKOST-FFQ. However, estimated time in “moderate to vigorous intensity,” “vigorous intensity,” and “sedentary time” showed acceptable correlations and good agreement between the methods. The DIGIKOST-FFQ was able to identify adherence to the Norwegian FBDG and physical activity recommendations.

Conclusions: The DIGIKOST-FFQ gave valid estimates of dietary intakes and was able to identify individuals with different degrees of adherence to the Norwegian FBDG and physical activity recommendations. Moderate physical activity was underreported, water was overreported, and vegetables showed poor correlation, which are important to consider when interpreting the data. Good agreement was observed between the methods in estimating dietary intakes and time in “moderate to vigorous physical activity,” “sedentary time,” and “sleep.”

Introduction

Monitoring dietary intake and physical activity with valid and feasible methods in a population is of great importance, as there is a growing body of evidence on the strong association between lifestyle and risk of several chronic diseases [ 1 - 5 ].

Even though self-administered web-based 24-hour dietary recalls and food are increasingly used in epidemiological studies [ 6 , 7 ], the most common type of dietary assessment tool in observational and intervention studies is still the food frequency questionnaire (FFQ) [ 8 , 9 ]. Generally, the FFQs assess the frequency of food intakes and portion size of the food consumed, are typically self-administrated, and vary with regard to which food items or nutrients are included, portion sizes, frequency categories, length of the questionnaire, and duration of the registration period (ie, weeks, months, or years) [ 6 , 10 - 12 ]. The FFQs may be used to estimate habitual dietary intakes over a longer period of time and to rank individuals according to dietary intakes and may generate data on dietary intakes on a group level. Thus, FFQs are valuable in assessing the degree of adherence to dietary recommendations in a population [ 8 , 9 , 13 - 15 ].

As with all dietary assessment methods, FFQs are prone to reporting errors, particularly systematic errors, often related to memory, knowledge of portion sizes, and understanding of the questions [ 11 , 12 , 16 , 17 ]. This can result in over- or underestimation of dietary intakes [ 18 , 19 ], which may attenuate the observed association between diet and health outcomes [ 20 , 21 ]. Systematic error may be reduced with clear instruction and the use of relevant examples, such as pictures of portion sizes and informative text.

The emerging use of digital applications has resulted in modifications of traditional printed assessment tools into web-based tools in both research and the clinic [ 22 , 23 ]. Web-based dietary assessment tools have demonstrated good validity and give comparable data with printed versions [ 6 , 23 - 25 ].

We have developed a short digital semiquantitative food and lifestyle frequency questionnaire (the DIGIKOST-FFQ) based on a validated printed questionnaire called NORDIET-FFQ [ 26 ] developed at the Department of Nutrition, University of Oslo, Norway. The DIGIKOST-FFQ is designed to assess adherence to the Norwegian food–based dietary guidelines (FBDGs) and national lifestyle guidelines [ 3 , 27 ] and has been revised according to the results from a qualitative evaluation study (ie, focus group interviews and usability testing) [ 27 ].

Generally, the DIGIKOST-FFQ delivers similar data on the main dietary and physical activity recommendations as the NORDIET-FFQ. However, some adjustments have been made based on the results from the validation study of the NORDIET-FFQ ( Multimedia Appendix 1 [ 3 , 26 - 38 ]). In addition, the DIGIKOST-FFQ is based on a digital platform constituting several technical functions to improve the feasibility and the understandability of the questions [ 27 ].

Therefore, investigating the relative validity of this new digital assessment tool is of great importance to current and future studies applying the assessment method for dietary data collection. Thus, in this project, we aimed to investigate the relative validity of the DIGIKOST-FFQ in assessing dietary intake, physical activity, and adherence to the Norwegian FBDG in an adult and healthy population in Norway, with weighed food record (WR) and activity sensor as the reference methods.

Study Design and Participants

Participants were recruited between April and September 2021 and randomly extracted by the National Registry of Norway and by advertisements on Facebook. Adult (18 years and older) men and women living in the Oslo region in Norway were eligible for the study. The study design was cross-sectional. All participants completed the DIGIKOST-FFQ, and after 1-2 months, they filled in a 7-day WR and used an activity sensor (SenseWear Armband Mini [SWA]; BodyMedia). In addition to the WR, the participants received a digital scale and an instruction on how to weigh and record all foods and beverages consumed during a period of 7 consecutive days. Due to the COVID-19 pandemic, the instructions were given in a video meeting (Zoom; Zoom Video Communications) by the responsible researchers in the study. Then, the participants were invited to meet outside the study center at the Department of Nutrition, University of Oslo at a pickup point to receive the WR, a digital scale, SWA, and the prepaid postal envelopes for returning the equipment after use.

During the WR registration period, the participants also wore the SWA to record all physical activity, sedentary time, and sleep. By the end of the registration period, the participants returned the completed WRs and the SWA to the study center by postal mail. At the end of the study, all participants were offered to voluntarily fill out the DIGIKOST-FFQ once more in order to receive an individual feedback report benchmarking their dietary intakes and physical activity against the Norwegian FBDG. In addition, they could be randomly selected to win a gift certificate of NOK 500 (US $56.91). The digital scale was also given to the participants as a gift.

The first version of the DIGIKOST-FFQ underwent evaluation in focus group interviews and usability testing [ 27 ]. The final version of the DIGIKOST-FFQ is described in Multimedia Appendix 1 . In brief, the DIGIKOST is a digital diet and lifestyle assessment tool designed to assess dietary intake and other lifestyle factors and evaluate these according to the Norwegian FBDG. DIGIKOST also has a report function, the DIGIKOST report, which is an individual feedback report on respondents’ adherence to the Norwegian FBDG, with specific and personalized advice on how to fulfill the recommendations . The DIGIKOST-FFQ is based on a software platform called Nettskjema, developed and administered by the University Information Technology Center at the University of Oslo, Norway [ 28 ]. The main login option in DIGIKOST is the ID-port (e-ID used by the Norwegian Agency for Public Management and eGovernment [Difi]) [ 29 , 30 ]. The DIGIKOST-FFQ takes approximately 20 minutes to complete [ 27 ] and includes 103 food and lifestyle items, of which 78 questions about food items (grams per day), 7 questions about physical activity (minutes per week), sedentary time, and sleep (hours per day), 8 questions about tobacco use, and 10 questions about body weight and demographic data. The main food groups cover foods rich in fiber (ie, fruits and berries, vegetables, and whole grain products), fish, dairy products, meat, oils, margarine, and beverages (Table S1 in Multimedia Appendix 1 ). The responses from the DIGIKOST-FFQ are directly transferred into a secure server called services for sensitive data, and the crude variables are automatically transformed by unique algorithms into food groups, activities, and lifestyle indices according to the national recommendations [ 26 ]. The DIGIKOST data set also contains 2 indices, the Norwegian diet index and the Norwegian lifestyle index [ 32 ] ( Multimedia Appendix 1 ). The Norwegian diet index consists of 12 components corresponding to the Norwegian FBDG with a 3-level scoring approach including 3 categories representing low, intermediate, and high adherence, giving a composite diet index ranging in scores from 0 (lowest adherence) to 20 (highest adherence) points. The Norwegian lifestyle index consists of 5 components (ie, diet, physical activity, normal weight, alcoholic drinks, and tobacco use) with a 3-level equal scoring approach, and a composite lifestyle index ranging from 0 to 5 points.

Seven-Day WR

Dietary data were retrieved from the WR and manually coded and imported into the food and nutrient calculation system, KBS (version 7.3, 2018, AE-10 database), developed at the Department of Nutrition, University of Oslo [ 39 ].

Physical Activity Sensor

We used the activity sensors called SWA to objectively measure different activity intensities. The SWA was placed on the nondominant arm at the upper part around the triceps muscle. The participants were instructed to use the SWA all day and night and to only remove it during water-based activities like showers or swimming. All activity data generated from the SWA were exported to a computer, and all calculations on activities were conducted in the SenseWear Professional software (version 7.0; BodyMedia Inc).

Sample Size

An adequate number of participants in validation studies are shown to be 80-100 [ 10 , 11 , 40 , 41 ]. This ensured statistical power of 80% to detect differences of 1 portion of “fruit” or “vegetable” per day (1 portion=100 g) between test and reference methods, assuming an SD of 1.6 portions (or 160 g) and a significance level of 5% [ 42 , 43 ]. Moreover, in order to detect a Pearson correlation coefficient of 0.5 or higher between the DIGIKOST-FFQ and the reference methods, a sample size of 38 men and 38 women is required to achieve a significance level of 5% and a power of 90% [ 44 ].

Statistical Analyses

All analyses were performed using SPSS software (version 29; IBM Corp). Normal distribution for continuous variables was checked by visual inspection of histograms and quantile-quantile plots. Dietary and activity variables were presented in median and IQR. Comparisons between the methods were presented as median differences, and categorical variables were presented in frequencies and percentages. Paired 2-tailed t tests were used to compare differences in normal distributed variables, and Wilcoxon signed rank test in nonnormal distributed variables. P values <.05 were considered statistically significant. Ranking of individual dietary intakes and activity levels were tested by Spearman ρ for nonparametric variables and the 95% CI for the correlation estimate. A correlation below 0.3 was defined as poor, whereas a correlation between 0.3 and 0.49 was fair, and above 0.5 was satisfactory, according to Hankin et al [ 45 ].

Cross-classification of individual intakes between the methods was estimated by ranking participants’ intakes, dividing them into quartiles, and calculating how many were classified in the same and same plus adjacent, misclassified with 2 quartiles, or classified in the opposite quartile (grossly misclassified, 3 quartiles apart).

Bland-Altman plots were used to explore and visualize bias, such as under- or overreporting (mean differences), limits of agreement (mean difference and 1.96 SD), and the presence of outliers in the data. Sensitivity and specificity were calculated to evaluate the participants’ adherence to the Norwegian FBDGs identified by both methods. The sensitivity was defined as the proportion of participants who were classified as not fulfilling the recommendations both by the DIGIKOST-FFQ and the WR or SWA divided by the number of participants not fulfilling the recommendations according to the WR or SWA only. Specificity was defined as the proportion of participants who were classified as fulfilling the recommendations both by the DIGIKOST-FFQ and WR or SWA divided by the number of participants fulfilling the recommendations according to the WR or SWA only. Values above 60% were defined as good for both measures. Adherence to the Norwegian diet index and physical activity recommendations were examined on a continuous scale. For all the different diet and physical activity recommendations, participants were assigned points of adherence, defined as high (1 point), intermediate (0.5 point), and low (0 point) to each specific recommendation.

Ethical Considerations

This study was carried out in accordance with the Helsinki Declaration. The Norwegian Data Protection Services (Sikt, project registration 277679) has approved the DIGIKOST protocol and the informed consent. The participant signed the informed consent electronically prior to participating in the study. They were also informed about the ability to opt out of the study as well as information regarding the safe storage of all personal data. All data were analyzed in the safe and approved server, and with restricted access, called services for sensitive data at the University of Oslo [ 28 ]. Only tables and figures without individual sensitive data were exported. No data were collected about the invited participants who did not participate.

Participant Characteristics

In total, 77 (male: n=16 and female: n=61) participants, corresponding to 6% (n=77) of the 1249 who were invited and 59% (n=77) of the 131 who consented, participated in this study. All of these 77 participants fulfilled the inclusion criterion by the completion of both the DIGIKOST-FFQ and the WR. Of these, 56 (73%) also used the SWA ( Figure 1 and Table 1 ). The mean age was 45 (SD 14.6) years, and the mean BMI was 24.5 (SD 3.9) kg/m 2 . Most of the participants were highly educated (college or higher: n=70, 91%) and currently employed (n=53, 69%). Total daily energy intake estimated from the WR was on mean 8057 (SD 2157) kJ, and energy expenditure measured from the SWA was on mean 10,365 (SD 1827) kJ/day ( Table 1 ).

Dietary Intakes From the DIGIKOST-FFQ and WR

Estimates of median dietary intakes from DIGIKOST-FFQ and WR, median differences, correlation coefficients, and cross-classifications are presented in Table 2 . All estimated intakes of food groups from the DIGIKOST-FFQ were significantly different from the WR estimated intakes, except for processed meat, sugar-rich beverages, and dietary supplements.

a FFQ: food frequency questionnaire.

b WR: weighed food record.

c For dietary supplements, n=40 (52%) and n=14 (18%) for DIGIKOST-FFQ and WR, respectively, and Fischer exact test was performed ( P <.39).

d Wilcoxon signed rank test.

The median differences were however generally small and below a regular portion size for the different food groups. The ranking of individual intakes was fair and satisfactory for all food groups and poor for “vegetables.” Based on the Bland-Altman plot evaluations, the agreement between the 2 methods was good, with most of the differences within the 95% limit of agreement for each food group and evenly distributed above and below the mean difference ( Figures 2 and 3 ). In total, 69% or more of the participants were classified into the same or adjacent quartile for all food groups, and gross misclassification ranged from 0 for legumes, fatty fish, and alcohol to 13% for unsalted nuts ( Table 2 ). The sensitivity and specificity of the DIGIKOST-FFQ are presented in Table 3 . The sensitivity for different food groups ranged from 32% for red meat to 90% for low-fat dairy products. For specificity, the food groups ranged from 40% for unsalted nuts to 100% for sugar- and fat-rich foods. Most of the participants achieved intermediate adherence to the diet score recommendation with both methods ( Table 4 ) [ 32 ]. More detailed results are presented in the sections below.

a MVPA: moderate to vigorous physical activity.

b Fischer exact test ( P <.4).

b No adherence.

c Intermediate adherence.

d High adherence.

e FFQ: food frequency questionnaire.

f SWA: SenseWear Armband Mini.

g WR: weighed food record.

Fruits, Berries, Vegetables, Whole Grains, and Unsalted Nuts

The DIGIKOST-FFQ slightly overestimated intakes of “fruits and berries” with a median difference of 31 g/day ( Table 2 ). The correlation was satisfactory ( r =0.64), and 87% of the individual intakes were classified in the correct or adjacent quartile, and only 1% was grossly misclassified. For intakes up to 200 g/day, the limits of agreement were 200 g above and below the mean difference, which increased with higher intakes ( Figure 2 A). “Vegetables” were overestimated with 45 g/day and with a similar distribution of limits of agreement as for “fruits and berries” ( Figure 2 B). In total, 69% of the individual intakes of “vegetables” were classified in the correct or adjacent quartile, and 10% were grossly misclassified. The ranking of individual intakes of “vegetables” was poor ( r =0.24). The median difference for intakes of “whole grains” was 43 g/day, the correlation was satisfactory ( r =0.51), and 81% of the participants were classified in the same or adjacent quartile, whereas 1% was grossly misclassified. Limits of agreements ranged from –48 to 133 g/day, and the plot indicated a trend for overestimation with increased intakes ( Figure 2 D). “Unsalted nuts” were estimated with a median difference of 2.3 g/day and with a borderline satisfactory correlation ( r =0.48). In total, 74% of the participants were classified in the same or adjacent quartile, and 13% were grossly misclassified ( Table 2 ). The distribution of the differences increased with higher intakes, and the limits of agreement were wide ( Figure 2 C).

Fish, Dairy Products, and Meat

Median differences for “fish” and “fatty fish” intakes were 22 and 10 g/day, respectively ( Table 2 ). Ranking of individual intakes was satisfactory with correlations of r =0.52 and r =0.61 for “fish” and “fatty fish,” respectively. Limits of agreements ranged from –68 to 117 g/day, and the distribution of the differences increased with higher mean intakes up to about 100 g/day of “fish.” For “fatty fish,” the agreement was good with evenly distributed differences above and below the mean difference and narrow limits of agreement (–49 to 71 g/day; Figure 2 E and F). Classification of individual intakes was good for both “fish” and “fatty fish” with low gross misclassification ( Table 2 ).

The DIGIKOST-FFQ showed a minor underestimation of dairy products on the group level, acceptable ranking ( r =0.65), low gross misclassification of individual intakes, and good agreement between the methods ( Table 2 and Figure 3 A and B). The DIGIKOST-FFQ was able to estimate intakes of “red meat” and “processed meat” on the group level and to rank individual intakes of both food groups ( Table 2 and Figure 2 G and H). More than 82% of the participants were classified in the same or adjacent quartile, and few were grossly misclassified.

Sugar- and Fat-Rich Foods and Beverages, Other Beverages, and Dietary Supplements

There were good agreements between the methods in estimating intakes of “sugar- and fat-rich foods” and “sugar-rich beverages” on the group level, fair ranking of individual intakes, and good quartile classifications ( Table 2 and Figure 3 G and H). Intakes of “water” were overestimated by the DIGIKOST-FFQ, and the ranking of individual intake was fair ( Table 2 and Figure 3 E). The DIGIKOST-FFQ was able to estimate the intake of “ethanol” both on group and individual levels ( Table 2 ). More than 79% of the participants were classified in the same or adjacent quartile for intakes of “sugar- and fat-rich foods,” “sugar-rich beverages,” “water,” and “ethanol” ( Table 2 ).

Physical Activity, Sleep, and Sedentary Time

The DIGIKOST-FFQ underestimated time in moderate intensity (median –468 min/week), which increased with higher amounts of time in moderate physical intensity ( Table 5 and Figure 4 A). The correlation between the methods was borderline fair ( r =0.29; Table 4 ). In total, 73% of the participants were classified in the correct or adjacent quartile, and 5% were grossly misclassified. On the group level, vigorous intensity was overestimated by 18 min/week, and the individual differences increased with increased time in “vigorous physical activity” ( Table 5 and Figure 4 B). There was a fair correlation between the methods for vigorous intensity ( P =.40), and no gross misclassification. The DIGIKOST-FFQ was able to rank individual time in “moderate to vigorous intensity” ( P =.64), and 79% were classified in the correct or adjacent quartile, and few were grossly misclassified ( Table 5 ). On the group level, time in “moderate to vigorous intensity” was underestimated by DIGIKOST-FFQ, which also increased with higher amounts of time in this intensity ( Table 5 and Figure 4 C). The underreporting of “moderate to vigorous intensity” was particularly seen among individuals with BMI <25 kg/m 2 ( Figure 4 D).

b SWA: SenseWear Armband Mini.

c Wilcoxon signed rank test.

d MPA: moderate physical activity.

e VPA: vigorous physical activity.

f MVPA: moderate to vigorous physical activity.

The DIGIKOST-FFQ was able to estimate time in “sedentary” activities on the group level, with a trend for overreporting with higher amounts of time in this activity, and the correlation between the methods was poor ( r =0.27; Table 5 and Figure 4 E). A high number of participants (75%) were classified in the correct or adjacent quartile, and 9% were grossly misclassified. Median time in “sleep” was estimated to be 7 hours per day with both methods, with a trend for underreporting by the DIGIKOST-FFQ with increased time in “sleep” ( Table 5 and Figure 4 F). The correlation was poor ( r =0.19), but the classification of individual time in quartiles was good (71%; Table 5 ).

Adherence to Dietary Guidelines and Physical Activity Recommendations

The sensitivity was above 60% for all dietary guidelines except for “whole grains,” “fish,” “red meat,” and “drinks with added sugar” ( Table 3 ). The specificity was above 60% for all dietary guidelines except for “unsalted nuts,” “processed meat,” “low-fat dairy products,” and “dietary supplements” ( Table 3 ). The total mean diet score of the Norwegian diet index (ranging from 0 to 20 points) was estimated to be 11.2 (SD 3.2) and 10.4 (SD 3.1) points with the DIGIKOST-FFQ and WR, respectively ( Multimedia Appendix 2 ) [ 32 ]. Adherences to each component in the Norwegian diet index measured by DIGIKOST-FFQ, WR, and SWA are shown in Tables 4 and 6 . Intermediate adherences were the most frequent for the composite “diet score,” “fruits and berries,” “vegetables,” and “margarine, butter, and oils” estimated with both DIGIKOST-FFQ and WR. Low adherences were shown by both methods for “unsalted nuts,” “processed meat,” and “sugar- and fat-rich foods,” whereas high adherences were shown for “fish,” “sugar-rich drinks,” and “dietary supplements.” Approximately, evenly distributed adherences (ie, low, intermediate, and high) with both methods were shown for “red meat” ( Table 6 ). High adherence to the recommendation of physical activity was estimated as the most frequent by both methods ( Table 4 ).

a No adherence.

b Intermediate adherence.

c High adherence.

d FFQ: food frequency questionnaire.

e WR: weighed food record.

f Not available.

Principal Findings

We investigated the relative validity of the new digital assessment tool, DIGIKOST-FFQ, with 7-day WR and activity sensors as reference methods. Generally, the DIGIKOST-FFQ was able to rank individual intakes of foods and time spent in physical activities adequately and also able to evaluate dietary intakes and adherences to the Norwegian FBDG and physical activity recommendation.

The DIGIKOST-FFQ is based on the paper form NORDIET-FFQ, and improvements were made to enhance the quality of the DIGIKOST-FFQ estimates. For example, in the DIGIKOST-FFQ, questions about fruits and berries were presented as single food items as opposed to aggregated questions in the paper FFQ, which underreported the fruit, berry, and vegetable intakes [ 26 ]. In this validation study, improved correlations and agreement between the test and reference methods for estimated fruit, berry, and vegetable intakes were observed. Moreover, WR has been documented to underreport energy and food intake [ 46 , 47 ]. Thus, the overreporting of fruit and vegetable intakes in the DIGIKOST-FFQ along with the improved correlations and agreement between the methods in this validation study indicate a higher accuracy of the intake estimates from the DIGIKOST-FFQ for these food groups as compared to the NORDIET-FFQ. In particular, the DIGIKOST-FFQ showed a borderline fair correlation in ranking individual intakes of vegetables. However, the median difference between the methods was well below 1 portion of vegetable (ie, 100 g), the agreement between the methods was good, and the DIGIKOST-FFQ was able to identify adherence to the recommendation of vegetables.

Few participants reported intakes of unsalted nuts, and low amounts were reported for those with intakes. This may explain the high share grossly misclassification of nut intakes in this study.

Adding single-item questions regarding whole grain products along with automatic counting of slices of bread consumed per week to help the responder to report intakes of bread in the DIGIKOST-FFQ ( Multimedia Appendix 1 ) may have successfully contributed to the improvement of the reported intakes of whole grains. In addition, legumes were added as a new item in the DIGIKOST-FFQ, and the good agreement and correlation observed for intakes of legumes contributed to the overall good ability of DIGIKOST-FFQ to identify individuals with different adherences to the recommendations for all fiber-rich foods.

Reported intakes of fish, meat, dairy products, sugar- and fat-rich foods, sugar-rich beverages, ethanol, water, and dietary supplements were good with the DIGIKOST-FFQ. We hypothesize that the overreporting of water, corresponding to 1 glass of water per day, might be explained by the warmer climate in June 2021, during which most of the participants completed the DIGIKOST-FFQ, as compared to the colder climate in August 2021 when they completed the WR, thus resulting in higher intakes of water in June.

Underreporting of physical activity by questionnaires is a common issue when compared to objective activity sensors, which record all activities within 24 hours a day, whereas questionnaires rely only on a few questions supported by others [ 48 , 49 ]. In particular, sex and BMI have been shown to alter the association between self-reported and objectively measured physical activity [ 49 ]. Additionally, questionnaires are subjected to memory and conceptualization of different degree of intensities, such as the difference between moderate and vigorous intensities. In this study, there was a trend for normal-weighed individuals to underreport moderate physical activity more than overweighed or obese individuals. Overall, on the group level, the DIGIKOST-FFQ underreported moderate to vigorous physical activity by 450 minutes per week when compared to SWA. In particular, low and moderate intensities of physical activity, such as transport to work (eg, biking), gardening, and housework, have been challenging to report in questionnaires [ 50 , 51 ], which was also observed in this study. Moreover, the aim of the DIGIKOST tool is to estimate adherence to the recommendations and to give advice in the individual report in order to spend more time in physical activity. Therefore, ranking individual time in different activities and classifying the individuals according to adherence are important abilities of the assessment tool.

Adding illustrations of different intensities with images and text for the participants to better differentiate between the performance of physical activity during the day and week may increase the quality of the estimated time in moderate physical activity. Generally, the results from this study showed that the DIGIKOST-FFQ was able to rank individual dietary intakes and identify adherences to most of the dietary guidelines and physical activity recommendations. This conforms to previous studies showing the ability of FFQs to classify individuals according to different intakes of foods and time in activities [ 8 , 9 ].

Strengths and Limitations

We included 77 participants in the study, which is an adequate number in validation studies. The announcement by email lists from the National Registry of Norway was accompanied by an advertisement in Facebook. Recruitment through Facebook has shown to be the most successful strategy. Generally, the use of social media for recruitment in health research has shown to be effective in hard-to-reach populations [ 52 , 53 ]. In this study, most of the participants were women with higher education than the general Norwegian population [ 54 ]. This might compromise the external validity of this study.

In this study, the DIGIKOST-FFQ (test method) was administered prior to the reference methods in order to avoid the learning effect from the reference method [ 11 ]. Moreover, the DIGIKOST-FFQ asks for dietary intakes and activities the last 2 months, whereas the reference methods (WR and SWA) prospectively recorded dietary intake and activities for 7 consecutive days and completed within 1-2 months after fulfilling the DIGIKOST-FFQ. The average range of days between completion of DIGIKOST-FFQ and reference methods was 19 (SD 16.8) days. The expectancy of meaningful changes in dietary intakes or activities during this period is low.

The measurement errors usually associated with printed questionnaires have also been documented in digital questionnaires such as underreporting and recall errors [ 6 , 10 , 12 , 26 , 31 , 41 , 55 , 56 ]. Closed questions, predefined categories of frequency and amounts, and foods not included in the questionnaire may result in misclassification and underreporting. In DIGIKOST-FFQ, underreporting was documented for some food groups and activities, but there were few gross misclassifications, which indicates the good ability to identify individuals with different adherence to the recommendations. Ideally, in validation studies, the error in the reference method should be independent from the test method [ 16 ]. This study used a self-reported instrument, that is, the WR, as the reference method against the DIGIKOST-FFQ. Since both methods are self-reported, the errors may not be independent, thus inflating the validity correlation coefficient and an optimistic evaluation of the validity of the FFQ. Even though the WR is commonly used as a reference method in the validation of FFQs [ 11 , 12 ] and does not rely on the memory of food intakes, no interpretation of portion size is needed due to the weighing. Moreover, the WR provides data on foods that are really consumed, which are not provided by using biomarkers as a reference method. The DIGIKOST-FFQ does not cover a whole diet, since the aim of the tool is to estimate adherence to the recommendations, thus biomarkers for energy intake or nutrients are not used in this study. In the future, revisions of the DIGIKOST tool developments may be conducted to enable estimations of energy and food intakes; however, that is not the design and aim of this version.

Conclusions

The DIGIKOST-FFQ was able to assess dietary intake and physical activity and identify individuals with different adherences to the Norwegian FBDG and physical activity recommendation. However, moderate physical activity was underreported, water was overreported, and vegetables showed a low correlation between the methods, which is important to take into account when interpreting the data in future studies. The new digital questionnaire contributes with valuable data for research projects focusing on improved lifestyle, prevention of diseases, and increased quality of life and survival, as well as a screening tool in the clinic.

Acknowledgments

The authors would like to acknowledge project coordinator Nina Nordberg Gjerrestad and the participants in the study. The authors would also like to thank Professor Sveinung Berntsen for generating the physical activity data from the SenseWear Armband. The Throne Holst Foundation for Nutrition Research and Aktieselskabet Freia Chocolade Fabriks Medisinske Fond funded this study.

Authors' Contributions

HBH had the main responsibility for writing the paper. HBH, MDK, MHC, AH, and RB contributed to the conception and the design of the study and drafting of the paper. MDK and HBH contributed to the acquisition of data. HBH, MDK, and MHC contributed to analyses. All authors contributed to the interpretation of the data and the writing and approval of the final paper.

Conflicts of Interest

None declared.

DIGIKOST: a digital application for diet and lifestyle assessment and benchmarking against national guidelines.

Adherence to total diet score and each single component measured with both assessment tools.

• Diet, nutrition and the prevention of chronic diseases: report of a joint WHO/FAO expert consultation. World Health Organization. 2003. URL: https://www.who.int/publications/i/item/924120916X [accessed 2024-03-27]
• Food, nutrition, physical activity, and the prevention of cancer: a global perspective. World Cancer Research Fund/American Institute for Cancer Research. 2007. URL: https://www3.paho.org/hq/dmdocuments/2011/nutrition-AICR-WCR-food-physical-activ.pdf [accessed 2024-03-27]
• Kostråd for å Fremme Folkehelsen og Forebygge Kroniske Sykdommer Metodologi og Vitenskapelig Kunnskapsgrunnlag: Nasjonalt Råd for Ernæring 2011. Oslo. Helsedirektoratet; 2011;353.
• Diet, nutrition, physical activity and cancer: a global prespective. World Cancer Research Fund/American Institute for Cancer Research. Continuous Update Project Expert Report 2018. 2018. URL: https://www.wcrf.org/wp-content/uploads/2021/02/Summary-of-Third-Expert-Report-2018.pdf [accessed 2024-03-27]
• Scientific report of the 2020 Dietary Guidelines Advisory Committee: Advisory report to the Secretary of Agriculture and the Secretary of Health and Human Services. Dietary Guidelines Advisory Committee. 2020. URL: https:/​/www.​dietaryguidelines.gov/​sites/​default/​files/​2020-07/​ScientificReport_of_the_2020DietaryGuidelinesAdvisoryCommittee_first-print.​pdf [accessed 2024-04-17]
• Cade JE. Measuring diet in the 21st century: use of new technologies. Proc Nutr Soc. 2017;76(3):276-282. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Conrad J, Nöthlings U. Innovative approaches to estimate individual usual dietary intake in large-scale epidemiological studies. Proc Nutr Soc. 2017;76(3):213-219. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Shim JS, Oh K, Kim HC. Dietary assessment methods in epidemiologic studies. Epidemiol Health. 2014;36:e2014009. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Veierød MB, Thelle D, Hjartåker A, Laake P. Epidemiologiske og Kliniske Forskningsmetoder. Oslo. Gyldendal Akademisk; 2007;33-44.
• Cade J, Thompson R, Burley V, Warm D. Development, validation and utilisation of food-frequency questionnaires—a review. Public Health Nutr. 2002;5(4):567-587. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Margetts BM, Nelson M. Design Concepts in Nutritional Epidemiology. New York. Oxford University Press; 1997.
• Willett W. Nutritional Epidemiology. Oxford. Oxford University Press; 2013.
• Araujo MC, Yokoo EM, Pereira RA. Validation and calibration of a semiquantitative food frequency questionnaire designed for adolescents. J Am Diet Assoc. 2010;110(8):1170-1177. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Dehghan M, del Cerro S, Zhang X, Cuneo JM, Linetzky B, Diaz R, et al. Validation of a semi-quantitative food frequency questionnaire for Argentinean adults. PLoS One. 2012;7(5):e37958. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tucker KL, Smith CE, Lai CQ, Ordovas JM. Quantifying diet for nutrigenomic studies. Annu Rev Nutr. 2013;33:349-371. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kirkpatrick SI, Baranowski T, Subar AF, Tooze JA, Frongillo EA. Best practices for conducting and interpreting studies to validate self-report dietary assessment methods. J Acad Nutr Diet. 2019;119(11):1801-1816. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Park Y, Dodd KW, Kipnis V, Thompson FE, Potischman N, Schoeller DA, et al. Comparison of self-reported dietary intakes from the automated self-administered 24-h recall, 4-d food records, and food-frequency questionnaires against recovery biomarkers. Am J Clin Nutr. 2018;107(1):80-93. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kirkpatrick SI, Troiano RP, Barrett B, Cunningham C, Subar AF, Park Y, et al. Measurement error affecting web- and paper-based dietary assessment instruments: insights from the multi-cohort eating and activity study for understanding reporting error. Am J Epidemiol. 2022;191(6):1125-1139. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kipnis V, Midthune D, Freedman LS, Bingham S, Schatzkin A, Subar A, et al. Empirical evidence of correlated biases in dietary assessment instruments and its implications. Am J Epidemiol. 2001;153(4):394-403. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Adamson AJ, Collerton J, Davies K, Foster E, Jagger C, Stamp E, et al. Nutrition in advanced age: dietary assessment in the Newcastle 85+ study. Eur J Clin Nutr. 2009;63(Suppl 1):S6-S18. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Liu L, Wang PP, Roebothan B, Ryan A, Tucker CS, Colbourne J, et al. Assessing the validity of a self-administered food-frequency questionnaire (FFQ) in the adult population of Newfoundland and Labrador, Canada. Nutr J. 2013;12:49. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Falomir Z, Arregui M, Madueño F, Corella D, Coltell Ó. Automation of food questionnaires in medical studies: a state-of-the-art review and future prospects. Comput Biol Med. 2012;42(10):964-974. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Forster H, Fallaize R, Gallagher C, O'Donovan CB, Woolhead C, Walsh MC, et al. Online dietary intake estimation: the Food4Me food frequency questionnaire. J Med Internet Res. 2014;16(6):e150. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Carlsen MH, Andersen LF, Hjartåker A. Reproducibility and feasibility of an online self-administered food frequency questionnaire for use among adult Norwegians. Food Nutr Res. 2021;65:7561. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kristal AR, Kolar AS, Fisher JL, Plascak JJ, Stumbo PJ, Weiss R, et al. Evaluation of web-based, self-administered, graphical food frequency questionnaire. J Acad Nutr Diet. 2014;114(4):613-621. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Henriksen HB, Carlsen MH, Paur I, Berntsen S, Bøhn SK, Skjetne AJ, et al. Relative validity of a short food frequency questionnaire assessing adherence to the Norwegian dietary guidelines among colorectal cancer patients. Food Nutr Res. 2018;62:1306. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Henriksen HB, Knudsen MD, Carlsen MH, Hjartåker A, Blomhoff R. A short digital food frequency questionnaire (DIGIKOST-FFQ) assessing dietary intake and other lifestyle factors among Norwegians: qualitative evaluation with focus group interviews and usability testing. JMIR Form Res. 2022;6(11):e35933. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Usability testing. University Center for Information Technology (USIT). 2017. URL: https://www.usit.uio.no/om/organisasjon/bnt/web/ux/blogg/2017/brukertest.html [accessed 2024-03-27]
• Norwegian Digitalisation Agency. 2023. URL: https://www.digdir.no/digdir/about-norwegian-digitalisation-agency/887 [accessed 2024-03-27]
• Difi. 2023. URL: https://eid.difi.no/nb/id-porten [accessed 2024-03-27]
• Henriksen HB, Berntsen S, Paur I, Zucknick M, Skjetne AJ, Bøhn SK, et al. Validation of two short questionnaires assessing physical activity in colorectal cancer patients. BMC Sports Sci Med Rehabil. 2018;10:8-20. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Henriksen HB, Berg HB, Andersen LF, Weedon-Fekjær H, Blomhoff R. Development of the Norwegian diet index and the Norwegian lifestyle index and evaluation in a national survey. Food Nutr Res. 2023;67:-11. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Nettskjema. 2023. URL: https://nettskjema.no/ [accessed 2024-04-15]
• Dalane J, Bergvatn TAM, Kielland E, Carlsen MH. Mål, vekt og porsjonsstørrelser for matvarer [Weights, measures and portion sizes for foods]. Universitetet i Oslo Helsedirektoratet. 2015. URL: https://tinyurl.com/4mjn3vj2 [accessed 2024-04-15]
• Norwegian Directorate of Health. 2021. URL: https:/​/www.​helsenorge.no/​kosthold-og-ernaring/​sma-grep-for-et-sunt-kosthold/​dagens-maltider/​#tallerkenmodellen [accessed 2024-04-15]
• Norwegian Institute of Public Health. 2021. URL: https://www.fhi.no/en/ [accessed 2024-04-15]
• World Health Organization (WHO), Europe. URL: https:/​/www.​euro.who.int/​en/​health-topics/​disease-prevention/​nutrition/​a-healthy-lifestyle/​body-mass-index-bmi?source=post_page [accessed 2024-04-15]
• Web-form. University of Oslo. 2021. URL: https://www.uio.no/english/services/it/adm-services/nettskjema/ [accessed 2024-04-15]
• Rimestad AH, Løken EB, Nordbotten A. Den norske matvaretabellen og beregningsdatabasen ved institutt for ernæringsforskning. Nor J Epidemiol. 2009;10(1):2000. [ FREE Full text ] [ CrossRef ]
• Willett WC. Diet and cancer: an evolving picture. JAMA. 2005;293(2):233-234. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Willett WC, Sampson L, Stampfer MJ, Rosner B, Bain C, Witschi J, et al. Reproducibility and validity of a semiquantitative food frequency questionnaire. Am J Epidemiol. 1985;122(1):51-65. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Glasson C, Chapman K, James E. Fruit and vegetables should be targeted separately in health promotion programmes: differences in consumption levels, barriers, knowledge and stages of readiness for change. Public Health Nutr. 2011;14(4):694-701. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Totland TH, Melnæs BK, Lundberg-Hallén N, Helland-Kigen KM, Lund-Blix NA, Myhre JB, et al. Norkost 3: en Landsomfattende Kostholdsundersøkelse Blant Menn og Kvinner i Norge i Alderen 18-70 år, 2010-11. Oslo. Helsedirektoratet; 2012.
• Bland M. An Introduction to Medical Statistics. Oxford. Oxford University Press; 2000.
• Hankin JH, Wilkens LR, Kolonel LN, Yoshizawa CN. Validation of a quantitative diet history method in Hawaii. Am J Epidemiol. 1991;133(6):616-628. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Garden L, Clark H, Whybrow S, Stubbs RJ. Is misreporting of dietary intake by weighed food records or 24-hour recalls food specific? Eur J Clin Nutr. 2018;72(7):1026-1034. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Stubbs RJ, O'Reilly LM, Whybrow S, Fuller Z, Johnstone AM, Livingstone MBE, et al. Measuring the difference between actual and reported food intakes in the context of energy balance under laboratory conditions. Br J Nutr. 2014;111(11):2032-2043. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Warren JM, Ekelund U, Besson H, Mezzani A, Geladas N, Vanhees L, et al. Experts Panel. Assessment of physical activity—a review of methodologies with reference to epidemiological research: a report of the exercise physiology section of the European Association of Cardiovascular Prevention and Rehabilitation. Eur J Cardiovasc Prev Rehabil. 2010;17(2):127-139. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Quinlan C, Rattray B, Pryor D, Northey JM, Anstey KJ, Butterworth P, et al. The accuracy of self-reported physical activity questionnaires varies with sex and body mass index. PLoS One. 2021;16(8):e0256008. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Cleland JA. The qualitative orientation in medical education research. Korean J Med Educ. 2017;29(2):61-71. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Prince SA, Adamo KB, Hamel ME, Hardt J, Gorber SC, Tremblay M. A comparison of direct versus self-report measures for assessing physical activity in adults: a systematic review. Int J Behav Nutr Phys Act. 2008;5:56-70. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Darko EM, Kleib M, Olson J. Social media use for research participant recruitment: integrative literature review. J Med Internet Res. 2022;24(8):e38015-e38037. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Gelinas L, Pierce R, Winkler S, Cohen IG, Lynch HF, Bierer BE. Using social media as a research recruitment tool: ethical issues and recommendations. Am J Bioeth. 2017;17(3):3-14. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Educational attainment of the population. Statistics Norway. 2020. URL: https://www.ssb.no/en/utdanning/utdanningsniva/statistikk/befolkningens-utdanningsniva [accessed 2024-02-05]
• Carlsen MH, Lillegaard ITL, Karlsen A, Blomhoff R, Drevon CA, Andersen LF. Evaluation of energy and dietary intake estimates from a food frequency questionnaire using independent energy expenditure measurement and weighed food records. Nutr J. Sep 15, 2010;9:37-46. [ FREE Full text ] [ CrossRef ] [ Medline ]
• McClung HL, Ptomey LT, Shook RP, Aggarwal A, Gorczyca AM, Sazonov ES, et al. Dietary intake and physical activity assessment: current tools, techniques, and technologies for use in adult populations. Am J Prev Med. Oct 01, 2018;55(4):e93-e104. [ FREE Full text ] [ CrossRef ] [ Medline ]

Abbreviations

Edited by G Eysenbach, T de Azevedo Cardoso; submitted 07.10.23; peer-reviewed by D Brassard; comments to author 19.12.23; revised version received 12.02.24; accepted 09.03.24; published 30.04.24.

©Hege Berg Henriksen, Markus Dines Knudsen, Anette Hjartåker, Rune Blomhoff, Monica Hauger Carlsen. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 30.04.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.