references in scientific research

• Privacy Policy

Buy Me a Coffee

Home » References in Research – Types, Examples and Writing Guide

References in Research – Types, Examples and Writing Guide

Table of Contents

References in Research

Definition:

References in research are a list of sources that a researcher has consulted or cited while conducting their study. They are an essential component of any academic work, including research papers, theses, dissertations, and other scholarly publications.

Types of References

There are several types of references used in research, and the type of reference depends on the source of information being cited. The most common types of references include:

References to books typically include the author’s name, title of the book, publisher, publication date, and place of publication.

Example: Smith, J. (2018). The Art of Writing. Penguin Books.

Journal Articles

References to journal articles usually include the author’s name, title of the article, name of the journal, volume and issue number, page numbers, and publication date.

Example: Johnson, T. (2021). The Impact of Social Media on Mental Health. Journal of Psychology, 32(4), 87-94.

Web sources

References to web sources should include the author or organization responsible for the content, the title of the page, the URL, and the date accessed.

Example: World Health Organization. (2020). Coronavirus disease (COVID-19) advice for the public. Retrieved from https://www.who.int/emergencies/disease/novel-coronavirus-2019/advice-for-public

Conference Proceedings

References to conference proceedings should include the author’s name, title of the paper, name of the conference, location of the conference, date of the conference, and page numbers.

Example: Chen, S., & Li, J. (2019). The Future of AI in Education. Proceedings of the International Conference on Educational Technology, Beijing, China, July 15-17, pp. 67-78.

References to reports typically include the author or organization responsible for the report, title of the report, publication date, and publisher.

Example: United Nations. (2020). The Sustainable Development Goals Report. United Nations.

Formats of References

Some common Formates of References with their examples are as follows:

APA (American Psychological Association) Style

The APA (American Psychological Association) Style has specific guidelines for formatting references used in academic papers, articles, and books. Here are the different reference formats in APA style with examples:

Author, A. A. (Year of publication). Title of book. Publisher.

Example : Smith, J. K. (2005). The psychology of social interaction. Wiley-Blackwell.

Journal Article

Author, A. A., Author, B. B., & Author, C. C. (Year of publication). Title of article. Title of Journal, volume number(issue number), page numbers.

Example : Brown, L. M., Keating, J. G., & Jones, S. M. (2012). The role of social support in coping with stress among African American adolescents. Journal of Research on Adolescence, 22(1), 218-233.

Author, A. A. (Year of publication or last update). Title of page. Website name. URL.

Example : Centers for Disease Control and Prevention. (2020, December 11). COVID-19: How to protect yourself and others. https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting-sick/prevention.html

Magazine article

Author, A. A. (Year, Month Day of publication). Title of article. Title of Magazine, volume number(issue number), page numbers.

Example : Smith, M. (2019, March 11). The power of positive thinking. Psychology Today, 52(3), 60-65.

Newspaper article:

Author, A. A. (Year, Month Day of publication). Title of article. Title of Newspaper, page numbers.

Example: Johnson, B. (2021, February 15). New study shows benefits of exercise on mental health. The New York Times, A8.

Edited book

Editor, E. E. (Ed.). (Year of publication). Title of book. Publisher.

Example : Thompson, J. P. (Ed.). (2014). Social work in the 21st century. Sage Publications.

Chapter in an edited book:

Author, A. A. (Year of publication). Title of chapter. In E. E. Editor (Ed.), Title of book (pp. page numbers). Publisher.

Example : Johnson, K. S. (2018). The future of social work: Challenges and opportunities. In J. P. Thompson (Ed.), Social work in the 21st century (pp. 105-118). Sage Publications.

MLA (Modern Language Association) Style

The MLA (Modern Language Association) Style is a widely used style for writing academic papers and essays in the humanities. Here are the different reference formats in MLA style:

Author’s Last name, First name. Title of Book. Publisher, Publication year.

Example : Smith, John. The Psychology of Social Interaction. Wiley-Blackwell, 2005.

Journal article

Author’s Last name, First name. “Title of Article.” Title of Journal, volume number, issue number, Publication year, page numbers.

Example : Brown, Laura M., et al. “The Role of Social Support in Coping with Stress among African American Adolescents.” Journal of Research on Adolescence, vol. 22, no. 1, 2012, pp. 218-233.

Author’s Last name, First name. “Title of Webpage.” Website Name, Publication date, URL.

Example : Centers for Disease Control and Prevention. “COVID-19: How to Protect Yourself and Others.” CDC, 11 Dec. 2020, https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting-sick/prevention.html.

Author’s Last name, First name. “Title of Article.” Title of Magazine, Publication date, page numbers.

Example : Smith, Mary. “The Power of Positive Thinking.” Psychology Today, Mar. 2019, pp. 60-65.

Newspaper article

Author’s Last name, First name. “Title of Article.” Title of Newspaper, Publication date, page numbers.

Example : Johnson, Bob. “New Study Shows Benefits of Exercise on Mental Health.” The New York Times, 15 Feb. 2021, p. A8.

Editor’s Last name, First name, editor. Title of Book. Publisher, Publication year.

Example : Thompson, John P., editor. Social Work in the 21st Century. Sage Publications, 2014.

Chapter in an edited book

Author’s Last name, First name. “Title of Chapter.” Title of Book, edited by Editor’s First Name Last name, Publisher, Publication year, page numbers.

Example : Johnson, Karen S. “The Future of Social Work: Challenges and Opportunities.” Social Work in the 21st Century, edited by John P. Thompson, Sage Publications, 2014, pp. 105-118.

Chicago Manual of Style

The Chicago Manual of Style is a widely used style for writing academic papers, dissertations, and books in the humanities and social sciences. Here are the different reference formats in Chicago style:

Example : Smith, John K. The Psychology of Social Interaction. Wiley-Blackwell, 2005.

Author’s Last name, First name. “Title of Article.” Title of Journal volume number, no. issue number (Publication year): page numbers.

Example : Brown, Laura M., John G. Keating, and Sarah M. Jones. “The Role of Social Support in Coping with Stress among African American Adolescents.” Journal of Research on Adolescence 22, no. 1 (2012): 218-233.

Author’s Last name, First name. “Title of Webpage.” Website Name. Publication date. URL.

Example : Centers for Disease Control and Prevention. “COVID-19: How to Protect Yourself and Others.” CDC. December 11, 2020. https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting-sick/prevention.html.

Author’s Last name, First name. “Title of Article.” Title of Magazine, Publication date.

Example : Smith, Mary. “The Power of Positive Thinking.” Psychology Today, March 2019.

Author’s Last name, First name. “Title of Article.” Title of Newspaper, Publication date.

Example : Johnson, Bob. “New Study Shows Benefits of Exercise on Mental Health.” The New York Times, February 15, 2021.

Example : Thompson, John P., ed. Social Work in the 21st Century. Sage Publications, 2014.

Author’s Last name, First name. “Title of Chapter.” In Title of Book, edited by Editor’s First Name Last Name, page numbers. Publisher, Publication year.

Example : Johnson, Karen S. “The Future of Social Work: Challenges and Opportunities.” In Social Work in the 21st Century, edited by John P. Thompson, 105-118. Sage Publications, 2014.

Harvard Style

The Harvard Style, also known as the Author-Date System, is a widely used style for writing academic papers and essays in the social sciences. Here are the different reference formats in Harvard Style:

Author’s Last name, First name. Year of publication. Title of Book. Place of publication: Publisher.

Example : Smith, John. 2005. The Psychology of Social Interaction. Oxford: Wiley-Blackwell.

Author’s Last name, First name. Year of publication. “Title of Article.” Title of Journal volume number (issue number): page numbers.

Example: Brown, Laura M., John G. Keating, and Sarah M. Jones. 2012. “The Role of Social Support in Coping with Stress among African American Adolescents.” Journal of Research on Adolescence 22 (1): 218-233.

Author’s Last name, First name. Year of publication. “Title of Webpage.” Website Name. URL. Accessed date.

Example : Centers for Disease Control and Prevention. 2020. “COVID-19: How to Protect Yourself and Others.” CDC. https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting-sick/prevention.html. Accessed April 1, 2023.

Author’s Last name, First name. Year of publication. “Title of Article.” Title of Magazine, month and date of publication.

Example : Smith, Mary. 2019. “The Power of Positive Thinking.” Psychology Today, March 2019.

Author’s Last name, First name. Year of publication. “Title of Article.” Title of Newspaper, month and date of publication.

Example : Johnson, Bob. 2021. “New Study Shows Benefits of Exercise on Mental Health.” The New York Times, February 15, 2021.

Editor’s Last name, First name, ed. Year of publication. Title of Book. Place of publication: Publisher.

Example : Thompson, John P., ed. 2014. Social Work in the 21st Century. Thousand Oaks, CA: Sage Publications.

Author’s Last name, First name. Year of publication. “Title of Chapter.” In Title of Book, edited by Editor’s First Name Last Name, page numbers. Place of publication: Publisher.

Example : Johnson, Karen S. 2014. “The Future of Social Work: Challenges and Opportunities.” In Social Work in the 21st Century, edited by John P. Thompson, 105-118. Thousand Oaks, CA: Sage Publications.

Vancouver Style

The Vancouver Style, also known as the Uniform Requirements for Manuscripts Submitted to Biomedical Journals, is a widely used style for writing academic papers in the biomedical sciences. Here are the different reference formats in Vancouver Style:

Author’s Last name, First name. Title of Book. Edition number. Place of publication: Publisher; Year of publication.

Example : Smith, John K. The Psychology of Social Interaction. 2nd ed. Oxford: Wiley-Blackwell; 2005.

Author’s Last name, First name. Title of Article. Abbreviated Journal Title. Year of publication; volume number(issue number):page numbers.

Example : Brown LM, Keating JG, Jones SM. The Role of Social Support in Coping with Stress among African American Adolescents. J Res Adolesc. 2012;22(1):218-233.

Author’s Last name, First name. Title of Webpage. Website Name [Internet]. Publication date. [cited date]. Available from: URL.

Example : Centers for Disease Control and Prevention. COVID-19: How to Protect Yourself and Others [Internet]. 2020 Dec 11. [cited 2023 Apr 1]. Available from: https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting-sick/prevention.html.

Author’s Last name, First name. Title of Article. Title of Magazine. Year of publication; month and day of publication:page numbers.

Example : Smith M. The Power of Positive Thinking. Psychology Today. 2019 Mar 1:32-35.

Author’s Last name, First name. Title of Article. Title of Newspaper. Year of publication; month and day of publication:page numbers.

Example : Johnson B. New Study Shows Benefits of Exercise on Mental Health. The New York Times. 2021 Feb 15:A4.

Editor’s Last name, First name, editor. Title of Book. Edition number. Place of publication: Publisher; Year of publication.

Example: Thompson JP, editor. Social Work in the 21st Century. 1st ed. Thousand Oaks, CA: Sage Publications; 2014.

Author’s Last name, First name. Title of Chapter. In: Editor’s Last name, First name, editor. Title of Book. Edition number. Place of publication: Publisher; Year of publication. page numbers.

Example : Johnson KS. The Future of Social Work: Challenges and Opportunities. In: Thompson JP, editor. Social Work in the 21st Century. 1st ed. Thousand Oaks, CA: Sage Publications; 2014. p. 105-118.

Turabian Style

Turabian style is a variation of the Chicago style used in academic writing, particularly in the fields of history and humanities. Here are the different reference formats in Turabian style:

Author’s Last name, First name. Title of Book. Place of publication: Publisher, Year of publication.

Example : Smith, John K. The Psychology of Social Interaction. Oxford: Wiley-Blackwell, 2005.

Author’s Last name, First name. “Title of Article.” Title of Journal volume number, no. issue number (Year of publication): page numbers.

Example : Brown, LM, Keating, JG, Jones, SM. “The Role of Social Support in Coping with Stress among African American Adolescents.” J Res Adolesc 22, no. 1 (2012): 218-233.

Author’s Last name, First name. “Title of Webpage.” Name of Website. Publication date. Accessed date. URL.

Example : Centers for Disease Control and Prevention. “COVID-19: How to Protect Yourself and Others.” CDC. December 11, 2020. Accessed April 1, 2023. https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting-sick/prevention.html.

Author’s Last name, First name. “Title of Article.” Title of Magazine, Month Day, Year of publication, page numbers.

Example : Smith, M. “The Power of Positive Thinking.” Psychology Today, March 1, 2019, 32-35.

Author’s Last name, First name. “Title of Article.” Title of Newspaper, Month Day, Year of publication.

Example : Johnson, B. “New Study Shows Benefits of Exercise on Mental Health.” The New York Times, February 15, 2021.

Editor’s Last name, First name, ed. Title of Book. Place of publication: Publisher, Year of publication.

Example : Thompson, JP, ed. Social Work in the 21st Century. Thousand Oaks, CA: Sage Publications, 2014.

Author’s Last name, First name. “Title of Chapter.” In Title of Book, edited by Editor’s Last name, First name, page numbers. Place of publication: Publisher, Year of publication.

Example : Johnson, KS. “The Future of Social Work: Challenges and Opportunities.” In Social Work in the 21st Century, edited by Thompson, JP, 105-118. Thousand Oaks, CA: Sage Publications, 2014.

IEEE (Institute of Electrical and Electronics Engineers) Style

IEEE (Institute of Electrical and Electronics Engineers) style is commonly used in engineering, computer science, and other technical fields. Here are the different reference formats in IEEE style:

Author’s Last name, First name. Book Title. Place of Publication: Publisher, Year of publication.

Example : Oppenheim, A. V., & Schafer, R. W. Discrete-Time Signal Processing. Upper Saddle River, NJ: Prentice Hall, 2010.

Author’s Last name, First name. “Title of Article.” Abbreviated Journal Title, vol. number, no. issue number, pp. page numbers, Month year of publication.

Example: Shannon, C. E. “A Mathematical Theory of Communication.” Bell System Technical Journal, vol. 27, no. 3, pp. 379-423, July 1948.

Conference paper

Author’s Last name, First name. “Title of Paper.” In Title of Conference Proceedings, Place of Conference, Date of Conference, pp. page numbers, Year of publication.

Example: Gupta, S., & Kumar, P. “An Improved System of Linear Discriminant Analysis for Face Recognition.” In Proceedings of the 2011 International Conference on Computer Science and Network Technology, Harbin, China, Dec. 2011, pp. 144-147.

Author’s Last name, First name. “Title of Webpage.” Name of Website. Date of publication or last update. Accessed date. URL.

Example : National Aeronautics and Space Administration. “Apollo 11.” NASA. July 20, 1969. Accessed April 1, 2023. https://www.nasa.gov/mission_pages/apollo/apollo11.html.

Technical report

Author’s Last name, First name. “Title of Report.” Name of Institution or Organization, Report number, Year of publication.

Example : Smith, J. R. “Development of a New Solar Panel Technology.” National Renewable Energy Laboratory, NREL/TP-6A20-51645, 2011.

Author’s Last name, First name. “Title of Patent.” Patent number, Issue date.

Example : Suzuki, H. “Method of Producing Carbon Nanotubes.” US Patent 7,151,019, December 19, 2006.

Standard Title. Standard number, Publication date.

Example : IEEE Standard for Floating-Point Arithmetic. IEEE Std 754-2008, August 29, 2008

ACS (American Chemical Society) Style

ACS (American Chemical Society) style is commonly used in chemistry and related fields. Here are the different reference formats in ACS style:

Author’s Last name, First name; Author’s Last name, First name. Title of Article. Abbreviated Journal Title Year, Volume, Page Numbers.

Example : Wang, Y.; Zhao, X.; Cui, Y.; Ma, Y. Facile Preparation of Fe3O4/graphene Composites Using a Hydrothermal Method for High-Performance Lithium Ion Batteries. ACS Appl. Mater. Interfaces 2012, 4, 2715-2721.

Author’s Last name, First name. Book Title; Publisher: Place of Publication, Year of Publication.

Example : Carey, F. A. Organic Chemistry; McGraw-Hill: New York, 2008.

Author’s Last name, First name. Chapter Title. In Book Title; Editor’s Last name, First name, Ed.; Publisher: Place of Publication, Year of Publication; Volume number, Chapter number, Page Numbers.

Example : Grossman, R. B. Analytical Chemistry of Aerosols. In Aerosol Measurement: Principles, Techniques, and Applications; Baron, P. A.; Willeke, K., Eds.; Wiley-Interscience: New York, 2001; Chapter 10, pp 395-424.

Author’s Last name, First name. Title of Webpage. Website Name, URL (accessed date).

Example : National Institute of Standards and Technology. Atomic Spectra Database. https://www.nist.gov/pml/atomic-spectra-database (accessed April 1, 2023).

Author’s Last name, First name. Patent Number. Patent Date.

Example : Liu, Y.; Huang, H.; Chen, H.; Zhang, W. US Patent 9,999,999, December 31, 2022.

Author’s Last name, First name; Author’s Last name, First name. Title of Article. In Title of Conference Proceedings, Publisher: Place of Publication, Year of Publication; Volume Number, Page Numbers.

Example : Jia, H.; Xu, S.; Wu, Y.; Wu, Z.; Tang, Y.; Huang, X. Fast Adsorption of Organic Pollutants by Graphene Oxide. In Proceedings of the 15th International Conference on Environmental Science and Technology, American Chemical Society: Washington, DC, 2017; Volume 1, pp 223-228.

AMA (American Medical Association) Style

AMA (American Medical Association) style is commonly used in medical and scientific fields. Here are the different reference formats in AMA style:

Author’s Last name, First name. Article Title. Journal Abbreviation. Year; Volume(Issue):Page Numbers.

Example : Jones, R. A.; Smith, B. C. The Role of Vitamin D in Maintaining Bone Health. JAMA. 2019;321(17):1765-1773.

Author’s Last name, First name. Book Title. Edition number. Place of Publication: Publisher; Year.

Example : Guyton, A. C.; Hall, J. E. Textbook of Medical Physiology. 13th ed. Philadelphia, PA: Saunders; 2015.

Author’s Last name, First name. Chapter Title. In: Editor’s Last name, First name, ed. Book Title. Edition number. Place of Publication: Publisher; Year: Page Numbers.

Example: Rajakumar, K. Vitamin D and Bone Health. In: Holick, M. F., ed. Vitamin D: Physiology, Molecular Biology, and Clinical Applications. 2nd ed. New York, NY: Springer; 2010:211-222.

Author’s Last name, First name. Webpage Title. Website Name. URL. Published date. Updated date. Accessed date.

Example : National Cancer Institute. Breast Cancer Prevention (PDQ®)–Patient Version. National Cancer Institute. https://www.cancer.gov/types/breast/patient/breast-prevention-pdq. Published October 11, 2022. Accessed April 1, 2023.

Author’s Last name, First name. Conference presentation title. In: Conference Title; Conference Date; Place of Conference.

Example : Smith, J. R. Vitamin D and Bone Health: A Meta-Analysis. In: Proceedings of the Annual Meeting of the American Society for Bone and Mineral Research; September 20-23, 2022; San Diego, CA.

Thesis or dissertation

Author’s Last name, First name. Title of Thesis or Dissertation. Degree level [Doctoral dissertation or Master’s thesis]. University Name; Year.

Example : Wilson, S. A. The Effects of Vitamin D Supplementation on Bone Health in Postmenopausal Women [Doctoral dissertation]. University of California, Los Angeles; 2018.

ASCE (American Society of Civil Engineers) Style

The ASCE (American Society of Civil Engineers) style is commonly used in civil engineering fields. Here are the different reference formats in ASCE style:

Author’s Last name, First name. “Article Title.” Journal Title, volume number, issue number (year): page numbers. DOI or URL (if available).

Example : Smith, J. R. “Evaluation of the Effectiveness of Sustainable Drainage Systems in Urban Areas.” Journal of Environmental Engineering, vol. 146, no. 3 (2020): 04020010. https://doi.org/10.1061/(ASCE)EE.1943-7870.0001668.

Example : McCuen, R. H. Hydrologic Analysis and Design. 4th ed. Upper Saddle River, NJ: Pearson Education; 2013.

Author’s Last name, First name. “Chapter Title.” In: Editor’s Last name, First name, ed. Book Title. Edition number. Place of Publication: Publisher; Year: page numbers.

Example : Maidment, D. R. “Floodplain Management in the United States.” In: Shroder, J. F., ed. Treatise on Geomorphology. San Diego, CA: Academic Press; 2013: 447-460.

Author’s Last name, First name. “Paper Title.” In: Conference Title; Conference Date; Location. Place of Publication: Publisher; Year: page numbers.

Example: Smith, J. R. “Sustainable Drainage Systems for Urban Areas.” In: Proceedings of the ASCE International Conference on Sustainable Infrastructure; November 6-9, 2019; Los Angeles, CA. Reston, VA: American Society of Civil Engineers; 2019: 156-163.

Author’s Last name, First name. “Report Title.” Report number. Place of Publication: Publisher; Year.

Example : U.S. Army Corps of Engineers. “Hurricane Sandy Coastal Risk Reduction Program, New York and New Jersey.” Report No. P-15-001. Washington, DC: U.S. Army Corps of Engineers; 2015.

CSE (Council of Science Editors) Style

The CSE (Council of Science Editors) style is commonly used in the scientific and medical fields. Here are the different reference formats in CSE style:

Author’s Last name, First Initial. Middle Initial. “Article Title.” Journal Title. Year;Volume(Issue):Page numbers.

Example : Smith, J.R. “Evaluation of the Effectiveness of Sustainable Drainage Systems in Urban Areas.” Journal of Environmental Engineering. 2020;146(3):04020010.

Author’s Last name, First Initial. Middle Initial. Book Title. Edition number. Place of Publication: Publisher; Year.

Author’s Last name, First Initial. Middle Initial. “Chapter Title.” In: Editor’s Last name, First Initial. Middle Initial., ed. Book Title. Edition number. Place of Publication: Publisher; Year:Page numbers.

Author’s Last name, First Initial. Middle Initial. “Paper Title.” In: Conference Title; Conference Date; Location. Place of Publication: Publisher; Year.

Example : Smith, J.R. “Sustainable Drainage Systems for Urban Areas.” In: Proceedings of the ASCE International Conference on Sustainable Infrastructure; November 6-9, 2019; Los Angeles, CA. Reston, VA: American Society of Civil Engineers; 2019.

Author’s Last name, First Initial. Middle Initial. “Report Title.” Report number. Place of Publication: Publisher; Year.

Bluebook Style

The Bluebook style is commonly used in the legal field for citing legal documents and sources. Here are the different reference formats in Bluebook style:

Case citation

Case name, volume source page (Court year).

Example : Brown v. Board of Education, 347 U.S. 483 (1954).

Statute citation

Name of Act, volume source § section number (year).

Example : Clean Air Act, 42 U.S.C. § 7401 (1963).

Regulation citation

Name of regulation, volume source § section number (year).

Example: Clean Air Act, 40 C.F.R. § 52.01 (2019).

Book citation

Author’s Last name, First Initial. Middle Initial. Book Title. Edition number (if applicable). Place of Publication: Publisher; Year.

Example: Smith, J.R. Legal Writing and Analysis. 3rd ed. New York, NY: Aspen Publishers; 2015.

Journal article citation

Author’s Last name, First Initial. Middle Initial. “Article Title.” Journal Title. Volume number (year): first page-last page.

Example: Garcia, C. “The Right to Counsel: An International Comparison.” International Journal of Legal Information. 43 (2015): 63-94.

Website citation

Author’s Last name, First Initial. Middle Initial. “Page Title.” Website Title. URL (accessed month day, year).

Example : United Nations. “Universal Declaration of Human Rights.” United Nations. https://www.un.org/en/universal-declaration-human-rights/ (accessed January 3, 2023).

Oxford Style

The Oxford style, also known as the Oxford referencing system or the documentary-note citation system, is commonly used in the humanities, including literature, history, and philosophy. Here are the different reference formats in Oxford style:

Author’s Last name, First name. Book Title. Place of Publication: Publisher, Year of Publication.

Example : Smith, John. The Art of Writing. New York: Penguin, 2020.

Author’s Last name, First name. “Article Title.” Journal Title volume, no. issue (year): page range.

Example: Garcia, Carlos. “The Role of Ethics in Philosophy.” Philosophy Today 67, no. 3 (2019): 53-68.

Chapter in an edited book citation

Author’s Last name, First name. “Chapter Title.” In Book Title, edited by Editor’s Name, page range. Place of Publication: Publisher, Year of Publication.

Example : Lee, Mary. “Feminism in the 21st Century.” In The Oxford Handbook of Feminism, edited by Jane Smith, 51-69. Oxford: Oxford University Press, 2018.

Author’s Last name, First name. “Page Title.” Website Title. URL (accessed day month year).

Example : Jones, David. “The Importance of Learning Languages.” Oxford Language Center. https://www.oxfordlanguagecenter.com/importance-of-learning-languages/ (accessed 3 January 2023).

Dissertation or thesis citation

Author’s Last name, First name. “Title of Dissertation/Thesis.” PhD diss., University Name, Year of Publication.

Example : Brown, Susan. “The Art of Storytelling in American Literature.” PhD diss., University of Oxford, 2020.

Newspaper article citation

Author’s Last name, First name. “Article Title.” Newspaper Title, Month Day, Year.

Example : Robinson, Andrew. “New Developments in Climate Change Research.” The Guardian, September 15, 2022.

AAA (American Anthropological Association) Style

The American Anthropological Association (AAA) style is commonly used in anthropology research papers and journals. Here are the different reference formats in AAA style:

Author’s Last name, First name. Year of Publication. Book Title. Place of Publication: Publisher.

Example : Smith, John. 2019. The Anthropology of Food. New York: Routledge.

Author’s Last name, First name. Year of Publication. “Article Title.” Journal Title volume, no. issue: page range.

Example : Garcia, Carlos. 2021. “The Role of Ethics in Anthropology.” American Anthropologist 123, no. 2: 237-251.

Author’s Last name, First name. Year of Publication. “Chapter Title.” In Book Title, edited by Editor’s Name, page range. Place of Publication: Publisher.

Example: Lee, Mary. 2018. “Feminism in Anthropology.” In The Oxford Handbook of Feminism, edited by Jane Smith, 51-69. Oxford: Oxford University Press.

Author’s Last name, First name. Year of Publication. “Page Title.” Website Title. URL (accessed day month year).

Example : Jones, David. 2020. “The Importance of Learning Languages.” Oxford Language Center. https://www.oxfordlanguagecenter.com/importance-of-learning-languages/ (accessed January 3, 2023).

Author’s Last name, First name. Year of Publication. “Title of Dissertation/Thesis.” PhD diss., University Name.

Example : Brown, Susan. 2022. “The Art of Storytelling in Anthropology.” PhD diss., University of California, Berkeley.

Author’s Last name, First name. Year of Publication. “Article Title.” Newspaper Title, Month Day.

Example : Robinson, Andrew. 2021. “New Developments in Anthropology Research.” The Guardian, September 15.

AIP (American Institute of Physics) Style

The American Institute of Physics (AIP) style is commonly used in physics research papers and journals. Here are the different reference formats in AIP style:

Example : Johnson, S. D. 2021. “Quantum Computing and Information.” Journal of Applied Physics 129, no. 4: 043102.

Example : Feynman, Richard. 2018. The Feynman Lectures on Physics. New York: Basic Books.

Example : Jones, David. 2020. “The Future of Quantum Computing.” In The Handbook of Physics, edited by John Smith, 125-136. Oxford: Oxford University Press.

Conference proceedings citation

Author’s Last name, First name. Year of Publication. “Title of Paper.” Proceedings of Conference Name, date and location: page range. Place of Publication: Publisher.

Example : Chen, Wei. 2019. “The Applications of Nanotechnology in Solar Cells.” Proceedings of the 8th International Conference on Nanotechnology, July 15-17, Tokyo, Japan: 224-229. New York: AIP Publishing.

Example : American Institute of Physics. 2022. “About AIP Publishing.” AIP Publishing. https://publishing.aip.org/about-aip-publishing/ (accessed January 3, 2023).

Patent citation

Author’s Last name, First name. Year of Publication. Patent Number.

Example : Smith, John. 2018. US Patent 9,873,644.

References Writing Guide

Here are some general guidelines for writing references:

• Follow the citation style guidelines: Different disciplines and journals may require different citation styles (e.g., APA, MLA, Chicago). It is important to follow the specific guidelines for the citation style required.
• Include all necessary information : Each citation should include enough information for readers to locate the source. For example, a journal article citation should include the author(s), title of the article, journal title, volume number, issue number, page numbers, and publication year.
• Use proper formatting: Citation styles typically have specific formatting requirements for different types of sources. Make sure to follow the proper formatting for each citation.
• Order citations alphabetically: If listing multiple sources, they should be listed alphabetically by the author’s last name.
• Be consistent: Use the same citation style throughout the entire paper or project.
• Check for accuracy: Double-check all citations to ensure accuracy, including correct spelling of author names and publication information.
• Use reputable sources: When selecting sources to cite, choose reputable and authoritative sources. Avoid sources that are biased or unreliable.
• Include all sources: Make sure to include all sources used in the research, including those that were not directly quoted but still informed the work.
• Use online tools : There are online tools available (e.g., citation generators) that can help with formatting and organizing references.

Purpose of References in Research

References in research serve several purposes:

• To give credit to the original authors or sources of information used in the research. It is important to acknowledge the work of others and avoid plagiarism.
• To provide evidence for the claims made in the research. References can support the arguments, hypotheses, or conclusions presented in the research by citing relevant studies, data, or theories.
• To allow readers to find and verify the sources used in the research. References provide the necessary information for readers to locate and access the sources cited in the research, which allows them to evaluate the quality and reliability of the information presented.
• To situate the research within the broader context of the field. References can show how the research builds on or contributes to the existing body of knowledge, and can help readers to identify gaps in the literature that the research seeks to address.

Importance of References in Research

References play an important role in research for several reasons:

• Credibility : By citing authoritative sources, references lend credibility to the research and its claims. They provide evidence that the research is based on a sound foundation of knowledge and has been carefully researched.
• Avoidance of Plagiarism : References help researchers avoid plagiarism by giving credit to the original authors or sources of information. This is important for ethical reasons and also to avoid legal repercussions.
• Reproducibility : References allow others to reproduce the research by providing detailed information on the sources used. This is important for verification of the research and for others to build on the work.
• Context : References provide context for the research by situating it within the broader body of knowledge in the field. They help researchers to understand where their work fits in and how it builds on or contributes to existing knowledge.
• Evaluation : References provide a means for others to evaluate the research by allowing them to assess the quality and reliability of the sources used.

Advantages of References in Research

There are several advantages of including references in research:

• Acknowledgment of Sources: Including references gives credit to the authors or sources of information used in the research. This is important to acknowledge the original work and avoid plagiarism.
• Evidence and Support : References can provide evidence to support the arguments, hypotheses, or conclusions presented in the research. This can add credibility and strength to the research.
• Reproducibility : References provide the necessary information for others to reproduce the research. This is important for the verification of the research and for others to build on the work.
• Context : References can help to situate the research within the broader body of knowledge in the field. This helps researchers to understand where their work fits in and how it builds on or contributes to existing knowledge.
• Evaluation : Including references allows others to evaluate the research by providing a means to assess the quality and reliability of the sources used.
• Ongoing Conversation: References allow researchers to engage in ongoing conversations and debates within their fields. They can show how the research builds on or contributes to the existing body of knowledge.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Data Collection – Methods Types and Examples

Delimitations in Research – Types, Examples and...

Research Process – Steps, Examples and Tips

Research Design – Types, Methods and Examples

Institutional Review Board – Application Sample...

Evaluating Research – Process, Examples and...

The CSE Manual for Authors, Editors, and Publishers

• MANUSCRIPT PREPARATION
• MANUSCRIPT AND PROOF MARKUP
• SAMPLE CORRESPONDENCE
• EDITORIAL OFFICE PRACTICES (PDF)
• PROMOTING INTEGRITY IN SCIENTIFIC JOURNAL PUBLICATIONS
• SCIENTIFIC STYLE AND FORMAT CITATION QUICK GUIDE

Scientific Style and Format Citation Quick Guide

Scientific Style and Format presents three systems for referring to references (also known as citations) within the text of a journal article, book, or other scientific publication: 1) citation–sequence; 2) name–year; and 3) citation–name. These abbreviated references are called in-text references. They refer to a list of references at the end of the document.

The system of in-text references that you use will determine the order of references at the end of your document. These end references have essentially the same format in all three systems, except for the placement of the date of publication in the name–year system.

Though Scientific Style and Format now uses citation–sequence for its own references, each system is widely used in scientific publishing. Consult your publisher to determine which system you will need to follow.

Click on the tabs below for more information and to see some common examples of materials cited in each style, including examples of electronic sources. For numerous specific examples, see Chapter 29 of the 8th edition of Scientific Style and Format .

Citation–Sequence and Citation–Name

The following examples illustrate the citation–sequence and citation–name systems. The two systems are identical except for the order of references. In both systems, numbers within the text refer to the end references.

In citation–sequence, the end references are listed in the sequence in which they first appear within the text. For example, if a reference by Smith is the first one mentioned in the text, then the complete reference to the Smith work will be number 1 in the end references. The same number is used for subsequent in-text references to the same document.

In citation–name, the end references are listed alphabetically by author. Multiple works by the same author are listed alphabetically by title. The references are numbered in that sequence, such that a work authored by Adam is number 1, Brown is number 2, and so on. Numbers assigned to the end references are used for the in-text references regardless of the sequence in which they appear in the text of the work. For example, if a work by Zielinski is number 56 in the reference list, each in-text reference to Zielinski will be number 56 also.

List authors in the order in which they appear in the original text, followed by a period. Periods also follow article and journal title and volume or issue information. Separate the date from volume and issue by a semicolon. The location (usually the page range for the article) is preceded by a colon.

Author(s). Article title. Journal title. Date;volume(issue):location.

Journal titles are generally abbreviated according to the List of Title Word Abbreviations maintained by the ISSN International Centre. See Appendix 29.1 in Scientific Style and Format for more information.

For articles with more than 1 author, names are separated by a comma.

Smart N, Fang ZY, Marwick TH. A practical guide to exercise training for heart failure patients. J Card Fail. 2003;9(1):49–58.

For articles with more than 10 authors, list the first 10 followed by “et al.”

Pizzi C, Caraglia M, Cianciulli M, Fabbrocini A, Libroia A, Matano E, Contegiacomo A, Del Prete S, Abbruzzese A, Martignetti A, et al. Low-dose recombinant IL-2 induces psychological changes: monitoring by Minnesota Multiphasic Personality Inventory (MMPI). Anticancer Res. 2002;22(2A):727–732.

Volume with no issue or other subdivision

Laskowski DA. Physical and chemical properties of pyrethroids. Rev Environ Contam Toxicol. 2002;174:49–170.

Volume with issue and supplement

Gardos G, Cole JO, Haskell D, Marby D, Paine SS, Moore P. The natural history of tardive dyskinesia. J Clin Pharmacol. 1988;8(4 Suppl):31S–37S

Volume with supplement but no issue

Heemskerk J, Tobin AJ, Ravina B. From chemical to drug: neurodegeneration drug screening and the ethics of clinical trials. Nat Neurosci. 2002;5 Suppl:1027–1029.

Multiple issue numbers

Ramstrom O, Bunyapaiboonsri T, Lohmann S, Lehn JM. Chemical biology of dynamic combinatorial libraries. Biochim Biophys Acta. 2002;1572(2–3):178–186.

Issue with no volume

Sabatier R. Reorienting health and social services. AIDS STD Health Promot Exch. 1995;(4):1–3.

Separate information about author(s), title, edition, and publication by periods. The basic format is as follows:

Author(s). Title. Edition. Place of publication: publisher; date. Extent. Notes.

Extent can include information about pagination or number of volumes and is considered optional. Notes can include information of interest to the reader, such as language of publication other than English; such notes are optional.

Essential notes provide information about location, such as a URL for online works. See Chapter 29 for more information.

For books with more than 1 author, names are separated by a comma.

Ferrozzi F, Garlaschi G, Bova D. CT of metastases. New York (NY): Springer; 2000.

For books with more than 10 authors, list the first 10 followed by “et al.”

Wenger NK, Sivarajan Froelicher E, Smith LK, Ades PA, Berra K, Blumenthal JA, Certo CME, Dattilo AM, Davis D, DeBusk RF, et al. Cardiac rehabilitation. Rockville (MD): Agency for Health Care Policy and Research (US); 1995.

Organization as author

Advanced Life Support Group. Acute medical emergencies: the practical approach. London (England): BMJ Books; 2001.

Author(s) plus editor(s) or translator(s)

Klarsfeld A, Revah F. The biology of death: origins of mortality. Brady L, translator. Ithaca (NY): Cornell University Press; 2003.

Luzikov VN. Mitochondrial biogenesis and breakdown. Galkin AV, translator; Roodyn DB, editor. New York (NY): Consultants Bureau; 1985.

Chapter or other part of a book, same author(s)

Gawande A. The checklist manifesto: how to get things right. New York (NY): Metropolitan Books; 2010. Chapter 3, The end of the master builder; p. 48–71.

Chapter or other part of a book, different authors

Rapley R. Recombinant DNA and genetic analysis. In: Wilson K, Walker J, editors. Principles and techniques of biochemistry and molecular biology. 7th ed. New York (NY): Cambridge University Press; 2010. p. 195–262.

Multivolume work as a whole

Alkire LG, editor. Periodical title abbreviations. 16th ed. Detroit (MI): Thompson Gale; 2006. 2 vol. Vol. 1, By abbreviation; vol. 2, By title.

Dissertations and Theses

Lutz M. 1903: American nervousness and the economy of cultural change [dissertation]. [Stanford (CA)]: Stanford University; 1989.

Blanco EE, Meade JC, Richards WD, inventors; Ophthalmic Ventures, assignee. Surgical stapling system. United States patent US 4,969,591. 1990 Nov 13.

Weiss R. Study shows problems in cloning people: researchers find replicating primates will be harder than other mammals. Washington Post (Home Ed.). 2003 Apr 11;Sect. A:12 (col. 1).

Indicate a copyright date with a lowercase “c”.

Johnson D, editor. Surgical techniques in orthopaedics: anterior cruciate ligament reconstruction [DVD]. Rosemont (IL): American Academy of Orthopaedic Surgeons; c2002. 1 DVD.

Websites and Other Online Formats

References to websites and other online formats follow the same general principles as for printed references, with the addition of a date of update/revision (if available) along with an access date and a URL.

Title of Homepage. Edition. Place of publication: publisher; date of publication [date updated; date accessed]. Notes.

If no date of publication can be determined, use a copyright date (if available), preceded by “c”. Include the URL in the notes.

APSnet: plant pathology. St Paul (MN): American Phytopathological Association; c1994–2005 [accessed 2005 Jun 20]. http://www.apsnet.org/.

Online journal article

Author(s) of article. Title of article. Title of journal (edition). Date of publication [date updated; date accessed];volume(issue):location. Notes.

A DOI (Digital Object Identifier) may be included in the notes in addition to a URL, if available:

Savage E, Ramsay M, White J, Beard S, Lawson H, Hunjan R, Brown D. Mumps outbreaks across England and Wales in 2004: observational study. BMJ. 2005 [accessed 2005 May 31];330(7500):1119–1120. http://bmj.bmjjournals.com/cgi/reprint/330/7500/1119. doi:10.1136/bmj.330.7500.1119.

Author(s). Title of book. Edition. Place of publication: publisher; date of publication [date updated; date accessed]. Notes.

Brogden KA, Guthmille JM, editors. Polymicrobial diseases. Washington (DC): ASM Press; 2002 [accessed February 28, 2014]. http://www.ncbi.nlm.nih.gov/books/NBK2475/.

Author’s name. Title of post [descriptive word]. Title of blog. Date of publication. [accessed date]. URL.

Fogarty M. Formatting titles on Twitter and Facebook [blog]. Grammar Girl: Quick and Dirty Tips for Better Writing. 2012 Aug 14. [accessed 2012 Oct 19]. http://grammar.quickanddirtytips.com/formatting-titles-on-twitter-and-facebook.aspx.

Forthcoming or Unpublished Material

Not all forthcoming or unpublished sources are suitable for inclusion in reference lists. Check with your publisher if in doubt.

Forthcoming journal article or book

Journal article:

Farley T, Galves A, Dickinson LM, Perez MJ. Stress, coping, and health: a comparison of Mexican immigrants, Mexican-Americans, and non-Hispanic whites. J Immigr Health. Forthcoming 2005 Jul.

Goldstein DS. Adrenaline and the inner world: an introduction to scientific integrative medicine. Baltimore (MD): Johns Hopkins University Press. Forthcoming 2006.

Paper or poster presented at meeting

Unpublished presentations are cited as follows:

Antani S, Long LR, Thoma GR, Lee DJ. Anatomical shape representation in spine x-ray images. Paper presented at: VIIP 2003. Proceedings of the 3rd IASTED International Conference on Visualization, Imaging and Image Processing; 2003 Sep 8–10; Benalmadena, Spain.

Charles L, Gordner R. Analysis of MedlinePlus en Español customer service requests. Poster session presented at: Futuro magnifico! Celebrating our diversity. MLA ’05: Medical Library Association Annual Meeting; 2005 May 14–19; San Antonio, TX.

References to published presentations are cited much like contributions to books, with the addition of information about the date and place of the conference. See Chapter 29 for more information.

Personal communication

References to personal communication are placed in running text rather than as formal end references.

Permission is usually required and should be acknowledged in an “Acknowledgment” or “Notes” section at the end of the document.

. . . and most of these meningiomas proved to be inoperable (2003 letter from RS Grant to me; unreferenced, see “Notes”) while a few were not.

Name–Year

The following examples illustrate the name–year system. In this system (sometimes called the Harvard system), in-text references consist of the surname of the author or authors and the year of publication of the document. End references are unnumbered and appear in alphabetical order by author and year of publication, with multiple works by the same author listed in chronological order.

Each example of an end reference is accompanied here by an example of a corresponding in-text reference. For more details and many more examples, see Chapter 29 of Scientific Style and Format .

For the end reference, list authors in the order in which they appear in the original text. The year of publication follows the author list. Use periods to separate each element, including author(s), date of publication, article and journal title, and volume or issue information. Location (usually the page range for the article) is preceded by a colon.

Author(s). Date. Article title. Journal title. Volume(issue):location.

For the in-text reference, use parentheses and list author(s) by surname followed by year of publication.

(Author(s) Year)

For articles with 2 authors, names are separated by a comma in the end reference but by “and” in the in-text reference.

Mazan MR, Hoffman AM. 2001. Effects of aerosolized albuterol on physiologic responses to exercise in standardbreds. Am J Vet Res. 62(11):1812–1817.

(Mazan and Hoffman 2001)

For articles with 3 to 10 authors, list all authors in the end reference; in the in-text reference, list only the first, followed by “et al.”

Smart N, Fang ZY, Marwick TH. 2003. A practical guide to exercise training for heart failure patients. J Card Fail. 9(1):49–58.

(Smart et al. 2003)

For articles with more than 10 authors, list the first 10 in the end reference, followed by “et al.”

Pizzi C, Caraglia M, Cianciulli M, Fabbrocini A, Libroia A, Matano E, Contegiacomo A, Del Prete S, Abbruzzese A, Martignetti A, et al. 2002. Low-dose recombinant IL-2 induces psychological changes: monitoring by Minnesota Multiphasic Personality Inventory (MMPI). Anticancer Res. 22(2A):727–732.

(Pizzi et al. 2002)

Laskowski DA. 2002. Physical and chemical properties of pyrethroids. Rev Environ Contam Toxicol. 174:49–170.

(Laskowski 2002)

Gardos G, Cole JO, Haskell D, Marby D, Paine SS, Moore P. 1988. The natural history of tardive dyskinesia. J Clin Pharmacol. 8(4 Suppl):31S–37S.

(Gardos et al. 1988)

Heemskerk J, Tobin AJ, Ravina B. 2002. From chemical to drug: neurodegeneration drug screening and the ethics of clinical trials. Nat Neurosci. 5 Suppl:1027–1029.

(Heemskerk et al. 2002)

Ramstrom O, Bunyapaiboonsri T, Lohmann S, Lehn JM. 2002. Chemical biology of dynamic combinatorial libraries. Biochim Biophys Acta. 1572(2–3):178–186.

(Ramstrom et al. 2002)

Sabatier R. 1995. Reorienting health and social services. AIDS STD Health Promot Exch. (4):1–3.

(Sabatier 1995)

In the end reference, separate information about author(s), date, title, edition, and publication by periods. The basic format is as follows:

Author(s). Date. Title. Edition. Place of publication: publisher. Extent. Notes.

Extent can include information about pagination or number of volumes and is considered optional. Notes can include information of interest to the reader, such as language of publication other than English; such notes are optional. Essential notes provide information about location, such as a URL for online works. See Chapter 29 for more information.

For books with 2 authors, names are separated by a comma in the end reference but by “and” in the in-text reference.

Leboffe MJ, Pierce BE. 2010. Microbiology: laboratory theory and application. Englewood (CO): Morton Publishing Company.

(Leboffe and Pierce 2010)

For books with 3 to 10 authors, list all authors in the end reference; in the in-text reference, list only the first, followed by “et al.”

Ferrozzi F, Garlaschi G, Bova D. 2000. CT of metastases. New York (NY): Springer.

(Ferrozzi et al. 2000)

For books with more than 10 authors, list the first 10 in the end reference, followed by “et al.”

Wenger NK, Sivarajan Froelicher E, Smith LK, Ades PA, Berra K, Blumenthal JA, Certo CME, Dattilo AM, Davis D, DeBusk RF, et al. 1995. Cardiac rehabilitation. Rockville (MD): Agency for Health Care Policy and Research (US).

(Wenger et al. 1995)

[ALSG] Advanced Life Support Group. 2001. Acute medical emergencies: the practical approach. London (England): BMJ Books.

(ALSG 2001)

Klarsfeld A, Revah F. 2003. The biology of death: origins of mortality. Brady L, translator. Ithaca (NY): Cornell University Press.

Luzikov VN. 1985. Mitochondrial biogenesis and breakdown. Galkin AV, translator; Roodyn DB, editor. New York (NY): Consultants Bureau.

(Klarsfeld and Revah 2003)

(Luzikov 1985)

Gawande A. 2010. The checklist manifesto: how to get things right. New York (NY): Metropolitan Books. Chapter 3, The end of the master builder; p. 48–71.

(Gawande 2010)

Rapley R. 2010. Recombinant DNA and genetic analysis. In: Wilson K, Walker J, editors. Principles and techniques of biochemistry and molecular biology. 7th ed. New York (NY): Cambridge University Press. p. 195–262.

(Rapley 2010)

Alkire LG, editor. 2006. Periodical title abbreviations. 16th ed. Detroit (MI): Thompson Gale. 2 vol. Vol. 1, By abbreviation; vol. 2, By title.

(Alkire 2006)

Lutz M. 1989. 1903: American nervousness and the economy of cultural change [dissertation]. [Stanford (CA)]: Stanford University.

(Lutz 1989)

Blanco EE, Meade JC, Richards WD, inventors; Ophthalmic Ventures, assignee. 1990 Nov 13. Surgical stapling system. United States patent US 4,969,591.

(Blanco et al. 1990)

Weiss R. 2003 Apr 11. Study shows problems in cloning people: researchers find replicating primates will be harder than other mammals. Washington Post (Home Ed.). Sect. A:12 (col. 1).

(Weiss 2003)

Johnson D, editor. c2002. Surgical techniques in orthopaedics: anterior cruciate ligament reconstruction [DVD]. Rosemont (IL): American Academy of Orthopaedic Surgeons. 1 DVD.

(Johnson c2002)

Format for end reference:

Title of Homepage. Date of publication. Edition. Place of publication: publisher; [date updated; date accessed]. Notes.

APSnet: plant pathology online. c1994–2005. St Paul (MN): American Phytopathological Association; [accessed 2005 Jun 20]. http://www.apsnet.org/.

For the in-text reference, include only the first word or two of the title (enough to distinguish it from other titles in the reference list), followed by an ellipsis.

(APSnet . . . c1994–2005)

Author(s) of article. Date of publication. Title of article. Title of journal (edition). [date updated; date accessed];Volume(issue):location. Notes.

Savage E, Ramsay M, White J, Beard S, Lawson H, Hunjan R, Brown D. 2005. Mumps outbreaks across England and Wales in 2004: observational study. BMJ. [accessed 2005 May 31];330(7500):1119–1120. http://bmj.bmjjournals.com/cgi/reprint/330/7500/1119. doi:10.1136/bmj.330.7500.1119.

(Savage et al. 2005)

Author(s). Date of publication. Title of book. Edition. Place of publication: publisher; [date updated; date accessed]. Notes.

Brogden KA, Guthmille JM, editors. 2002. Polymicrobial diseases. Washington (DC): ASM Press; [accessed February 28, 2014]. http://www.ncbi.nlm.nih.gov/books/NBK2475/.

(Brogden and Guthmille 2002)

Author’s name. Date of publication. Title of post [descriptive word]. Title of blog. [accessed date]. URL.

Fogarty M. 2012 Aug 14. Formatting titles on Twitter and Facebook [blog]. Grammar Girl: Quick and Dirty Tips for Better Writing. [accessed 2012 Oct 19]. http://grammar.quickanddirtytips.com/formatting-titles-on-twitter-and-facebook.aspx.

(Fogarty 2012)

Farley T, Galves A, Dickinson LM, Perez MJ. Forthcoming 2005 Jul. Stress, coping, and health: a comparison of Mexican immigrants, Mexican-Americans, and non-Hispanic whites. J Immigr Health.

(Farley et al. 2005)

Goldstein DS. Forthcoming 2006. Adrenaline and the inner world: an introduction to scientific integrative medicine. Baltimore (MD): Johns Hopkins University Press.

(Goldstein 2006)

Antani S, Long LR, Thoma GR, Lee DJ. 2003. Anatomical shape representation in spine x-ray images. Paper presented at: VIIP 2003. Proceedings of the 3rd IASTED International Conference on Visualization, Imaging and Image Processing; Benalmadena, Spain.

Charles L, Gordner R. 2005. Analysis of MedlinePlus en Español customer service requests. Poster session presented at: Futuro magnifico! Celebrating our diversity. MLA ’05: Medical Library Association Annual Meeting; San Antonio, TX.

(Atani et al. 2003)

(Charles and Gordner 2005)

References to personal communication are placed in running text rather than as formal end references. Permission is usually required and should be acknowledged in an “Acknowledgment” or “Notes” section at the end of the document.

Scientific Style and Format, 8th Edition text © 2014 by the Council of Science Editors. Scientific Style and Format Online © 2014 by the Council of Science Editors.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base
• Citing sources
• How to Cite a Journal Article | APA, MLA, & Chicago Examples

How to Cite a Journal Article | APA, MLA, & Chicago Examples

Published on March 9, 2021 by Jack Caulfield . Revised on January 17, 2024.

To cite an article from an academic journal, you need an in-text citation and a corresponding reference listing the name(s) of the author(s), the publication date, the article title and journal name, the volume and issue numbers, the page range, and the URL or DOI .

Different citation styles present this information differently. The main citation styles are APA , MLA , and Chicago style .

You can use the interactive example generator to explore the format for APA and MLA journal article citations.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

Table of contents

Citing an article in apa style, citing an article in mla style, citing an article in chicago style, frequently asked questions about citations.

In an APA Style journal article reference , the article title is in plain text and sentence case, while the journal name appears in italics, in title case.

The in-text citation lists up to two authors; for three or more, use “ et al. ”

When citing a journal article in print or from a database, don’t include a URL. You can still include the DOI if available.

You can also cite a journal article using our free APA Citation Generator . Search by title or DOI to automatically generate a correct citation.

Generate accurate APA citations with Scribbr

Here's why students love scribbr's proofreading services.

Discover proofreading & editing

In an MLA Works Cited entry for a journal article , the article title appears in quotation marks, the name of the journal in italics—both in title case.

List up to two authors in both the in-text citation and the Works Cited entry. For three or more, use “et al.”

A DOI is always included when available; a URL appears if no DOI is available but the article was accessed online . If you accessed the article in print and no DOI is available, you can omit this part.

You can also use our free MLA Citation Generator to create your journal article citations.

Generate accurate MLA citations with Scribbr

In Chicago notes and bibliography style, you include a bibliography entry for each source, and cite them in the text using footnotes .

A bibliography entry for a journal article lists the title of the article in quotation marks and the journal name in italics—both in title case. List up to 10 authors in full; use “et al.” for 11 or more.

In the footnote, use “et al.” for four or more authors.

A DOI or URL (preferably a DOI) is included for articles consulted online; for articles consulted in print, omit this part.

Chicago also offers an alternative author-date style of citation. Examples of how to cite journal articles in this style can be found here .

The elements included in journal article citations across APA , MLA , and Chicago style are the name(s) of the author(s), the title of the article, the year of publication, the name of the journal, the volume and issue numbers, the page range of the article, and, when accessed online, the DOI or URL.

In MLA and Chicago style, you also include the specific month or season of publication alongside the year, when this information is available.

The DOI is usually clearly visible when you open a journal article on an academic database. It is often listed near the publication date, and includes “doi.org” or “DOI:”. If the database has a “cite this article” button, this should also produce a citation with the DOI included.

If you can’t find the DOI, you can search on Crossref using information like the author, the article title, and the journal name.

The abbreviation “ et al. ” (Latin for “and others”) is used to shorten citations of sources with multiple authors.

“Et al.” is used in APA in-text citations of sources with 3+ authors, e.g. (Smith et al., 2019). It is not used in APA reference entries .

Use “et al.” for 3+ authors in MLA in-text citations and Works Cited entries.

Use “et al.” for 4+ authors in a Chicago in-text citation , and for 10+ authors in a Chicago bibliography entry.

Check if your university or course guidelines specify which citation style to use. If the choice is left up to you, consider which style is most commonly used in your field.

• APA Style is the most popular citation style, widely used in the social and behavioral sciences.
• MLA style is the second most popular, used mainly in the humanities.
• Chicago notes and bibliography style is also popular in the humanities, especially history.
• Chicago author-date style tends to be used in the sciences.

Other more specialized styles exist for certain fields, such as Bluebook and OSCOLA for law.

The most important thing is to choose one style and use it consistently throughout your text.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Caulfield, J. (2024, January 17). How to Cite a Journal Article | APA, MLA, & Chicago Examples. Scribbr. Retrieved April 3, 2024, from https://www.scribbr.com/citing-sources/cite-a-journal-article/

Is this article helpful?

Jack Caulfield

Other students also liked, how to cite a book | apa, mla, & chicago examples, how to cite a website | mla, apa & chicago examples, how to cite a wikipedia article | apa, mla & chicago, what is your plagiarism score.

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

5 Accessing Scientific Literature and Referencing

An essential skill for all scientists to master is the ability to access relevant and reliable scientific information from a variety of sources.

You will need access to scientific literature for a variety of reasons:

• designing an experiment
• writing an article or essay
• designing a poster.

All of these tasks involved sourcing reliable, authoritative literature, and you’ll need to know how to reference it.

This chapter will provide student scientists with assistance in navigating the many avenues for locating scientific literature and referencing it, including using the reference management software EndNote.

5.1 Types of scientific literature

The two main types of scientific literature are original investigations and literature reviews.

Original investigations ( Figure 5.1 ) are published accounts of new studies undertaken on a particular topic. They will generally step the reader through the stages of the study:

• introduction

Published literature reviews ( Figure 5.2 ) present a synthesis and evaluation of the existing literature on a particular topic with the aim of gaining a new, deeper understanding of the topic. The review article will be structured around themes rather than stages of the scientific method.

5.2 Accessing scientific literature

You can locate scientific literature via Google Scholar , online databases such as PubMed, and the University of Southern Queensland library website when you are looking for a specific article or searching for literature on a specific topic.

Google Scholar

Google Scholar provides a simple way to perform a broad search for scholarly literature. From one place, you can search across many disciplines and sources:

• court opinions.

Academic publishers, professional societies, online repositories, universities and other web sites all publish these types of literature. Google Scholar helps you find relevant work across the world of scholarly research.

Box 5.1: Search tips

• Set your preferences to retrieve your university library resources: For example, select Settings > Library Link > add University of Southern Queensland Library > Save
• Use the asterisk * (e.g. child* will find child, children, childhood, childless)
• Use the asterisk as a placeholder to find a word within words e.g. “acquired * injury” finds acquired brain injury
• Use quotes to search by phrase (e.g. “type 2 diabetes” or “social media”)

Try this search in Google Scholar: “patient information” AND “back pain”

Extra help with using Google scholar is at this webpage. 

Databases, what are they and how do I use them to find information?

Databases are another way to find quality academic and scholarly information. The USQ library subscribes to many databases that are relevant to your studies in human physiology, such as:

• Web of Science
• ClinicalKey
• ScienceDirect

PubMed is a database that comprises of more than 26 million citations for biomedical literature from MEDLINE, life science journals and online books. Citations may include links to full-text content from PubMed Central and publisher websites.

Journals contain scholarly articles written by experts in specific disciplines. This tutorial explains what scholarly journals are and how to access them from the USQ library.

5.3 Determining if an article is scholarly or peer reviewed

It can be hard to work out if a journal is scholarly or peer reviewed. There is a lot of information online that looks like proper science, but isn’t! These tips can help you determine if you are accessing reliable information.

If searching in a library database:

• Check to see if there is a box on the database search page that allows you to limit your search results to refereed or peer-reviewed journal articles.

If you already have a journal article or title, use these option to check if it is scholarly or peer reviewed: 

• Look at the article itself for a header or similar that indicates refereed or reviewed.
• Look at the table of contents of the journal. Often items are grouped under a heading like ‘reviewed articles’.
• Check the journal’s website to see if a statement is made about the content being peer reviewed or refereed. However, be aware that not all the contents of a refereed journal will be refereed (e.g. books reviews, practice, commentaries, editorials are not peer reviewed).
• Look for the Ulrichsweb database in your library catalogue. If you have access, use the ‘Quick Search’ drop down and select ‘Title (keyword)’ and type in the journal title. Next to journal titles that include at least some refereed content is the image of a black and white striped  ‘referee’s shirt’.  You can also click on the journal title and you will see ‘Refereed – yes or no’.

5.4 Library website resources to assist with searching for authoritative information

Your university library will provide tutorials and resources to help you search for authoritative information.

5.5 Referencing

Anyone who reads your work will need to know where you got your information from if you didn’t generate it yourself (e.g. the results of your experiment). The reference section provides a list of the references that you cited in the body of your work, whether it be a literature review, original investigation research article or essay.

It is important to accurately cite references in research papers to acknowledge your sources and ensure credit is appropriately given to authors of work you have referred to. An accurate and comprehensive reference list also shows your readers that you are well-read in your topic area and are aware of the key papers that provide the context to your research.

It is important to keep track of your resources and to reference them consistently in the format required by the publication in which your work will appear. Most scientists will use referencing software to store details of all of the journal articles (and other sources) they use while writing their review article. This software also automates the process of adding in-text references and creating a reference list.

In-text citations indicate where (within your sentences) you have used the ideas of other scientists. The in-text citations will either be provided as a number, or as the name of the author and date of publication. A reference list is a list of all the sources that you have used as in-text references in your scientific paper that enables the reader of your work to locate and verify the sources you have use.

Here are two basic formats for a reference list:

• an alphabetical listing by first author’s last name (author–date system)
• a numerical listing that reflects the order of the citations in the body of the paper (number format).

The format will depend on the journal of publication, as each journal has their own specific referencing format.

A bibliography tends to use the author–date format, as the works might not be cited in the text.

Author–date system

Author-date reference styles indicate in-text citations by placing the author’s surname and the date of publication in brackets, and the reference list is in alphabetical order by author’s surname. Harvard and APA are examples of author date styles.

The associated images show a section of the discussion, and a section of the reference list of a research article (Bain et al., 2014) that has used an author-date system.

Your university library will provide guidance and examples of the referencing styles you are expected to use. Ask your tutor which style you are to use for your assignment.

This short video shows you the basics of Harvard referencing. Please note there are different interpretations of the style and use the resources provided by your university library when composing your own in-text citations and reference lists.

Click the drop down below to review the terms learned from this chapter.

Bain, A.R., Morrison, S.A., & Ainslie, P.N. (2014). Cerebral oxygenation and hyperthermia. Frontiers in Physiology, 5 , 92.

How To Do Science Copyright © 2022 by University of Southern Queensland is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

Loading metrics

Open Access

Ten simple rules for responsible referencing

* E-mail: [email protected]

Affiliation Maastricht University, Care and Public Health Research Institute (CAPHRI), Department of Health, Ethics & Society, Maastricht, the Netherlands

• Bart Penders

Published: April 12, 2018

• https://doi.org/10.1371/journal.pcbi.1006036
• Reader Comments

Citation: Penders B (2018) Ten simple rules for responsible referencing. PLoS Comput Biol 14(4): e1006036. https://doi.org/10.1371/journal.pcbi.1006036

Editor: Scott Markel, Dassault Systemes BIOVIA, UNITED STATES

Copyright: © 2018 Bart Penders. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Funding: The work that lead to this publication was, in part, supported by the ZonMW programme Fostering Responsible Research Practices, grant no. 45001005. The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

We researchers aim to read and write publications containing high-quality prose, exceptional data, arguments, and conclusions, embedded firmly in existing literature while making abundantly clear what we are adding to it. Through the inclusion of references, we demonstrate the foundation upon which our studies rest as well as how they are different from previous work. That difference can include literature we dispute or disprove, arguments or claims we expand, and new ideas, suggestions, and hypotheses we base upon published work. This leads to the question of how to decide which study or author to cite, and in what way.

Writing manuscripts requires, among so much more, decisions on which previous studies to include and exclude, as well as decisions on how exactly that inclusion takes place. A well-referenced manuscript places the authors’ argument in the proper knowledge context and thereby can support its novelty, its value, and its visibility. Citations link one study to others, creating a web of knowledge that carries meaning and allows other researchers to identify work as relevant in general and relevant to them in particular.

On the one hand, citation practices create value by tying together relevant scientific contributions, regardless of whether they are large or small. In the process, they confer or withhold credit, contributing to the relative status of published work in the literature. On the other hand, citation practices exist in the context of current regimes of evaluating science. While it may go unnoticed in daily writing practices, the act of including a single reference in a study is thus subject to value-based criteria internal to science (e.g., content, relevance, credit) and external to science (e.g., accountability, performance).

Accordingly, referencing is not a neutral act. Citations are a form of scientific currency, actively conferring or denying value. Citing certain sources—and especially citing them often—legitimises ideas, solidifies theories, and establishes claims as facts. References also create transparency by allowing others to retrace your steps. Referencing is thus a moral issue, an issue upon which multiple values in science converge. Citing competitors adds to their profiles, citing papers from a specific journal adds to its impact factor, citing supervisors or lab mates helps build your own profile, and citing the right papers helps establish your familiarity with the field. All of these translate into pressures on scientists to cite specific sources, from peers, editors, and others. Fong and Wilhite demonstrate the abundance of so-called coercive citation practices [ 1 ]. Also, citation-based metrics have proliferated as proxies for quality and impact over the years [ 2 – 4 ], only to be currently subjected to significant and highly relevant critique [ 5 – 8 ]. To cite well, or to reference responsibly, is thus a matter of concern to all scientists.

Here, I offer 10 simple rules for responsible referencing. Scientists as authors produce references, and as readers and reviewers, they assess and evaluate references. Through this symmetrical relationship to literature that all scientists share, they take responsibility for tying together all knowledge it contains. Producing and evaluating references are, however, distinct processes, warranting different responsibilities. Respecting this dual relationship researchers have with literature, the first six rules primarily refer to producing a citation and the responsibilities this entails. The second set of four rules refers to evaluating citations and the meaning they have or acquire once they have become part of a text.

Rule 1: Include relevant citations

All scholarly writing requires a demonstration of the relevance of the questions asked, a display of the methods used, a rationale for the use of materials, and a discussion of issues relevant to the content of the publication. All of these are done, at least in large part, by including citations to relevant previous work. Omitting such references can wrongfully suggest that your own publication is the origin of an idea, a question, a method, or a critique, thereby illegitimately appropriating them. Citations identify where ideas have come from, and consulting the cited works allows readers of your text to study them more closely, as well as to evaluate whether your use of them is appropriate.

A single exception exists when facts, findings, or methods have become part of scientific or scholarly canon. There is no need to include a citation on the claim that DNA is built out of four bases, nor do you have to cite Kjell Kleppe or Kary Mullis every time you use PCR (neither do I right now). However, the decision as to when something truly becomes part of canon can be quite difficult and will include periods of adjustment (with irregular citation) and negotiation (on whether to cite or not).

Rule 2: Read the publications you cite

Citation is not an administrative task. First, a single paper can be cited for multiple reasons, ranging from reported data to methods, and can be cited both positively and negatively in the literature. The only way to identify whether its content is relevant as support for your claim is to read it in full.

Second, the collection of citations included to support your work and argument is one of the elements from which your work draws credibility. The same goes for the citations you include to criticise, dispute, or disprove. As a consequence, a chain is only as strong as its weakest link. The quality of the publication you trust and upon which you confer authority codetermines the quality and credibility of your work. Citation rates, especially on the journal level, do not correspond well to research quality [ 9 ], and they conflate positive and negative citations, not distinguishing authority conferred or authority that is challenged. To cite meaningfully and credibly requires that you consult the content of a publication rather than whether others have cited it, as a criterion for citation.

Rule 3: Cite in accordance with content

If, at some phase in the research, you have decided that a specific study merits citation, the issue of specifically how and where to cite it deserves explicit consideration. Mere inclusion does not suffice. Sources deserve credit for the exact contribution they offer, not their contribution in general. This may mean that you need to cite a single source multiple times throughout your own argument, including explanations or indications why.

A specific way to break Rule 3 is in the form of the so-called ‘Trojan citation’ [ 10 ]. The Trojan citation arises when a publication reporting similar findings to your own is cited in the context of a discussion of a minor issue, ignoring (sometimes deliberately) its key argument or contribution. By focussing on a trivial detail, the Trojan citation obscures the true significance of the cited work. As a consequence, it hides that your work is not as novel as it seems. As a questionable citation practice, a Trojan citation can be used to satisfy reviewers’ or editors’ requests to include a reference to a relevant paper. Alternatively, a Trojan citation may emerge unknowingly when (1) you are unaware of the content of a cited publication (not adhering to Rule 2 creates a very significant risk of being unable to follow Rule 3) or (2) disputes exist in the scientific community or among the authors on the contribution and/or quality of a scientific publication (in which case, Rule 4 will help).

Rule 4: Cite transparently, not neutrally

Citing, even in accordance with content, requires context. This is especially important when it happens as part of the article’s argument. Not all citations are a part of an article’s argument. Citations to data, resources, materials, and established methods require less, if any, context. As part of the argument, however, the mere inclusion of a citation, even when in the right spot, does not convey the value of the reference and, accordingly, the rationale for including it. In a recent editorial, the Nature Genetics editors argued against so-called neutral citation. This citation practice, they argue, appears neutral or procedural yet lacks required displays of context of the cited source or rationale for including [ 11 ]. Rather, citations should mention assessments of value, worth, relevance, or significance in the context of whether findings support or oppose reported data or conclusions.

This flows from the realisation that citations are political, even though that term is rarely used in this context. Researchers can use them to accurately represent, inflate, or deflate contributions, based on (1) whether they are included and (2) whether their contributions are qualified. Context or rationale can be qualified by using the right verbs. The contribution of a specific reference can be inflated or deflated through the absence of or use of the wrong qualifying term (‘the authors suggest’ versus ‘the authors establish’; ‘this excellent study shows’ versus ‘this pilot study shows’). If intentional, it is a form of deception, rewriting the content of scientific canon. If unintentional, it is the result of sloppy writing. Ask yourself why you are citing prior work and which value you are attributing to it, and whether the answers to these questions are accessible to your readers.

Rule 5: Cite yourself when required

In the context of critical discussions of citations and evaluations of citation-based metrics, self-citation has almost become a taboo. It is important to realise, though, that self-citation serves an important function by showing incremental iterative advancement of your work [ 12 ]. As a consequence, your previous work or that of the group in which you are embedded should be cited in accordance with all of the rules above. The amount of acceptable self-citation is very likely to differ between fields; smaller fields (niche fields) are likely to (legitimately) exhibit more.

This does not mean that self-citation is always unproblematic. For instance, excessive self-citation can suggest salami slicing, a publication strategy in which elements of a single study are published separately [ 13 ]. This questionable research practice, in tandem with self-citation, aims to inflate publication and citation metrics.

Rule 6: Prioritise the citations you include

Many journals have restrictions on the number of references authors are allowed to include. The exact number varies per publisher, journal, and article type and can be as low as three (for a correspondence item in Nature ). Even if no reference limit exists, other journals impose a word limit that includes references, effectively also capping the amount of references. Coping with these limits sometimes requires difficult decisions to omit citations you may feel are legitimate or even necessary. In order to deal with this issue and avoid random removal of references, all desired citations require prioritisation. A few rules of thumb, shown in Box 1 , will help decisions on reference priority.

Box 1: Reference prioritisation

‘Ten simple sub-rules for prioritising references’ can help to facilitate prioritisation. In most cases, a subset of the 10 sub-rules will suffice. First, prioritise anew for each publication. Prioritisations cannot (easily) be copied from one study to another. Second, prioritise per section (e.g., introduction, methods, discussion), not across the entire paper. Different sections require different types of support. Third, for the introduction, prioritise reviews, allowing broad context for relevance and aim. Fourth, for the discussion, prioritise empirical papers, allowing detailed accounts of relative contribution. Fifth, prioritise reviewed over un- or prereviewed papers (e.g., editorials, preprints, etc.). Sixth, deprioritise self-citations. Seventh, limit the number of citations to support a specific claim, if necessary, to a single citation. Eighth, move methodological citations to supplementary (online) information. Ninth, in cases of equal relevance, prioritise citation of female first or last authors to help repair gender imbalances in science. Tenth, request the inclusion of additional references with the editors, arguing that you have used all of the previous nine sub-rules.

Rule 7: Evaluate citations as the choices that they are

Research publications are not mere vessels of data or findings. They convey a narrative explaining why questions are worth asking, what their answers may mean, how these answers were reached, why they are to be trusted, and more. They also have a purpose in the sense that they will act as support for other studies to come. Each of the elements of their story is supported by links to other studies, and each of those links is the result of an active choice by the author(s) in the context of the goal they wish to achieve by their inclusion.

At the other end of the narrative, readers assess and evaluate the story constantly, asking whether it could have been told differently. The realisation that narratives can be told differently, supported by other citations to other prior work, does not disqualify them. Both the story and the choice of citations are political choices meant to provide the argument with as much power, credibility, and legitimacy the author(s) can muster. They are tailored to the audience the authors seek to convince: their peers. The choice to include or exclude a reference can only be evaluated in the context of that narrative and the role they play in it. Peritz has provided a classification of citation roles to assist this evaluation [ 14 ].

Rule 8: Evaluate citations in their rhetorical context

Rhetorical strategies serve to convince and persuade. Narratives are but one of the tools that can be used to persuade audiences. Metaphors, numbers, and associations all feature in our research papers as tools to convince our readers. The genre of the scientific article has had centuries to evolve to incorporate many of them, with the goal of convincing readers that the author is right. Bazerman has literally written the book on this [ 15 ] and urges us to consider academic texts and their features as part of social and intellectual endeavours. Citations are a part of the social fabric of science in the sense that through citing specific sources, authors show their allegiance to schools of thought, communities, or, in the context of scientific controversies, which paradigm they consider themselves part of. Other rhetorical uses of citations include explicit citations to notable figures and their work, which can serve as appeals to authority, while long lists of citations can serve as proxies for well-studied subjects.

Consider the following: Authors can describe a field as well-studied and include three references—X, Y, and Z—as support for their claim. Alternatively, they can argue that a field is understudied but that three exceptions exist, i.e., X, Y, and Z. Understanding the value attributed to X, Y, and Z in that particular text requires assessment of the rhetorical strategies of the author(s).

Rule 9: Evaluate citations as framed communication

Authors use words to accomplish things and, in service of those goals, position their work and that of others. They frame prior work in a very specific way, supporting the arguments made. We all do. The positioning of X, Y, and Z either as the norm or as exceptions, as shown in Rule 8, is an example of framing. It is important to recognise such framing and that X, Y, and Z acquire meaning in the text as the result of the frame. There is no frameless communication, as Goffman [ 16 ] demonstrated. All messages and texts contain and require a frame—a structure of definitions and assumptions that help organise coherence, connections, and, ultimately, meaning—or in other words, a perspective on reality.

As a result, a citation is not a neutral line drawn between publications A and B. Rather, the representation of cited article A only acquires meaning in the context of citing in article B. Article A can be framed differently when cited in work B or C. It can be framed as innovative in B or dogmatic in C. Framing usually is not lying or deceiving; it is a normative positioning of evidence in context. Hence, a citation is a careful translation of a source’s relevant elements, which acquire meaning in that context only.

An important consequence of this is that merely counting citations of article A in the literature does not inform us of the value (or many types of value or lack thereof) of article A to the scientific community. This point also appears as the first principle in the Leiden Manifesto, which argues that quantitative metrics can only support qualitative metrics (i.e., reading with an attentive eye for politics, rhetoric, context, and frame—or as adhering to Rules 7–9). The Leiden Manifesto was published by bibliometricians and scholars of research evaluation following the 2014 conference on Science and Technology Indicators in Leiden, the Netherlands. It warns against the abuse of, among other things, citation-based research metrics [ 9 ].

Rule 10: Accept that citation cultures differ across boundaries

Despite critiques of the system, science is organised in such a way that citations continue to act as a currency that is represented as being universal [ 4 ]. However, citation practices are, for the most part, local practices, whether local to laboratories or department or local to disciplines. The average number of citations per paper differs between disciplines, and the way that citations are represented in the text and the value of being cited also differ radically [ 17 ]. What counts as proper citation practice in molecular biology—for instance, the inclusion of multiple references following a statement—is considered unacceptable in research ethics or science policy, in which single references require paragraphs of contextualisation and translation (see Rule 9 ). When reading a paper from an adjacent discipline, respect its different norms and conventions for responsible referencing and proper citation. If you are cited by a scientist from another discipline, assess that act as existing in a (however slightly) different citation culture.

Acknowledgments

I thank Maurice Zeegers and his team, who work on citation analyses, for stimulating me to think about the issue of citation more clearly, deeply, and critically, resulting in the considerations above. I also thank David Shaw for critical comments, moral support, and editorial assistance. As a closing note, as the human being that I am, I too have quite possibly referenced imperfectly in my previous work.

• View Article
• PubMed/NCBI
• Google Scholar
• 3. Garfield E, Merton R. Citation indexing: Its theory and application in science, technology, and humanities: New York: Wiley; 1979.
• 4. Wouters P. The citation culture. Amsterdam: University of Amsterdam; 1999.
• 5. Dahler-Larsen P. Constitutive effects of performance indicator systems. Dilemmas of engagement: Evaluation and the new public management. Emerald Group Publishing Limited; 2007. p. 17–35.
• 15. Bazerman C. Shaping written knowledge: The genre and activity of the experimental article in science. Madison, WI: University of Wisconsin Press; 1988.
• 16. Goffman E. Frame analysis: An essay on the organization of experience.Cambrdige, MA: Harvard University Press; 1974.

• Manuscript Preparation

How to write your references quickly and easily

• 3 minute read
• 229.3K views

Table of Contents

Every scientific paper builds on previous research – even if it’s in a new field, related studies will have preceded and informed it. In peer-reviewed articles, authors must give credit to this previous research, through citations and references. Not only does this show clearly where the current research came from, but it also helps readers understand the content of the paper better.

There is no optimum number of references for an academic article but depending on the subject you could be dealing with more than 100 different papers, conference reports, video articles, medical guidelines or any number of other resources.

That’s a lot of content to manage. Before submitting your manuscript, this needs to be checked, cross-references in the text and the list, organized and formatted.

The exact content and format of the citations and references in your paper will depend on the journal you aim to publish in, so the first step is to check the journal’s Guide for Authors before you submit.

There are two main points to pay attention to – consistency and accuracy. When you go through your manuscript to edit or proofread it, look closely at the citations within the text. Are they all the same? For example, if the journal prefers the citations to be in the format (name, year), make sure they’re all the same: (Smith, 2016).

Your citations must also be accurate and complete. Do they match your references list? Each citation should be included in the list, so cross-checking is important. It’s also common for journals to prefer that most, if not all, of the articles listed in your references be cited within the text – after all, these should be studies that contributed to the knowledge underpinning your work, not just your bedtime reading. So go through them carefully, noting any missing references or citations and filling the gaps.

Each journal has its own requirements when it comes to the content and format of references, as well as where and how you should include them in your submission, so double-check before you hit send!

In general, a reference will include authors’ names and initials, the title of the article, name of the journal, volume and issue, date, page numbers and DOI. On ScienceDirect, articles are linked to their original source (if also published on ScienceDirect) or to their Scopus record, so including the DOI can help link to the correct article.

A spotless reference list

Luckily, compiling and editing the references in your scientific manuscript can be easy – and it no longer has to be manual. Management tools like Mendeley can keep track of all your references, letting you share them with your collaborators. With the Word plugin, it’s possible to select the right citation style for the journal you’re submitting to and the tool will format your references automatically.

Like with any other part of your manuscript, it’s important to make sure your reference list has been checked and edited. Elsevier Author Services Language Editing can help, with professional manuscript editing that will help make sure your references don’t hold you back from publication.

Why it’s best to ask a professional when it comes to translation

• Research Process

How to Choose a Journal to Submit an Article

You may also like.

Make Hook, Line, and Sinker: The Art of Crafting Engaging Introductions

Can Describing Study Limitations Improve the Quality of Your Paper?

A Guide to Crafting Shorter, Impactful Sentences in Academic Writing

6 Steps to Write an Excellent Discussion in Your Manuscript

How to Write Clear and Crisp Civil Engineering Papers? Here are 5 Key Tips to Consider

The Clear Path to An Impactful Paper: ②

The Essentials of Writing to Communicate Research in Medicine

Changing Lines: Sentence Patterns in Academic Writing

Input your search keywords and press Enter.

University Libraries

• Research Guides
• Blackboard Learn
• Interlibrary Loan
• Study Rooms
• University of Arkansas

Science Fair Resources

Citing your sources.

• The Scientific Method
• Books in Our Collection
• Online Encyclopedias
• General Science Journals
• Other Helpful Resources
• Find Articles
• Poster Techniques
• Community Borrower Privileges This link opens in a new window
• Fayetteville Public Library

When you use ideas that are not your own, it is important to credit or cite the author(s) or source, even if you do not quote their idea or words exactly as written. Citing your sources allows your reader to identify the works you have consulted and to understand the scope of your research. There are many different citation styles available. You may be required to use a particular style or you may choose one.

One of the commonly used styles is the APA (American Psychological Association) Style.

APA style stipulates that authors use brief references in the text of a work with full bibliographic details supplied in a Reference List (typically at the end of your document). In text, the reference is very brief and usually consists simply of the author's last name and a date.For example:

...Sheep milk has been proved to contain more nutrients than cow milk (Johnson, 2005).

In a Reference list, the reference contains full bibliographic details written in a format that depends on the type of reference. Examples of formats for some common types of references are listed below. For additional information, visit the University of Arkansas libraries webpage on citing your sources . Another useful web-site on this topic is here.

Author last name, Author First Initial. Author Second Initial. (Publication Year). Title of article. Title of Journal, volume(issue) (if issue numbered), pages.

Bass, M. A., Enochs, W. K., & DiBrezzo, R. (2002). Comparison of two exercise programs on general well-being of college students. Psychological Reports, 91(3), 1195-1201.

Author Last Name, Author First Initial. Author Second Initial. (if there is no author move entry title to first position) (Publication year). Title of article or entry. In Work title. (Vol. number, pp. pages). Place: Publisher.

"Ivory-billed woodpecker." (2002). In The new encyclopædia britannica. (Vol. 5, p. ). 15th ed. Chicago: Encyclopædia Britannica.

Author Last Name, Author First Initial. Author Second Initial. (if there is no author move entry title to first position) (Publication year). Title of article or entry. In Work title. Retrieved from (database name or URL).

Ivory-billed woodpecker. (2006). In Encyclopædia britannica online. Retrieved from http://search.eb.com/eb/article-9043081

Author last name, Author First Initial. Author Second Initial. (Publication Year, Month Day). Title of article. Title of Magazine,volume, pages.

Holloway, M. (2005, August). When extinct isn't. Scientific American, 293, 22-23.

Author last name, Author First Initial. Author Second Initial. (Publication Year, Month Day). Title of article. Title of Magazine. volume, pages. Retrieved from (database name or URL).

Holloway, M. (2005, August). When extinct isn't. Scientific American, 293, 22-23. Retrieved from Academic Search Premier database.

Page Author Last Name, Page Author First Initial. Page Author Second Initial. Page title [nature of work - web site, blog, forum posting, etc.]. (Publication Year). Retrieved from (URL)

Sabo, G., et al. Rock art in Arkansas [Web site]. (2001). Retrieved from http://arkarcheology.uark.edu/rockart/index.html

• << Previous: Find Articles
• Next: Poster Techniques >>
• Last Updated: Mar 28, 2024 4:54 PM
• URL: https://uark.libguides.com/sciencefair
• See us on Instagram
• Follow us on Twitter
• Phone: 479-575-4104

Compiled by Timothy T. Allen , revised 2000. This paper greatly expands upon a handout originally prepared by an unknown author for distribution to students in introductory earth science courses at Dartmouth College. The work is presented here without copyright, although acknowledgement is (of course) appreciated. This document is also available in in Adobe Acrobat Format

Introduction

It is important to properly and appropriately cite references in scientific research papers in order to acknowledge your sources and give credit where credit is due. Science moves forward only by building upon the work of others. There are, however, other reasons for citing references in scientific research papers. Citations to appropriate sources show that you've done your homework and are aware of the background and context into which your work fits, and they help lend validity to your arguments. Reference citations also provide avenues for interested readers to follow up on aspects of your work -- they help weave the web of science. You may wish to include citations for sources that add relevant information to your own work, or that present alternate views. The reference citation style described here is a version of the "Author, Date" scientific style, adapted from Hansen (1991) and the Council of Biology Editors (1994). Harnack & Kleppinger (2000) have adapted "CBE style" to cite and document online sources . When to Cite References in Scientific Research Papers You should acknowledge a source any time (and every time) you use a fact or an idea that you obtained from that source. Thus, clearly, you need to cite sources for all direct quotations. But you also need to cite sources from which you paraphrase or summarize facts or ideas -- whether you've put the fact or idea into your own words or not, you got the fact or idea from somebody else and you need to give them proper acknowledgement (even if an idea might be considered "common knowledge," but you didn't know it until you found it in a particular source).

If you have more than one source by the same author published in the same year, distinguish them both in the in-text citation and in the reference list, by appending the letters a, b, c... to the year, in the order in which the different references appear in your paper. (For example: Allen 1996a, 1996b.)

If your source of information has no individual identifiable author, use the name of the organization to which the work can be attributed in place of the author's name:

For internet sources without any identifiable author or date, simply use the URL address as the in-text citation:

As New England is located at the convergence of several distinct storm tracks ( http://www.mountwashington.org/mtw_mtn.htm ), we expect to find clear differences in isotopic composition among seasons and potentially among different rain storm events (Fig. 1).

Such a source would be omitted from your References Cited or Bibliography section.

References Cited (in this document)

Council of Biology Editors, 1994, Scientific style and format: the CBE manual for authors, editors, and publishers, 6th edition, Cambridge University Press, New York. 825 p.

Harnack, A., and Kleppinger, E., 2000, Online! A reference guide to using internet sources , Bedford/St. Martin's, http://www.bedfordstmartins.com/online/index.html (August 1, 2000).

Reference Management in Scientific Writing

• First Online: 10 April 2022

Cite this chapter

• Fahimeh Tabatabaei 3 &
• Lobat Tayebi 3  

672 Accesses

The reference list is the last section of a proposal or scientific article. As we explained in the previous Chaps. 3 , 4 , and 5 , when preparing a systematic review, a proposal, a scientific report, or an original article, you must consult and refer to the validated articles published or accepted for publication, which are related to your subject. Using these previously published articles would allow you to argue and enrich your proposal or manuscript. The objective of a reference list/bibliography is to let the reader find these consulted articles. The researcher is obliged to observe ethical and scientific principles in reference writing.

• Bibliography
• Journal impact factor

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info
• Durable hardcover edition

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

I. Soares-Rusu, C.A. Villavicencio-Espinoza, N.A. de Oliveira, L. Wang, H.M. Honório, J.H. Rubo, et al., Clinical evaluation of lithium disilicate veneers manufactured by CAD/CAM compared with heat-pressed methods: Randomized controlled clinical trial. Oper. Dent. 46 (1), 4–14 (2021)

Article   Google Scholar  

D.T. Graves, J.D. Corrêa, T.A. Silva, The oral microbiota is modified by systemic diseases. J. Dent. Res. 98 (2), 148–156 (2019)

Additional Resources

https://clarivate.com/webofsciencegroup/support/endnote/

https://www.mendeley.com/guides

Download references

Author information

Authors and affiliations.

School of Dentistry, Marquette University, Milwaukee, WI, USA

Fahimeh Tabatabaei & Lobat Tayebi

You can also search for this author in PubMed   Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this chapter

Tabatabaei, F., Tayebi, L. (2022). Reference Management in Scientific Writing. In: Research Methods in Dentistry. Springer, Cham. https://doi.org/10.1007/978-3-030-98028-3_6

Download citation

DOI : https://doi.org/10.1007/978-3-030-98028-3_6

Published : 10 April 2022

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-98027-6

Online ISBN : 978-3-030-98028-3

eBook Packages : Engineering Engineering (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Characteristics of References

• Provides the reader with information about who conducted the research, when it was published and the journal that published the work.
• Provides detailed information about author names, article title, journal name, volume, issue and page numbers so that readers can easily find the source of the information.
• Acknowledges the scientist(s) who conducted the research and/or the journal article where the research was originally published.  

The References (or Bibliography) section should list all the sources of information that were used in the poster.   This section appears at the end of the poster.   The References section ( Figs. 2 and 8 ) typically contains all journal articles (i.e., primary sources) but it can also contain secondary sources (e.g., newspapers, documentaries, government reports).   References tell the reader where the original data, information, technique, and/or method can be obtained, who conducted the work and when the paper was published.    

In posters, in-text citations are used to tell the reader where information was obtained.   An in-text citation should appear after every sentence in the poster that describes the work of others.   This includes all sentences that describe discoveries, findings, data, information, experiments, results, techniques, methods, dates, locations, etc.  

In-text citations can be done using either (1) superscript numbers or (2) authors last name, followed by year published.  

• Polar bear cubs were 25% larger when fed a high-protein diet compared to high-sugar diet. 1      
• Polar bear cubs were 25% larger when fed a high-protein diet compared to high-sugar diet (Jones and Smith, 2018).  

The “1” and “Jones and Smith, 2018” both refer to the same journal article:   E.J. Jones and W. A. Smith (2018), Journal of Natural Science, Vol. 53, Issue 12, pages 36-45.   Both types of in-text citations are acceptable for use in posters.   Authors typically choose superscript numbers to save space.      

Formatting References

You have likely been taught about MLA (Modern Language Association of America) or APA (American Psychological Association) formatting and style guide in middle or high school.   Many of you are likely proficient in these styles.   For most posters you likely will not follow the MLA nor APA styles when citing your sources.   There is a practical explanation for why these two styles often are not used in a poster. It is because there are thousands of different professional scientific societies around the world and each society has its own preferred formatting style that they use in publications for their journals and conferences.   Therefore, citation styles will vary depending on where a poster is presented.   In fact, many scientists use software, that, with a click of a button, will transform all of their citations into the proper style and format for any journal or conference.

Nonetheless, we provide 15 examples below of how one could cite primary sources (examples 1-5 below) and secondary sources (examples 6-15 below) of information in a scientific poster.  

Figure 8. References List

Citing Primary Sources

Peer-reviewed journal articles are considered primary sources.   Patents and Published Technical Reports from Government Agencies and Universities are also considered primary sources of information.   Five examples of how to cite primary sources are numbered below 1-5.  

1. Journal Article in Print:   Most journals are printed on paper others are entirely available online.   Authors Names. (Year Published in parenthesis). Article Title.   Journal Name, Volume Number (Issue Number in parenthesis): Page Numbers.  

1A. Journal Article with one or two authors:

McMurran, M. and Christopher, G. (2009). Bayes factors increases criminal sentence recommendations. Legal & Criminological Psychology, 14(1):101-107.

1B. Journal Article with more than two authors:  

Post, E., et al. (2009). Genome studies of quorum sensing organisms. Science, 325(5946):1355-1358.

2. Online Journal Article:   These journals are electronic and not printed on paper.   Authors Names. (Year Published in parenthesis). Title of article. Journal name. Volume number and or page numbers.   Include complete URL link in full or DOI if known.    

Dionne, M.S. and Schneider, D.S. (2002). Adaptive mutability in targeted microRNA infections. Genome Biol. 3:10.3559. http://genomebiology.com/2002/3/4/reviews/1010  

3. Government Technical Report in Print: Author names or name of organization. (Year Published in parenthesis). Report title.   Report Number. Name of government agency that published report, Place of publication.  

Smith, G.I. and Chen Y.P. (2018). Growth stages and tolerable fire intervals for Georgia’s native vegetation data sets. Report no. 247. U.S. Department of Interior. New York, NY, USA.

4. Government Technical Report Published Online: Author names or name of organization. (Year Published in parenthesis). Report title.   Report Number. Name of government agency that published report. Place of publication.   Date retrieved followed by complete URL link in full or DOI if known.  

Spandone, H.K. et al. (2017). Energy futures for Midwestern wind farms. Report no. C2.4715.12. U.S. Department of Energy. Washington D.C., USA.   Retrieved on February 15, 2017 from https://www.energy.gov/science-innovation/energy-sources/renewable-energy/wind

5. Patent : Author names. Date in parenthesis. Title of patented item, technique, method or process. Patent number.

Odell, J.C. (1970, April). Process for batch culturing. U.S. patent 484,363,770.

Citing Secondary Sources

Secondary sources report on and interpret results that have been presented in primary sources.   Secondary sources include books, documentaries, magazines, newspapers, podcasts, webpages from government agencies and universities.   Ten examples of how to cite secondary sources are numbered below 6-15.      

6. Book Chapter : Authors names. (Date of publication in parenthesis). Chapter title, page numbers. Editors of book, Book Title, Place of publication.   Name of publisher.  

Forman, M.S., and Valsamakis, A. (2003). Specimen collection, transport, and processing: virology, p. 1227-1241. Murray, P.R., et al. (Eds.), Manual of clinical microbiology, 8th ed, Washington, D.C. Penguin Press.  

Anderegg, D. (2007). Nerds: Who they are and why we need more of them. New York, NY. Jeremy P. Tarcher, Penguin Press.  

8. Magazine Article in Print :

Road map to a great deal. (2009, October). Consumer Reports, 74(10), 44-47.

9. Magazine Article Published Online :

Taibbi, M. (2009, September 3). Sick and wrong. Rolling Stone, 1086, 58-65. Retrieved on February 22, 2020 from http://www.rollingstone.com  

10. Newspaper Article in Print:

Lucchetti, A. & Craig, S. (2009, September 11). Morgan Stanley taps new boss. The Wall Street Journal, pp. A1, A16.  

11. Newspaper Article Published Online :

Moran, S. (2009, September 7). If you don’t snooze, you lose: Most Americans aren’t getting enough sleep. And for both adults and students, there are health consequences. Star Tribune. Retrieved on August 6, 2019 from http://www.startribune.com /  

12. Podcast :

Nature (Producer). (2009, July 16). Moon gazing in the Southern hemisphere, Audio podcast. Retrieved on November 5,2009, from http://www.nature.com/nature/podcast/index-2009-07-16.html  

13. Documentary, Video or Movie :

Donner, R. & Lee, S. (Producers), & Hood, G. (Director). (2009). X-Men Origins: Wolverine [DVD]. USA: Twentieth Century-Fox Film Corporation.  

14. Personal Web Page : In most instances a web page is not used as a reference in a poster.  

Wilson, E.O. (1999, September). Biological Diversity: The Oldest Human Heritage, New York State Museum, Albany. Retrieved on July 12, 2020 from https://eowilsonfoundation.org/e-o-wilson/  

15. Web Page of Organization or Group of Authors : In most instances, a webpage is not used as a reference in a poster.  

National Museum of American History. (2006, July 7). National museum of American history displays recent hip-hop acquisitions. Retrieved from https://americanhistory.si.edu  

Scientific Posters: A Learner's Guide Copyright © 2020 by Ella Weaver; Kylienne A. Shaul; Henry Griffy; and Brian H. Lower is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License , except where otherwise noted.

Share This Book

The appropriate use of references in a scientific research paper

Affiliation.

• 1 Emergency Medicine Research, Emergency Department, Royal Melbourne Hospital, Parkville, Victoria, Australia. [email protected]
• PMID: 12147114
• DOI: 10.1046/j.1442-2026.2002.00312.x

References have an important and varied role in any scientific paper. Unfortunately, many authors do not appreciate this importance and errors within reference lists are frequently encountered. Most reference errors involve spelling, numerical and punctuation mistakes, although the use of too many, too few or even inappropriate references is often seen. The consequences of reference errors include difficulty in reference retrieval, limitation for the reader to read more widely, failure to credit the cited authors, and inaccuracies in citation indexes. This paper discusses the value of accurate reference lists and provides guidelines for their preparation.

• Bibliographies as Topic*
• Publishing*

Help | Advanced Search

Computer Science > Computation and Language

Title: realm: reference resolution as language modeling.

Abstract: Reference resolution is an important problem, one that is essential to understand and successfully handle context of different kinds. This context includes both previous turns and context that pertains to non-conversational entities, such as entities on the user's screen or those running in the background. While LLMs have been shown to be extremely powerful for a variety of tasks, their use in reference resolution, particularly for non-conversational entities, remains underutilized. This paper demonstrates how LLMs can be used to create an extremely effective system to resolve references of various types, by showing how reference resolution can be converted into a language modeling problem, despite involving forms of entities like those on screen that are not traditionally conducive to being reduced to a text-only modality. We demonstrate large improvements over an existing system with similar functionality across different types of references, with our smallest model obtaining absolute gains of over 5% for on-screen references. We also benchmark against GPT-3.5 and GPT-4, with our smallest model achieving performance comparable to that of GPT-4, and our larger models substantially outperforming it.

Submission history

Access paper:.

• HTML (experimental)
• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 26 March 2024

Predicting and improving complex beer flavor through machine learning

• Michiel Schreurs   ORCID: orcid.org/0000-0002-9449-5619 1 , 2 , 3   na1 ,
• Supinya Piampongsant 1 , 2 , 3   na1 ,
• Miguel Roncoroni   ORCID: orcid.org/0000-0001-7461-1427 1 , 2 , 3   na1 ,
• Lloyd Cool   ORCID: orcid.org/0000-0001-9936-3124 1 , 2 , 3 , 4 ,
• Beatriz Herrera-Malaver   ORCID: orcid.org/0000-0002-5096-9974 1 , 2 , 3 ,
• Christophe Vanderaa   ORCID: orcid.org/0000-0001-7443-5427 4 ,
• Florian A. Theßeling 1 , 2 , 3 ,
• Łukasz Kreft   ORCID: orcid.org/0000-0001-7620-4657 5 ,
• Alexander Botzki   ORCID: orcid.org/0000-0001-6691-4233 5 ,
• Philippe Malcorps 6 ,
• Luk Daenen 6 ,
• Tom Wenseleers   ORCID: orcid.org/0000-0002-1434-861X 4 &
• Kevin J. Verstrepen   ORCID: orcid.org/0000-0002-3077-6219 1 , 2 , 3  

Nature Communications volume  15 , Article number:  2368 ( 2024 ) Cite this article

48k Accesses

846 Altmetric

Metrics details

• Chemical engineering
• Gas chromatography
• Machine learning
• Metabolomics
• Taste receptors

The perception and appreciation of food flavor depends on many interacting chemical compounds and external factors, and therefore proves challenging to understand and predict. Here, we combine extensive chemical and sensory analyses of 250 different beers to train machine learning models that allow predicting flavor and consumer appreciation. For each beer, we measure over 200 chemical properties, perform quantitative descriptive sensory analysis with a trained tasting panel and map data from over 180,000 consumer reviews to train 10 different machine learning models. The best-performing algorithm, Gradient Boosting, yields models that significantly outperform predictions based on conventional statistics and accurately predict complex food features and consumer appreciation from chemical profiles. Model dissection allows identifying specific and unexpected compounds as drivers of beer flavor and appreciation. Adding these compounds results in variants of commercial alcoholic and non-alcoholic beers with improved consumer appreciation. Together, our study reveals how big data and machine learning uncover complex links between food chemistry, flavor and consumer perception, and lays the foundation to develop novel, tailored foods with superior flavors.

Similar content being viewed by others

BitterSweet: Building machine learning models for predicting the bitter and sweet taste of small molecules

Rudraksh Tuwani, Somin Wadhwa & Ganesh Bagler

Sensory lexicon and aroma volatiles analysis of brewing malt

Xiaoxia Su, Miao Yu, … Tianyi Du

Predicting odor from molecular structure: a multi-label classification approach

Kushagra Saini & Venkatnarayan Ramanathan

Introduction

Predicting and understanding food perception and appreciation is one of the major challenges in food science. Accurate modeling of food flavor and appreciation could yield important opportunities for both producers and consumers, including quality control, product fingerprinting, counterfeit detection, spoilage detection, and the development of new products and product combinations (food pairing) 1 , 2 , 3 , 4 , 5 , 6 . Accurate models for flavor and consumer appreciation would contribute greatly to our scientific understanding of how humans perceive and appreciate flavor. Moreover, accurate predictive models would also facilitate and standardize existing food assessment methods and could supplement or replace assessments by trained and consumer tasting panels, which are variable, expensive and time-consuming 7 , 8 , 9 . Lastly, apart from providing objective, quantitative, accurate and contextual information that can help producers, models can also guide consumers in understanding their personal preferences 10 .

Despite the myriad of applications, predicting food flavor and appreciation from its chemical properties remains a largely elusive goal in sensory science, especially for complex food and beverages 11 , 12 . A key obstacle is the immense number of flavor-active chemicals underlying food flavor. Flavor compounds can vary widely in chemical structure and concentration, making them technically challenging and labor-intensive to quantify, even in the face of innovations in metabolomics, such as non-targeted metabolic fingerprinting 13 , 14 . Moreover, sensory analysis is perhaps even more complicated. Flavor perception is highly complex, resulting from hundreds of different molecules interacting at the physiochemical and sensorial level. Sensory perception is often non-linear, characterized by complex and concentration-dependent synergistic and antagonistic effects 15 , 16 , 17 , 18 , 19 , 20 , 21 that are further convoluted by the genetics, environment, culture and psychology of consumers 22 , 23 , 24 . Perceived flavor is therefore difficult to measure, with problems of sensitivity, accuracy, and reproducibility that can only be resolved by gathering sufficiently large datasets 25 . Trained tasting panels are considered the prime source of quality sensory data, but require meticulous training, are low throughput and high cost. Public databases containing consumer reviews of food products could provide a valuable alternative, especially for studying appreciation scores, which do not require formal training 25 . Public databases offer the advantage of amassing large amounts of data, increasing the statistical power to identify potential drivers of appreciation. However, public datasets suffer from biases, including a bias in the volunteers that contribute to the database, as well as confounding factors such as price, cult status and psychological conformity towards previous ratings of the product.

Classical multivariate statistics and machine learning methods have been used to predict flavor of specific compounds by, for example, linking structural properties of a compound to its potential biological activities or linking concentrations of specific compounds to sensory profiles 1 , 26 . Importantly, most previous studies focused on predicting organoleptic properties of single compounds (often based on their chemical structure) 27 , 28 , 29 , 30 , 31 , 32 , 33 , thus ignoring the fact that these compounds are present in a complex matrix in food or beverages and excluding complex interactions between compounds. Moreover, the classical statistics commonly used in sensory science 34 , 35 , 36 , 37 , 38 , 39 require a large sample size and sufficient variance amongst predictors to create accurate models. They are not fit for studying an extensive set of hundreds of interacting flavor compounds, since they are sensitive to outliers, have a high tendency to overfit and are less suited for non-linear and discontinuous relationships 40 .

In this study, we combine extensive chemical analyses and sensory data of a set of different commercial beers with machine learning approaches to develop models that predict taste, smell, mouthfeel and appreciation from compound concentrations. Beer is particularly suited to model the relationship between chemistry, flavor and appreciation. First, beer is a complex product, consisting of thousands of flavor compounds that partake in complex sensory interactions 41 , 42 , 43 . This chemical diversity arises from the raw materials (malt, yeast, hops, water and spices) and biochemical conversions during the brewing process (kilning, mashing, boiling, fermentation, maturation and aging) 44 , 45 . Second, the advent of the internet saw beer consumers embrace online review platforms, such as RateBeer (ZX Ventures, Anheuser-Busch InBev SA/NV) and BeerAdvocate (Next Glass, inc.). In this way, the beer community provides massive data sets of beer flavor and appreciation scores, creating extraordinarily large sensory databases to complement the analyses of our professional sensory panel. Specifically, we characterize over 200 chemical properties of 250 commercial beers, spread across 22 beer styles, and link these to the descriptive sensory profiling data of a 16-person in-house trained tasting panel and data acquired from over 180,000 public consumer reviews. These unique and extensive datasets enable us to train a suite of machine learning models to predict flavor and appreciation from a beer’s chemical profile. Dissection of the best-performing models allows us to pinpoint specific compounds as potential drivers of beer flavor and appreciation. Follow-up experiments confirm the importance of these compounds and ultimately allow us to significantly improve the flavor and appreciation of selected commercial beers. Together, our study represents a significant step towards understanding complex flavors and reinforces the value of machine learning to develop and refine complex foods. In this way, it represents a stepping stone for further computer-aided food engineering applications 46 .

To generate a comprehensive dataset on beer flavor, we selected 250 commercial Belgian beers across 22 different beer styles (Supplementary Fig.  S1 ). Beers with ≤ 4.2% alcohol by volume (ABV) were classified as non-alcoholic and low-alcoholic. Blonds and Tripels constitute a significant portion of the dataset (12.4% and 11.2%, respectively) reflecting their presence on the Belgian beer market and the heterogeneity of beers within these styles. By contrast, lager beers are less diverse and dominated by a handful of brands. Rare styles such as Brut or Faro make up only a small fraction of the dataset (2% and 1%, respectively) because fewer of these beers are produced and because they are dominated by distinct characteristics in terms of flavor and chemical composition.

Extensive analysis identifies relationships between chemical compounds in beer

For each beer, we measured 226 different chemical properties, including common brewing parameters such as alcohol content, iso-alpha acids, pH, sugar concentration 47 , and over 200 flavor compounds (Methods, Supplementary Table  S1 ). A large portion (37.2%) are terpenoids arising from hopping, responsible for herbal and fruity flavors 16 , 48 . A second major category are yeast metabolites, such as esters and alcohols, that result in fruity and solvent notes 48 , 49 , 50 . Other measured compounds are primarily derived from malt, or other microbes such as non- Saccharomyces yeasts and bacteria (‘wild flora’). Compounds that arise from spices or staling are labeled under ‘Others’. Five attributes (caloric value, total acids and total ester, hop aroma and sulfur compounds) are calculated from multiple individually measured compounds.

As a first step in identifying relationships between chemical properties, we determined correlations between the concentrations of the compounds (Fig.  1 , upper panel, Supplementary Data  1 and 2 , and Supplementary Fig.  S2 . For the sake of clarity, only a subset of the measured compounds is shown in Fig.  1 ). Compounds of the same origin typically show a positive correlation, while absence of correlation hints at parameters varying independently. For example, the hop aroma compounds citronellol, and alpha-terpineol show moderate correlations with each other (Spearman’s rho=0.39 and 0.57), but not with the bittering hop component iso-alpha acids (Spearman’s rho=0.16 and −0.07). This illustrates how brewers can independently modify hop aroma and bitterness by selecting hop varieties and dosage time. If hops are added early in the boiling phase, chemical conversions increase bitterness while aromas evaporate, conversely, late addition of hops preserves aroma but limits bitterness 51 . Similarly, hop-derived iso-alpha acids show a strong anti-correlation with lactic acid and acetic acid, likely reflecting growth inhibition of lactic acid and acetic acid bacteria, or the consequent use of fewer hops in sour beer styles, such as West Flanders ales and Fruit beers, that rely on these bacteria for their distinct flavors 52 . Finally, yeast-derived esters (ethyl acetate, ethyl decanoate, ethyl hexanoate, ethyl octanoate) and alcohols (ethanol, isoamyl alcohol, isobutanol, and glycerol), correlate with Spearman coefficients above 0.5, suggesting that these secondary metabolites are correlated with the yeast genetic background and/or fermentation parameters and may be difficult to influence individually, although the choice of yeast strain may offer some control 53 .

Spearman rank correlations are shown. Descriptors are grouped according to their origin (malt (blue), hops (green), yeast (red), wild flora (yellow), Others (black)), and sensory aspect (aroma, taste, palate, and overall appreciation). Please note that for the chemical compounds, for the sake of clarity, only a subset of the total number of measured compounds is shown, with an emphasis on the key compounds for each source. For more details, see the main text and Methods section. Chemical data can be found in Supplementary Data  1 , correlations between all chemical compounds are depicted in Supplementary Fig.  S2 and correlation values can be found in Supplementary Data  2 . See Supplementary Data  4 for sensory panel assessments and Supplementary Data  5 for correlation values between all sensory descriptors.

Interestingly, different beer styles show distinct patterns for some flavor compounds (Supplementary Fig.  S3 ). These observations agree with expectations for key beer styles, and serve as a control for our measurements. For instance, Stouts generally show high values for color (darker), while hoppy beers contain elevated levels of iso-alpha acids, compounds associated with bitter hop taste. Acetic and lactic acid are not prevalent in most beers, with notable exceptions such as Kriek, Lambic, Faro, West Flanders ales and Flanders Old Brown, which use acid-producing bacteria ( Lactobacillus and Pediococcus ) or unconventional yeast ( Brettanomyces ) 54 , 55 . Glycerol, ethanol and esters show similar distributions across all beer styles, reflecting their common origin as products of yeast metabolism during fermentation 45 , 53 . Finally, low/no-alcohol beers contain low concentrations of glycerol and esters. This is in line with the production process for most of the low/no-alcohol beers in our dataset, which are produced through limiting fermentation or by stripping away alcohol via evaporation or dialysis, with both methods having the unintended side-effect of reducing the amount of flavor compounds in the final beer 56 , 57 .

Besides expected associations, our data also reveals less trivial associations between beer styles and specific parameters. For example, geraniol and citronellol, two monoterpenoids responsible for citrus, floral and rose flavors and characteristic of Citra hops, are found in relatively high amounts in Christmas, Saison, and Brett/co-fermented beers, where they may originate from terpenoid-rich spices such as coriander seeds instead of hops 58 .

Tasting panel assessments reveal sensorial relationships in beer

To assess the sensory profile of each beer, a trained tasting panel evaluated each of the 250 beers for 50 sensory attributes, including different hop, malt and yeast flavors, off-flavors and spices. Panelists used a tasting sheet (Supplementary Data  3 ) to score the different attributes. Panel consistency was evaluated by repeating 12 samples across different sessions and performing ANOVA. In 95% of cases no significant difference was found across sessions ( p  > 0.05), indicating good panel consistency (Supplementary Table  S2 ).

Aroma and taste perception reported by the trained panel are often linked (Fig.  1 , bottom left panel and Supplementary Data  4 and 5 ), with high correlations between hops aroma and taste (Spearman’s rho=0.83). Bitter taste was found to correlate with hop aroma and taste in general (Spearman’s rho=0.80 and 0.69), and particularly with “grassy” noble hops (Spearman’s rho=0.75). Barnyard flavor, most often associated with sour beers, is identified together with stale hops (Spearman’s rho=0.97) that are used in these beers. Lactic and acetic acid, which often co-occur, are correlated (Spearman’s rho=0.66). Interestingly, sweetness and bitterness are anti-correlated (Spearman’s rho = −0.48), confirming the hypothesis that they mask each other 59 , 60 . Beer body is highly correlated with alcohol (Spearman’s rho = 0.79), and overall appreciation is found to correlate with multiple aspects that describe beer mouthfeel (alcohol, carbonation; Spearman’s rho= 0.32, 0.39), as well as with hop and ester aroma intensity (Spearman’s rho=0.39 and 0.35).

Similar to the chemical analyses, sensorial analyses confirmed typical features of specific beer styles (Supplementary Fig.  S4 ). For example, sour beers (Faro, Flanders Old Brown, Fruit beer, Kriek, Lambic, West Flanders ale) were rated acidic, with flavors of both acetic and lactic acid. Hoppy beers were found to be bitter and showed hop-associated aromas like citrus and tropical fruit. Malt taste is most detected among scotch, stout/porters, and strong ales, while low/no-alcohol beers, which often have a reputation for being ‘worty’ (reminiscent of unfermented, sweet malt extract) appear in the middle. Unsurprisingly, hop aromas are most strongly detected among hoppy beers. Like its chemical counterpart (Supplementary Fig.  S3 ), acidity shows a right-skewed distribution, with the most acidic beers being Krieks, Lambics, and West Flanders ales.

Tasting panel assessments of specific flavors correlate with chemical composition

We find that the concentrations of several chemical compounds strongly correlate with specific aroma or taste, as evaluated by the tasting panel (Fig.  2 , Supplementary Fig.  S5 , Supplementary Data  6 ). In some cases, these correlations confirm expectations and serve as a useful control for data quality. For example, iso-alpha acids, the bittering compounds in hops, strongly correlate with bitterness (Spearman’s rho=0.68), while ethanol and glycerol correlate with tasters’ perceptions of alcohol and body, the mouthfeel sensation of fullness (Spearman’s rho=0.82/0.62 and 0.72/0.57 respectively) and darker color from roasted malts is a good indication of malt perception (Spearman’s rho=0.54).

Heatmap colors indicate Spearman’s Rho. Axes are organized according to sensory categories (aroma, taste, mouthfeel, overall), chemical categories and chemical sources in beer (malt (blue), hops (green), yeast (red), wild flora (yellow), Others (black)). See Supplementary Data  6 for all correlation values.

Interestingly, for some relationships between chemical compounds and perceived flavor, correlations are weaker than expected. For example, the rose-smelling phenethyl acetate only weakly correlates with floral aroma. This hints at more complex relationships and interactions between compounds and suggests a need for a more complex model than simple correlations. Lastly, we uncovered unexpected correlations. For instance, the esters ethyl decanoate and ethyl octanoate appear to correlate slightly with hop perception and bitterness, possibly due to their fruity flavor. Iron is anti-correlated with hop aromas and bitterness, most likely because it is also anti-correlated with iso-alpha acids. This could be a sign of metal chelation of hop acids 61 , given that our analyses measure unbound hop acids and total iron content, or could result from the higher iron content in dark and Fruit beers, which typically have less hoppy and bitter flavors 62 .

Public consumer reviews complement expert panel data

To complement and expand the sensory data of our trained tasting panel, we collected 180,000 reviews of our 250 beers from the online consumer review platform RateBeer. This provided numerical scores for beer appearance, aroma, taste, palate, overall quality as well as the average overall score.

Public datasets are known to suffer from biases, such as price, cult status and psychological conformity towards previous ratings of a product. For example, prices correlate with appreciation scores for these online consumer reviews (rho=0.49, Supplementary Fig.  S6 ), but not for our trained tasting panel (rho=0.19). This suggests that prices affect consumer appreciation, which has been reported in wine 63 , while blind tastings are unaffected. Moreover, we observe that some beer styles, like lagers and non-alcoholic beers, generally receive lower scores, reflecting that online reviewers are mostly beer aficionados with a preference for specialty beers over lager beers. In general, we find a modest correlation between our trained panel’s overall appreciation score and the online consumer appreciation scores (Fig.  3 , rho=0.29). Apart from the aforementioned biases in the online datasets, serving temperature, sample freshness and surroundings, which are all tightly controlled during the tasting panel sessions, can vary tremendously across online consumers and can further contribute to (among others, appreciation) differences between the two categories of tasters. Importantly, in contrast to the overall appreciation scores, for many sensory aspects the results from the professional panel correlated well with results obtained from RateBeer reviews. Correlations were highest for features that are relatively easy to recognize even for untrained tasters, like bitterness, sweetness, alcohol and malt aroma (Fig.  3 and below).

RateBeer text mining results can be found in Supplementary Data  7 . Rho values shown are Spearman correlation values, with asterisks indicating significant correlations ( p  < 0.05, two-sided). All p values were smaller than 0.001, except for Esters aroma (0.0553), Esters taste (0.3275), Esters aroma—banana (0.0019), Coriander (0.0508) and Diacetyl (0.0134).

Besides collecting consumer appreciation from these online reviews, we developed automated text analysis tools to gather additional data from review texts (Supplementary Data  7 ). Processing review texts on the RateBeer database yielded comparable results to the scores given by the trained panel for many common sensory aspects, including acidity, bitterness, sweetness, alcohol, malt, and hop tastes (Fig.  3 ). This is in line with what would be expected, since these attributes require less training for accurate assessment and are less influenced by environmental factors such as temperature, serving glass and odors in the environment. Consumer reviews also correlate well with our trained panel for 4-vinyl guaiacol, a compound associated with a very characteristic aroma. By contrast, correlations for more specific aromas like ester, coriander or diacetyl are underrepresented in the online reviews, underscoring the importance of using a trained tasting panel and standardized tasting sheets with explicit factors to be scored for evaluating specific aspects of a beer. Taken together, our results suggest that public reviews are trustworthy for some, but not all, flavor features and can complement or substitute taste panel data for these sensory aspects.

Models can predict beer sensory profiles from chemical data

The rich datasets of chemical analyses, tasting panel assessments and public reviews gathered in the first part of this study provided us with a unique opportunity to develop predictive models that link chemical data to sensorial features. Given the complexity of beer flavor, basic statistical tools such as correlations or linear regression may not always be the most suitable for making accurate predictions. Instead, we applied different machine learning models that can model both simple linear and complex interactive relationships. Specifically, we constructed a set of regression models to predict (a) trained panel scores for beer flavor and quality and (b) public reviews’ appreciation scores from beer chemical profiles. We trained and tested 10 different models (Methods), 3 linear regression-based models (simple linear regression with first-order interactions (LR), lasso regression with first-order interactions (Lasso), partial least squares regressor (PLSR)), 5 decision tree models (AdaBoost regressor (ABR), extra trees (ET), gradient boosting regressor (GBR), random forest (RF) and XGBoost regressor (XGBR)), 1 support vector regression (SVR), and 1 artificial neural network (ANN) model.

To compare the performance of our machine learning models, the dataset was randomly split into a training and test set, stratified by beer style. After a model was trained on data in the training set, its performance was evaluated on its ability to predict the test dataset obtained from multi-output models (based on the coefficient of determination, see Methods). Additionally, individual-attribute models were ranked per descriptor and the average rank was calculated, as proposed by Korneva et al. 64 . Importantly, both ways of evaluating the models’ performance agreed in general. Performance of the different models varied (Table  1 ). It should be noted that all models perform better at predicting RateBeer results than results from our trained tasting panel. One reason could be that sensory data is inherently variable, and this variability is averaged out with the large number of public reviews from RateBeer. Additionally, all tree-based models perform better at predicting taste than aroma. Linear models (LR) performed particularly poorly, with negative R 2 values, due to severe overfitting (training set R 2  = 1). Overfitting is a common issue in linear models with many parameters and limited samples, especially with interaction terms further amplifying the number of parameters. L1 regularization (Lasso) successfully overcomes this overfitting, out-competing multiple tree-based models on the RateBeer dataset. Similarly, the dimensionality reduction of PLSR avoids overfitting and improves performance, to some extent. Still, tree-based models (ABR, ET, GBR, RF and XGBR) show the best performance, out-competing the linear models (LR, Lasso, PLSR) commonly used in sensory science 65 .

GBR models showed the best overall performance in predicting sensory responses from chemical information, with R 2 values up to 0.75 depending on the predicted sensory feature (Supplementary Table  S4 ). The GBR models predict consumer appreciation (RateBeer) better than our trained panel’s appreciation (R 2 value of 0.67 compared to R 2 value of 0.09) (Supplementary Table  S3 and Supplementary Table  S4 ). ANN models showed intermediate performance, likely because neural networks typically perform best with larger datasets 66 . The SVR shows intermediate performance, mostly due to the weak predictions of specific attributes that lower the overall performance (Supplementary Table  S4 ).

Model dissection identifies specific, unexpected compounds as drivers of consumer appreciation

Next, we leveraged our models to infer important contributors to sensory perception and consumer appreciation. Consumer preference is a crucial sensory aspects, because a product that shows low consumer appreciation scores often does not succeed commercially 25 . Additionally, the requirement for a large number of representative evaluators makes consumer trials one of the more costly and time-consuming aspects of product development. Hence, a model for predicting chemical drivers of overall appreciation would be a welcome addition to the available toolbox for food development and optimization.

Since GBR models on our RateBeer dataset showed the best overall performance, we focused on these models. Specifically, we used two approaches to identify important contributors. First, rankings of the most important predictors for each sensorial trait in the GBR models were obtained based on impurity-based feature importance (mean decrease in impurity). High-ranked parameters were hypothesized to be either the true causal chemical properties underlying the trait, to correlate with the actual causal properties, or to take part in sensory interactions affecting the trait 67 (Fig.  4A ). In a second approach, we used SHAP 68 to determine which parameters contributed most to the model for making predictions of consumer appreciation (Fig.  4B ). SHAP calculates parameter contributions to model predictions on a per-sample basis, which can be aggregated into an importance score.

A The impurity-based feature importance (mean deviance in impurity, MDI) calculated from the Gradient Boosting Regression (GBR) model predicting RateBeer appreciation scores. The top 15 highest ranked chemical properties are shown. B SHAP summary plot for the top 15 parameters contributing to our GBR model. Each point on the graph represents a sample from our dataset. The color represents the concentration of that parameter, with bluer colors representing low values and redder colors representing higher values. Greater absolute values on the horizontal axis indicate a higher impact of the parameter on the prediction of the model. C Spearman correlations between the 15 most important chemical properties and consumer overall appreciation. Numbers indicate the Spearman Rho correlation coefficient, and the rank of this correlation compared to all other correlations. The top 15 important compounds were determined using SHAP (panel B).

Both approaches identified ethyl acetate as the most predictive parameter for beer appreciation (Fig.  4 ). Ethyl acetate is the most abundant ester in beer with a typical ‘fruity’, ‘solvent’ and ‘alcoholic’ flavor, but is often considered less important than other esters like isoamyl acetate. The second most important parameter identified by SHAP is ethanol, the most abundant beer compound after water. Apart from directly contributing to beer flavor and mouthfeel, ethanol drastically influences the physical properties of beer, dictating how easily volatile compounds escape the beer matrix to contribute to beer aroma 69 . Importantly, it should also be noted that the importance of ethanol for appreciation is likely inflated by the very low appreciation scores of non-alcoholic beers (Supplementary Fig.  S4 ). Despite not often being considered a driver of beer appreciation, protein level also ranks highly in both approaches, possibly due to its effect on mouthfeel and body 70 . Lactic acid, which contributes to the tart taste of sour beers, is the fourth most important parameter identified by SHAP, possibly due to the generally high appreciation of sour beers in our dataset.

Interestingly, some of the most important predictive parameters for our model are not well-established as beer flavors or are even commonly regarded as being negative for beer quality. For example, our models identify methanethiol and ethyl phenyl acetate, an ester commonly linked to beer staling 71 , as a key factor contributing to beer appreciation. Although there is no doubt that high concentrations of these compounds are considered unpleasant, the positive effects of modest concentrations are not yet known 72 , 73 .

To compare our approach to conventional statistics, we evaluated how well the 15 most important SHAP-derived parameters correlate with consumer appreciation (Fig.  4C ). Interestingly, only 6 of the properties derived by SHAP rank amongst the top 15 most correlated parameters. For some chemical compounds, the correlations are so low that they would have likely been considered unimportant. For example, lactic acid, the fourth most important parameter, shows a bimodal distribution for appreciation, with sour beers forming a separate cluster, that is missed entirely by the Spearman correlation. Additionally, the correlation plots reveal outliers, emphasizing the need for robust analysis tools. Together, this highlights the need for alternative models, like the Gradient Boosting model, that better grasp the complexity of (beer) flavor.

Finally, to observe the relationships between these chemical properties and their predicted targets, partial dependence plots were constructed for the six most important predictors of consumer appreciation 74 , 75 , 76 (Supplementary Fig.  S7 ). One-way partial dependence plots show how a change in concentration affects the predicted appreciation. These plots reveal an important limitation of our models: appreciation predictions remain constant at ever-increasing concentrations. This implies that once a threshold concentration is reached, further increasing the concentration does not affect appreciation. This is false, as it is well-documented that certain compounds become unpleasant at high concentrations, including ethyl acetate (‘nail polish’) 77 and methanethiol (‘sulfury’ and ‘rotten cabbage’) 78 . The inability of our models to grasp that flavor compounds have optimal levels, above which they become negative, is a consequence of working with commercial beer brands where (off-)flavors are rarely too high to negatively impact the product. The two-way partial dependence plots show how changing the concentration of two compounds influences predicted appreciation, visualizing their interactions (Supplementary Fig.  S7 ). In our case, the top 5 parameters are dominated by additive or synergistic interactions, with high concentrations for both compounds resulting in the highest predicted appreciation.

To assess the robustness of our best-performing models and model predictions, we performed 100 iterations of the GBR, RF and ET models. In general, all iterations of the models yielded similar performance (Supplementary Fig.  S8 ). Moreover, the main predictors (including the top predictors ethanol and ethyl acetate) remained virtually the same, especially for GBR and RF. For the iterations of the ET model, we did observe more variation in the top predictors, which is likely a consequence of the model’s inherent random architecture in combination with co-correlations between certain predictors. However, even in this case, several of the top predictors (ethanol and ethyl acetate) remain unchanged, although their rank in importance changes (Supplementary Fig.  S8 ).

Next, we investigated if a combination of RateBeer and trained panel data into one consolidated dataset would lead to stronger models, under the hypothesis that such a model would suffer less from bias in the datasets. A GBR model was trained to predict appreciation on the combined dataset. This model underperformed compared to the RateBeer model, both in the native case and when including a dataset identifier (R 2  = 0.67, 0.26 and 0.42 respectively). For the latter, the dataset identifier is the most important feature (Supplementary Fig.  S9 ), while most of the feature importance remains unchanged, with ethyl acetate and ethanol ranking highest, like in the original model trained only on RateBeer data. It seems that the large variation in the panel dataset introduces noise, weakening the models’ performances and reliability. In addition, it seems reasonable to assume that both datasets are fundamentally different, with the panel dataset obtained by blind tastings by a trained professional panel.

Lastly, we evaluated whether beer style identifiers would further enhance the model’s performance. A GBR model was trained with parameters that explicitly encoded the styles of the samples. This did not improve model performance (R2 = 0.66 with style information vs R2 = 0.67). The most important chemical features are consistent with the model trained without style information (eg. ethanol and ethyl acetate), and with the exception of the most preferred (strong ale) and least preferred (low/no-alcohol) styles, none of the styles were among the most important features (Supplementary Fig.  S9 , Supplementary Table  S5 and S6 ). This is likely due to a combination of style-specific chemical signatures, such as iso-alpha acids and lactic acid, that implicitly convey style information to the original models, as well as the low number of samples belonging to some styles, making it difficult for the model to learn style-specific patterns. Moreover, beer styles are not rigorously defined, with some styles overlapping in features and some beers being misattributed to a specific style, all of which leads to more noise in models that use style parameters.

Model validation

To test if our predictive models give insight into beer appreciation, we set up experiments aimed at improving existing commercial beers. We specifically selected overall appreciation as the trait to be examined because of its complexity and commercial relevance. Beer flavor comprises a complex bouquet rather than single aromas and tastes 53 . Hence, adding a single compound to the extent that a difference is noticeable may lead to an unbalanced, artificial flavor. Therefore, we evaluated the effect of combinations of compounds. Because Blond beers represent the most extensive style in our dataset, we selected a beer from this style as the starting material for these experiments (Beer 64 in Supplementary Data  1 ).

In the first set of experiments, we adjusted the concentrations of compounds that made up the most important predictors of overall appreciation (ethyl acetate, ethanol, lactic acid, ethyl phenyl acetate) together with correlated compounds (ethyl hexanoate, isoamyl acetate, glycerol), bringing them up to 95 th percentile ethanol-normalized concentrations (Methods) within the Blond group (‘Spiked’ concentration in Fig.  5A ). Compared to controls, the spiked beers were found to have significantly improved overall appreciation among trained panelists, with panelist noting increased intensity of ester flavors, sweetness, alcohol, and body fullness (Fig.  5B ). To disentangle the contribution of ethanol to these results, a second experiment was performed without the addition of ethanol. This resulted in a similar outcome, including increased perception of alcohol and overall appreciation.

Adding the top chemical compounds, identified as best predictors of appreciation by our model, into poorly appreciated beers results in increased appreciation from our trained panel. Results of sensory tests between base beers and those spiked with compounds identified as the best predictors by the model. A Blond and Non/Low-alcohol (0.0% ABV) base beers were brought up to 95th-percentile ethanol-normalized concentrations within each style. B For each sensory attribute, tasters indicated the more intense sample and selected the sample they preferred. The numbers above the bars correspond to the p values that indicate significant changes in perceived flavor (two-sided binomial test: alpha 0.05, n  = 20 or 13).

In a last experiment, we tested whether using the model’s predictions can boost the appreciation of a non-alcoholic beer (beer 223 in Supplementary Data  1 ). Again, the addition of a mixture of predicted compounds (omitting ethanol, in this case) resulted in a significant increase in appreciation, body, ester flavor and sweetness.

Predicting flavor and consumer appreciation from chemical composition is one of the ultimate goals of sensory science. A reliable, systematic and unbiased way to link chemical profiles to flavor and food appreciation would be a significant asset to the food and beverage industry. Such tools would substantially aid in quality control and recipe development, offer an efficient and cost-effective alternative to pilot studies and consumer trials and would ultimately allow food manufacturers to produce superior, tailor-made products that better meet the demands of specific consumer groups more efficiently.

A limited set of studies have previously tried, to varying degrees of success, to predict beer flavor and beer popularity based on (a limited set of) chemical compounds and flavors 79 , 80 . Current sensitive, high-throughput technologies allow measuring an unprecedented number of chemical compounds and properties in a large set of samples, yielding a dataset that can train models that help close the gaps between chemistry and flavor, even for a complex natural product like beer. To our knowledge, no previous research gathered data at this scale (250 samples, 226 chemical parameters, 50 sensory attributes and 5 consumer scores) to disentangle and validate the chemical aspects driving beer preference using various machine-learning techniques. We find that modern machine learning models outperform conventional statistical tools, such as correlations and linear models, and can successfully predict flavor appreciation from chemical composition. This could be attributed to the natural incorporation of interactions and non-linear or discontinuous effects in machine learning models, which are not easily grasped by the linear model architecture. While linear models and partial least squares regression represent the most widespread statistical approaches in sensory science, in part because they allow interpretation 65 , 81 , 82 , modern machine learning methods allow for building better predictive models while preserving the possibility to dissect and exploit the underlying patterns. Of the 10 different models we trained, tree-based models, such as our best performing GBR, showed the best overall performance in predicting sensory responses from chemical information, outcompeting artificial neural networks. This agrees with previous reports for models trained on tabular data 83 . Our results are in line with the findings of Colantonio et al. who also identified the gradient boosting architecture as performing best at predicting appreciation and flavor (of tomatoes and blueberries, in their specific study) 26 . Importantly, besides our larger experimental scale, we were able to directly confirm our models’ predictions in vivo.

Our study confirms that flavor compound concentration does not always correlate with perception, suggesting complex interactions that are often missed by more conventional statistics and simple models. Specifically, we find that tree-based algorithms may perform best in developing models that link complex food chemistry with aroma. Furthermore, we show that massive datasets of untrained consumer reviews provide a valuable source of data, that can complement or even replace trained tasting panels, especially for appreciation and basic flavors, such as sweetness and bitterness. This holds despite biases that are known to occur in such datasets, such as price or conformity bias. Moreover, GBR models predict taste better than aroma. This is likely because taste (e.g. bitterness) often directly relates to the corresponding chemical measurements (e.g., iso-alpha acids), whereas such a link is less clear for aromas, which often result from the interplay between multiple volatile compounds. We also find that our models are best at predicting acidity and alcohol, likely because there is a direct relation between the measured chemical compounds (acids and ethanol) and the corresponding perceived sensorial attribute (acidity and alcohol), and because even untrained consumers are generally able to recognize these flavors and aromas.

The predictions of our final models, trained on review data, hold even for blind tastings with small groups of trained tasters, as demonstrated by our ability to validate specific compounds as drivers of beer flavor and appreciation. Since adding a single compound to the extent of a noticeable difference may result in an unbalanced flavor profile, we specifically tested our identified key drivers as a combination of compounds. While this approach does not allow us to validate if a particular single compound would affect flavor and/or appreciation, our experiments do show that this combination of compounds increases consumer appreciation.

It is important to stress that, while it represents an important step forward, our approach still has several major limitations. A key weakness of the GBR model architecture is that amongst co-correlating variables, the largest main effect is consistently preferred for model building. As a result, co-correlating variables often have artificially low importance scores, both for impurity and SHAP-based methods, like we observed in the comparison to the more randomized Extra Trees models. This implies that chemicals identified as key drivers of a specific sensory feature by GBR might not be the true causative compounds, but rather co-correlate with the actual causative chemical. For example, the high importance of ethyl acetate could be (partially) attributed to the total ester content, ethanol or ethyl hexanoate (rho=0.77, rho=0.72 and rho=0.68), while ethyl phenylacetate could hide the importance of prenyl isobutyrate and ethyl benzoate (rho=0.77 and rho=0.76). Expanding our GBR model to include beer style as a parameter did not yield additional power or insight. This is likely due to style-specific chemical signatures, such as iso-alpha acids and lactic acid, that implicitly convey style information to the original model, as well as the smaller sample size per style, limiting the power to uncover style-specific patterns. This can be partly attributed to the curse of dimensionality, where the high number of parameters results in the models mainly incorporating single parameter effects, rather than complex interactions such as style-dependent effects 67 . A larger number of samples may overcome some of these limitations and offer more insight into style-specific effects. On the other hand, beer style is not a rigid scientific classification, and beers within one style often differ a lot, which further complicates the analysis of style as a model factor.

Our study is limited to beers from Belgian breweries. Although these beers cover a large portion of the beer styles available globally, some beer styles and consumer patterns may be missing, while other features might be overrepresented. For example, many Belgian ales exhibit yeast-driven flavor profiles, which is reflected in the chemical drivers of appreciation discovered by this study. In future work, expanding the scope to include diverse markets and beer styles could lead to the identification of even more drivers of appreciation and better models for special niche products that were not present in our beer set.

In addition to inherent limitations of GBR models, there are also some limitations associated with studying food aroma. Even if our chemical analyses measured most of the known aroma compounds, the total number of flavor compounds in complex foods like beer is still larger than the subset we were able to measure in this study. For example, hop-derived thiols, that influence flavor at very low concentrations, are notoriously difficult to measure in a high-throughput experiment. Moreover, consumer perception remains subjective and prone to biases that are difficult to avoid. It is also important to stress that the models are still immature and that more extensive datasets will be crucial for developing more complete models in the future. Besides more samples and parameters, our dataset does not include any demographic information about the tasters. Including such data could lead to better models that grasp external factors like age and culture. Another limitation is that our set of beers consists of high-quality end-products and lacks beers that are unfit for sale, which limits the current model in accurately predicting products that are appreciated very badly. Finally, while models could be readily applied in quality control, their use in sensory science and product development is restrained by their inability to discern causal relationships. Given that the models cannot distinguish compounds that genuinely drive consumer perception from those that merely correlate, validation experiments are essential to identify true causative compounds.

Despite the inherent limitations, dissection of our models enabled us to pinpoint specific molecules as potential drivers of beer aroma and consumer appreciation, including compounds that were unexpected and would not have been identified using standard approaches. Important drivers of beer appreciation uncovered by our models include protein levels, ethyl acetate, ethyl phenyl acetate and lactic acid. Currently, many brewers already use lactic acid to acidify their brewing water and ensure optimal pH for enzymatic activity during the mashing process. Our results suggest that adding lactic acid can also improve beer appreciation, although its individual effect remains to be tested. Interestingly, ethanol appears to be unnecessary to improve beer appreciation, both for blond beer and alcohol-free beer. Given the growing consumer interest in alcohol-free beer, with a predicted annual market growth of >7% 84 , it is relevant for brewers to know what compounds can further increase consumer appreciation of these beers. Hence, our model may readily provide avenues to further improve the flavor and consumer appreciation of both alcoholic and non-alcoholic beers, which is generally considered one of the key challenges for future beer production.

Whereas we see a direct implementation of our results for the development of superior alcohol-free beverages and other food products, our study can also serve as a stepping stone for the development of novel alcohol-containing beverages. We want to echo the growing body of scientific evidence for the negative effects of alcohol consumption, both on the individual level by the mutagenic, teratogenic and carcinogenic effects of ethanol 85 , 86 , as well as the burden on society caused by alcohol abuse and addiction. We encourage the use of our results for the production of healthier, tastier products, including novel and improved beverages with lower alcohol contents. Furthermore, we strongly discourage the use of these technologies to improve the appreciation or addictive properties of harmful substances.

The present work demonstrates that despite some important remaining hurdles, combining the latest developments in chemical analyses, sensory analysis and modern machine learning methods offers exciting avenues for food chemistry and engineering. Soon, these tools may provide solutions in quality control and recipe development, as well as new approaches to sensory science and flavor research.

Beer selection

250 commercial Belgian beers were selected to cover the broad diversity of beer styles and corresponding diversity in chemical composition and aroma. See Supplementary Fig.  S1 .

Chemical dataset

Sample preparation.

Beers within their expiration date were purchased from commercial retailers. Samples were prepared in biological duplicates at room temperature, unless explicitly stated otherwise. Bottle pressure was measured with a manual pressure device (Steinfurth Mess-Systeme GmbH) and used to calculate CO 2 concentration. The beer was poured through two filter papers (Macherey-Nagel, 500713032 MN 713 ¼) to remove carbon dioxide and prevent spontaneous foaming. Samples were then prepared for measurements by targeted Headspace-Gas Chromatography-Flame Ionization Detector/Flame Photometric Detector (HS-GC-FID/FPD), Headspace-Solid Phase Microextraction-Gas Chromatography-Mass Spectrometry (HS-SPME-GC-MS), colorimetric analysis, enzymatic analysis, Near-Infrared (NIR) analysis, as described in the sections below. The mean values of biological duplicates are reported for each compound.

HS-GC-FID/FPD

HS-GC-FID/FPD (Shimadzu GC 2010 Plus) was used to measure higher alcohols, acetaldehyde, esters, 4-vinyl guaicol, and sulfur compounds. Each measurement comprised 5 ml of sample pipetted into a 20 ml glass vial containing 1.75 g NaCl (VWR, 27810.295). 100 µl of 2-heptanol (Sigma-Aldrich, H3003) (internal standard) solution in ethanol (Fisher Chemical, E/0650DF/C17) was added for a final concentration of 2.44 mg/L. Samples were flushed with nitrogen for 10 s, sealed with a silicone septum, stored at −80 °C and analyzed in batches of 20.

The GC was equipped with a DB-WAXetr column (length, 30 m; internal diameter, 0.32 mm; layer thickness, 0.50 µm; Agilent Technologies, Santa Clara, CA, USA) to the FID and an HP-5 column (length, 30 m; internal diameter, 0.25 mm; layer thickness, 0.25 µm; Agilent Technologies, Santa Clara, CA, USA) to the FPD. N 2 was used as the carrier gas. Samples were incubated for 20 min at 70 °C in the headspace autosampler (Flow rate, 35 cm/s; Injection volume, 1000 µL; Injection mode, split; Combi PAL autosampler, CTC analytics, Switzerland). The injector, FID and FPD temperatures were kept at 250 °C. The GC oven temperature was first held at 50 °C for 5 min and then allowed to rise to 80 °C at a rate of 5 °C/min, followed by a second ramp of 4 °C/min until 200 °C kept for 3 min and a final ramp of (4 °C/min) until 230 °C for 1 min. Results were analyzed with the GCSolution software version 2.4 (Shimadzu, Kyoto, Japan). The GC was calibrated with a 5% EtOH solution (VWR International) containing the volatiles under study (Supplementary Table  S7 ).

HS-SPME-GC-MS

HS-SPME-GC-MS (Shimadzu GCMS-QP-2010 Ultra) was used to measure additional volatile compounds, mainly comprising terpenoids and esters. Samples were analyzed by HS-SPME using a triphase DVB/Carboxen/PDMS 50/30 μm SPME fiber (Supelco Co., Bellefonte, PA, USA) followed by gas chromatography (Thermo Fisher Scientific Trace 1300 series, USA) coupled to a mass spectrometer (Thermo Fisher Scientific ISQ series MS) equipped with a TriPlus RSH autosampler. 5 ml of degassed beer sample was placed in 20 ml vials containing 1.75 g NaCl (VWR, 27810.295). 5 µl internal standard mix was added, containing 2-heptanol (1 g/L) (Sigma-Aldrich, H3003), 4-fluorobenzaldehyde (1 g/L) (Sigma-Aldrich, 128376), 2,3-hexanedione (1 g/L) (Sigma-Aldrich, 144169) and guaiacol (1 g/L) (Sigma-Aldrich, W253200) in ethanol (Fisher Chemical, E/0650DF/C17). Each sample was incubated at 60 °C in the autosampler oven with constant agitation. After 5 min equilibration, the SPME fiber was exposed to the sample headspace for 30 min. The compounds trapped on the fiber were thermally desorbed in the injection port of the chromatograph by heating the fiber for 15 min at 270 °C.

The GC-MS was equipped with a low polarity RXi-5Sil MS column (length, 20 m; internal diameter, 0.18 mm; layer thickness, 0.18 µm; Restek, Bellefonte, PA, USA). Injection was performed in splitless mode at 320 °C, a split flow of 9 ml/min, a purge flow of 5 ml/min and an open valve time of 3 min. To obtain a pulsed injection, a programmed gas flow was used whereby the helium gas flow was set at 2.7 mL/min for 0.1 min, followed by a decrease in flow of 20 ml/min to the normal 0.9 mL/min. The temperature was first held at 30 °C for 3 min and then allowed to rise to 80 °C at a rate of 7 °C/min, followed by a second ramp of 2 °C/min till 125 °C and a final ramp of 8 °C/min with a final temperature of 270 °C.

Mass acquisition range was 33 to 550 amu at a scan rate of 5 scans/s. Electron impact ionization energy was 70 eV. The interface and ion source were kept at 275 °C and 250 °C, respectively. A mix of linear n-alkanes (from C7 to C40, Supelco Co.) was injected into the GC-MS under identical conditions to serve as external retention index markers. Identification and quantification of the compounds were performed using an in-house developed R script as described in Goelen et al. and Reher et al. 87 , 88 (for package information, see Supplementary Table  S8 ). Briefly, chromatograms were analyzed using AMDIS (v2.71) 89 to separate overlapping peaks and obtain pure compound spectra. The NIST MS Search software (v2.0 g) in combination with the NIST2017, FFNSC3 and Adams4 libraries were used to manually identify the empirical spectra, taking into account the expected retention time. After background subtraction and correcting for retention time shifts between samples run on different days based on alkane ladders, compound elution profiles were extracted and integrated using a file with 284 target compounds of interest, which were either recovered in our identified AMDIS list of spectra or were known to occur in beer. Compound elution profiles were estimated for every peak in every chromatogram over a time-restricted window using weighted non-negative least square analysis after which peak areas were integrated 87 , 88 . Batch effect correction was performed by normalizing against the most stable internal standard compound, 4-fluorobenzaldehyde. Out of all 284 target compounds that were analyzed, 167 were visually judged to have reliable elution profiles and were used for final analysis.

Discrete photometric and enzymatic analysis

Discrete photometric and enzymatic analysis (Thermo Scientific TM Gallery TM Plus Beermaster Discrete Analyzer) was used to measure acetic acid, ammonia, beta-glucan, iso-alpha acids, color, sugars, glycerol, iron, pH, protein, and sulfite. 2 ml of sample volume was used for the analyses. Information regarding the reagents and standard solutions used for analyses and calibrations is included in Supplementary Table  S7 and Supplementary Table  S9 .

NIR analyses

NIR analysis (Anton Paar Alcolyzer Beer ME System) was used to measure ethanol. Measurements comprised 50 ml of sample, and a 10% EtOH solution was used for calibration.

Correlation calculations

Pairwise Spearman Rank correlations were calculated between all chemical properties.

Sensory dataset

Trained panel.

Our trained tasting panel consisted of volunteers who gave prior verbal informed consent. All compounds used for the validation experiment were of food-grade quality. The tasting sessions were approved by the Social and Societal Ethics Committee of the KU Leuven (G-2022-5677-R2(MAR)). All online reviewers agreed to the Terms and Conditions of the RateBeer website.

Sensory analysis was performed according to the American Society of Brewing Chemists (ASBC) Sensory Analysis Methods 90 . 30 volunteers were screened through a series of triangle tests. The sixteen most sensitive and consistent tasters were retained as taste panel members. The resulting panel was diverse in age [22–42, mean: 29], sex [56% male] and nationality [7 different countries]. The panel developed a consensus vocabulary to describe beer aroma, taste and mouthfeel. Panelists were trained to identify and score 50 different attributes, using a 7-point scale to rate attributes’ intensity. The scoring sheet is included as Supplementary Data  3 . Sensory assessments took place between 10–12 a.m. The beers were served in black-colored glasses. Per session, between 5 and 12 beers of the same style were tasted at 12 °C to 16 °C. Two reference beers were added to each set and indicated as ‘Reference 1 & 2’, allowing panel members to calibrate their ratings. Not all panelists were present at every tasting. Scores were scaled by standard deviation and mean-centered per taster. Values are represented as z-scores and clustered by Euclidean distance. Pairwise Spearman correlations were calculated between taste and aroma sensory attributes. Panel consistency was evaluated by repeating samples on different sessions and performing ANOVA to identify differences, using the ‘stats’ package (v4.2.2) in R (for package information, see Supplementary Table  S8 ).

Online reviews from a public database

The ‘scrapy’ package in Python (v3.6) (for package information, see Supplementary Table  S8 ). was used to collect 232,288 online reviews (mean=922, min=6, max=5343) from RateBeer, an online beer review database. Each review entry comprised 5 numerical scores (appearance, aroma, taste, palate and overall quality) and an optional review text. The total number of reviews per reviewer was collected separately. Numerical scores were scaled and centered per rater, and mean scores were calculated per beer.

For the review texts, the language was estimated using the packages ‘langdetect’ and ‘langid’ in Python. Reviews that were classified as English by both packages were kept. Reviewers with fewer than 100 entries overall were discarded. 181,025 reviews from >6000 reviewers from >40 countries remained. Text processing was done using the ‘nltk’ package in Python. Texts were corrected for slang and misspellings; proper nouns and rare words that are relevant to the beer context were specified and kept as-is (‘Chimay’,’Lambic’, etc.). A dictionary of semantically similar sensorial terms, for example ‘floral’ and ‘flower’, was created and collapsed together into one term. Words were stemmed and lemmatized to avoid identifying words such as ‘acid’ and ‘acidity’ as separate terms. Numbers and punctuation were removed.

Sentences from up to 50 randomly chosen reviews per beer were manually categorized according to the aspect of beer they describe (appearance, aroma, taste, palate, overall quality—not to be confused with the 5 numerical scores described above) or flagged as irrelevant if they contained no useful information. If a beer contained fewer than 50 reviews, all reviews were manually classified. This labeled data set was used to train a model that classified the rest of the sentences for all beers 91 . Sentences describing taste and aroma were extracted, and term frequency–inverse document frequency (TFIDF) was implemented to calculate enrichment scores for sensorial words per beer.

The sex of the tasting subject was not considered when building our sensory database. Instead, results from different panelists were averaged, both for our trained panel (56% male, 44% female) and the RateBeer reviews (70% male, 30% female for RateBeer as a whole).

Beer price collection and processing

Beer prices were collected from the following stores: Colruyt, Delhaize, Total Wine, BeerHawk, The Belgian Beer Shop, The Belgian Shop, and Beer of Belgium. Where applicable, prices were converted to Euros and normalized per liter. Spearman correlations were calculated between these prices and mean overall appreciation scores from RateBeer and the taste panel, respectively.

Pairwise Spearman Rank correlations were calculated between all sensory properties.

Machine learning models

Predictive modeling of sensory profiles from chemical data.

Regression models were constructed to predict (a) trained panel scores for beer flavors and quality from beer chemical profiles and (b) public reviews’ appreciation scores from beer chemical profiles. Z-scores were used to represent sensory attributes in both data sets. Chemical properties with log-normal distributions (Shapiro-Wilk test, p  <  0.05 ) were log-transformed. Missing chemical measurements (0.1% of all data) were replaced with mean values per attribute. Observations from 250 beers were randomly separated into a training set (70%, 175 beers) and a test set (30%, 75 beers), stratified per beer style. Chemical measurements (p = 231) were normalized based on the training set average and standard deviation. In total, three linear regression-based models: linear regression with first-order interaction terms (LR), lasso regression with first-order interaction terms (Lasso) and partial least squares regression (PLSR); five decision tree models, Adaboost regressor (ABR), Extra Trees (ET), Gradient Boosting regressor (GBR), Random Forest (RF) and XGBoost regressor (XGBR); one support vector machine model (SVR) and one artificial neural network model (ANN) were trained. The models were implemented using the ‘scikit-learn’ package (v1.2.2) and ‘xgboost’ package (v1.7.3) in Python (v3.9.16). Models were trained, and hyperparameters optimized, using five-fold cross-validated grid search with the coefficient of determination (R 2 ) as the evaluation metric. The ANN (scikit-learn’s MLPRegressor) was optimized using Bayesian Tree-Structured Parzen Estimator optimization with the ‘Optuna’ Python package (v3.2.0). Individual models were trained per attribute, and a multi-output model was trained on all attributes simultaneously.

Model dissection

GBR was found to outperform other methods, resulting in models with the highest average R 2 values in both trained panel and public review data sets. Impurity-based rankings of the most important predictors for each predicted sensorial trait were obtained using the ‘scikit-learn’ package. To observe the relationships between these chemical properties and their predicted targets, partial dependence plots (PDP) were constructed for the six most important predictors of consumer appreciation 74 , 75 .

The ‘SHAP’ package in Python (v0.41.0) was implemented to provide an alternative ranking of predictor importance and to visualize the predictors’ effects as a function of their concentration 68 .

Validation of causal chemical properties

To validate the effects of the most important model features on predicted sensory attributes, beers were spiked with the chemical compounds identified by the models and descriptive sensory analyses were carried out according to the American Society of Brewing Chemists (ASBC) protocol 90 .

Compound spiking was done 30 min before tasting. Compounds were spiked into fresh beer bottles, that were immediately resealed and inverted three times. Fresh bottles of beer were opened for the same duration, resealed, and inverted thrice, to serve as controls. Pairs of spiked samples and controls were served simultaneously, chilled and in dark glasses as outlined in the Trained panel section above. Tasters were instructed to select the glass with the higher flavor intensity for each attribute (directional difference test 92 ) and to select the glass they prefer.

The final concentration after spiking was equal to the within-style average, after normalizing by ethanol concentration. This was done to ensure balanced flavor profiles in the final spiked beer. The same methods were applied to improve a non-alcoholic beer. Compounds were the following: ethyl acetate (Merck KGaA, W241415), ethyl hexanoate (Merck KGaA, W243906), isoamyl acetate (Merck KGaA, W205508), phenethyl acetate (Merck KGaA, W285706), ethanol (96%, Colruyt), glycerol (Merck KGaA, W252506), lactic acid (Merck KGaA, 261106).

Significant differences in preference or perceived intensity were determined by performing the two-sided binomial test on each attribute.

Reporting summary

Further information on research design is available in the  Nature Portfolio Reporting Summary linked to this article.

Data availability

The data that support the findings of this work are available in the Supplementary Data files and have been deposited to Zenodo under accession code 10653704 93 . The RateBeer scores data are under restricted access, they are not publicly available as they are property of RateBeer (ZX Ventures, USA). Access can be obtained from the authors upon reasonable request and with permission of RateBeer (ZX Ventures, USA).  Source data are provided with this paper.

Code availability

The code for training the machine learning models, analyzing the models, and generating the figures has been deposited to Zenodo under accession code 10653704 93 .

Tieman, D. et al. A chemical genetic roadmap to improved tomato flavor. Science 355 , 391–394 (2017).

Article   ADS   CAS   PubMed   Google Scholar  

Plutowska, B. & Wardencki, W. Application of gas chromatography–olfactometry (GC–O) in analysis and quality assessment of alcoholic beverages – A review. Food Chem. 107 , 449–463 (2008).

Article   CAS   Google Scholar  

Legin, A., Rudnitskaya, A., Seleznev, B. & Vlasov, Y. Electronic tongue for quality assessment of ethanol, vodka and eau-de-vie. Anal. Chim. Acta 534 , 129–135 (2005).

Loutfi, A., Coradeschi, S., Mani, G. K., Shankar, P. & Rayappan, J. B. B. Electronic noses for food quality: A review. J. Food Eng. 144 , 103–111 (2015).

Ahn, Y.-Y., Ahnert, S. E., Bagrow, J. P. & Barabási, A.-L. Flavor network and the principles of food pairing. Sci. Rep. 1 , 196 (2011).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Bartoshuk, L. M. & Klee, H. J. Better fruits and vegetables through sensory analysis. Curr. Biol. 23 , R374–R378 (2013).

Article   CAS   PubMed   Google Scholar  

Piggott, J. R. Design questions in sensory and consumer science. Food Qual. Prefer. 3293 , 217–220 (1995).

Article   Google Scholar  

Kermit, M. & Lengard, V. Assessing the performance of a sensory panel-panellist monitoring and tracking. J. Chemom. 19 , 154–161 (2005).

Cook, D. J., Hollowood, T. A., Linforth, R. S. T. & Taylor, A. J. Correlating instrumental measurements of texture and flavour release with human perception. Int. J. Food Sci. Technol. 40 , 631–641 (2005).

Chinchanachokchai, S., Thontirawong, P. & Chinchanachokchai, P. A tale of two recommender systems: The moderating role of consumer expertise on artificial intelligence based product recommendations. J. Retail. Consum. Serv. 61 , 1–12 (2021).

Ross, C. F. Sensory science at the human-machine interface. Trends Food Sci. Technol. 20 , 63–72 (2009).

Chambers, E. IV & Koppel, K. Associations of volatile compounds with sensory aroma and flavor: The complex nature of flavor. Molecules 18 , 4887–4905 (2013).

Pinu, F. R. Metabolomics—The new frontier in food safety and quality research. Food Res. Int. 72 , 80–81 (2015).

Danezis, G. P., Tsagkaris, A. S., Brusic, V. & Georgiou, C. A. Food authentication: state of the art and prospects. Curr. Opin. Food Sci. 10 , 22–31 (2016).

Shepherd, G. M. Smell images and the flavour system in the human brain. Nature 444 , 316–321 (2006).

Meilgaard, M. C. Prediction of flavor differences between beers from their chemical composition. J. Agric. Food Chem. 30 , 1009–1017 (1982).

Xu, L. et al. Widespread receptor-driven modulation in peripheral olfactory coding. Science 368 , eaaz5390 (2020).

Kupferschmidt, K. Following the flavor. Science 340 , 808–809 (2013).

Billesbølle, C. B. et al. Structural basis of odorant recognition by a human odorant receptor. Nature 615 , 742–749 (2023).

Article   ADS   PubMed   PubMed Central   Google Scholar  

Smith, B. Perspective: Complexities of flavour. Nature 486 , S6–S6 (2012).

Pfister, P. et al. Odorant receptor inhibition is fundamental to odor encoding. Curr. Biol. 30 , 2574–2587 (2020).

Moskowitz, H. W., Kumaraiah, V., Sharma, K. N., Jacobs, H. L. & Sharma, S. D. Cross-cultural differences in simple taste preferences. Science 190 , 1217–1218 (1975).

Eriksson, N. et al. A genetic variant near olfactory receptor genes influences cilantro preference. Flavour 1 , 22 (2012).

Ferdenzi, C. et al. Variability of affective responses to odors: Culture, gender, and olfactory knowledge. Chem. Senses 38 , 175–186 (2013).

Article   PubMed   Google Scholar  

Lawless, H. T. & Heymann, H. Sensory evaluation of food: Principles and practices. (Springer, New York, NY). https://doi.org/10.1007/978-1-4419-6488-5 (2010).

Colantonio, V. et al. Metabolomic selection for enhanced fruit flavor. Proc. Natl. Acad. Sci. 119 , e2115865119 (2022).

Fritz, F., Preissner, R. & Banerjee, P. VirtualTaste: a web server for the prediction of organoleptic properties of chemical compounds. Nucleic Acids Res 49 , W679–W684 (2021).

Tuwani, R., Wadhwa, S. & Bagler, G. BitterSweet: Building machine learning models for predicting the bitter and sweet taste of small molecules. Sci. Rep. 9 , 1–13 (2019).

Dagan-Wiener, A. et al. Bitter or not? BitterPredict, a tool for predicting taste from chemical structure. Sci. Rep. 7 , 1–13 (2017).

Pallante, L. et al. Toward a general and interpretable umami taste predictor using a multi-objective machine learning approach. Sci. Rep. 12 , 1–11 (2022).

Malavolta, M. et al. A survey on computational taste predictors. Eur. Food Res. Technol. 248 , 2215–2235 (2022).

Lee, B. K. et al. A principal odor map unifies diverse tasks in olfactory perception. Science 381 , 999–1006 (2023).

Mayhew, E. J. et al. Transport features predict if a molecule is odorous. Proc. Natl. Acad. Sci. 119 , e2116576119 (2022).

Niu, Y. et al. Sensory evaluation of the synergism among ester odorants in light aroma-type liquor by odor threshold, aroma intensity and flash GC electronic nose. Food Res. Int. 113 , 102–114 (2018).

Yu, P., Low, M. Y. & Zhou, W. Design of experiments and regression modelling in food flavour and sensory analysis: A review. Trends Food Sci. Technol. 71 , 202–215 (2018).

Oladokun, O. et al. The impact of hop bitter acid and polyphenol profiles on the perceived bitterness of beer. Food Chem. 205 , 212–220 (2016).

Linforth, R., Cabannes, M., Hewson, L., Yang, N. & Taylor, A. Effect of fat content on flavor delivery during consumption: An in vivo model. J. Agric. Food Chem. 58 , 6905–6911 (2010).

Guo, S., Na Jom, K. & Ge, Y. Influence of roasting condition on flavor profile of sunflower seeds: A flavoromics approach. Sci. Rep. 9 , 11295 (2019).

Ren, Q. et al. The changes of microbial community and flavor compound in the fermentation process of Chinese rice wine using Fagopyrum tataricum grain as feedstock. Sci. Rep. 9 , 3365 (2019).

Hastie, T., Friedman, J. & Tibshirani, R. The Elements of Statistical Learning. (Springer, New York, NY). https://doi.org/10.1007/978-0-387-21606-5 (2001).

Dietz, C., Cook, D., Huismann, M., Wilson, C. & Ford, R. The multisensory perception of hop essential oil: a review. J. Inst. Brew. 126 , 320–342 (2020).

CAS   Google Scholar  

Roncoroni, Miguel & Verstrepen, Kevin Joan. Belgian Beer: Tested and Tasted. (Lannoo, 2018).

Meilgaard, M. Flavor chemistry of beer: Part II: Flavor and threshold of 239 aroma volatiles. in (1975).

Bokulich, N. A. & Bamforth, C. W. The microbiology of malting and brewing. Microbiol. Mol. Biol. Rev. MMBR 77 , 157–172 (2013).

Dzialo, M. C., Park, R., Steensels, J., Lievens, B. & Verstrepen, K. J. Physiology, ecology and industrial applications of aroma formation in yeast. FEMS Microbiol. Rev. 41 , S95–S128 (2017).

Article   PubMed   PubMed Central   Google Scholar  

Datta, A. et al. Computer-aided food engineering. Nat. Food 3 , 894–904 (2022).

American Society of Brewing Chemists. Beer Methods. (American Society of Brewing Chemists, St. Paul, MN, U.S.A.).

Olaniran, A. O., Hiralal, L., Mokoena, M. P. & Pillay, B. Flavour-active volatile compounds in beer: production, regulation and control. J. Inst. Brew. 123 , 13–23 (2017).

Verstrepen, K. J. et al. Flavor-active esters: Adding fruitiness to beer. J. Biosci. Bioeng. 96 , 110–118 (2003).

Meilgaard, M. C. Flavour chemistry of beer. part I: flavour interaction between principal volatiles. Master Brew. Assoc. Am. Tech. Q 12 , 107–117 (1975).

Briggs, D. E., Boulton, C. A., Brookes, P. A. & Stevens, R. Brewing 227–254. (Woodhead Publishing). https://doi.org/10.1533/9781855739062.227 (2004).

Bossaert, S., Crauwels, S., De Rouck, G. & Lievens, B. The power of sour - A review: Old traditions, new opportunities. BrewingScience 72 , 78–88 (2019).

Google Scholar  

Verstrepen, K. J. et al. Flavor active esters: Adding fruitiness to beer. J. Biosci. Bioeng. 96 , 110–118 (2003).

Snauwaert, I. et al. Microbial diversity and metabolite composition of Belgian red-brown acidic ales. Int. J. Food Microbiol. 221 , 1–11 (2016).

Spitaels, F. et al. The microbial diversity of traditional spontaneously fermented lambic beer. PLoS ONE 9 , e95384 (2014).

Blanco, C. A., Andrés-Iglesias, C. & Montero, O. Low-alcohol Beers: Flavor Compounds, Defects, and Improvement Strategies. Crit. Rev. Food Sci. Nutr. 56 , 1379–1388 (2016).

Jackowski, M. & Trusek, A. Non-Alcohol. beer Prod. – Overv. 20 , 32–38 (2018).

Takoi, K. et al. The contribution of geraniol metabolism to the citrus flavour of beer: Synergy of geraniol and β-citronellol under coexistence with excess linalool. J. Inst. Brew. 116 , 251–260 (2010).

Kroeze, J. H. & Bartoshuk, L. M. Bitterness suppression as revealed by split-tongue taste stimulation in humans. Physiol. Behav. 35 , 779–783 (1985).

Mennella, J. A. et al. A spoonful of sugar helps the medicine go down”: Bitter masking bysucrose among children and adults. Chem. Senses 40 , 17–25 (2015).

Wietstock, P., Kunz, T., Perreira, F. & Methner, F.-J. Metal chelation behavior of hop acids in buffered model systems. BrewingScience 69 , 56–63 (2016).

Sancho, D., Blanco, C. A., Caballero, I. & Pascual, A. Free iron in pale, dark and alcohol-free commercial lager beers. J. Sci. Food Agric. 91 , 1142–1147 (2011).

Rodrigues, H. & Parr, W. V. Contribution of cross-cultural studies to understanding wine appreciation: A review. Food Res. Int. 115 , 251–258 (2019).

Korneva, E. & Blockeel, H. Towards better evaluation of multi-target regression models. in ECML PKDD 2020 Workshops (eds. Koprinska, I. et al.) 353–362 (Springer International Publishing, Cham, 2020). https://doi.org/10.1007/978-3-030-65965-3_23 .

Gastón Ares. Mathematical and Statistical Methods in Food Science and Technology. (Wiley, 2013).

Grinsztajn, L., Oyallon, E. & Varoquaux, G. Why do tree-based models still outperform deep learning on tabular data? Preprint at http://arxiv.org/abs/2207.08815 (2022).

Gries, S. T. Statistics for Linguistics with R: A Practical Introduction. in Statistics for Linguistics with R (De Gruyter Mouton, 2021). https://doi.org/10.1515/9783110718256 .

Lundberg, S. M. et al. From local explanations to global understanding with explainable AI for trees. Nat. Mach. Intell. 2 , 56–67 (2020).

Ickes, C. M. & Cadwallader, K. R. Effects of ethanol on flavor perception in alcoholic beverages. Chemosens. Percept. 10 , 119–134 (2017).

Kato, M. et al. Influence of high molecular weight polypeptides on the mouthfeel of commercial beer. J. Inst. Brew. 127 , 27–40 (2021).

Wauters, R. et al. Novel Saccharomyces cerevisiae variants slow down the accumulation of staling aldehydes and improve beer shelf-life. Food Chem. 398 , 1–11 (2023).

Li, H., Jia, S. & Zhang, W. Rapid determination of low-level sulfur compounds in beer by headspace gas chromatography with a pulsed flame photometric detector. J. Am. Soc. Brew. Chem. 66 , 188–191 (2008).

Dercksen, A., Laurens, J., Torline, P., Axcell, B. C. & Rohwer, E. Quantitative analysis of volatile sulfur compounds in beer using a membrane extraction interface. J. Am. Soc. Brew. Chem. 54 , 228–233 (1996).

Molnar, C. Interpretable Machine Learning: A Guide for Making Black-Box Models Interpretable. (2020).

Zhao, Q. & Hastie, T. Causal interpretations of black-box models. J. Bus. Econ. Stat. Publ. Am. Stat. Assoc. 39 , 272–281 (2019).

Article   MathSciNet   Google Scholar  

Hastie, T., Tibshirani, R. & Friedman, J. The Elements of Statistical Learning. (Springer, 2019).

Labrado, D. et al. Identification by NMR of key compounds present in beer distillates and residual phases after dealcoholization by vacuum distillation. J. Sci. Food Agric. 100 , 3971–3978 (2020).

Lusk, L. T., Kay, S. B., Porubcan, A. & Ryder, D. S. Key olfactory cues for beer oxidation. J. Am. Soc. Brew. Chem. 70 , 257–261 (2012).

Gonzalez Viejo, C., Torrico, D. D., Dunshea, F. R. & Fuentes, S. Development of artificial neural network models to assess beer acceptability based on sensory properties using a robotic pourer: A comparative model approach to achieve an artificial intelligence system. Beverages 5 , 33 (2019).

Gonzalez Viejo, C., Fuentes, S., Torrico, D. D., Godbole, A. & Dunshea, F. R. Chemical characterization of aromas in beer and their effect on consumers liking. Food Chem. 293 , 479–485 (2019).

Gilbert, J. L. et al. Identifying breeding priorities for blueberry flavor using biochemical, sensory, and genotype by environment analyses. PLOS ONE 10 , 1–21 (2015).

Goulet, C. et al. Role of an esterase in flavor volatile variation within the tomato clade. Proc. Natl. Acad. Sci. 109 , 19009–19014 (2012).

Article   ADS   CAS   PubMed   PubMed Central   Google Scholar  

Borisov, V. et al. Deep Neural Networks and Tabular Data: A Survey. IEEE Trans. Neural Netw. Learn. Syst. 1–21 https://doi.org/10.1109/TNNLS.2022.3229161 (2022).

Statista. Statista Consumer Market Outlook: Beer - Worldwide.

Seitz, H. K. & Stickel, F. Molecular mechanisms of alcoholmediated carcinogenesis. Nat. Rev. Cancer 7 , 599–612 (2007).

Voordeckers, K. et al. Ethanol exposure increases mutation rate through error-prone polymerases. Nat. Commun. 11 , 3664 (2020).

Goelen, T. et al. Bacterial phylogeny predicts volatile organic compound composition and olfactory response of an aphid parasitoid. Oikos 129 , 1415–1428 (2020).

Article   ADS   Google Scholar  

Reher, T. et al. Evaluation of hop (Humulus lupulus) as a repellent for the management of Drosophila suzukii. Crop Prot. 124 , 104839 (2019).

Stein, S. E. An integrated method for spectrum extraction and compound identification from gas chromatography/mass spectrometry data. J. Am. Soc. Mass Spectrom. 10 , 770–781 (1999).

American Society of Brewing Chemists. Sensory Analysis Methods. (American Society of Brewing Chemists, St. Paul, MN, U.S.A., 1992).

McAuley, J., Leskovec, J. & Jurafsky, D. Learning Attitudes and Attributes from Multi-Aspect Reviews. Preprint at https://doi.org/10.48550/arXiv.1210.3926 (2012).

Meilgaard, M. C., Carr, B. T. & Carr, B. T. Sensory Evaluation Techniques. (CRC Press, Boca Raton). https://doi.org/10.1201/b16452 (2014).

Schreurs, M. et al. Data from: Predicting and improving complex beer flavor through machine learning. Zenodo https://doi.org/10.5281/zenodo.10653704 (2024).

Download references

Acknowledgements

We thank all lab members for their discussions and thank all tasting panel members for their contributions. Special thanks go out to Dr. Karin Voordeckers for her tremendous help in proofreading and improving the manuscript. M.S. was supported by a Baillet-Latour fellowship, L.C. acknowledges financial support from KU Leuven (C16/17/006), F.A.T. was supported by a PhD fellowship from FWO (1S08821N). Research in the lab of K.J.V. is supported by KU Leuven, FWO, VIB, VLAIO and the Brewing Science Serves Health Fund. Research in the lab of T.W. is supported by FWO (G.0A51.15) and KU Leuven (C16/17/006).

Author information

These authors contributed equally: Michiel Schreurs, Supinya Piampongsant, Miguel Roncoroni.

Authors and Affiliations

VIB—KU Leuven Center for Microbiology, Gaston Geenslaan 1, B-3001, Leuven, Belgium

Michiel Schreurs, Supinya Piampongsant, Miguel Roncoroni, Lloyd Cool, Beatriz Herrera-Malaver, Florian A. Theßeling & Kevin J. Verstrepen

CMPG Laboratory of Genetics and Genomics, KU Leuven, Gaston Geenslaan 1, B-3001, Leuven, Belgium

Leuven Institute for Beer Research (LIBR), Gaston Geenslaan 1, B-3001, Leuven, Belgium

Laboratory of Socioecology and Social Evolution, KU Leuven, Naamsestraat 59, B-3000, Leuven, Belgium

Lloyd Cool, Christophe Vanderaa & Tom Wenseleers

VIB Bioinformatics Core, VIB, Rijvisschestraat 120, B-9052, Ghent, Belgium

Łukasz Kreft & Alexander Botzki

AB InBev SA/NV, Brouwerijplein 1, B-3000, Leuven, Belgium

Philippe Malcorps & Luk Daenen

You can also search for this author in PubMed   Google Scholar

Contributions

S.P., M.S. and K.J.V. conceived the experiments. S.P., M.S. and K.J.V. designed the experiments. S.P., M.S., M.R., B.H. and F.A.T. performed the experiments. S.P., M.S., L.C., C.V., L.K., A.B., P.M., L.D., T.W. and K.J.V. contributed analysis ideas. S.P., M.S., L.C., C.V., T.W. and K.J.V. analyzed the data. All authors contributed to writing the manuscript.

Corresponding author

Correspondence to Kevin J. Verstrepen .

Ethics declarations

Competing interests.

K.J.V. is affiliated with bar.on. The other authors declare no competing interests.

Peer review

Peer review information.

Nature Communications thanks Florian Bauer, Andrew John Macintosh and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. A peer review file is available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary information, peer review file, description of additional supplementary files, supplementary data 1, supplementary data 2, supplementary data 3, supplementary data 4, supplementary data 5, supplementary data 6, supplementary data 7, reporting summary, source data, source data, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Schreurs, M., Piampongsant, S., Roncoroni, M. et al. Predicting and improving complex beer flavor through machine learning. Nat Commun 15 , 2368 (2024). https://doi.org/10.1038/s41467-024-46346-0

Download citation

Received : 30 October 2023

Accepted : 21 February 2024

Published : 26 March 2024

DOI : https://doi.org/10.1038/s41467-024-46346-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing: Translational Research newsletter — top stories in biotechnology, drug discovery and pharma.

share this!

April 5, 2024

This article has been reviewed according to Science X's editorial process and policies . Editors have highlighted the following attributes while ensuring the content's credibility:

fact-checked

trusted source

People who work from home are less likely to get pay rises and promotions, finds research

by Tony Trueman, British Sociological Association

People who work from home all or part of the time are less likely to get pay rises and promotions, the first post-COVID research project into the WFH phenomenon has found.

A survey of 937 UK managers found that they were 11% less likely to give a promotion to staff who worked entirely from home than to those who were completely office-based.

Hybrid workers—those working partly in the office and partly at home—were on average 7% less likely to be promoted.

Managers were 9% less likely to give a pay rise to staff working entirely from home than to those who were completely office-based, and 7% less likely to give one to hybrid workers.

The research found a gender gap : managers were 15% less likely to promote men who worked entirely from home than those who were completely office-based, and 10% less likely to give a pay increase. The figures for women were 7% and 8%, respectively.

Agnieszka Kasperska, Professor Anna Matysiak and Dr. Ewa Cukrowska-Torzewska, all from the University of Warsaw, carried out the research, the first study since the lockdowns began in the UK on how working from home affects careers.

They presented 937 managers employed in various businesses and industries within the UK with two profiles of hypothetical full-time staff members who worked either five days at the office a week, five days at home, or three days at the office and two at home. The managers then chose which one they were likely to promote, and also which one they would give a pay rise to.

Agnieszka Kasperska told the British Sociological Association's online annual conference 5 April, 2024 that "the recent COVID-19 pandemic has triggered a substantial shift towards working from home, potentially influencing employers' attitudes and companies' readiness to manage remote employees.

"However, our findings indicate that individuals working from home still encounter career penalties, irrespective of the widespread adoption of this mode of work.

"Both male and female remote workers experience career penalties, but they are substantially larger for men."

They found that in organizations with very demanding work cultures, the managers were around 30% less likely to promote and 19% less likely to give a pay rise to men who worked entirely from home than to men who worked solely in the office. The figures for women were 15% and 19%, respectively. In organizations with more supportive environments, no penalty to staff for flexible working was found.

"In more supportive organizations, so where there is less pressure and long working days and where family-friendly policies exist, we don't find such negative consequences of remote work," she said.

The profiles given to managers also included different characteristics such as gender, age, experience in the sector, skill level and family situation. The raw data were adjusted to remove the influence of these from the final results so that the effects of working from home could be studied in isolation.

Provided by British Sociological Association

Explore further

Feedback to editors

Study: Focusing immediately on the benefits of waiting might help people improve their self-control

5 hours ago

Utilizing palladium for addressing contact issues of buried oxide thin film transistors

Ocean waves propel PFAS back to land, new study finds

eDNA methods give a real-time look at coral reef health

Prehistoric henge reveals centuries-old sacred site in Lincolnshire

Kerr-enhanced optical spring for next-generation gravitational wave detectors

6 hours ago

New micromaterial releases nanoparticles that selectively destroy cancer cells

Simulating magnetization in a Heisenberg quantum spin chain

Chemical reactions can scramble quantum information as well as black holes

Researchers say neutron stars are key to understanding elusive dark matter

7 hours ago

Relevant PhysicsForums posts

Today's fusion music: t square, cassiopeia, rei & kanade sato, cover songs versus the original track, which ones are better.

2 hours ago

Interesting anecdotes in the history of physics?

Apr 4, 2024

What are your favorite Disco "Classics"?

Apr 2, 2024

How did ‘concern’ semantically shift to mean ‘commercial enterprise' ?

Wars of the roses (lancaster, red rose - york, white rose), 1455-1487.

More from Art, Music, History, and Linguistics

Related Stories

Working from home is producing economic benefits return-to-office rules would quash

Feb 22, 2024

Over a third of office workers are hybrid 'misfits'

Oct 19, 2022

Couples don't have the same experience when both work from home

Jan 3, 2023

How COVID has changed the way Americans work, and how much money they have

Jan 25, 2024

What's it worth to work from home? For some, it's as much as one-third of their wage

Jan 16, 2024

Working from home has worked for people with disability. The back-to-the-office push could wind back gains

Jul 25, 2023

Recommended for you

Giving eyeglasses to workers in developing countries boosts income

9 hours ago

More than money, family and community bonds prep teens for college success: Study

8 hours ago

Study on the psychology of blame points to promising strategies for reducing animosity within political divide

Apr 3, 2024

Characterizing social networks by the company they keep

Song lyrics have become simpler and more repetitive since 1980, study finds

Mar 28, 2024

Let us know if there is a problem with our content

Use this form if you have come across a typo, inaccuracy or would like to send an edit request for the content on this page. For general inquiries, please use our contact form . For general feedback, use the public comments section below (please adhere to guidelines ).

Please select the most appropriate category to facilitate processing of your request

Thank you for taking time to provide your feedback to the editors.

Your feedback is important to us. However, we do not guarantee individual replies due to the high volume of messages.

E-mail the story

Your email address is used only to let the recipient know who sent the email. Neither your address nor the recipient's address will be used for any other purpose. The information you enter will appear in your e-mail message and is not retained by Phys.org in any form.

Newsletter sign up

Get weekly and/or daily updates delivered to your inbox. You can unsubscribe at any time and we'll never share your details to third parties.

More information Privacy policy

Donate and enjoy an ad-free experience

We keep our content available to everyone. Consider supporting Science X's mission by getting a premium account.

E-mail newsletter

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Turk J Anaesthesiol Reanim
• v.44(4); 2016 Aug

What is Scientific Research and How Can it be Done?

Scientific researches are studies that should be systematically planned before performing them. In this review, classification and description of scientific studies, planning stage randomisation and bias are explained.

Research conducted for the purpose of contributing towards science by the systematic collection, interpretation and evaluation of data and that, too, in a planned manner is called scientific research: a researcher is the one who conducts this research. The results obtained from a small group through scientific studies are socialised, and new information is revealed with respect to diagnosis, treatment and reliability of applications. The purpose of this review is to provide information about the definition, classification and methodology of scientific research.

Before beginning the scientific research, the researcher should determine the subject, do planning and specify the methodology. In the Declaration of Helsinki, it is stated that ‘the primary purpose of medical researches on volunteers is to understand the reasons, development and effects of diseases and develop protective, diagnostic and therapeutic interventions (method, operation and therapies). Even the best proven interventions should be evaluated continuously by investigations with regard to reliability, effectiveness, efficiency, accessibility and quality’ ( 1 ).

The questions, methods of response to questions and difficulties in scientific research may vary, but the design and structure are generally the same ( 2 ).

Classification of Scientific Research

Scientific research can be classified in several ways. Classification can be made according to the data collection techniques based on causality, relationship with time and the medium through which they are applied.

• Observational
• Experimental
• Descriptive
• Retrospective
• Prospective
• Cross-sectional
• Social descriptive research ( 3 )

Another method is to classify the research according to its descriptive or analytical features. This review is written according to this classification method.

I. Descriptive research

• Case series
• Surveillance studies

II. Analytical research

• Observational studies: cohort, case control and cross- sectional research
• Interventional research: quasi-experimental and clinical research
• Case Report: it is the most common type of descriptive study. It is the examination of a single case having a different quality in the society, e.g. conducting general anaesthesia in a pregnant patient with mucopolysaccharidosis.
• Case Series: it is the description of repetitive cases having common features. For instance; case series involving interscapular pain related to neuraxial labour analgesia. Interestingly, malignant hyperthermia cases are not accepted as case series since they are rarely seen during historical development.
• Surveillance Studies: these are the results obtained from the databases that follow and record a health problem for a certain time, e.g. the surveillance of cross-infections during anaesthesia in the intensive care unit.

Moreover, some studies may be experimental. After the researcher intervenes, the researcher waits for the result, observes and obtains data. Experimental studies are, more often, in the form of clinical trials or laboratory animal trials ( 2 ).

Analytical observational research can be classified as cohort, case-control and cross-sectional studies.

Firstly, the participants are controlled with regard to the disease under investigation. Patients are excluded from the study. Healthy participants are evaluated with regard to the exposure to the effect. Then, the group (cohort) is followed-up for a sufficient period of time with respect to the occurrence of disease, and the progress of disease is studied. The risk of the healthy participants getting sick is considered an incident. In cohort studies, the risk of disease between the groups exposed and not exposed to the effect is calculated and rated. This rate is called relative risk. Relative risk indicates the strength of exposure to the effect on the disease.

Cohort research may be observational and experimental. The follow-up of patients prospectively is called a prospective cohort study . The results are obtained after the research starts. The researcher’s following-up of cohort subjects from a certain point towards the past is called a retrospective cohort study . Prospective cohort studies are more valuable than retrospective cohort studies: this is because in the former, the researcher observes and records the data. The researcher plans the study before the research and determines what data will be used. On the other hand, in retrospective studies, the research is made on recorded data: no new data can be added.

In fact, retrospective and prospective studies are not observational. They determine the relationship between the date on which the researcher has begun the study and the disease development period. The most critical disadvantage of this type of research is that if the follow-up period is long, participants may leave the study at their own behest or due to physical conditions. Cohort studies that begin after exposure and before disease development are called ambidirectional studies . Public healthcare studies generally fall within this group, e.g. lung cancer development in smokers.

• Case-Control Studies: these studies are retrospective cohort studies. They examine the cause and effect relationship from the effect to the cause. The detection or determination of data depends on the information recorded in the past. The researcher has no control over the data ( 2 ).

Cross-sectional studies are advantageous since they can be concluded relatively quickly. It may be difficult to obtain a reliable result from such studies for rare diseases ( 2 ).

Cross-sectional studies are characterised by timing. In such studies, the exposure and result are simultaneously evaluated. While cross-sectional studies are restrictedly used in studies involving anaesthesia (since the process of exposure is limited), they can be used in studies conducted in intensive care units.

• Quasi-Experimental Research: they are conducted in cases in which a quick result is requested and the participants or research areas cannot be randomised, e.g. giving hand-wash training and comparing the frequency of nosocomial infections before and after hand wash.
• Clinical Research: they are prospective studies carried out with a control group for the purpose of comparing the effect and value of an intervention in a clinical case. Clinical study and research have the same meaning. Drugs, invasive interventions, medical devices and operations, diets, physical therapy and diagnostic tools are relevant in this context ( 6 ).

Clinical studies are conducted by a responsible researcher, generally a physician. In the research team, there may be other healthcare staff besides physicians. Clinical studies may be financed by healthcare institutes, drug companies, academic medical centres, volunteer groups, physicians, healthcare service providers and other individuals. They may be conducted in several places including hospitals, universities, physicians’ offices and community clinics based on the researcher’s requirements. The participants are made aware of the duration of the study before their inclusion. Clinical studies should include the evaluation of recommendations (drug, device and surgical) for the treatment of a disease, syndrome or a comparison of one or more applications; finding different ways for recognition of a disease or case and prevention of their recurrence ( 7 ).

Clinical Research

In this review, clinical research is explained in more detail since it is the most valuable study in scientific research.

Clinical research starts with forming a hypothesis. A hypothesis can be defined as a claim put forward about the value of a population parameter based on sampling. There are two types of hypotheses in statistics.

• H 0 hypothesis is called a control or null hypothesis. It is the hypothesis put forward in research, which implies that there is no difference between the groups under consideration. If this hypothesis is rejected at the end of the study, it indicates that a difference exists between the two treatments under consideration.
• H 1 hypothesis is called an alternative hypothesis. It is hypothesised against a null hypothesis, which implies that a difference exists between the groups under consideration. For example, consider the following hypothesis: drug A has an analgesic effect. Control or null hypothesis (H 0 ): there is no difference between drug A and placebo with regard to the analgesic effect. The alternative hypothesis (H 1 ) is applicable if a difference exists between drug A and placebo with regard to the analgesic effect.

The planning phase comes after the determination of a hypothesis. A clinical research plan is called a protocol . In a protocol, the reasons for research, number and qualities of participants, tests to be applied, study duration and what information to be gathered from the participants should be found and conformity criteria should be developed.

The selection of participant groups to be included in the study is important. Inclusion and exclusion criteria of the study for the participants should be determined. Inclusion criteria should be defined in the form of demographic characteristics (age, gender, etc.) of the participant group and the exclusion criteria as the diseases that may influence the study, age ranges, cases involving pregnancy and lactation, continuously used drugs and participants’ cooperation.

The next stage is methodology. Methodology can be grouped under subheadings, namely, the calculation of number of subjects, blinding (masking), randomisation, selection of operation to be applied, use of placebo and criteria for stopping and changing the treatment.

I. Calculation of the Number of Subjects

The entire source from which the data are obtained is called a universe or population . A small group selected from a certain universe based on certain rules and which is accepted to highly represent the universe from which it is selected is called a sample and the characteristics of the population from which the data are collected are called variables. If data is collected from the entire population, such an instance is called a parameter . Conducting a study on the sample rather than the entire population is easier and less costly. Many factors influence the determination of the sample size. Firstly, the type of variable should be determined. Variables are classified as categorical (qualitative, non-numerical) or numerical (quantitative). Individuals in categorical variables are classified according to their characteristics. Categorical variables are indicated as nominal and ordinal (ordered). In nominal variables, the application of a category depends on the researcher’s preference. For instance, a female participant can be considered first and then the male participant, or vice versa. An ordinal (ordered) variable is ordered from small to large or vice versa (e.g. ordering obese patients based on their weights-from the lightest to the heaviest or vice versa). A categorical variable may have more than one characteristic: such variables are called binary or dichotomous (e.g. a participant may be both female and obese).

If the variable has numerical (quantitative) characteristics and these characteristics cannot be categorised, then it is called a numerical variable. Numerical variables are either discrete or continuous. For example, the number of operations with spinal anaesthesia represents a discrete variable. The haemoglobin value or height represents a continuous variable.

Statistical analyses that need to be employed depend on the type of variable. The determination of variables is necessary for selecting the statistical method as well as software in SPSS. While categorical variables are presented as numbers and percentages, numerical variables are represented using measures such as mean and standard deviation. It may be necessary to use mean in categorising some cases such as the following: even though the variable is categorical (qualitative, non-numerical) when Visual Analogue Scale (VAS) is used (since a numerical value is obtained), it is classified as a numerical variable: such variables are averaged.

Clinical research is carried out on the sample and generalised to the population. Accordingly, the number of samples should be correctly determined. Different sample size formulas are used on the basis of the statistical method to be used. When the sample size increases, error probability decreases. The sample size is calculated based on the primary hypothesis. The determination of a sample size before beginning the research specifies the power of the study. Power analysis enables the acquisition of realistic results in the research, and it is used for comparing two or more clinical research methods.

Because of the difference in the formulas used in calculating power analysis and number of samples for clinical research, it facilitates the use of computer programs for making calculations.

It is necessary to know certain parameters in order to calculate the number of samples by power analysis.

• Type-I (α) and type-II (β) error levels
• Difference between groups (d-difference) and effect size (ES)
• Distribution ratio of groups
• Direction of research hypothesis (H1)

a. Type-I (α) and Type-II (β) Error (β) Levels

Two types of errors can be made while accepting or rejecting H 0 hypothesis in a hypothesis test. Type-I error (α) level is the probability of finding a difference at the end of the research when there is no difference between the two applications. In other words, it is the rejection of the hypothesis when H 0 is actually correct and it is known as α error or p value. For instance, when the size is determined, type-I error level is accepted as 0.05 or 0.01.

Another error that can be made during a hypothesis test is a type-II error. It is the acceptance of a wrongly hypothesised H 0 hypothesis. In fact, it is the probability of failing to find a difference when there is a difference between the two applications. The power of a test is the ability of that test to find a difference that actually exists. Therefore, it is related to the type-II error level.

Since the type-II error risk is expressed as β, the power of the test is defined as 1–β. When a type-II error is 0.20, the power of the test is 0.80. Type-I (α) and type-II (β) errors can be intentional. The reason to intentionally make such an error is the necessity to look at the events from the opposite perspective.

b. Difference between Groups and ES

ES is defined as the state in which statistical difference also has clinically significance: ES≥0.5 is desirable. The difference between groups is the absolute difference between the groups compared in clinical research.

c. Allocation Ratio of Groups

The allocation ratio of groups is effective in determining the number of samples. If the number of samples is desired to be determined at the lowest level, the rate should be kept as 1/1.

d. Direction of Hypothesis (H1)

The direction of hypothesis in clinical research may be one-sided or two-sided. While one-sided hypotheses hypothesis test differences in the direction of size, two-sided hypotheses hypothesis test differences without direction. The power of the test in two-sided hypotheses is lower than one-sided hypotheses.

After these four variables are determined, they are entered in the appropriate computer program and the number of samples is calculated. Statistical packaged software programs such as Statistica, NCSS and G-Power may be used for power analysis and calculating the number of samples. When the samples size is calculated, if there is a decrease in α, difference between groups, ES and number of samples, then the standard deviation increases and power decreases. The power in two-sided hypothesis is lower. It is ethically appropriate to consider the determination of sample size, particularly in animal experiments, at the beginning of the study. The phase of the study is also important in the determination of number of subjects to be included in drug studies. Usually, phase-I studies are used to determine the safety profile of a drug or product, and they are generally conducted on a few healthy volunteers. If no unacceptable toxicity is detected during phase-I studies, phase-II studies may be carried out. Phase-II studies are proof-of-concept studies conducted on a larger number (100–500) of volunteer patients. When the effectiveness of the drug or product is evident in phase-II studies, phase-III studies can be initiated. These are randomised, double-blinded, placebo or standard treatment-controlled studies. Volunteer patients are periodically followed-up with respect to the effectiveness and side effects of the drug. It can generally last 1–4 years and is valuable during licensing and releasing the drug to the general market. Then, phase-IV studies begin in which long-term safety is investigated (indication, dose, mode of application, safety, effectiveness, etc.) on thousands of volunteer patients.

II. Blinding (Masking) and Randomisation Methods

When the methodology of clinical research is prepared, precautions should be taken to prevent taking sides. For this reason, techniques such as randomisation and blinding (masking) are used. Comparative studies are the most ideal ones in clinical research.

Blinding Method

A case in which the treatments applied to participants of clinical research should be kept unknown is called the blinding method . If the participant does not know what it receives, it is called a single-blind study; if even the researcher does not know, it is called a double-blind study. When there is a probability of knowing which drug is given in the order of application, when uninformed staff administers the drug, it is called in-house blinding. In case the study drug is known in its pharmaceutical form, a double-dummy blinding test is conducted. Intravenous drug is given to one group and a placebo tablet is given to the comparison group; then, the placebo tablet is given to the group that received the intravenous drug and intravenous drug in addition to placebo tablet is given to the comparison group. In this manner, each group receives both the intravenous and tablet forms of the drug. In case a third party interested in the study is involved and it also does not know about the drug (along with the statistician), it is called third-party blinding.

Randomisation Method

The selection of patients for the study groups should be random. Randomisation methods are used for such selection, which prevent conscious or unconscious manipulations in the selection of patients ( 8 ).

No factor pertaining to the patient should provide preference of one treatment to the other during randomisation. This characteristic is the most important difference separating randomised clinical studies from prospective and synchronous studies with experimental groups. Randomisation strengthens the study design and enables the determination of reliable scientific knowledge ( 2 ).

The easiest method is simple randomisation, e.g. determination of the type of anaesthesia to be administered to a patient by tossing a coin. In this method, when the number of samples is kept high, a balanced distribution is created. When the number of samples is low, there will be an imbalance between the groups. In this case, stratification and blocking have to be added to randomisation. Stratification is the classification of patients one or more times according to prognostic features determined by the researcher and blocking is the selection of a certain number of patients for each stratification process. The number of stratification processes should be determined at the beginning of the study.

As the number of stratification processes increases, performing the study and balancing the groups become difficult. For this reason, stratification characteristics and limitations should be effectively determined at the beginning of the study. It is not mandatory for the stratifications to have equal intervals. Despite all the precautions, an imbalance might occur between the groups before beginning the research. In such circumstances, post-stratification or restandardisation may be conducted according to the prognostic factors.

The main characteristic of applying blinding (masking) and randomisation is the prevention of bias. Therefore, it is worthwhile to comprehensively examine bias at this stage.

Bias and Chicanery

While conducting clinical research, errors can be introduced voluntarily or involuntarily at a number of stages, such as design, population selection, calculating the number of samples, non-compliance with study protocol, data entry and selection of statistical method. Bias is taking sides of individuals in line with their own decisions, views and ideological preferences ( 9 ). In order for an error to lead to bias, it has to be a systematic error. Systematic errors in controlled studies generally cause the results of one group to move in a different direction as compared to the other. It has to be understood that scientific research is generally prone to errors. However, random errors (or, in other words, ‘the luck factor’-in which bias is unintended-do not lead to bias ( 10 ).

Another issue, which is different from bias, is chicanery. It is defined as voluntarily changing the interventions, results and data of patients in an unethical manner or copying data from other studies. Comparatively, bias may not be done consciously.

In case unexpected results or outliers are found while the study is analysed, if possible, such data should be re-included into the study since the complete exclusion of data from a study endangers its reliability. In such a case, evaluation needs to be made with and without outliers. It is insignificant if no difference is found. However, if there is a difference, the results with outliers are re-evaluated. If there is no error, then the outlier is included in the study (as the outlier may be a result). It should be noted that re-evaluation of data in anaesthesiology is not possible.

Statistical evaluation methods should be determined at the design stage so as not to encounter unexpected results in clinical research. The data should be evaluated before the end of the study and without entering into details in research that are time-consuming and involve several samples. This is called an interim analysis . The date of interim analysis should be determined at the beginning of the study. The purpose of making interim analysis is to prevent unnecessary cost and effort since it may be necessary to conclude the research after the interim analysis, e.g. studies in which there is no possibility to validate the hypothesis at the end or the occurrence of different side effects of the drug to be used. The accuracy of the hypothesis and number of samples are compared. Statistical significance levels in interim analysis are very important. If the data level is significant, the hypothesis is validated even if the result turns out to be insignificant after the date of the analysis.

Another important point to be considered is the necessity to conclude the participants’ treatment within the period specified in the study protocol. When the result of the study is achieved earlier and unexpected situations develop, the treatment is concluded earlier. Moreover, the participant may quit the study at its own behest, may die or unpredictable situations (e.g. pregnancy) may develop. The participant can also quit the study whenever it wants, even if the study has not ended ( 7 ).

In case the results of a study are contrary to already known or expected results, the expected quality level of the study suggesting the contradiction may be higher than the studies supporting what is known in that subject. This type of bias is called confirmation bias. The presence of well-known mechanisms and logical inference from them may create problems in the evaluation of data. This is called plausibility bias.

Another type of bias is expectation bias. If a result different from the known results has been achieved and it is against the editor’s will, it can be challenged. Bias may be introduced during the publication of studies, such as publishing only positive results, selection of study results in a way to support a view or prevention of their publication. Some editors may only publish research that extols only the positive results or results that they desire.

Bias may be introduced for advertisement or economic reasons. Economic pressure may be applied on the editor, particularly in the cases of studies involving drugs and new medical devices. This is called commercial bias.

In recent years, before beginning a study, it has been recommended to record it on the Web site www.clinicaltrials.gov for the purpose of facilitating systematic interpretation and analysis in scientific research, informing other researchers, preventing bias, provision of writing in a standard format, enhancing contribution of research results to the general literature and enabling early intervention of an institution for support. This Web site is a service of the US National Institutes of Health.

The last stage in the methodology of clinical studies is the selection of intervention to be conducted. Placebo use assumes an important place in interventions. In Latin, placebo means ‘I will be fine’. In medical literature, it refers to substances that are not curative, do not have active ingredients and have various pharmaceutical forms. Although placebos do not have active drug characteristic, they have shown effective analgesic characteristics, particularly in algology applications; further, its use prevents bias in comparative studies. If a placebo has a positive impact on a participant, it is called the placebo effect ; on the contrary, if it has a negative impact, it is called the nocebo effect . Another type of therapy that can be used in clinical research is sham application. Although a researcher does not cure the patient, the researcher may compare those who receive therapy and undergo sham. It has been seen that sham therapies also exhibit a placebo effect. In particular, sham therapies are used in acupuncture applications ( 11 ). While placebo is a substance, sham is a type of clinical application.

Ethically, the patient has to receive appropriate therapy. For this reason, if its use prevents effective treatment, it causes great problem with regard to patient health and legalities.

Before medical research is conducted with human subjects, predictable risks, drawbacks and benefits must be evaluated for individuals or groups participating in the study. Precautions must be taken for reducing the risk to a minimum level. The risks during the study should be followed, evaluated and recorded by the researcher ( 1 ).

After the methodology for a clinical study is determined, dealing with the ‘Ethics Committee’ forms the next stage. The purpose of the ethics committee is to protect the rights, safety and well-being of volunteers taking part in the clinical research, considering the scientific method and concerns of society. The ethics committee examines the studies presented in time, comprehensively and independently, with regard to ethics and science; in line with the Declaration of Helsinki and following national and international standards concerning ‘Good Clinical Practice’. The method to be followed in the formation of the ethics committee should be developed without any kind of prejudice and to examine the applications with regard to ethics and science within the framework of the ethics committee, Regulation on Clinical Trials and Good Clinical Practice ( www.iku.com ). The necessary documents to be presented to the ethics committee are research protocol, volunteer consent form, budget contract, Declaration of Helsinki, curriculum vitae of researchers, similar or explanatory literature samples, supporting institution approval certificate and patient follow-up form.

Only one sister/brother, mother, father, son/daughter and wife/husband can take charge in the same ethics committee. A rector, vice rector, dean, deputy dean, provincial healthcare director and chief physician cannot be members of the ethics committee.

Members of the ethics committee can work as researchers or coordinators in clinical research. However, during research meetings in which members of the ethics committee are researchers or coordinators, they must leave the session and they cannot sign-off on decisions. If the number of members in the ethics committee for a particular research is so high that it is impossible to take a decision, the clinical research is presented to another ethics committee in the same province. If there is no ethics committee in the same province, an ethics committee in the closest settlement is found.

Thereafter, researchers need to inform the participants using an informed consent form. This form should explain the content of clinical study, potential benefits of the study, alternatives and risks (if any). It should be easy, comprehensible, conforming to spelling rules and written in plain language understandable by the participant.

This form assists the participants in taking a decision regarding participation in the study. It should aim to protect the participants. The participant should be included in the study only after it signs the informed consent form; the participant can quit the study whenever required, even when the study has not ended ( 7 ).

Peer-review: Externally peer-reviewed.

Author Contributions: Concept - C.Ö.Ç., A.D.; Design - C.Ö.Ç.; Supervision - A.D.; Resource - C.Ö.Ç., A.D.; Materials - C.Ö.Ç., A.D.; Analysis and/or Interpretation - C.Ö.Ç., A.D.; Literature Search - C.Ö.Ç.; Writing Manuscript - C.Ö.Ç.; Critical Review - A.D.; Other - C.Ö.Ç., A.D.

Conflict of Interest: No conflict of interest was declared by the authors.

Financial Disclosure: The authors declared that this study has received no financial support.