use chatgpt for research paper

'ZDNET Recommends': What exactly does it mean?

ZDNET's recommendations are based on many hours of testing, research, and comparison shopping. We gather data from the best available sources, including vendor and retailer listings as well as other relevant and independent reviews sites. And we pore over customer reviews to find out what matters to real people who already own and use the products and services we’re assessing.

When you click through from our site to a retailer and buy a product or service, we may earn affiliate commissions. This helps support our work, but does not affect what we cover or how, and it does not affect the price you pay. Neither ZDNET nor the author are compensated for these independent reviews. Indeed, we follow strict guidelines that ensure our editorial content is never influenced by advertisers.

ZDNET's editorial team writes on behalf of you, our reader. Our goal is to deliver the most accurate information and the most knowledgeable advice possible in order to help you make smarter buying decisions on tech gear and a wide array of products and services. Our editors thoroughly review and fact-check every article to ensure that our content meets the highest standards. If we have made an error or published misleading information, we will correct or clarify the article. If you see inaccuracies in our content, please report the mistake via this form .

How to use ChatGPT to summarize a book, article, or research paper

AI chatbots like ChatGPT can be used to make summarizing long articles, research papers, and books an easier job. If you're tasked with writing a summary for school or work about a body of written text, and you're pinched for time, ChatGPT can help you understand the necessary components.

You should remember that ChatGPT is a tool that can help you further understand a topic, and it may not be in your best interest to have it write your work for you.

Also:   How to make ChatGPT provide sources and citations

If you're a student writing a research paper, someone who is keen to discover more about a lengthy article, or someone who wants to dive into a complicated subject, you can use ChatGPT to simplify the process.

How ChatGPT can create summaries for you

Materials needed : You'll need a device that can connect to the internet, an OpenAI account , and a basic understanding of the article, research paper, or book you want to summarize. 

Also:  This AI chatbot can sum up any PDF and answer any question you have about it

The process should take about one to three minutes.

1. Find your article, paper, or book to summarize

If you need ChatGPT to help summarize an article or research paper, find the body of text online and keep it open in a separate tab.

2. Open your web browser and log in to OpenAI

Open your web browser of choice and type in chat.openai.com/chat . 

Also: How to use ChatGPT to write Excel formulas

Log in or sign up for an account. 

3. Type your request

In the chat box, type in TLDR: followed by the title of the book you want summarized. TLDR stands for too long, didn't read. 

Also: How to use ChatGPT: Everything you need to know

You can also ask ChatGPT: "Summarize [book title]." 

You cannot paste a URL into ChatGPT. If you do so, you'll get a response explaining that the chatbot cannot access websites. If you want ChatGPT to summarize research, you'll need a basic understanding of the topic to ensure you aren't including inaccurate information in your report.

For example, if you're writing a paper about how twin sibling dynamics affect their marriages, you can ask ChatGPT: "Explain how twin relationships affect their marriages."

Also:   The best AI chatbots: ChatGPT and other noteworthy alternatives 

ChatGPT will offer you a few bullet points of both positive and negative effects a twin bond can have on the twins' marriages. You'll still need to read key points of the research, as ChatGPT won't provide you with specific statistics or scientific conclusions. But ChatGPT can help you understand the context surrounding the research you're interested in.

If you're having trouble comprehending specific passages in an article, book, or research paper, you can copy parts of the text and paste them into ChatGPT.

Think of this method as reshuffling the words you're currently reading to help you make more sense of the text in front of you.

This is an accurate summary of 'The Hunger Games: Mockingjay'.

What are ChatGPT's limitations?

If you're using ChatGPT to summarize an article, book, or piece of research, keep in mind that ChatGPT isn't aware of events that occurred after September 2021. 

Also:   4 things Claude AI can do that ChatGPT can't

ChatGPT is a large language model that uses queues and millions of data points to mimic human responses. This form of mimicry is why ChatGPT will answer questions even when it doesn't output the correct answer. So, make sure you're not using any information from ChatGPT without fact-checking it.

If you try to get around this obstacle and provide ChatGPT with an article that contains information post-2021, it might hallucinate. Here, I asked the chatbot to summarize an article about a new app I wrote about , and it made up a few details. 

Lemon8 is a new app from TikTok's parent company, ByteDance. Although the TikTok trend may exist, that's not what the article is about.

Can ChatGPT summarize a PDF?

Copying and pasting a URL into ChatGPT won't yield you the best results. If there is a specific paragraph or sentence you're struggling with, you can copy and paste it into ChatGPT and ask it to explain it to you. Still, it's best to read the PDF and use the chatbot as a summary tool and not as an educator. 

Also:  How to use ChatGPT to write code

If you're looking for an AI chatbot that you can regularly rely on to give you an accurate summary of a PDF, consider using ChatPDF . You can summarize up to three PDFs of up to 120 pages per day, and an upgraded plan is available for $5 per month.

Can ChatGPT summarize an email thread?

Sort of. If you want to copy and paste every single email, ChatGPT can summarize the thread's contents for you. It would be more helpful to scan an email thread yourself and ask ChatGPT to help you write a response based on the key points you know about the conversation. 

Editor's note: We've added additional context to the step concerning ChatGPT summarizing articles by URL. 

More on AI tools

How to make chatgpt provide sources and citations, how to use chatgpt, how to use chatgpt to write a cover letter (and why you should).

Upcoming Summer 2024 Application Deadline is May 12, 2024.  

Click here to apply.

Featured Posts

Pace University's Pre-College Program: Our Honest Review

Building a College Portfolio? Here are 8 Things You Should Know

Should You Apply to Catapult Incubator as a High Schooler?

8 Criminal Justice Internships for High School Students

10 Reasons to Apply to Scripps' Research Internship for High School Students

10 Free Business Programs for High School Students

Spike Lab - Is It a Worthwhile Incubator Program for High School Students?

10 Business Programs for High School Students in NYC

11 Summer Film Programs for High School Students

10 Photography Internships for High School Students

How to Use ChatGPT To Write A Research Paper

Over the past three years, we’ve helped thousands of students write research papers with research mentors. And, since the public release of ChatGPT, we’ve been getting questions about how best to use (and not use) this new technology. In this article, we outline the best ways to use ChatGPT from our experience including the ways that ChatGPT can help out academic writing - and the ways that it can be limited. We’re researchers ourselves - our founders met at Harvard while doing a research program - so we’ll show you what we’re seeing and how you can use it for your own papers.

What is ChatGPT?

ChatGPT is an advanced language model developed by OpenAI. It is designed to generate human-like responses to any input you desire, making it a powerful tool for anyone. ChatGPT can understand social contexts, recognize patterns, and generate coherent messages. Here’s an example from our co-cofounder Stephen, when he used ChatGPT to try to recreate a paper one of his students wrote on high school employment.

As you can see from his article, ChatGPT was quite good at building a somewhat credible sounding literature review - but wasn’t as good at identifying accurate information. ChatGPT has gotten better since he wrote that piece (in December, 2022), but it still makes factual mistakes which can be dangerous for a researcher when using ChatGPT.

ChatGPT has already been used to write complex code for algorithms, transcribed new poems and stories, and

7 Ways to Use ChatGPT to Write A Research Paper

Below, we outline a few steps to use ChatGPT when writing a research paper. We begin with thinking about the structure of the paper and then moving to discuss how you can level up the existing writing you’ve done.

1. Summarize Other Research Papers

As an up-and-coming researcher, other scientific papers may be extremely dense and difficult to read. Have ChatGPT summarize the article for you.

Us: Write a paragraph summary of "Acceleration of 1I/‘Oumuamua from radiolytically produced H2 in H2O ice" by Bergner.

ChatGPT: In the paper "Acceleration of 1I/‘Oumuamua from radiolytically produced H2 in H2O ice", Bergner explores the possibility that the interstellar object 'Oumuamua could have been propelled by gas released from its surface due to the decomposition of water ice by cosmic radiation. The author presents a theoretical model and calculations to support this hypothesis and shows that the acceleration observed in 'Oumuamua's trajectory could be explained by the release of hydrogen gas from the object's surface. The study suggests that the radiolytic decomposition of water ice could be a common mechanism for the propulsion of interstellar objects and highlights the importance of further investigating the properties and behavior of such objects.

2. Edit Your Style and Grammar

ChatGPT’s ability to deeply understand English can be used as a grammar checker by inputting any text into the chat window and receiving feedback on any grammatical errors. This can already be done with other existing software, but the most exciting thing is that ChatGPT can transform any input you enter into the prompt.

If you believe that your essay sounds too informal, utilizes rudimentary vocabulary that you want to upgrade, or you want to add more technical terms, you can say just that. Copy your essay into the prompt, with your request, such as “ Use fancier scientific knowledge/terminology to applicable places in my essay. ” or “Edit my essay to increase the brevity of this paragraph.” or “Make this essay more at the level for 8th graders/college-level research paper.”

Outside of research papers, you could even use this to change the tone or style of any writing pieces. “Transform this essay into a long prose in the style of a limerick. “Write this paragraph about my day yesterday in the style of a Ph.D research paper.”

3. Generating an First Outline

Writing a large research paper can be extremely overwhelming, and sometimes, you may not even know where to start. An outline can be extremely helpful, and can be generated by ChatGPT. You can even specialize and customize this outline based on your needs and specialities, as each paper and style of paper will have its unique structures.

Us: Draft an outline for a biology research paper at a college level

I. Introduction

A. Background information on the topic

B. Significance of the research question

C. Research objectives and hypotheses

D. Brief overview of the structure of the paper

II. Literature review

A. Overview of the relevant literature

B. Key concepts and theories related to the topic

C. Previous research on the topic

D. Gaps in the literature and the need for further research

III. Methods

A. Description of the study design and methodology

B. Sampling strategy and sample size

C. Data collection techniques and instrumentation

D. Data analysis methods

… and it continues on until the end of the paper.

4. Build a First Draft to Your Paper

Similar to above, we can have ChatGPT generate ideas and new content based off of your outlines for any research papers. With any idea you may have, you can have ChatGPT write a couple sentences that could kickstart your thinking process or use as experimentation. For example,

Us: Write three sentences on the effect and most prevalent gut microbiota regulating the immune system

ChatGPT: The gut microbiota is known to modulate the immune system via complex interactions with immune cells, including regulatory T cells and antigen-presenting cells. Alterations in gut microbial composition, such as a decrease in beneficial bacteria like Bacteroidetes and Firmicutes, and an increase in pathogenic bacteria, have been linked to several immune-mediated diseases. Therefore, maintaining a healthy balance of gut microbiota is essential for immune homeostasis and preventing autoimmune disorders.

From your outline, you can begin using this to begin bolstering your essay with actual content, utilizing this generated content as a text-filled outline to begin writing the rest of the paper. Be sure to edit, fact check anything that you may not recognize, and do not purely just copy and paste.

5. Expand and Draw Conclusions From Experiments

Sometimes during research, you run into a roadblock and cannot figure out where to go further from your recent experiment. You may just need another person, or better yet, a computer to help you out.

Us: What conclusions can I draw from a western blot and what experiments should I conduct afterwards?

ChatGPT: A Western blot is a widely used technique in molecular biology to detect and quantify specific proteins in a sample. The conclusions that can be drawn from a Western blot depend on the research question being addressed, but generally, the presence or absence of a target protein in the sample can be inferred from the signal intensity of the band observed on the blot. Other conclusions, such as the size and post-translational modifications of the protein of interest, can also be deduced based on the molecular weight and mobility of the band.

After obtaining initial results from a Western blot, it is important to conduct further experiments to confirm and extend the findings. For example, one could perform additional Western blots using different antibodies to validate the specificity of the signal, or use techniques such as co-immunoprecipitation or mass spectrometry to identify interacting partners or modifications of the target protein. It may also be useful to compare the levels of the target protein across different samples or experimental conditions to gain insight into its regulation and function. Additionally, follow-up experiments could involve perturbing the expression or activity of the protein of interest to investigate its role in cellular processes or disease states.

6. Correct Misconceptions As an Encyclopedia

It can be easy to mix up pieces of information, or just accidentally mess up something. Use the internet as an encyclopedia for your convenience.

Us: Cesium fluoride has been shown to exhibit unique catalytic properties in organic reactions due to its high reactivity and low ionization potential, making it an attractive alternative to traditional catalysts.

ChatGPT: The sentence is partially true, as Cesium fluoride (CsF) is indeed known for its high reactivity and low ionization potential. However, it is not commonly used as a catalyst in organic reactions because it is a strong base and can cause unwanted side reactions. In addition, CsF is also expensive and can be difficult to handle due to its high reactivity with moisture in the air. Therefore, CsF is not a common alternative to traditional catalysts in organic reactions.

7. Use as a Writing Partner Throughout Your Process

How many times have you wished that you had someone to banter with, to run ideas through, to ask if this paragraph makes sense in the structure of your paper. While ChatGPT may be just a computer, its natural language processing origins means that it can be great to answer these questions above. ChatGPT can assess readability, make sure that a sentence fits into place with the “story” of your research, and do much of what another person “reading over” your essay can do.

How to NOT Use ChatGPT?

While being such a valuable resource, there are many, many reasons and scenarios to not use ChatGPT. ChatGPT, at its heart, can be an aid to progressing your education, but never a replacement for using critical thinking skills or purely – to learn.

1. Write Your Entire Research Paper For You Through Copy and Paste

It may be tempting to just ask all of your research questions and copy and paste what ChatGPT generates for you, but is taking the “easy” way out the best way to go? Remember that ChatGPT is trained from data and continues to be an ever-evolving model, but not perfect and never fully accurate. It can easily misdirect you with information that may seem correct, but are fully factually inaccurate. Additionally, think of all of the moral issues with submitting work that was not created by you and conclusions not synthesized by you.

2. Analyze Your Data

ChatGPT cannot synthesize new information and conclusions based on posted data, due to its limitations as a language model. ChatGPT is just an advanced version of existing language models, which can only predict the next word based off of the previous word based off of previous data it has learned from. It can be difficult, or flat out wrong, to synthesize conclusions based on what ChatGPT has analyzed.

If you’re looking for the opportunity to research under the guidance of a mentor that is not ChatGPT, then consider applying to the Lumiere Research Scholar Program . Last year over 2100 students applied for about 500 spots in the program. You can find the application form here.

Aaron Zheng is a sophomore at Harvard University, studying Bioengineering. He is passionate about biotechnology, business development, and aiding students get to college, regardless of background. In his spare time, he looks to read, journal, and explore the world.

Image source: Stock image.

Interesting and useful! I am very interested in this question. By the way, my job is such that I often turn to the services of translators. I have been using this platform for quite some time https://www.translate.com/italian-english They offer professional translation and every order goes through three stages. In this case, we can conclude that these specialists take their work responsibly and can be trusted. What do you think about it?

Now more and more people are starting to use automation while writing texts, and this is great. Too bad it didn't happen before. Although, better than a human, a robot will not write. Do you agree? Personally, I went to https://www.fellowshippersonalstatement.com/ I'm delighted because my article was written for me from scratch, without any templates. My best qualities and achievements were emphasized. Obviously, they only used my information from the completed questionnaire. They are responsible professionals.

A very useful article, I like it, but in my unprofessional opinion, it is too early to trust chat with such tasks, as for me, if you need help with writing, it is better to contact specialists. For example, I turned to the guys who provided me with a document editing service at a very affordable price, doing everything themselves, carefully checking everything, in short, professionals in their field, I trust them more than chat, it still needs to progress so that additional algorithms are installed. In the meantime, it cannot replace human work in the field of working with documents, texts, etc.

How to Use ChatGPT for Research and Essays

Want to use ChatGPT to help out with research and essays? To avoid inaccurate information, there are certain ways you can use it to your advantage.

For better or worse, OpenAI’s ChatGPT has become a big part of our lives. It’s become integral to gathering information, researching topics, and creating written works. And frankly, not using it might put you at a disadvantage academically.

However, this AI is not flawless; there is a method to use it to help with your essays and research. Using it the right way will help you avoid plagiarism issues or inaccurate information. Here, we’ll show you how to use ChatGPT to write an essay ethically, so it comes out as a quality, factual, and original piece.

1. Draw Your Outline Without ChatGPT

Assuming you already have an idea for your project, the first thing you should do before you hop on ChatGPT’s website is to prepare your thesis and outline without using AI. It’s a vital step to making sure the core of your essay actually comes from you.

If you use ChatGPT to prompt an outline for you, it could promote idea laziness, and you will find ChatGPT’s suggestions may have replaced the ones you could have produced on your own. So, get a piece of paper or a blank word processor page and create an outline for your essay.

Also, ChatGPT will avoid controversial topics—even in an outline. Therefore, you might find yourself with an outline missing vital pieces of historical sections if you rely on it. This bias is one of the major problems with OpenAI’s ChatGPT .

2. Prompt ChatGPT to Draw a Parallel Outline

Before you begin this section, if this is your first time using ChatGPT, you should read our guide on how to use ChatGPT . It will help you with creating an account and defining its capabilities. Once that’s out of the way, you can move on to the next step.

Now that you have an original outline, it doesn’t matter how basic it looks; you can use ChatGPT to create another outline. Craft a prompt with this template:

ChatGPT’s result is far more detailed than our outline. Here, you should adopt the parts of ChatGPT’s outline you would like to integrate into your own. Combine the best of the two and flesh out an outline that will guide you best as you write.

3. Create a ChatGPT Prompt for Each Section

Now that you have an outline with sections, you can begin to hack away at it section by section. Start with your introduction, where you will include your thesis statement. Ask ChatGPT to create multiple thesis statements on your idea, and choose the one that best encapsulates the major point you’re trying to communicate in your essay.

You can do something similar for all the other sections as well. Tell the AI to generate written pieces on your section topics. Don’t forget to add that it communicates the point in the tone you want. In most cases, essays should sound academic. Therefore, our prompt for each section looked like this:

Do not just copy and paste the information it generates; the next step is a vital second part of this methodology.

4. Confirm the Information With a Reputable Source

As you begin to write, you must check if the information you’re getting from ChatGPT is indeed correct. You must do this because ChatGPT occasionally hallucinates , coming up with its own facts and making up sources when you ask it to direct you to where it got its information. In some cases, it blatantly refuses to tell you at all where it got its data.

The free version of ChatGPT is not actively connected to the internet and cannot fetch information after September 2021. If you need this functionality, then it could be time to upgrade to ChatGPT Plus —especially if this is a tool you will be using heavily for work or school.

The bottom line is that you should use a reputable source (such as a book from an authority or an official website) to support every important statement you put down. You can also learn how to use Google to fact-check information .

5. Use ChatGPT for Examples and Breakdowns

It’s counterproductive to write about ideas that you don’t even understand yourself. When you reach a part of your essay that you don’t properly understand, you should go to ChatGPT and have it broken down for you.

That’s really the one thing you can count on ChatGPT to do in this process. It has good skill in crafting useful examples and explaining complicated ideas in a form you can easily understand. A popular way of phrasing a prompt to make ChatGPT explain something complicated is adding “explain like I’m five” to your prompt.

Here we used ChatGPT to break down hip and elbow dysplasia conditions in a way that is easier to understand. And it used analogies like jigsaw puzzles to paint a clearer image of what it might look like.

6. Write the Essay Yourself

As we’ve stated earlier, under no condition should you ask ChatGPT to write your whole essay for you. Not only is that lazy and deceitful, but it could also open you up to plagiarism and submitting incorrect information.

Ensure that every line in your essay is typed with your hands. Besides, if you’re writing an academic essay where you must cite sources, you will still need to corroborate all the points you’ve made with a corresponding authority. That means you’ll have to go and fact-check everything ChatGPT has written and find a corroborating source.

This could even take more time than just writing it yourself because the AI could have hallucinated some of its facts, leading you on a wild goose chase as you try to find a citation for something that doesn’t exist.

Lastly, ChatGPT doesn’t match the nuanced knowledge of a human professional in heavily specialized fields or journals that need up-to-date information. Reading a book or article by an expert, internalizing it, and writing it in your own words will give you far better results than relying on ChatGPT.

7. Polish the Style and Citation With ChatGPT

After writing, you can paste portions of your essay and ask ChatGPT to shorten, lengthen, or optimize the style. ChatGPT is good at mimicking popular styles, and you can use that to your advantage. If you want your writing to sound a little more professional, it can help rephrase it.

If you’re having trouble with citation styles and how to integrate them into your essay, you can present your essay to ChatGPT, give it your sources, and ask it to weld them together for you.

ChatGPT Is a Tool, Not a Solution

You can’t “ChatGPT” away academic or professional work; you must use it the same way you would use a tool. Take a calculator, for instance; it doesn’t replace the mathematician but improves the mathematician. You should use ChatGPT the same way a mathematician will use a calculator: for the boring, repetitive, rote work.

Let the ideas and story come from you and your experiences. And if you want to keep using ChatGPT as a student, make sure you know what you shouldn't do.

11 Ways To Use ChatGPT To Write A Research Paper

All these tasks that would have taken you hours, maybe even days, and sometimes longer, can now take just a few minutes or even shorter when you use ChatGPT.

Today I want to talk about the top things that ChatGPT can help you with when it comes to research papers so that you can write your paper much faster.

Before we dive into the specific tips and show you how to use ChatGPT when writing research papers, let me preface by saying that it is just a tool and is limited in many ways. It’s not good enough yet to replace you and replace the researcher in writing the research paper.

You’ve got to understand that you can’t just tell ChatGPT to write a couple of paragraphs for you and then use it as ChatGPT writes it and then submit it. It’s just not going to work like that. Plus, it would kind of be plagiarism although AI isn’t human.

So what you’ve got to do is remember about the limitations of the technology:

• One main limitation is that it was trained on data up to January 2022, which means that more recent papers and more recent data are not going to be there.
• Another crucial limitation is that it doesn’t pull the references for most things. So when you’re writing the literature review, ChatGPT can write paragraphs for you real quick, but there are no references for them, and you can’t really pull the references for it.
• And remember: what ChatGPT writes isn’t necessarily perfect or good enough to be published in a top journal. You will most likely have to take ChatGPT text, improve upon it, revise it before you finally submit it.

So, let’s dive in and let’s see exactly how you can accelerate your progress with your research papers using ChatGPT.

1. How to Write Research Paper Title with ChatGPT

The first thing that we’re gonna do is get ChatGPT to give us some good research paper titles using ChatGPT.

I’m going to copy and paste the abstract that I’ve already got, and then I’m going to tell ChatGPT to suggest a title for a research paper based on this abstract .

Here is the title of the research paper that ChatGPT generated: “ Unmasking the Academic Impersonator: A Multifaceted Investigation into Detecting LLM-Generated Writing ”.

I think ChatGPT’s title is very good.

Now let’s see if ChatGPT can maybe give us some alternative so that we can choose.

ChatGPT message : Can you suggest two additional titles for the research paper based on that same abstract?

Title 2: “ Guardians of Academic Integrity: Unveiling LLM-Generated Content through Dataset, User Study, and Detection Algorithms “.

Title 3: “ Deciphering the Enigma: Insights into Detecting LLM-Generated Academic Writing via Dataset, User Perception, and Advanced Algorithms “.

ChatGPT message : Can you suggest two more titles which will be phrased as questions based on that same abstract?

Title 4: “ Can Humans Spot the Imposter? Exploring the Challenge of Detecting LLM-Generated Academic Writing “.

Title 5: “ Is AI Outsmarting Academia? Investigating the Detection of LLM-Generated Content in Research Papers “.

So ChatGPT can clearly give you titles. The way to do it is just to paste the abstract and ask ChatGPT to give you titles. An alternative, if you’ve got the whole introduction written, you can also give ChatGPT the introduction, and it will give you good titles as well.

2. How to Use ChatGPT to Write an Abstract

Now let’s see if ChatGPT can do the reverse.

Can ChatGPT write an abstract based on the title of your paper?

ChatGPT message : Write an abstract for a research paper based on this title.

The result is really good. It follows the exact structure of an abstract.

3. How to Write Research Outline with ChatGPT

Now, you can ask ChatGPT to write an outline that includes as many details of your task as possible. For example, I used the title suggested by ChatGPT in the first step.

ChatGPT prompt : Write an outline for a research paper that is [NUMBER] pages long and needs to use [NUMBER] primary sources on this topic: [TITLE]

ChatGPT generated a 33-point outline that details the areas I should cover in my research paper.

4. How to Write an Introduction Using ChatGPT

Let’s ask ChatGPT to write an introduction to a research paper and see what comes out of it.

ChatGPT prompt : Write a [NUMBER] paragraph introduction to a research paper entitled: [TITLE]

ChatGPT wrote a 4 paragraph introduction and it looks very good again.

5. How to Write a Thesis Statement Using ChatGPT

Writing a strong thesis statement is essential for any research paper, as it serves as the backbone of your argument or analysis. With ChatGPT you can simplify the writing process.

ChatGPT message : Write a thesis statement for a research paper based on this title.

ChatGPT generates a thesis statement based on the information provided. Review the response carefully, considering its clarity, relevance, and alignment with your research goals. If necessary, refine the statement by providing additional guidance or asking follow-up questions.

6. Find The Research Gap Using ChatGPT

Now we can get ChatGPT to do is to work on a research gap. This is what many PhD students and researchers do.

Let’s see if ChatGPT can suggest us any good research gaps.

ChatGPT message : Suggest a research gap on the topic of detecting ChatGPT-generated academic writing.

This is really well presented.

7. Writing a Research Question Using ChatGPT

If you have a research gap, you can just give ChatGPT the research gap, and from that, you can get research questions.

ChatGPT message : Suggest two different research questions based on the following research gap.

Both of these research questions are fantastic. And both of them should be investigated.

8. Making a Research Question More Specific with ChatGPT

A big problem that a lot of people have with research questions or aims is that they are too vague or too general.

What ChatGPT suggests here is pretty good already, but let’s see if it’s capable of making a research question more specific.

ChatGPT message : Make the following research question more specific and precise.

Question #1

Question #2

Right, this is pretty good. This is definitely more specific.

I think ChatGPT can help you to specify your research questions.

If you’re struggling with your aim or research question is too vague, then you can use ChatGPT to make it more specific.

9. How To Use ChatGPT To Write A Literature Review

You’re just kind of starting out with a topic, and you want to see what are some of the key publications that you should read on that topic just to get yourself started.

Let’s see how ChatGPT can help us accelerate the literature review process.

ChatGPT prompt : I am investigating [TOPIC]. What are the key research papers on this topic I should read?

I have to say that I’m not very impressed. I think the result was affected by the limitations of the data on which the algorithm was trained until 2022.

Nevertheless, you can definitely use ChatGPT to get you the literature that you should start reading.

10. Writing a Methodology For a Research Paper Using ChatGPT

After reviewing the literature, I ask ChatGPT to suggest the methodology section for the topic. This section will include research approaches and sampling techniques for data collection.

ChatGPT message : Write a research methodology for a research paper based on this title.

As you can see, ChatGPT can help generate the overall structure of the research methodology.

11. How to use ChatGPT to summarize a research paper

The last thing which I feel ChatGPT could be very useful in research is asking for summarization.

We can ask ChatGPT to summarize a particular paper or the key takeaways that could be important for us from that paper.

In that way, you can just note it down in a proper way with proper citations, which you could use as a literature review in your research writing or academic writing.

ChatGPT prompt: Can you provide a summary of [PAPER NAME]

Can ChatGPT write research papers?

To answer this question, I turned to ChatGPT, here is its answer:

“I can indeed assist in generating content for research papers, including providing structured information, drafting paragraphs, and even offering suggestions for methodology and analysis.However, it’s crucial to note that while I can generate text based on the input and parameters provided, I lack true understanding, creativity, or original thought. Therefore, while I can help with the mechanical aspects of writing a research paper, the conceptualization, critical analysis, and interpretation of research findings are best left to human researchers who possess domain expertise and critical thinking skills. Ultimately, the quality and validity of a research paper rely on the expertise and integrity of its human authors.”

Using the power of ChatGPT can be transformative for those grappling with the daunting task of writing research papers.

ChatGPT won’t write your research paper for you from scratch, but it can make the writing process much easier.

It gives you some content to work with. And then you can rewrite it in your own words.

You’ve got a starting point now.

Related articles

• 7 Reasons Why Copywriters Must Be Shameless
• 5 Factors that Distinguish Great Copywriters
• 10 Things Epic Copywriters Do
• Career Advice for Writers: How to Become a Copywriter
• 5 Things You Need to Know About Copywriting

A professional freelance web copywriter with several years’ experience in web marketing and SEO copywriting.

Other posts by Christina Walker

Comments (0)

Navigation Menu

Search code, repositories, users, issues, pull requests..., provide feedback.

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly.

To see all available qualifiers, see our documentation .

• Notifications

50 use cases of ChatGPT for research work

hollobit/ResearchChatGPT

Folders and files, repository files navigation, how can i use chatgpt in my research work.

There are many ways you can use ChatGPT in your research work. In this repository, I've collected a list of 50 ways and examples of how you can use ChatGPT in your research work.

What is ChatGPT?

ChatGPT is an artificial intelligence language model that uses machine learning algorithms to generate human-like text. ChatGPT can be used to generate text in a variety of contexts, including answering questions, summarizing text, and generating new ideas.

What are ChatGPT Prompts?

ChatGPT is an AI language model that can generate human-like responses to prompts given to it. It is trained on a massive corpus of text data and can understand the context of a given prompt to generate relevant responses. By using ChatGPT prompts, you can generate topic ideas for your research paper based on keywords or subject areas.

Here are 50 examples of how ChatGPT can help your research.

• Generate topic ideas based on keywords or subject areas
• Provide guidance on conducting comprehensive literature reviews
• Suggest appropriate research methodologies and data collection techniques
• Help refine and narrow research questions and hypotheses
• Provide feedback on the clarity, coherence, and organization of writing
• Offer suggestions for improving writing style and tone
• Help with developing introductions, conclusions, and thesis statements
• Generate topic sentences and main points for paragraphs
• Assist with formatting and citation style
• Provide feedback on the validity and reliability of research findings
• Suggest strategies for addressing research limitations and gaps
• Provide guidance on analyzing and interpreting research data
• Help with identifying and addressing potential ethical issues in research
• Offer strategies for presenting research findings in a clear and concise manner
• Assist with developing tables, graphs, and other visual aids
• Provide feedback on the structure and clarity of arguments
• Suggest strategies for improving the coherence and flow of writing
• Help with proofreading and editing for grammar, punctuation, and spelling errors
• Provide recommendations for organizing and structuring the paper
• Assist with integrating primary and secondary sources
• Help with identifying and avoiding bias in research
• Provide guidance on selecting appropriate research participants or samples
• Assist with developing and refining research instruments (e.g. surveys, questionnaires)
• Suggest effective strategies for analyzing qualitative and quantitative data
• Provide guidance on data cleaning and preparation
• Offer feedback on the relevance and significance of research findings
• Suggest strategies for identifying and addressing potential confounding variables in research
• Help with crafting compelling and engaging titles for research papers
• Provide feedback on the appropriateness of research questions and hypotheses
• Offer suggestions for developing effective and relevant research questions
• Provide guidance on selecting appropriate research designs
• Assist with developing and implementing research studies
• Suggest effective strategies for collecting and analyzing data
• Assist with identifying and avoiding common research pitfalls
• Provide recommendations for writing and submitting research papers for publication
• Help with navigating the peer review process
• Offer feedback on the appropriateness of research results
• Provide guidance on the process of interpreting research findings
• Assist with developing and refining research theories and models
• Suggest effective strategies for testing and refining research theories and models
• Provide recommendations for presenting and disseminating research findings
• Help with developing compelling and persuasive arguments in research papers
• Offer suggestions for writing effective abstracts for research papers
• Provide guidance on identifying and avoiding plagiarism in research papers
• Suggest strategies for effectively communicating research findings to different audiences
• Assist with identifying appropriate journals for publication
• Provide feedback on the significance and contribution of research findings to the field
• Help with addressing peer review feedback and revising research papers
• Offer guidance on developing and refining research proposals
• Provide recommendations for conducting ethical and rigorous research.

As always, if you have any suggestions or things you'd like to add, please let us know.

1. Generating Topic Ideas through ChatGPT Prompt

When it comes to writing a research paper, the first and foremost task is to find a topic that interests you and is relevant to your field of study. But sometimes, it can be challenging to come up with a research topic, especially if you're not sure where to start. That's where ChatGPT prompts come in handy. In this article, we'll explore how to use ChatGPT prompts to generate topic ideas based on keywords or subject areas.

How to Generate Topic Ideas using ChatGPT Prompts?

To generate topic ideas using ChatGPT prompts, follow these steps:

Identify the keywords or subject areas related to your research field.

Go to the ChatGPT website or use any other platform that offers similar features.

Enter your keywords or subject areas in the prompt box.

Click on the ""Generate"" button to get topic ideas related to your keywords or subject areas.

Review the generated responses and choose the ones that interest you the most.

Refine your topic ideas by adding or removing keywords and subject areas as needed.

Let's say you're a student studying psychology, and you're interested in researching the effects of social media on mental health. Here's how you can use ChatGPT prompts to generate topic ideas:

Keywords: social media, mental health, psychology.

Go to the ChatGPT website.

Enter the following prompt: ""Can you suggest research topics related to social media and mental health?""

Click on the ""Generate"" button.

ChatGPT generates several responses, such as:

• The impact of social media on adolescent mental health
• The role of social media in depression and anxiety
• Social media addiction and its effects on mental health
• The relationship between social media use and self-esteem
• How social media can be used to improve mental health outcomes

Review the generated responses and choose the one that interests you the most.

Refine your topic idea by adding or removing keywords and subject areas as needed.

In conclusion, using ChatGPT prompts can be an effective way to generate research topic ideas based on keywords or subject areas. By following the steps outlined in this article, you can use ChatGPT prompts to find a topic that interests you and is relevant to your field of study.

2. Guidance on Conducting Comprehensive Literature Reviews

Conducting a comprehensive literature review is an essential part of any research paper writing process. It helps researchers to understand the current state of knowledge, identify research gaps, and develop a theoretical framework for their study. The following ChatGPT prompts can be used to achieve this task:

Prompt 1: Identify Relevant Keywords

The first step in conducting a comprehensive literature review is to identify relevant keywords that are related to your research topic. This prompt can be used to generate a list of potential keywords:

• What are the main concepts or themes that relate to your research topic?
• What synonyms or related terms can be used to describe these concepts or themes?
• What are the common abbreviations or acronyms used in the field?

Example: If you are researching the impact of social media on mental health, the keywords could be social media, mental health, social networking sites, Facebook, Twitter, Instagram, depression, anxiety, etc.

Prompt 2: Search for Relevant Literature

Once you have identified the keywords, the next step is to search for relevant literature using various databases such as Google Scholar, Scopus, PubMed, etc. This prompt can be used to refine your search strategy:

• What are the most relevant databases for your research topic?
• What search operators can be used to refine your search?
• What are the inclusion and exclusion criteria for selecting the literature?

Example: If you want to search for literature on the impact of social media on mental health, you can use the following search strategy: ""social media AND mental health"" OR ""social networking sites AND depression"" OR ""Facebook AND anxiety.""

Prompt 3: Evaluate the Quality of Literature

It is essential to evaluate the quality of the literature before including it in your review. This prompt can be used to develop a systematic approach to evaluate the literature:

• What are the criteria for evaluating the quality of literature?
• What are the potential biases or limitations of the studies?
• What are the strengths and weaknesses of the studies?

Example: When evaluating the quality of literature on the impact of social media on mental health, you can use the following criteria: sample size, research design, measurement tools, statistical analysis, etc.

Prompt 4: Synthesize and Summarize the Literature

The final step in conducting a comprehensive literature review is to synthesize and summarize the literature. This prompt can be used to develop a framework for synthesizing and summarizing the literature:

• What are the main findings of the studies?
• What are the similarities and differences between the studies?
• What are the research gaps and future directions?

Example: After synthesizing and summarizing the literature on the impact of social media on mental health, you can conclude that social media has both positive and negative effects on mental health, and more research is needed to understand the underlying mechanisms and moderators.

In conclusion, conducting a comprehensive literature review requires a systematic and structured approach. The ChatGPT prompts can be used to guide researchers in identifying relevant keywords, searching for relevant literature, evaluating the quality of literature, and synthesizing and summarizing the literature.

3. Suggesting Appropriate Research Methodologies and Data Collection Techniques

When it comes to writing a research paper, selecting the right research methodology and data collection techniques is crucial. The following ChatGPT prompts can help you in achieving this task:

ChatGPT Prompt 1: ""What are the best research methodologies for different types of research questions?""

Research methodologies are the procedures and techniques used to conduct research. Depending on the research question, different methodologies can be applied. Here are some examples:

• Qualitative research: This methodology is used to explore and understand complex phenomena through in-depth analysis of data. Qualitative research is best suited for research questions that seek to understand the meaning, context, and experiences of individuals or groups. Examples of qualitative methods include interviews, focus groups, and case studies.
• Quantitative research: This methodology is used to measure and quantify variables through statistical analysis. Quantitative research is best suited for research questions that seek to establish cause-and-effect relationships, test hypotheses, or generalize findings to a larger population. Examples of quantitative methods include surveys, experiments, and observational studies.
• Mixed-methods research: This methodology combines both qualitative and quantitative methods to provide a more comprehensive understanding of a research question. Mixed-methods research is best suited for research questions that require both in-depth exploration and empirical validation. Examples of mixed-methods research include sequential explanatory design, concurrent triangulation design, and convergent design.

ChatGPT Prompt 2: ""What are the best data collection techniques for different types of data?""

Data collection techniques are the methods used to collect and record data. Depending on the type of data, different techniques can be applied. Here are some examples:

• Survey: This technique involves collecting data from a sample of individuals through questionnaires or interviews. Surveys are best suited for collecting quantitative data on attitudes, opinions, and behaviors. Examples of surveys include online surveys, phone surveys, and in-person surveys.
• Observation: This technique involves observing and recording data on behavior or phenomena in their natural settings. Observation is best suited for collecting qualitative data on behavior, social interactions, and environmental factors. Examples of observation include participant observation, non-participant observation, and structured observation.
• Interview: This technique involves collecting data through face-to-face or phone conversations with individuals or groups. Interviews are best suited for collecting qualitative data on experiences, attitudes, and opinions. Examples of interviews include structured interviews, semi-structured interviews, and unstructured interviews.
• Document analysis: This technique involves collecting data from written or electronic documents such as reports, articles, and social media posts. Document analysis is best suited for collecting qualitative data on policies, laws, and public opinion. Examples of document analysis include content analysis, discourse analysis, and narrative analysis.

In conclusion, selecting the appropriate research methodology and data collection techniques is crucial for conducting effective research. By using the above ChatGPT prompts, you can identify the best methodologies and techniques for your research question and data type.

4. Using ChatGPT to Refine and Narrow Research Questions and Hypotheses

Introduction.

Research papers require a clear and concise research question or hypothesis that guides the study. However, formulating a research question or hypothesis can be challenging, especially when the researcher is not sure where to start. In this paper, we will discuss how ChatGPT can be used to help refine and narrow research questions and hypotheses.

How to Use ChatGPT to Refine and Narrow Research Questions and Hypotheses

Start with a broad research question or hypothesis. Before using ChatGPT, it is important to have a general idea of the research question or hypothesis. This can be a broad question or hypothesis that needs to be refined and narrowed down.

Enter the research question or hypothesis into ChatGPT. Once you have a general idea of the research question or hypothesis, enter it into ChatGPT. ChatGPT will generate text that is related to the research question or hypothesis.

Read the generated text and identify key concepts. After ChatGPT generates text related to the research question or hypothesis, read the text and identify key concepts. These concepts can be used to refine and narrow down the research question or hypothesis.

Refine and narrow down the research question or hypothesis. Use the key concepts identified in the generated text to refine and narrow down the research question or hypothesis. This can be done by eliminating irrelevant information and focusing on the most important concepts.

Example of Using ChatGPT to Refine and Narrow Research Questions and Hypotheses

Suppose a researcher is interested in studying the impact of social media on mental health. The researcher has a general idea of the research question but is unsure how to refine and narrow it down. The researcher enters the following question into ChatGPT: ""How does social media impact mental health?""

ChatGPT generates the following text:

• Social media can have both positive and negative effects on mental health.
• Some studies have shown that social media use is associated with increased rates of depression and anxiety.
• Other studies have found that social media use can have positive effects on mental health, such as increased social support and reduced feelings of loneliness.

The researcher reads the generated text and identifies the key concepts: positive and negative effects, depression and anxiety, increased social support, and reduced loneliness. The researcher uses these concepts to refine and narrow down the research question to: ""What are the positive and negative effects of social media on mental health, specifically in relation to depression, anxiety, social support, and loneliness?""

By using ChatGPT, the researcher was able to refine and narrow down the research question, making it more focused and specific.

5. Using ChatGPT Prompts for Providing Feedback on Writing Clarity, Coherence, and Organization

When it comes to writing research papers, clarity, coherence, and organization are crucial elements that can make or break the quality of the paper. As a reviewer, providing feedback on these aspects can help improve the overall quality of the paper. ChatGPT can be used to generate prompts that can help achieve this task.

For example, the following ChatGPT prompts can be used for providing feedback on writing clarity, coherence, and organization:

""Can you simplify the language used in the paper? Is there any jargon or technical terms that are not explained clearly?""

This prompt can help identify areas where the language used in the paper may be too complex or difficult to understand. It can also highlight instances where jargon or technical terms are not explained adequately, leading to confusion among readers.

""Is there a clear thesis statement in the paper? Does the rest of the paper support this statement?""

This prompt can help assess the overall coherence of the paper. It can help identify whether the thesis statement is clear and whether the rest of the paper supports this statement. If the thesis statement is unclear or unsupported, it can be challenging for readers to understand the main argument of the paper.

""Is there a logical flow of ideas in the paper? Are the paragraphs and sections well-organized?""

This prompt can help evaluate the organization of the paper. It can help identify areas where the ideas in the paper may not flow logically or where the paragraphs and sections may not be well-organized. This can make it challenging for readers to follow the argument and can lead to confusion.

""Are there any inconsistencies or contradictions in the paper? Are all the sources cited appropriately?""

This prompt can help identify any inconsistencies or contradictions in the paper. It can also help ensure that all the sources used in the paper are cited appropriately. Inconsistencies or contradictions can undermine the credibility of the paper, while incorrect or insufficient citations can lead to accusations of plagiarism.

Using ChatGPT prompts for providing feedback on writing clarity, coherence, and organization can help reviewers evaluate the quality of a research paper more objectively. It can also help authors identify areas for improvement and produce a more polished final product.

6. Suggestions for Improving Writing Style and Tone

Writing style and tone are crucial aspects that determine the overall quality of a research paper. It is important to ensure that the language used in the paper is clear, concise, and engaging to the readers. Here are some ChatGPT prompts that can be used to offer suggestions for improving writing style and tone in a research paper.

ChatGPT Prompts:

• What are some common writing mistakes that can be avoided in a research paper?
• How can the language used in the paper be made more engaging to the readers?
• What are some effective ways to vary sentence structure and avoid repetition in the paper?
• How can the tone of the paper be adjusted to suit the intended audience?
• What are some ways to ensure that the language used in the paper is accessible to all readers?

Prompt: What are some common writing mistakes that can be avoided in a research paper?

There are several common writing mistakes that can be avoided in a research paper. These include:

• Using passive voice: Using passive voice can make the language of the paper sound dull and unengaging. It is better to use active voice to make the writing more dynamic.
• Overusing jargon: Using too much jargon can make the paper difficult to understand for readers who are not familiar with the field. It is important to strike a balance between using technical terms and making the language accessible to all readers.
• Lack of clarity: Writing should be clear and concise to avoid confusion. Sentences should be well-structured and free from unnecessary words or phrases.
• Repetition: Repetition can make the paper sound monotonous. It is important to vary sentence structure and use synonyms to avoid repetition.
• Inconsistent tone: The tone of the paper should be consistent throughout. It is important to decide on the appropriate tone for the intended audience and maintain it throughout the paper.

By avoiding these common writing mistakes, the language of the research paper can be made more engaging and accessible to all readers.

7. Using ChatGPT Prompts to Develop Introductions, Conclusions, and Thesis Statements

When writing a research paper, it is essential to have a strong introduction, a clear thesis statement, and a convincing conclusion. These elements are what set the tone for your paper and convey the main idea of your research. ChatGPT prompts can help you develop these elements by providing you with ideas and guiding questions to include in your writing.

Introductions

Introductions are meant to grab the reader's attention, provide background information, and present the thesis statement. Here are some ChatGPT prompts that can help you develop a strong introduction:

• What is the significance of your research topic?
• What inspired you to research this topic?
• What is the current state of research on your topic?
• What is the problem or gap in knowledge that your research aims to address?

Topic: The Effect of Social Media on Mental Health

Introduction:

Social media has become a ubiquitous part of modern life. It is a tool that is used for communication, socialization, and entertainment. However, the impact of social media on mental health has been a subject of debate in recent years. This research aims to explore the relationship between social media use and mental health and to identify the factors that contribute to this relationship. By understanding the impact of social media on mental health, we can develop strategies to mitigate the negative effects and promote positive mental health outcomes.

Thesis Statements

The thesis statement is the core of your research paper. It presents the main idea of your research and guides the reader through your argument. Here are some ChatGPT prompts that can help you develop a clear and concise thesis statement:

• What is the main argument of your research?
• What is the significance of your research?
• What is the scope of your research?
• What is the problem that your research aims to solve?

Thesis statement:

This research aims to explore the relationship between social media use and mental health and to identify the factors that contribute to this relationship. By examining the impact of social media on mental health, we can develop effective strategies to promote positive mental health outcomes.

Conclusions

Conclusions are meant to summarize the main points of your research and restate the thesis statement. They also provide a space for reflection and discussion of future research directions. Here are some ChatGPT prompts that can help you develop a strong conclusion:

• What are the main findings of your research?
• What are the implications of your research?
• What are the limitations of your research?
• What are the future research directions?

Conclusion:

In conclusion, this research has shown that social media use has a significant impact on mental health. The findings suggest that social media use is associated with increased risk of depression, anxiety, and other mental health problems. However, the relationship between social media use and mental health is complex, and there are many factors that contribute to this relationship. Future research should focus on identifying these factors and developing strategies to mitigate the negative effects of social media on mental health.

8. Using ChatGPT prompts to generate topic sentences and main points for paragraphs

In research paper writing, generating topic sentences and main points for paragraphs is crucial for creating a clear and organized structure. One way to achieve this is by using ChatGPT prompts, which are pre-trained language models that can generate text based on a given prompt. Here's an example of how to use ChatGPT prompts for this task:

Step 1: Choose a prompt

Choose a prompt that is relevant to your research topic and can help you generate ideas for your paragraphs. For example:

• ""What are the main causes of climate change?""
• ""How has social media influenced political discourse in the 21st century?""
• ""What are the benefits and drawbacks of online learning?""

Step 2: Generate text using ChatGPT

Use a ChatGPT tool, such as OpenAI's GPT-3 or Hugging Face's GPT-2, to generate text based on the prompt. For example, using the prompt ""What are the main causes of climate change?"", you could get the following text:

• ""Climate change is caused by a variety of factors, including the burning of fossil fuels, deforestation, and industrial processes. These activities release greenhouse gases into the atmosphere, which trap heat and contribute to global warming. Other factors that contribute to climate change include agriculture, transportation, and waste management.""

Step 3: Extract topic sentences and main points

From the generated text, extract the topic sentences and main points that can be used for your paragraphs. For example:

• Topic sentence: Climate change is caused by a variety of factors.
• Burning of fossil fuels
• Deforestation
• Industrial processes
• Greenhouse gases trap heat and contribute to global warming
• Agriculture, transportation, and waste management also contribute to climate change.

Step 4: Use the topic sentences and main points in your paragraphs

Use the extracted topic sentences and main points to create clear and organized paragraphs in your research paper. For example:

Paragraph 1: Introduction to climate change

Paragraph 2: Greenhouse gases and global warming

• Topic sentence: Greenhouse gases trap heat and contribute to global warming.
• Explanation of greenhouse effect
• Examples of greenhouse gases
• Evidence of global warming

Paragraph 3: Other factors contributing to climate change

• Topic sentence: Agriculture, transportation, and waste management also contribute to climate change.
• Explanation of each factor
• Examples of how they contribute to climate change
• Possible solutions

Using ChatGPT prompts can be a useful tool for generating topic sentences and main points for your research paper paragraphs. By following the steps outlined above, you can create a clear and organized structure that effectively communicates your ideas to your readers.

9. Types of ChatGPT prompts for formatting and citation style

In academic writing, proper formatting and citation style are crucial aspects that help in ensuring the paper's clarity and credibility. Proper formatting and citation style help readers to locate and identify the sources used in the research paper. With the increasing number of citation styles and formatting requirements, it can be challenging for writers to keep up with the latest trends. Fortunately, ChatGPT prompts are a useful tool that can assist with formatting and citation style in research paper writing.

There are several types of ChatGPT prompts that can assist with formatting and citation style in research paper writing. These include:

1. Citation style prompts

Citation style prompts help writers format their citations according to the required citation style. For instance, if a writer is required to use the MLA citation style, they can use a ChatGPT prompt to generate a correct citation for a specific source.

Prompt: Generate a correct MLA citation for a book with the following details: Title: The Great Gatsby, Author: F. Scott Fitzgerald, Publisher: Scribner, Year of Publication: 1925.

Response: Fitzgerald, F. Scott. The Great Gatsby . Scribner, 1925.

2. Formatting prompts

Formatting prompts help writers format their research papers according to specific guidelines. For instance, if a writer is required to use a specific font or margin size, they can use a ChatGPT prompt to generate the correct format.

Prompt: Generate a formatting prompt for a research paper that requires Times New Roman font, 12-point size, and one-inch margins.

Response: Set the font to Times New Roman, 12-point size. Set the margins to one inch on all sides.

3. Reference management prompts

Reference management prompts help writers manage their references by organizing them in a specific order, such as alphabetical or chronological order.

Prompt: Organize the references in the research paper in alphabetical order.

Response: Alphabetize the references in the research paper according to the author's last name.

In conclusion, ChatGPT prompts are a useful tool that can assist with formatting and citation style in research paper writing. Citation style prompts, formatting prompts, and reference management prompts are some of the prompts that can be used for this purpose. By using ChatGPT prompts, writers can save time and ensure that their research papers are formatted and cited correctly.

10. Using ChatGPT prompts to provide feedback on the validity and reliability of research findings

In academic research, it is essential to ensure that the findings presented are both valid and reliable. Validity refers to the accuracy of the research results and whether they measure what they claim to measure. Reliability, on the other hand, refers to the consistency of the research results when they are repeated or tested.

To provide feedback on the validity and reliability of research findings, ChatGPT prompts can be used. ChatGPT is an AI language model that can generate responses to various prompts. Here are some examples of ChatGPT prompts that can be used:

Prompt 1: Can you explain how the study’s design and methodology contribute to the validity of the results?

• The study design and methodology should align with the research question and objectives.
• The sample size should be representative of the population being studied.
• The data collection methods should be reliable and valid.
• The data analysis methods should be appropriate and rigorous.

Example response: The study design and methodology appear to be appropriate for the research question and objectives. The sample size is sufficient and representative of the population being studied. Both quantitative and qualitative data collection methods were used, which increases the validity of the results. The data analysis methods were rigorous, and the statistical tests used were appropriate for the data.

Prompt 2: How do you assess the reliability of the findings presented in the study?

• The study should be replicable.
• The data should be consistent across different measures and time points.
• The research methods should be standardized.
• The results should be consistent with previous research on the topic.

Example response: The study appears to be reliable based on the consistency of the data across different measures and time points. The research methods were standardized, and the results are consistent with previous research on the topic. However, the study could be improved by providing more detail on the replicability of the study, such as providing information on the inter-rater reliability of the data collection methods.

Prompt 3: Can you identify any potential threats to the validity or reliability of the study?

• Threats to internal validity include confounding variables and selection bias.
• Threats to external validity include the generalizability of the results to other populations or situations.
• Threats to reliability include measurement error and inter-rater reliability.

Example response: One potential threat to the validity of the study is selection bias, as the sample was recruited through convenience sampling. There is also a risk of measurement error, as some of the data was self-reported. However, the study attempted to mitigate these threats by using standardized data collection methods and statistical analysis. The external validity of the study is limited due to the specific population being studied, which may limit the generalizability of the results to other situations or populations.

11. Strategies for Addressing Research Limitations and Gaps

Research limitations and gaps are a common issue in research studies. It is essential to identify and address these limitations and gaps to ensure the validity and reliability of the results. The following strategies can be used to address research limitations and gaps:

1. Acknowledge the limitations and gaps

It is essential to acknowledge the limitations and gaps in the research study. This will help to establish the scope of the study and the potential impact of the limitations and gaps on the results.

In a study on the effectiveness of a new teaching method, the researchers found that the sample size was small, which limited the generalizability of the results. The researchers acknowledged this limitation and highlighted the need for further research with larger sample sizes to validate the findings.

2. Provide alternative explanations

Another strategy for addressing research limitations and gaps is to provide alternative explanations for the results. This can help to mitigate the impact of the limitations and gaps on the validity of the results.

In a study on the impact of social media on mental health, the researchers found a correlation between social media use and depression. However, the researchers acknowledged that other factors, such as pre-existing mental health conditions, could also contribute to the results.

3. Conduct further research

Conducting further research is perhaps the most effective strategy for addressing research limitations and gaps. This can help to validate the findings and provide a more comprehensive understanding of the research topic.

In a study on the effectiveness of a new medication, the researchers found that the sample size was small, which limited the generalizability of the results. The researchers suggested conducting further research with a larger sample size to validate the findings.

4. Use multiple methods

Using multiple methods can help to address research limitations and gaps by providing a more comprehensive understanding of the research topic. This can help to mitigate the impact of the limitations and gaps on the validity of the results.

In a study on the impact of a new technology on productivity, the researchers used both quantitative and qualitative methods to gather data. This helped to provide a more comprehensive understanding of the impact of the technology on productivity.

5. Collaborate with other researchers

Collaborating with other researchers can help to address research limitations and gaps by providing access to additional resources and expertise. This can help to validate the findings and provide a more comprehensive understanding of the research topic.

In a study on the impact of a new treatment on patient outcomes, the researchers collaborated with other researchers to gather data from multiple sites. This helped to provide a more comprehensive understanding of the impact of the treatment on patient outcomes.

In conclusion, addressing research limitations and gaps is essential for ensuring the validity and reliability of the results. By acknowledging the limitations and gaps, providing alternative explanations, conducting further research, using multiple methods, and collaborating with other researchers, researchers can mitigate the impact of limitations and gaps on their research studies.

12. Guidance on Analyzing and Interpreting Research Data

Analyzing and interpreting research data is a crucial part of any research paper. It allows researchers to draw conclusions from their findings and communicate their results effectively. Here are some ChatGPT prompts that can help achieve this task:

Prompt 1: Exploring Data

This prompt can be used to analyze and explore data, which is the first step in data analysis. It helps researchers to identify patterns, trends, and relationships in their data. Here is an example:

Prompt: Use exploratory data analysis to identify patterns in the data.

Description:

• Identify the variables in the data set.
• Plot the data using different graphical techniques, such as histograms, scatter plots, and box plots.
• Look for patterns, trends, and relationships in the data.
• Use summary statistics, such as mean, median, and standard deviation, to describe the data.

Prompt 2: Hypothesis Testing

This prompt can be used to test hypotheses and draw conclusions from the data. It helps researchers to determine whether their findings are statistically significant and generalize to the population. Here is an example:

Prompt: Test the hypothesis that there is a significant difference in the mean scores of two groups.

• Define the null and alternative hypotheses.
• Select an appropriate test statistic, such as t-test or ANOVA.
• Calculate the p-value and compare it to the significance level.
• Interpret the results and draw conclusions.

Prompt 3: Regression Analysis

This prompt can be used to analyze the relationship between two or more variables. It helps researchers to determine the strength and direction of the relationship and make predictions based on the data. Here is an example:

Prompt: Conduct a regression analysis to determine the relationship between income and education level.

• Identify the dependent and independent variables.
• Select an appropriate regression model, such as linear regression or logistic regression.
• Calculate the regression coefficients and interpret their meaning.
• Evaluate the goodness of fit of the model, using measures such as R-squared and residual plots.
• Use the model to make predictions and draw conclusions.

In conclusion, using ChatGPT prompts can help researchers to analyze and interpret their data effectively. By following these guidelines, researchers can ensure that their findings are accurate, reliable, and meaningful.

13. Using ChatGPT to Identify and Address Potential Ethical Issues in Research

Research is critical in gaining knowledge and driving progress in various fields. However, ethical issues can arise during the research process that may compromise the integrity of the study or harm the participants. Therefore, it is crucial to identify and address potential ethical issues in research to ensure that the study is conducted ethically.

Using ChatGPT to Identify Ethical Issues

ChatGPT is an AI language model that can be used to generate prompts to identify potential ethical issues in research. The following are some examples of ChatGPT prompts that can be used to identify ethical issues in research:

• What are the potential consequences of the study on the participants?
• How can the study be conducted in a way that respects the autonomy of the participants?
• What are the potential conflicts of interest that may arise during the study?
• How can the study be conducted in a way that protects the privacy and confidentiality of the participants?
• What are the potential risks and benefits of the study, and how can they be addressed?

Using ChatGPT to Address Ethical Issues

Once potential ethical issues have been identified, ChatGPT can be used to generate prompts on how to address these issues. The following are some examples of ChatGPT prompts that can be used to address ethical issues in research:

• How can the researchers obtain informed consent from the participants?
• How can the researchers ensure that the study is conducted in a way that respects the cultural and social norms of the participants?
• How can the researchers minimize the potential harm to the participants?
• How can the researchers ensure that the study is conducted in a way that is transparent and accountable?
• How can the researchers ensure that the data collected is accurate and reliable?

Suppose a researcher wants to conduct a study on the effects of a new drug on cancer patients. The following are some potential ethical issues that may arise:

• The potential harm to the participants due to the side effects of the drug.
• The potential conflict of interest due to the researcher's affiliation with the drug company that produces the drug.
• The potential breach of confidentiality and privacy of the participants' medical records.
• The potential bias in the data collected due to the researcher's preconceived notions about the drug's effectiveness.

To address these issues, the researcher can use ChatGPT to generate prompts on how to obtain informed consent from the participants, how to minimize harm, how to ensure transparency and accountability, and how to minimize bias in the data collected.

In conclusion, ChatGPT can be a useful tool in identifying and addressing potential ethical issues in research. By generating prompts, researchers can ensure that their studies are conducted ethically and with the utmost respect for the participants' autonomy and well-being.

14. Strategies for Presenting Research Findings in a Clear and Concise Manner

When it comes to presenting research findings, it is essential to present them in a way that is easy to understand and engaging for your audience. Here are some strategies to help you present your research findings in a clear and concise manner:

Use Visuals

Visuals can help you communicate complex information in a clear and concise manner. Consider using graphs, charts, and images to help illustrate your findings. Here are some tips for using visuals effectively:

• Use simple and easy-to-understand graphics
• Choose the right type of graphic for the data you want to present
• Use color and contrast to highlight important points
• Use labels and captions to provide context and clarity

Use Plain Language

Using plain language can help you communicate your research findings in a way that is easy for your audience to understand. Here are some tips for using plain language:

• Avoid using jargon or technical terms that your audience may not be familiar with
• Use short and simple sentences
• Use active voice instead of passive voice
• Break up long paragraphs into shorter ones

Focus on the Key Findings

When presenting your research findings, it is important to focus on the key findings. Here are some tips for focusing on the key findings:

• Start with a clear and concise summary of your key findings
• Highlight the most important findings using visuals or other tools
• Avoid overwhelming your audience with too much information
• Focus on what is most relevant to your audience

Tell a Story

Telling a story can help you engage your audience and make your research findings more memorable. Here are some tips for telling a story:

• Start with a clear and compelling introduction that sets the stage for your research
• Use examples or case studies to illustrate your key findings
• Use anecdotes or personal stories to help your audience connect with your research
• End with a clear and concise conclusion that summarizes your key findings

Suppose you are conducting research on the impact of social media on mental health. You have collected a significant amount of data and now need to present your findings to a group of mental health professionals.

To present your findings in a clear and concise manner, you could use the following strategies:

• Use visuals such as graphs and charts to illustrate the data you have collected.
• Use plain language to explain the findings in a way that is easy to understand.
• Focus on the key findings, such as the correlation between social media use and depression.
• Tell a story that highlights the impact of social media on mental health, using anecdotes or case studies to make your findings more memorable.

By using these strategies, you can present your research findings in a way that is clear, concise, and engaging for your audience.

15. ChatGPT Prompt for Developing Tables, Graphs, and Other Visual Aids in Research Paper Writing

Research papers often require the inclusion of tables, graphs, and other visual aids to present data and findings in a clear and concise manner. ChatGPT offers several prompts that can assist with the development of these visual aids, including:

Prompt 1: ""Can you provide examples of tables and graphs that are commonly used in research papers?""

This prompt can be used to help writers identify the types of tables and graphs that are most commonly used in research papers. Examples of tables and graphs that may be discussed include:

• Bar graphs: used to compare data across categories
• Line graphs: used to show trends over time
• Scatter plots: used to show the relationship between two variables
• Pie charts: used to show proportions of a whole
• Tables: used to present data in an organized manner

Example response:

""Bar graphs are commonly used in research papers to compare data across categories. For example, if we were conducting a study on the effectiveness of different types of exercise on weight loss, we might use a bar graph to compare the average weight loss of participants who engaged in aerobic exercise, strength training, or a combination of both. Line graphs are often used to show trends over time, such as changes in a population's average income over a number of years. Scatter plots can be used to show the relationship between two variables, such as the relationship between a person's age and their level of education. Pie charts are commonly used to show proportions of a whole, such as the percentage of a company's revenue that comes from different product lines. Tables can be used to present data in an organized manner, such as a table that lists the demographic information of participants in a study.""

Prompt 2: ""Can you help me create a table/graph to present my data/findings?""

This prompt can be used to get assistance with creating a specific table or graph to present data or findings. The writer can provide the necessary information, such as the data to be included, the type of graph or table they need, and any formatting requirements.

""Sure, I can help you create a graph to present your findings. Can you tell me what type of data you have and what kind of graph you want to use? If you have data on the percentage of students who passed a test at different schools, a bar graph might be a good choice. You can have the schools listed on the x-axis and the percentage of students who passed the test listed on the y-axis. Alternatively, if you have data on the changes in temperature in different cities over time, a line graph might be a better choice. You can have the cities listed on the x-axis and the temperature listed on the y-axis, with different lines representing each city.""

Prompt 3: ""Can you provide feedback on my current table/graph and suggest improvements?""

This prompt can be used to get feedback on a table or graph that the writer has already created. The writer can provide the table or graph, and the ChatGPT can provide suggestions for improvements, such as changing the formatting or adding labels to make the data easier to understand.

""Thanks for sharing your graph with me. One suggestion I have is to add labels to the x- and y-axes to make it clear what the graph is showing. You might also want to change the color scheme to make it easier to read. Another suggestion is to add a title that summarizes the main point of the graph. For example, if your graph shows the relationship between age and income, your title could be 'Income by Age Group'.

16. Using ChatGPT to Achieve Better Research Paper Writing: Providing Feedback on Structure and Clarity of Arguments

One of the most essential aspects of writing a research paper is ensuring that the structure and arguments are clear and concise. To achieve this, feedback from peers and instructors is critical. However, getting feedback can be challenging, especially when dealing with complex topics and ideas. Fortunately, artificial intelligence (AI) tools such as ChatGPT can improve the writing process by providing useful feedback on the structure and clarity of arguments.

How can ChatGPT be used for providing feedback on the structure and clarity of arguments in research papers?

ChatGPT can provide feedback on the structure and clarity of arguments in research papers by analyzing the language used in the paper and evaluating its coherence, clarity, and logical flow. This can be done by inputting the paper into ChatGPT and analyzing the generated response. Here are some ways ChatGPT can be used to provide feedback on research paper structure and clarity:

1. Identifying unclear language

ChatGPT can identify unclear language by analyzing the text and suggesting alternative wordings, phrasings, or sentence structures. This can help improve the clarity of arguments in research papers.

2. Evaluating coherence and logical flow

ChatGPT can evaluate coherence and logical flow by analyzing the order of arguments, the use of transitional phrases, and the overall structure of the paper. It can suggest changes to improve the flow and ensure that the arguments are presented in a logical and coherent manner.

3. Checking for consistency

ChatGPT can check for consistency by analyzing the language used throughout the paper and identifying areas where the language is inconsistent or contradictory. It can suggest changes to ensure that the language used is consistent throughout the paper.

Let's say you are writing a research paper on the impact of social media on mental health. You have written the first draft of the paper, but you are not sure if the arguments are clear and well-structured. You can use ChatGPT to provide feedback on the structure and clarity of your arguments. Here is how:

• Input the paper into ChatGPT.
• Ask ChatGPT to evaluate the coherence and logical flow of the paper.
• ChatGPT suggests reordering some of the arguments to improve the flow.
• ChatGPT also suggests adding transitional phrases to improve the coherence of the paper.
• Finally, ChatGPT suggests rewording some of the sentences to improve the clarity of the arguments.

With these suggestions from ChatGPT, you can go back and revise your paper to improve the structure and clarity of your arguments.

In conclusion, ChatGPT is a valuable tool for providing feedback on the structure and clarity of arguments in research papers. It can help writers improve the coherence, clarity, and logical flow of their writing, making their arguments more persuasive and effective.

17. Strategies for Improving Coherence and Flow in Writing

When writing a research paper, it's important to ensure that the writing is easy to follow and understand. One way to achieve this is by improving coherence and flow. Coherence refers to the logical connections between ideas, while flow refers to the smooth transition from one idea to the next. Here are some strategies for improving coherence and flow in writing:

1. Use Transitional Words and Phrases

Transitional words and phrases can help to connect ideas and improve the flow of writing. Examples include:

• Furthermore
• In addition
• Nevertheless
• Consequently
• As a result

By using these transitions, the reader can follow the writer's train of thought and understand how different ideas relate to each other.

2. Use Topic Sentences

Topic sentences are a way to introduce the main idea of a paragraph. By including a clear and concise topic sentence, the reader can easily understand what the paragraph is about and how it relates to the overall argument.

""The prevalence of social media has led to a shift in the way that people communicate with each other. Specifically, social media has enabled people to connect with others who share similar interests, regardless of geographic location.""

3. Use Consistent Verb Tenses

Using consistent verb tenses can help to improve coherence in writing. When the verb tenses change unexpectedly, it can be confusing for the reader.

""The researchers conducted a survey and found that the majority of participants prefer coffee over tea. However, some participants reported that they drink tea more frequently than coffee.""

4. Use Parallel Structure

Parallel structure refers to using the same grammatical structure for similar ideas. This can help to improve coherence and make the writing easier to follow.

""The study found that participants who completed the intervention had lower levels of stress, were more physically active, and reported higher levels of life satisfaction than those who did not complete the intervention.""

5. Use Clear Pronoun References

Pronoun references can be confusing if it's not clear which noun they are referring to. To improve coherence, make sure that pronoun references are clear and unambiguous.

""The researchers conducted a study on the effects of caffeine on cognitive performance. They found that participants who consumed caffeine performed better on cognitive tasks than those who did not.""

By implementing these strategies, writers can improve the coherence and flow of their writing, making it easier for readers to follow their thought process and understand their argument.

18. Using ChatGPT for Proofreading and Editing Research Papers

As research paper writing is a complex and time-consuming process, it is common for writers to make grammatical, punctuation, and spelling errors. One way to overcome these challenges is by using ChatGPT to help proofread and edit research papers.

How can ChatGPT help with proofreading and editing?

ChatGPT can help in several ways with proofreading and editing research papers, including:

• Providing suggestions for grammar and syntax errors, such as incorrect verb tense or sentence structure.
• Offering alternatives for repetitive words or phrases.
• Identifying and correcting spelling errors.
• Providing recommendations for improving the readability and flow of the text.

Example of using ChatGPT for proofreading and editing

Here is an example of how ChatGPT can be used for proofreading and editing a research paper:

• Copy and paste the text of the research paper into the ChatGPT platform.
• Ask ChatGPT to proofread and edit the text.
• Review the suggestions and edits provided by ChatGPT.
• Make necessary changes to the research paper based on ChatGPT's recommendations.
• Review the paper again to ensure that all errors have been corrected.

By using ChatGPT for proofreading and editing, researchers can improve the quality of their writing and save time in the editing process. However, it is important to note that ChatGPT is not perfect and may not catch all errors, so it is still essential to review the text manually.

19. Provide recommendations for organizing and structuring the paper

When writing a research paper, it is important to structure your content in a logical and clear manner. This helps to ensure that your readers can easily understand your message and follow your argument. Here are some ChatGPT prompts that can be used to check whether your paper is well-organized and structured:

1. Does the introduction clearly state the purpose of the paper?

• The introduction should provide a clear and concise statement of the purpose of the paper.
• It should explain why the research was conducted and what questions the paper aims to answer.

Example: ""The purpose of this paper is to investigate the impact of social media on mental health. Specifically, we aim to explore whether social media use is associated with increased levels of depression and anxiety.""

2. Are the main points of the paper presented in a logical order?

• The main points of the paper should be presented in a logical order that makes sense to the reader.
• You may want to use headings and subheadings to help you organize your content.

Example: ""The paper is organized as follows. First, we provide a review of the literature on social media use and mental health. Next, we describe the methods used in our study. Then, we present our findings and discuss their implications. Finally, we offer some recommendations for future research.""

3. Are the conclusions supported by the evidence presented in the paper?

• The conclusions of the paper should be supported by the evidence presented in the paper.
• You should provide a clear and concise summary of your findings, and explain how they relate to your research question.

Example: ""In conclusion, our study found that social media use is positively associated with symptoms of depression and anxiety. These findings support the need for further research into the potential negative effects of social media on mental health. We suggest that future studies should explore the mechanisms through which social media use may affect mental health, and consider interventions that may help to mitigate these effects.""

In summary, organizing and structuring a research paper is essential for ensuring that your message is clear and coherent. Using ChatGPT prompts can help you to check whether your paper is well-organized and structured, and provide recommendations for improvement.

20. Using ChatGPT Prompts to Assist with Integrating Primary and Secondary Sources in Research Paper Writing

Research paper writing is a complex process that involves the use of both primary and secondary sources to support claims and arguments. Integrating these sources can be challenging, especially for students who are still learning how to conduct research and write academic papers. One way to overcome this challenge is to use ChatGPT prompts, which are designed to help writers generate ideas and organize their thoughts. In this paper, we will explore how ChatGPT prompts can be used to assist with integrating primary and secondary sources in research paper writing.

How can ChatGPT prompts assist with integrating primary and secondary sources?

ChatGPT prompts can assist with integrating primary and secondary sources by helping the writer identify relevant sources, extract key information from those sources, and integrate that information into their own writing. For example, here are some ChatGPT prompts that could be used to assist with integrating primary and secondary sources:

Prompt 1: ""What are the main arguments or claims made by the primary source?""

• This prompt can be used to help the writer identify the key arguments or claims made by a primary source. By focusing on the main arguments, the writer can more easily integrate the source into their own writing.

Prompt 2: ""What evidence does the secondary source provide to support the argument made by the primary source?""

• This prompt can be used to help the writer identify the evidence provided by a secondary source that supports the argument made by a primary source. By connecting the evidence from the secondary source to the argument from the primary source, the writer can strengthen their own argument.

Prompt 3: ""How does the primary source relate to the broader topic or research question?""

• This prompt can be used to help the writer contextualize the primary source within the broader topic or research question. By understanding how the primary source fits into the larger picture, the writer can more effectively integrate the source into their own writing.

Prompt 4: ""What are the limitations or weaknesses of the secondary source?""

• This prompt can be used to help the writer identify the limitations or weaknesses of a secondary source. By being aware of the limitations, the writer can more effectively evaluate the source and integrate it into their own writing.

In conclusion, using ChatGPT prompts can be an effective way to assist with integrating primary and secondary sources in research paper writing. By using prompts to identify relevant sources, extract key information, and integrate that information into their own writing, writers can strengthen their arguments and produce more effective research papers. As AI technology continues to advance, we can expect to see more tools like ChatGPT that can assist with various aspects of the research and writing process.

21. Using ChatGPT to Help with Identifying and Avoiding Bias in Research

Research papers are essential in the field of academics and research. It is necessary to ensure that research is conducted fairly and objectively. Bias in research can lead to inaccurate findings and conclusions. In this paper, we will explore how ChatGPT can be used to help identify and avoid bias in research.

How Can ChatGPT Help Identify and Avoid Bias in Research?

Prompt 1: ""what are some common biases in research"".

ChatGPT can provide a list of common biases in research. This can help researchers to identify and avoid these biases in their own research. Some common biases in research include:

• Confirmation bias: This is when a researcher only looks for evidence that confirms their hypothesis and ignores evidence that contradicts it.
• Sampling bias: This is when a sample is not representative of the population being studied.
• Publication bias: This is when studies with positive results are more likely to be published than studies with negative results.
• Selection bias: This is when participants are not randomly selected and are instead handpicked by the researcher.
• Reporting bias: This is when results are selectively reported, leaving out important information that could affect the conclusions drawn from the study.

Prompt 2: ""How can researchers avoid bias in their research?""

ChatGPT can provide a list of strategies that researchers can use to avoid bias in their research. Some strategies include:

• Use random sampling techniques to select participants.
• Use blind or double-blind study designs to reduce the impact of researcher bias.
• Be aware of your own biases and work to mitigate them.
• Use a control group to compare results.
• Use multiple measures to assess the same construct.
• Be transparent in reporting results, including negative results.

Suppose a researcher is conducting a study on the effectiveness of a new teaching method. They want to ensure that their research is not biased. The researcher can use ChatGPT to generate prompts such as ""What are some common biases in education research?"" and ""How can I avoid bias in my education research?"" to identify and avoid bias in their study. They may learn that confirmation bias is a common issue in education research and that they can mitigate it by being open-minded to all potential outcomes. They may also learn that using a blind study design can reduce the impact of their own biases.

Bias in research can have serious consequences, leading to inaccurate findings and conclusions. ChatGPT can be a useful tool for researchers to identify and avoid bias in their research. By generating prompts related to common biases and strategies to avoid them, researchers can conduct fair and objective research.

22. Selecting Appropriate Research Participants or Samples

Research design is the backbone of any research study. It is crucial to select the right participants or sample for research to gain valid and reliable results. The following ChatGPT prompts can be used to check whether the research study has selected appropriate research participants or samples:

ChatGPT Prompt 1: What is the sample size of the research study, and how was it determined?

The sample size of the research study should be appropriate to achieve the research objectives. The size of the sample depends on the following factors:

• The research design
• The level of significance (alpha)
• The desired power of the test
• The effect size
• The sampling method

The sample size should be large enough to achieve the desired power of the test and small enough to reduce the cost and time of the research. The sample size can be determined by using statistical formulas, power analysis, or sampling tables.

Example: The research study aims to investigate the effect of a new teaching method on the academic performance of high school students. The sample size of the study is determined by using power analysis. The level of significance is set at 0.05, the power of the test is set at 0.8, and the effect size is set at 0.5. The sample size is calculated to be 64 students.

ChatGPT Prompt 2: Are the research participants representative of the population under study?

The research participants should be representative of the population under study to generalize the research findings. The representativeness of the sample depends on the sampling method and the sampling frame. The sampling method should be unbiased and random to avoid selection bias. The sampling frame should include all the elements of the population under study.

Example: The research study aims to investigate the attitudes of university students towards online learning. The sample is selected by using stratified random sampling. The sampling frame includes all the students enrolled in the university. The sample is stratified based on the faculty, and the participants are selected randomly from each stratum. The research participants are representative of the university population.

ChatGPT Prompt 3: Are the research participants willing and able to participate in the research study?

The research participants should be willing and able to participate in the research study to avoid non-response bias. The willingness of the participants can be ensured by obtaining informed consent and providing incentives. The ability of the participants to participate can be ensured by selecting participants who meet the eligibility criteria.

Example: The research study aims to investigate the health-related quality of life of cancer patients who are receiving chemotherapy. The sample is selected from the cancer registry of a hospital. The eligibility criteria include age, cancer type, and chemotherapy regimen. The participants are contacted by phone, and informed consent is obtained from them. The participants who are willing and able to participate are included in the study.

Selecting appropriate research participants or samples is crucial to achieving valid and reliable research results. The above ChatGPT prompts can be used to check whether the research study has selected appropriate research participants or samples. By considering these prompts, researchers can ensure that their research is rigorous and meaningful.

23. Using ChatGPT Prompts to Assist with Developing and Refining Research Instruments

When it comes to research paper writing, developing and refining research instruments such as surveys and questionnaires can be a challenging task. Fortunately, ChatGPT prompts can be a useful tool to help with this process. By using prompts from ChatGPT, researchers can get ideas and insights on how to create effective research instruments that will yield accurate and valuable data.

Example of ChatGPT Prompt for Developing a Survey

Let's say you are conducting research on customer satisfaction with a particular product or service. You want to create a survey that will help you gather data on what customers like and dislike about the product or service. To get ideas for questions to include in the survey, you can use ChatGPT prompts. Here's an example of a prompt you could use:

Prompt: ""What are some effective questions to include in a customer satisfaction survey?""

Based on this prompt, ChatGPT might generate the following responses:

• ""How satisfied are you with the product/service overall?""
• ""What aspects of the product/service do you find most appealing?""
• ""What aspects of the product/service do you find least appealing?""
• ""How likely are you to recommend the product/service to others?""
• ""What improvements would you suggest for the product/service?""

Using these prompts, you can create a survey that is tailored to your research question and will yield valuable data.

Example of ChatGPT Prompt for Refining a Questionnaire

Let's say you are conducting research on the relationship between social media use and mental health. You have created a questionnaire that asks participants about their social media use and their mental health, but you're not sure if the questions are clear and effective. To get ideas for how to refine the questionnaire, you can use ChatGPT prompts. Here's an example of a prompt you could use:

Prompt: ""What are some effective ways to ask about social media use in a questionnaire?""

• ""How many hours per day do you spend on social media?""
• ""What social media platforms do you use most frequently?""
• ""How often do you post on social media?""
• ""Have you ever felt pressure to present a certain image on social media?""
• ""Have you ever experienced cyberbullying on social media?""

Using these prompts, you can refine your questionnaire to ensure that it is clear and effective in gathering data on social media use and mental health.

Overall, ChatGPT prompts can be a valuable tool for researchers who are developing and refining research instruments. By using prompts to generate ideas and suggestions, researchers can create instruments that will yield accurate and valuable data.

24. Suggest effective strategies for analyzing qualitative and quantitative data

Analyzing qualitative and quantitative data is an essential step in research paper writing. Both types of data require different analytical strategies to extract meaningful insights. In this paper, we will discuss effective strategies for analyzing qualitative and quantitative data.

Analyzing Qualitative Data

Qualitative data is descriptive data that cannot be measured numerically. Analyzing qualitative data involves the following strategies:

• Coding: This involves categorizing data into themes or patterns. The researcher identifies key concepts and assigns codes to them. This makes it easier to identify patterns and themes within the data.
• Content Analysis: This involves analyzing the content of the data to identify themes, patterns, and meanings. The researcher reads and re-reads the data to identify the main ideas and themes.
• Triangulation: This involves using multiple data sources to verify the findings. The researcher can use a combination of interviews, focus groups, and observations to gather data. This helps to ensure the validity and reliability of the findings.

Analyzing Quantitative Data

Quantitative data is numerical data that can be measured and analyzed statistically. Analyzing quantitative data involves the following strategies:

• Descriptive Statistics: This involves summarizing and describing the data using measures such as mean, mode, and median. This helps to identify patterns and trends in the data.
• Inferential Statistics: This involves using statistical tests to make inferences about the data. The researcher can use tests such as t-tests and ANOVA to test hypotheses and identify significant differences between groups.
• Data Visualization: This involves using graphs and charts to present the data visually. This makes it easier to identify patterns and trends in the data.

ChatGPT Prompt Examples

Here are some examples of ChatGPT prompts that can be used to achieve the mission of suggesting effective strategies for analyzing qualitative and quantitative data:

• What are some effective strategies for coding qualitative data?
• How can content analysis be used to analyze qualitative data?
• What is triangulation and how can it be used to ensure the validity and reliability of qualitative data?
• What is descriptive statistics and how can it be used to analyze quantitative data?
• How can inferential statistics be used to test hypotheses and identify significant differences between groups?
• What are some effective ways to visualize quantitative data?"

25. Guidance on Data Cleaning and Preparation

Data cleaning and preparation is a crucial step in the research paper writing process. It involves transforming raw data into a format that is suitable for analysis. The goal of this process is to ensure that the data is accurate, complete, and consistent. In this paper, we will discuss the different types of ChatGPT prompts that can be used to achieve this mission.

Types of ChatGPT Prompts

1. prompt for identifying missing values.

One of the most common issues in data cleaning is missing values. These values can be identified using the following prompt:

List all the missing values in the dataset.

2. Prompt for handling missing values

Once missing values have been identified, they need to be handled appropriately. The following prompt can be used to guide this process:

What method should be used to handle missing values?

3. Prompt for identifying outliers

Outliers are data points that are significantly different from other data points in the dataset. They can be identified using the following prompt:

Identify all the outliers in the dataset.

4. Prompt for handling outliers

Outliers can be handled in different ways depending on the nature of the data. The following prompt can be used to guide this process:

What method should be used to handle outliers?

5. Prompt for identifying duplicates

Duplicates can occur when the same data is recorded multiple times in the dataset. They can be identified using the following prompt:

Identify all the duplicate data in the dataset.

6. Prompt for handling duplicates

Duplicate data can be handled in different ways depending on the nature of the data. The following prompt can be used to guide this process:

What method should be used to handle duplicate data?

Consider a research paper that analyzes the relationship between a student's GPA and their attendance. The dataset used in this study contains the following variables: student ID, GPA, attendance, gender, and age.

To clean and prepare this data, the following prompts can be used:

• Prompt for identifying missing values: List all the missing values in the dataset.
• Prompt for handling missing values: What method should be used to handle missing values?
• Prompt for identifying outliers: Identify all the outliers in the dataset.
• Prompt for handling outliers: What method should be used to handle outliers?
• Prompt for identifying duplicates: Identify all the duplicate data in the dataset.
• Prompt for handling duplicates: What method should be used to handle duplicate data?

By using these prompts, the researcher can ensure that the data is accurate, complete, and consistent, which will improve the quality of their analysis.

26. Using ChatGPT Prompts to Check Feedback on the Relevance and Significance of Research Findings

When conducting research, it is important to gather feedback on the relevance and significance of the findings. One effective way to do this is by using ChatGPT prompts. In this article, we will explore the types of prompts that can be used to accomplish this task.

Example of Research Summary Text

The research findings suggest that there is a significant correlation between regular exercise and improved mental health. The study involved 500 participants, all of whom engaged in some form of physical activity at least three times a week. The results showed that those who exercised regularly reported lower levels of stress, anxiety, and depression compared to those who did not exercise.

ChatGPT Prompts

Here are some examples of prompts that can be used to check feedback from given summary text on the relevance and significance of the research findings:

• What are your thoughts on the correlation between regular exercise and improved mental health from the given summary text?
• Do you think the sample size of 500 participants is sufficient to draw valid conclusions from the given summary text?
• How do you think these findings could impact public health policies related to physical activity from the given summary text?
• Can you suggest any other factors that may influence the relationship between exercise and mental health from the given summary text?
• Do you think these findings could be applicable to other populations, such as older adults or individuals with disabilities from the given summary text?
• What implications do you think these findings have for mental health treatment and prevention from the given summary text?

Using ChatGPT prompts can be a useful way to gather feedback on the relevance and significance of research findings. By asking targeted questions, researchers can gain valuable insights into how their findings are perceived and how they can be applied to real-world situations.

27. Strategies for Identifying and Addressing Potential Confounding Variables in Research

Confounding variables are variables that can affect the outcome of a research study, but are not the variables of interest. They can lead to false conclusions and hinder the accuracy and validity of research findings. To identify and address potential confounding variables in research, the following strategies can be used:

1. Define the research question and hypothesis

• Clearly define the research question and hypothesis to ensure that the variables of interest are identified and focused on in the study.
• Identify potential confounding variables that may affect the outcome of the study.

2. Conduct a literature review

• Review relevant literature to identify potential confounding variables that have been previously identified in similar studies.
• Consider the methods used in these studies to address confounding variables.

3. Collect data on potential confounding variables

• Collect data on potential confounding variables to determine their relationship with the variables of interest.
• Use standardized measures and methods to ensure consistency and accuracy in data collection.

4. Analyze data for confounding variables

• Use statistical methods to analyze the data and identify potential confounding variables.
• Control for confounding variables by including them as covariates in the analysis or by stratifying the analysis to address their effects.

5. Address confounding variables in study design

• Design the study to minimize the effects of potential confounding variables.
• Use randomization, matching, or blocking to ensure that confounding variables are equally distributed among groups.
• Use blinding and placebo controls to reduce bias.

By using these strategies, researchers can identify and address potential confounding variables in their studies, leading to more accurate and valid research findings.

Example ChatGPT prompts for identifying and addressing confounding variables in research:

• What are some potential confounding variables that may affect the outcome of the study from the given summary text?
• How can the study design be modified to control for potential confounding variables from the given summary text?
• What statistical methods can be used to analyze the data for potential confounding variables from the given summary text?
• Can the effects of confounding variables be controlled for by including them as covariates in the analysis or by stratifying the analysis from the given summary text?
• How can blinding and placebo controls be used to reduce bias in the study from the given summary text?

28. How ChatGPT Prompts can Help with Crafting Compelling and Engaging Titles for Research Papers

The title of a research paper is the first thing that readers see, and it can have a significant impact on whether they decide to read the paper or not. Crafting a compelling and engaging title is essential for getting your research noticed and read by others. ChatGPT prompts can help researchers achieve this goal by providing a variety of prompts that can be used to generate creative and effective titles.

Example of ChatGPT Prompts for Title Generation

Here are some examples of ChatGPT prompts that can be used for crafting compelling and engaging titles for research papers:

• Please rewrite the given candidate title with consideration "What is the most surprising finding from your research, and how can it change the way we think about the topic"?
• Please rewrite the given candidate title with consideration "How can your research help solve a real-world problem, and what are the implications of your findings"?
• Please rewrite the given candidate title with consideration "What are the most interesting or controversial aspects of your research, and how can you highlight these in your title"?
• Please rewrite the given candidate title with consideration "What are the key takeaways from your research, and how can you turn these into a catchy and memorable title"?
• Please rewrite the given candidate title with consideration "What are the most important unanswered questions in your field, and how can your research contribute to answering these questions"?

Benefits of Using ChatGPT Prompts for Title Generation

There are several benefits to using ChatGPT prompts for crafting compelling and engaging titles for research papers, including:

• Increased creativity: ChatGPT prompts can help researchers think outside of the box and come up with new and innovative ideas for titles.
• Improved clarity: ChatGPT prompts can help researchers clarify the key findings and implications of their research, which can be reflected in the title.
• Enhanced memorability: ChatGPT prompts can help researchers come up with titles that are catchy and memorable, making their research more likely to be shared and cited by others.

Crafting compelling and engaging titles for research papers is essential for getting your research noticed and read by others. ChatGPT prompts can help researchers achieve this goal by providing a variety of prompts that can be used to generate creative and effective titles. By using ChatGPT prompts, researchers can increase their creativity, improve clarity, and enhance memorability, ultimately leading to greater impact and visibility for their research.

29. Using ChatGPT for Feedback on Research Questions and Hypotheses

In research paper writing, it is crucial to have appropriate research questions and hypotheses. These questions guide the research process and help researchers to achieve their objectives. However, it can be challenging to determine whether research questions and hypotheses are appropriate or not. In this use case, we will explore how ChatGPT can be used to provide feedback on the appropriateness of research questions and hypotheses.

Preparing to get feedback

To use ChatGPT for providing feedback on the appropriateness of research questions and hypotheses, follow these steps:

• Prepare the research material: Collect the research abstract, research questions, and hypotheses you want to evaluate. Make sure the material is clearly presented and accessible for input.
• Formulate a clear prompt: Create a concise and explicit prompt that instructs ChatGPT to provide feedback on the research questions and hypotheses. For example:
"Please evaluate the following research questions and hypotheses for their appropriateness, relevance, clarity, and alignment with the study's objectives. Offer suggestions for improvement, if necessary. Here are the research abstract, questions, and hypotheses: ..."
Then, insert the research abstract, questions, and hypotheses into the prompt.
• Input the prompt into ChatGPT: Copy and paste your prompt into ChatGPT, and submit it for processing.
• Review the response: Carefully review the generated response to ensure it addresses the appropriateness, relevance, clarity, and alignment of the research questions and hypotheses. Check whether the feedback is constructive and provides useful suggestions for improvement.
• Refine the prompt, if necessary: If the initial response does not fully address your needs or requires more specific feedback, refine the prompt to better direct ChatGPT toward the desired outcome. You may need to do this iteratively to get the most helpful feedback.

Example 1: Review Research Questions and Hypotheses

Example 2: generate research question and hypotheses and request feedback, types of chatgpt prompts:.

Clarification Prompt: This type of prompt is used to seek clarification on unclear or ambiguous research questions and hypotheses. It is especially useful when the research questions and hypotheses are too broad or vague. For example, "Could you please clarify what you mean by 'effective communication' in your research question?"

Specificity Prompt: This type of prompt is used to encourage researchers to make their research questions and hypotheses more specific and focused. It is particularly useful when the research questions and hypotheses are too general or broad. For example, "Could you please provide more specific details on the age range of the participants in your study?"

Relevance Prompt: This type of prompt is used to assess the relevance of research questions and hypotheses to the research topic. It is especially useful when the research questions and hypotheses are not directly related to the research topic. For example, "How does your research question relate to the main research topic?"

Testability Prompt: This type of prompt is used to check the testability of research questions and hypotheses. It is particularly useful when the research questions and hypotheses are too vague or general. For example, "Can you provide a testable hypothesis for your research question?"

Significance Prompt: This type of prompt is used to assess the significance of research questions and hypotheses. It is especially useful when the research questions and hypotheses do not contribute significantly to the existing body of knowledge. For example, "How does your research question contribute to the existing body of knowledge?"

Get Additional Feedbacks

Researchers can additionally input their questions and comments into ChatGPT and receive feedback on the appropriateness of their questions and hypotheses. Here are some examples of prompts that can be used for feedback on research questions and hypotheses:

Prompts 1: Check the hypotheses' relevance to the research question

• Can you explain what you mean by research question using research abstract?
• How would you define key concept in the research question from research abstract?
• Is the research question clear and concise from research abstract?

Prompts 2: Check the hypotheses' relevance to the research question

• Do the hypotheses address the research question?
• Are the hypotheses relevant to the study's aims and objectives?
• Do the hypotheses align with the research gap?

Prompts 3: Evaluate the hypotheses' testability

• Can the hypotheses be tested with the available data and research methods?
• Are there any limitations to testing the hypotheses?
• Can the hypotheses be modified to make them more testable?

Prompts 4: Evaluate weakness and potential

• "What are the weaknesses of my research question/hypothesis?"
• "What are the potential biases in my research question/hypothesis?"
• "How can I improve my research question/hypothesis?"
• "What are the gaps in my research question/hypothesis?"

30. Suggestions for Developing Effective and Relevant Research Questions

Research questions are the foundation of any research paper. They guide the research process, and the quality of the research questions directly affects the quality of the research. Therefore, it is essential to develop effective and relevant research questions to ensure that your research is meaningful and valuable. In this article, we will provide you with some suggestions for developing effective and relevant research questions.

Suggestions

1. choose a topic of interest.

The first step in developing effective and relevant research questions is to choose a topic that you are interested in. It is easier to develop research questions about a topic that you are passionate about. Also, choosing a topic that you are familiar with will help you to develop better research questions.

For example, if you are interested in the effects of social media on mental health, you can develop research questions such as:

• What is the relationship between social media use and anxiety?
• How does social media use affect depression among teenagers?

2. Define the Research Problem

Once you have chosen a topic, the next step is to define the research problem. The research problem is the gap or the knowledge that you want to explore in your research. Defining the research problem will help you to develop research questions that are relevant and meaningful.

For example, if your research problem is the lack of understanding of the impact of social media on mental health, you can develop research questions such as:

• What are the positive and negative effects of social media on mental health?
• How can social media be used to promote mental health?

3. Use a Question Stem

Using a question stem can help you to develop effective and relevant research questions. A question stem is a phrase or a sentence that starts with words such as "what," "how," "why," or "when." Using a question stem will help you to focus on a specific aspect of your research problem.

For example, if your research problem is the impact of social media on mental health, you can use the following question stems to develop research questions:

• What are the effects of social media on mental health?
• How does social media use affect mental health?
• Why do people use social media, and what are the effects on mental health?

4. Consider the Research Design

The research design is the plan or the strategy that you will use to conduct your research. The research design will affect the research questions that you can develop. Therefore, it is essential to consider the research design when developing research questions.

For example, if you are planning to conduct a survey to explore the impact of social media on mental health, you can develop research questions such as:

• What is the prevalence of social media use among teenagers, and how does it affect their mental health?
• How do different social media platforms affect mental health differently?

Example of prompt use

31. guidance on selecting appropriate research designs.

Research is an essential aspect of research life, and it requires a proper plan to conduct it successfully. One of the critical components of research planning is selecting an appropriate research design. The research design is the blueprint for conducting the research, and it outlines the procedures, methods, and techniques that will be used to gather and analyze data. This guidance will provide the necessary information required to select appropriate research designs.

Understanding the Research Question

Before selecting a research design, you need to understand the research question. The research question will determine the type of research design you will use. Ask yourself the following questions:

• What is the research question?
• What is the purpose of the research?
• What are the expected outcomes of the research?

Types of Research Designs

There are different types of research designs, and each design has its advantages and disadvantages. Consider the following research designs:

Experimental Research Design

Experimental research design is used to investigate the cause and effect relationship between variables. It involves manipulating the independent variable and observing the effect on the dependent variable.

Quasi-Experimental Research Design

Quasi-experimental research design is similar to experimental research design, but it lacks randomization. It is used when it is not possible to randomize participants.

Correlational Research Design

Correlational research design is used to investigate the relationship between two or more variables. It does not involve manipulation of variables.

Survey Research Design

Survey research design is used to collect data from a large sample of people. It involves using questionnaires or interviews to gather data.

Case Study Research Design

Case study research design is used to investigate a particular case or phenomenon. It involves in-depth analysis of a particular case.

Example of Prompt

Example and result.

Copyright (C) 2023, Prompted by Jonghong Jeon, Text Drafted by ChatGPT

Using ChatGPT to assist with writing a research paper

Enhance your academic writing with AI! Explore how ChatGPT helps you draft your methods section, brainstorm counterargument ideas, and write a compelling conclusion.

Are you struggling with writing the methods section of your research paper? Learn how ChatGPT can make this task easier with these steps:

Step 1: Brainstorm

Start by creating a bullet-point list of key details about your research methodology. Don't worry about language or structure at this stage; focus on getting the information down.

Step 2: Ask ChatGPT to refine

Input your bullet points and ask ChatGPT to provide detailed descriptions. This approach saves time and effort and ensures a comprehensive description of your methodology.

Step 3: Review ChatGPT’s output Review the output from ChatGPT and put your own touch on it. Remember that you are collaborating with ChatGPT and using it as a partner and it may not give you the perfect output every time.

You can also use ChatGPT to develop counterarguments and the conclusion of your scientific paper.

Counterarguments

Input your arguments and ask ChatGPT to create counterarguments that strengthen your main argument and show different sides of the topic.

Start by drafting your conclusion, and then ask ChatGPT to rewrite your paragraph as a conclusion. Incorporate counterarguments in your conclusion to provide a well-rounded viewpoint and strengthen your argument.

Start the first chapter of our ChatGPT Essentials for Clinicians course for free

If you're not using ChatGPT and other language learning models (LLM) to enhance your abilities and streamline your work, you're missing out on the future of medicine! 

Let's dive into the basics of ChatGPT and master the art of effective prompting—the key to harnessing the potential of these AI tools. This will enable you to compose academic content, simplify information, organize data, and much more.

Use ChatGPT To Write A Research Paper: Chat GPT For Scientific Papers

Producing academic writing can be daunting, but with ChatGPT, OpenAI’s advanced AI language model, the process becomes more accessible and efficient.

Here, learn how ChatGPT transforms the traditional approach to writing research papers, offering innovative solutions from structuring outlines to crafting intricate arguments.

Whether you’re a seasoned researcher or a novice writer, discover how ChatGPT’s capabilities can enhance your research process, ensuring a comprehensive and well-articulated paper.

Use ChatGPT To Write A Scientific Research Paper 

Step 1: General Introduction

Imagine you’re about to write a scientific research paper. You have a specific topic in mind but are struggling with how to structure your paper or what essential points to include in your introduction.

This is where ChatGPT, OpenAI’s powerful AI language model, steps in to assist in the writing process.

Start by asking ChatGPT to create an outline. For instance, you could provide ChatGPT with your research question and any relevant details about your study.

The AI will generate an outline that covers key aspects of your introduction, helping you to organize your thoughts and structure your paper effectively.

After you’ve brainstormed and provided the necessary prompts, ChatGPT can help draft an initial version of your introduction. Remember, the output generated by ChatGPT may vary, and it’s essential to use ChatGPT in conjunction with your own expertise.

For example, you might tell ChatGPT to write an introduction for a paper on the impact of a certain technology in healthcare. ChatGPT can generate text based on your prompt, providing insights into:

• Existing research,
• Gaps in the literature, and
• The significance of your study.

This initial draft can serve as a valuable foundation for your research paper.

Some prompts that you can use include:

• Develop an introduction that captures the readers’ attention and provides a concise overview of the research topic: [research topic].
• Compose an introductory paragraph that effectively introduces the readers to the topic at hand: [research topic]. Provide background information, and context for the research.
• Create an introduction that sets the stage for an academic article centered around [research topic]. Highlight its rationals, and the structure of the article.

Step 2: Problem Statement

When you use ChatGPT to write a research paper, especially for crafting a problem statement, you begin by providing a clear prompt.

This includes outlining your research question and any specific details about your topic. ChatGPT’s AI language model then processes this information to generate a concise and coherent problem statement.

For instance, if your research paper is on the environmental impact of urban development, you would ask ChatGPT to focus on this specific topic.

The AI can assist by generating text that highlights existing research gaps, the significance of the study, and why addressing this particular issue is vital.

ChatGPT can help structure your paper, creating an outline that seamlessly leads from the problem statement to other sections of your paper.

Remember, the content generated by ChatGPT should be used as a starting point. It’s crucial to review and edit this to ensure originality and accuracy. Proofreading the paper for grammar and making necessary revisions is a must.

Step 3: Research Question :

Writing research questions for your paper can seem daunting, but with ChatGPT, you can streamline this critical part of the writing process. You can start by asking ChatGPT to brainstorm ideas specific to your topic.

Say your research paper is on climate change’s effects on agriculture. Tell ChatGPT, “Generate research questions about the impact of climate change on crop yield.”

ChatGPT’s output will include various questions, such as “How does increased temperature affect wheat production in the Midwest?” These prompts can spark further ideas and help you refine your research objectives.

Remember, while ChatGPT can assist in the research paper writing process, it cannot write the paper entirely. Use the content generated by ChatGPT as a foundation, and then build upon it with your research and insights.

It’s always vital to review and edit the AI-generated material to ensure it aligns with your research goals and fills the gaps in existing literature.

Some example prompts to generate research objectives include:

• Generate a list of potential research questions about [research topic]
• Suggest innovative research questions focusing on [research topic]
• Identify research questions exploring the relationship between [variable 1 in research] and [variable 2 in research]

Step 4: Literature Review

Next, ask ChatGPT to assist in creating an outline for your literature review. For example, if you input, “Create an outline for a literature review on AI and environmental sustainability,” ChatGPT can structure your paper effectively, outlining key:

• and subtopics.

This AI language model excels in organizing research, laying out a coherent structure for your review.

Now, onto the juicy part: generating content. You can provide ChatGPT with prompts like, “Summarise recent studies on AI and renewable energy sources.”

ChatGPT will respond with a synthesis of:

• Incorporating recent studies and
• Key findings.

While ChatGPT can generate text and even suggest citations, it’s always crucial to cross-check and validate these with your research materials.

Remember, ChatGPT’s output is a starting point. You still need to review and edit the content for grammar, punctuation, and relevance to your research topic.

It’s essential to use ChatGPT in conjunction with your expertise, especially when analysing and interpreting data. The AI model can provide a valuable foundation, but the depth and accuracy of your literature review depend on your critical analysis and human expertise.

• Analyze and summarize the existing literature on [research topic], identifying the major theories, methodologies and results used in previous research.
• Analyze the limitations in the existing literature on [research topic], suggesting gaps for further research and exploration.
• Conduct a systematic literature review on [research topic], organising and categorising the relevant research papers based on their research questions, methodologies, and findings.

Step 5: Research Methodology

The next step is structuring your paper, particularly the methodology section. Here, ChatGPT’s prowess in generating an organized outline comes into play.

You can ask ChatGPT to create an outline for your methodology section. This AI tool can suggest various research methods relevant to your topic, such as:

• Qualitative,
• Quantitative, or
• Mixed methods.

It’s like having a virtual research assistant who understands the nuances of scientific research paper writing.

When it comes to the actual content generation, ChatGPT can provide you with a basic draft. For example, you can prompt ChatGPT with, “Generate a draft for the methodology section on a paper about online learning effectiveness.”

ChatGPT will produce a text that covers key aspects like:

• Sample size,
• Data collection methods, and
• Analysis techniques.

While ChatGPT can generate content, it’s essential to use it in conjunction with your own expertise. The content needs to be reviewed and edited to fit your specific research context.

ChatGPT can assist in organising your research and generating initial drafts, but the accuracy and relevancy of the methodology depend on your input and analysis.

• Write a detailed methodology section for conducting a qualitative study on [research topic].
• Create a data collection plan and describe the data analysis techniques in a quantitative research study on [research topic]
• Suggest a research design and methodology for conducting an experimental study on [research topic].

Step 6: Findings

Once you’ve gathered your data, the next step is to make sense of it. Here, ChatGPT can be used to brainstorm different ways to analyse and interpret your data. 

If your research paper is on the effects of a new educational strategy on student performance, you can ask ChatGPT to suggest various statistical methods to analyze your results. 

ChatGPT’s ability to process natural language and generate text-based suggestions can provide you with unique insights that may not have been immediately apparent.

When it comes to writing about your findings, ChatGPT can assist in structuring this section of your paper. You can tell ChatGPT about the results you obtained and ask for help in creating an outline that effectively presents these findings.

The AI model can suggest how to organize your results, discuss their implications, and highlight key points that are relevant to your research question.

You will need to review, edit, and ensure that the findings are accurately represented and align with the collected data.

• Analyse the findings related to [research topic] below, and then present and interpret the quantitative data collected below in a comprehensive report.
• Discuss the reliability and validity of the data about  [research topic] in the study below, based on relevant literature.
• Describe the key patterns and trends observed in the qualitative data gathered during the research on [research topic].

Use Chat GPT To Brainstorm And Write A Research Paper

ChatGPT emerges as a revolutionary tool in the realm of academic writing, offering unparalleled assistance in crafting research papers. Its ability to generate ideas, structure content, and provide insights makes it an invaluable asset for researchers and students alike.

It’s important to remember that while ChatGPT is a powerful aid, the uniqueness and depth of a research paper ultimately depend on the writer’s critical thinking and expertise. Embrace ChatGPT as a collaborative partner in your scholarly endeavours for enhanced research and writing efficiency.

Dr Andrew Stapleton has a Masters and PhD in Chemistry from the UK and Australia. He has many years of research experience and has worked as a Postdoctoral Fellow and Associate at a number of Universities. Although having secured funding for his own research, he left academia to help others with his YouTube channel all about the inner workings of academia and how to make it work for you.

Thank you for visiting Academia Insider.

We are here to help you navigate Academia as painlessly as possible. We are supported by our readers and by visiting you are helping us earn a small amount through ads and affiliate revenue - Thank you!

2024 © Academia Insider

ChatGPT in higher education - a synthesis of the literature and a future research agenda

• Open access
• Published: 02 May 2024

Cite this article

You have full access to this open access article

• Pritpal Singh Bhullar 1 ,
• Mahesh Joshi 2 &
• Ritesh Chugh   ORCID: orcid.org/0000-0003-0061-7206 3  

1134 Accesses

21 Altmetric

Explore all metrics

ChatGPT has emerged as a significant subject of research and exploration, casting a critical spotlight on teaching and learning practices in the higher education domain. This study examines the most influential articles, leading journals, and productive countries concerning citations and publications related to ChatGPT in higher education, while also shedding light on emerging thematic and geographic clusters within research on ChatGPT’s role and challenges in teaching and learning at higher education institutions. Forty-seven research papers from the Scopus database were shortlisted for bibliometric analysis. The findings indicate that the use of ChatGPT in higher education, particularly issues of academic integrity and research, has been studied extensively by scholars in the United States, who have produced the largest volume of publications, alongside the highest number of citations. This study uncovers four distinct thematic clusters (academic integrity, learning environment, student engagement, and scholarly research) and highlights the predominant areas of focus in research related to ChatGPT in higher education, including student examinations, academic integrity, student learning, and field-specific research, through a country-based bibliographic analysis. Plagiarism is a significant concern in the use of ChatGPT, which may reduce students’ ability to produce imaginative, inventive, and original material. This study offers valuable insights into the current state of ChatGPT in higher education literature, providing essential guidance for scholars, researchers, and policymakers.

Similar content being viewed by others

Students’ voices on generative AI: perceptions, benefits, and challenges in higher education

Artificial Intelligence (AI) Student Assistants in the Classroom: Designing Chatbots to Support Student Success

Examining science education in chatgpt: an exploratory study of generative artificial intelligence.

Avoid common mistakes on your manuscript.

1 Introduction

ChatGPT, or Chat Generative Pre-trained Transformer, is a popular generative Artificial Intelligence (AI) chatbot developed by OpenAI, employing natural language processing to deliver interactive human-like conversational experiences (Jeon et al., 2023 ; Angelis et al., 2023 ). ChatGPT utilises a pre-trained language learning model, derived from an extensive big-data corpus, to predict outcomes based on a given prompt (Crawford et al., 2023 ; Geerling et al., 2023 ; Li et al., 2023 ). Since its inception, ChatGPT has attracted widespread attention and popularity and has the potential to disrupt the education sector (Rana, 2023 ). According to a research survey of adults conducted by the Pew Research Centre, approximately 60% of adults in the United States and 78% of adults in Asia possess knowledge of ChatGPT; furthermore, men are more familiar with ChatGPT than women (Vogels, 2023 ). The study also found that among ethnic groups globally, individuals of Asian descent have the highest level of familiarity with AI-based large language models (LLMs).

People have found value in using ChatGPT for a wide range of purposes, including generating creative content, answering questions, providing explanations, offering suggestions, and even having casual conversations (Crawford et al., 2023 ; Throp, 2023 ; Wu et al., 2023 ). Furthermore, ChatGPT is an effective digital assistant for facilitating a thorough understanding of diverse and intricate subjects using simple and accessible language. Given these features, ChatGPT has the potential to bring about a paradigm shift in traditional methods of delivering instruction and revolutionise the future of education (Tlili et al., 2023 ). ChatGPT stands out as a promising tool for open education, enhancing the independence and autonomy of autodidactic learners through personalised support, guidance, and feedback, potentially fostering increased motivation and engagement (Firat, 2023 ). Its capabilities encompass facilitating complex learning, asynchronous communication, feedback provision, and cognitive offloading (Memarian & Doleck, 2023 ).

However, the rapid expansion of ChatGPT has also aroused apprehensions in the academic world, particularly after reports surfaced that the New York Department of Education had unexpectedly imposed a ban on access to the tool due to concerns about academic integrity violations (Sun et al., 2023 ; Neumann et al., 2023 ; Crawford et al., 2023 ). Students who use ChatGPT to produce superior written assignments may have an unfair advantage over peers who lack access (Farrokhnia et al., 2023 ; Cotton et al., 2023 ). Ethical concerns about the deployment of LLMs include the potential for bias, effects on employment, misuse and unethical deployment, and loss of integrity. However, there has been little research on the potential dangers that a sophisticated chatbot such as ChatGPT poses in the realm of higher education, particularly through the lens of a systematic literature review and bibliometric techniques.

In this light, this paper explores the literature on the application of ChatGPT in higher education institutions and the obstacles encountered in various disciplines from the perspectives of both faculty and students. The paper aims to analyse the current state of the field by addressing the following overarching research questions using bibliographic coupling, co-occurrence analysis, citation analysis, and co-authorship analysis:

What are the most influential articles in terms of citations in research related to ChatGPT in education?

What are the top journals and countries in terms of publication productivity related to the implications of ChatGPT in higher education institutions?

What are the emerging thematic clusters in research on the role and challenges of ChatGPT in teaching and learning in higher education institutions?

What are the geographic clusters in research on the role and challenges of ChatGPT in teaching and learning in higher education institutions?

2 Methodology

In conducting this study, publications on the impact of ChatGPT on various aspects of higher education institutions were systematically identified through an extensive search using Elsevier’s Scopus database, a comprehensive repository hosting over 20,000 globally ranked, peer-reviewed journals (Mishra et al., 2017 ; Palomo et al., 2017 ; Vijaya & Mathur, 2023 ). Scopus is a widely used database for bibliometric analyses and is considered one of the “largest curated databases covering scientific journals” (pg. 5116) in different subject areas (Singh et al., 2021 ). Widely acclaimed for its comprehensive coverage, Scopus has been extensively employed in bibliometric analyses across diverse disciplines, as evidenced by studies in capital structure theories, business research, entrepreneurial orientation and blockchain security (Bajaj et al., 2020 ; Donthu et al., 2020 ; Gupta et al., 2021 ; Patrício & Ferreira, 2020 ). Notably, despite the “extremely high” correlation between the Web of Science and Scopus databases, Scopus’s status as a superior and versatile data source for literature extraction is reinforced by its broader coverage of subject areas and categories compared to the narrower journal scope of Web of Science, facilitating scholars in locating literature most pertinent to the review area (Archambault et al., 2009 ; Paul et al., 2021 ). To ensure a systematic literature review, we adhered to the preferred reporting items for systematic reviews and meta-analysis (PRISMA) guidelines (Page et al., 2021 ) for the search, identification, selection, reading, and data extraction from the articles retrieved through the Scopus database (Fig.  1 ). Reliance on a single database is acceptable within the PRISMA framework (Moher et al., 2009 ).

Employing Boolean-assisted search queries, we aimed to capture a comprehensive range of topics related to ChatGPT’s impact on higher education institutions. Specific search queries were carefully selected to ensure a broad yet relevant search scope and included the following:

“ChatGPT and Teaching learning in universities” OR “Effect of ChatGPT in higher education institution” OR “ChatGPT and student assessment in higher education” OR “ChatGPT and academic integrity” OR “ChatGPT and teaching pedagogy in higher education institution” OR “ChatGPT and cheating student course assignment” OR “ChatGPT and teaching in higher education” OR “Implications of ChatGPT in higher education institutions” OR “ChatGPT and evaluation criteria in higher education institution” OR “ChatGPT in universities” OR “ChatGPT and student learnings. ”

The study includes papers published and included in the Scopus database on or before May 26, 2023 on the theme of ChatGPT and higher education. This timeframe was chosen to encompass the most recent and relevant literature available up to the point of data retrieval. Papers identified through the search queries underwent inclusion or exclusion based on predetermined criteria. Specifically, only papers published in journals were considered for this study, as these undergo a peer-review process and are subject to stringent selection criteria set by the journals, ensuring their quality and reliability. Papers in conference proceedings were excluded from the start of the search. Only papers written in English were included to maintain consistency and clarity, whereas others were excluded. Of the 48 research papers that were initially identified, 47 were ultimately selected for the bibliometric analysis, which was conducted using VOSviewer, a bibliometric analysis tool.

PRISMA Flowchart

From the identified pool of 47 articles, the analysis uncovered a nuanced distribution of research methodologies. Specifically, 11 studies were grounded in quantitative research methodologies, underscoring a quantitative focus within the literature. In contrast, a substantial majority of 31 articles embraced a qualitative framework, showcasing a diverse spectrum that included pure qualitative research, editorials, letters to the editor, and opinion pieces. Furthermore, the review brought to light four literature reviews, signifying a synthesis of existing knowledge, and identified one study that strategically employed a mixed-methods approach, blending both qualitative and quantitative research techniques.

To address the research questions, the selected publications underwent analysis using various bibliometric techniques. For the first and second research questions, citation analysis was employed. For the third and fourth research questions, bibliographic analysis was performed in VOSviewer software to generate clusters.

3 Findings and discussion

3.1 publication trend.

Information from the Scopus database indicates that academics began focusing on investigating various aspects of ChatGPT’s potential in higher education in 2022, as they published their findings in 2023. All academic articles in reputable publications in the Scopus database were published in 2023.

3.2 Citation analysis

Table  1 presents the top ten articles according to the number of citations. The number of articles increased significantly in 2023, consistent with the emerging nature and growing relevance of the topic. Exploring the ramifications of ChatGPT in higher education is a recent focal point for scholars, with numerous aspects warranting deeper investigation. The limited citation count, as anticipated, underscores that publications from 2023 are in the early stages of gaining visibility and recognition within the academic community.

The article by Thorp ( 2023 ), entitled “ChatGPT is fun, but not an author”, has received the highest number of citations (79). Thorp stresses the risks associated with implementing ChatGPT in the classroom. Although ChatGPT is an innovative AI tool, significant barriers remain to its implementation in the field of education. According to Thorp, using ChatGPT in academic writing is still inefficient. Thorp also expresses concerns about the rising prevalence of ChatGPT in the fabrication of scientific publications. The second most-cited work, “How Does ChatGPT Perform on the United States Medical Licensing Examination?” by Gilson and colleagues, has received 27 citations. Gilson et al. ( 2023 ) evaluated the accuracy, speed and clarity of ChatGPT’s responses to questions on the United States Medical Licensing Examination’s Step 1 and Step 2 tests. The text responses generated by ChatGPT were evaluated using three qualitative metrics: the logical justification of the chosen answer, the inclusion of information relevant to the question, and the inclusion of information extraneous to the question. The model attained a level of proficiency comparable to that of a third-year medical student. The study demonstrates the potential utility of ChatGPT as an interactive educational resource in the field of medicine to facilitate the acquisition of knowledge and skills. Third is Kasneci et al.’s article “ChatGPT for good? On opportunities and challenges of large language models for education”, with 13 citations. This paper examines the benefits and drawbacks of using language models in the classroom from the perspectives of both teachers and students. The authors find that these comprehensive language models can serve as a supplement rather than a replacement for classroom instruction. Each of the remaining top-ten articles mentioned the impact of ChatGPT on academic integrity in education and had received fewer than ten citations at the time of analysis.

Table  2 presents the top 10 journals in terms of the number of citations of publications related to the topic of ChatGPT in higher education. The journal Science , which published “ChatGPT is fun, but not an author,” was deemed most influential because it received the highest number of citations (79). JMIR Medical Education has published two articles that have been cited by 30 other research articles on the same topic. Journal of University Teaching and Learning Practise has published the most articles: three. Innovations in Education and Teaching International has published two articles on this topic, which together have been cited by six articles.

As shown in Table  3 , the majority of research articles pertaining to ChatGPT and higher education have originated from countries in Asia. Six of the top 10 countries for publishing articles on this topic are located in the Asian continent. However, the most influential studies in terms of citations have been produced by the United States, Germany, Australia, and the United Kingdom. Combined, these countries have received a total of 63 citations, with individual counts of 36, 17, 7, and 7, respectively. These four countries have 90% of the total citations of the top 10 most productive countries in the field of research on higher education perspectives on ChatGPT.

3.3 Bibliographic coupling

3.3.1 thematic clusters.

Four thematic clusters (TCs) were identified from the included research articles, as shown in Table  4 . VOSviewer was used to perform clustering based on bibliographic coupling. This method identifies relations between documents by examining publications that cite the same sources (Boyack & Klavans, 2010 ). VOSviewer clusters articles with a common knowledge base, assigning each publication to exactly one cluster. To implement this clustering technique, we assessed the co-occurrence of bibliographic references among articles within our dataset. Co-occurrence was determined by identifying shared references between articles, indicating a thematic connection (Boyack & Klavans, 2010 ). Articles sharing common references were considered to co-occur, enabling us to quantify the extent of thematic relationships based on the frequency of shared references. We identified and categorised thematic clusters within our dataset through the combined approach of VOSviewer clustering and co-occurrence analysis. This method typically results in a distribution of clusters, with a limited number of larger clusters and a more substantial number of smaller clusters.

The clusters were derived through an analysis of subordinate articles extracted from the Scopus database. VOSviewer systematically organised similar articles into distinct clusters based on the shared patterns of bibliographic references (Van Eck & Waltman, 2010 ). To ensure methodological transparency and robustness, we established clear criteria and parameters for clustering. Specifically, keywords with a minimum frequency ( n  = 5) were included in the analysis, and co-occurrence was calculated based on a pairwise comparison method. This systematic approach ensured the meaningful representation of thematic relationships within the dataset, guided by insights from previous literature (Jarneving, 2007 ). Using cluster analysis techniques, the articles were organised into cohesive groups characterised by the degree of thematic homogeneity guided by the nature of the research findings. This approach ensured a robust representation of the underlying thematic structure (Jarneving, 2007 ).

Furthermore, to mitigate the risk of subjective bias in thematic categorisation, a counter-coding approach was employed. A second researcher independently categorised thematic clusters identified by VOSviewer to assess inter-rater agreement. The level of agreement between the two researchers was assessed using Cohen’s kappa coefficient, ensuring the reliability and validity of the thematic classification process. The resulting kappa coefficient (0.69) indicated substantial agreement, suggesting a high level of agreement beyond what would be expected by chance alone (Gisev et al., 2013 ). Furthermore, the nomenclature assigned to each cluster was finalised based on the predominant research theme emerging from the analysis, providing a concise and informative label for each group.

TC1: ChatGPT and Academic Integrity: Cotton et al. ( 2023 ) describe ChatGPT as a double-edged sword that potentially threatens academic integrity. AI essay writing systems are programmed to churn out essays based on specific guidelines or prompts, and it can be difficult to distinguish between human and machine-generated writing. Thus, students could potentially use these systems to cheat by submitting essays that are not their original work (Dehouche, 2021 ). Kasneci et al. ( 2023 ) argue that effective pedagogical practices must be developed in order to implement large language models in classrooms. These skills include not only a deep understanding of the technology but also an appreciation of its constraints and the vulnerability of complex systems in general. In addition, educational institutions need to develop a clearly articulated plan for the successful integration and optimal use of big language models in educational contexts and teaching curricula. In addition, students need to be taught how to verify information through a teaching strategy emphasising critical thinking effectively. Possible bias in the generated output, the need for continuous human supervision, and the likelihood of unforeseen effects are just a few of the challenges that come with the employment of AI systems. Continuous monitoring and transparency are necessary to ensure academic integrity while using ChatGPT. Lim et al. ( 2023 ) report that ChatGPT poses academic integrity challenges for the faculty of higher education institutions, who must verify whether academic work (assignments, research reports, etc.) submitted by students is derived from the fresh perspective of data analysis or plagiarised and recycled (copying and pasting original work) by ChatGPT. ChatGPT may threaten student learning and classroom engagement if students have access to information and course assignments without assessing their integrity. Perkins ( 2023 ) also expresses concerns regarding academic integrity in the use of ChatGPT. Students are utilising ChatGPT to complete their course assignments without attribution rather than producing original work. Higher education institutions must establish clear boundaries regarding academic integrity and plagiarism in light of the growing utilisation of AI tools in academic and research settings. In addition, the challenges posed by AI essay writing systems like ChatGPT necessitate a multifaceted approach to safeguard academic integrity. Educational institutions should invest in comprehensive educational programs that not only teach students the ethical use of technology but also incorporate rigorous assessments of critical thinking skills. Additionally, integrating AI literacy into the curriculum, with a focus on understanding the limitations and potential biases of big language models, can empower students to discern between human and machine-generated content.

TC2: ChatGPT and Learning Environment: According to Crawford et al. ( 2023 ), increased stress levels and peer pressure among university students have created a favourable environment for the use of AI tools. ChatGPT provides enhanced educational opportunities for college-level students. It can help students identify areas they may have overlooked, offer guidance on additional reading materials, and enhance existing peer and teacher connections. In addition, ChatGPT can propose alternative methods of evaluating students beyond conventional assignments. Crawford et al. ( 2023 ) recommend providing practical assignments incorporating ChatGPT as a supplementary tool to reduce plagiarism. Su ( 2023 ) documents that ChatGPT can provide students with a personalised learning experience based on their specific needs. In addition, the ChatGPT platform can be used to create a virtual coaching system that offers prompt feedback to educators during their classroom evaluations. This approach fosters critical thinking and supports early childhood educators in refining their teaching methodologies to optimise interactive learning outcomes for students. Tang ( 2023b ) proposes that bolstering research integrity can be achieved by imposing restrictions on the utilisation of NLP-generated content in research papers. Additionally, the author advocates for transparency from researchers, emphasising the importance of explicitly stating the proportion of NLP-generated content incorporated in their papers. This recommendation prompts a critical examination of the role of AI-generated content in scholarly work, emphasising the importance of nurturing independent research and writing skills for both students and researchers.

TC3: ChatGPT and Student Engagement: Lee ( 2023 ) examines the ability of ChatGPT to provide an interactive learning experience and boost student engagement beyond textbook pedagogy. Iskender ( 2023 ) explains that ChatGPT provides a mechanism for students to generate and investigate diverse concepts expeditiously, thereby helping them engage in imaginative and evaluative thinking on specific subject matter. This approach has the potential to optimise time management for students and allow them to concentrate on more advanced cognitive activities. AI tools such as ChatGPT can potentially enhance the personalisation of learning materials by providing visual aids and summaries that can aid the learning process and significantly improve students’ competencies. Hence, leveraging ChatGPT in education can revolutionise learning by facilitating interactive experiences, nurturing imaginative thinking, and optimising time management for students.

TC4: ChatGPT and Scholarly Research: Ivanov and Soliman ( 2023 ) and Yan ( 2023 ) focus on the practical applications and implications of LLMs like ChatGPT in educational settings and scholarly research within the context of language learning, writing, and tourism. Yan’s investigation into ChatGPT’s application in second-language writing examines its effectiveness in addressing specific writing tasks at the undergraduate level. The findings underscore the nuanced balance between the strengths of ChatGPT and the inherent limitations in handling demanding academic writing tasks. Nevertheless, ChatGPT is also labelled as an ‘all-in-one’ solution for scholarly research and writing (Yan, 2023 ). In parallel, Ivanov and Soliman ( 2023 ) highlight that ChatGPT can assist scholars in the field of tourism research by composing preliminary literature reviews, substantiating their chosen methodologies, and creating visual aids such as tables and charts. Furthermore, the researchers outline that ChatGPT could provide valuable methodological ideas and insights by helping researchers generate questions and corresponding scales for inclusion in questionnaires. Hence, ChatGPT has the potential to become a valuable ally as a facilitator in academic writing processes and has the potential to transform the research workflow.

3.3.2 Geographic clusters

The results of the country-based bibliographic analysis are summarised in Table  5 . The present study utilised the prevailing research theme in the existing literature as a framework for categorising the countries into four distinct clusters on the basis of the number of documents published from different countries.

Cluster 1: Implications of ChatGPT for Student Examinations and Education : Cluster 1 is composed of five countries: Germany, Ireland, South Korea, Taiwan, and the United States. Researchers in these countries have emphasised the potential role of ChatGPT in higher education within the context of AI language models. Eleven research articles related to this theme were published by researchers based in the United States, the most in this cluster. The top three articles in Table  1 are from the United States. The study entitled “Opportunities and Challenges of Large Language Models for Education,” was authored by German researchers (Kasneci et al., 2023 ) and has been widely cited in the academic community (13 citations). The remaining studies were conducted by researchers from South Korea and Taiwan and focused on the impact of ChatGPT on the education sector and its associated opportunities and challenges. This cluster demonstrates that students could benefit greatly from using ChatGPT in performing various academic tasks, such as reviewing and revising their work, verifying the accuracy of homework answers, and improving the quality of their essays. It has also aided postgraduates whose first language is not English improve their writing, as ChatGPT can be instructed to rewrite a paragraph in a scholarly tone from scratch. The outcomes have demonstrated significant efficacy, thereby alleviating the cognitive load associated with translation for these students, enabling them to concentrate on the substance of their writing rather than the intricacies of composing in an unfamiliar language. To harness the potential benefits, future research could focus on developing targeted training programs for students and educators that emphasise the effective utilisation of ChatGPT to enhance not only academic tasks but also language proficiency for non-native English speakers, addressing both cognitive load and language intricacies.

Cluster 2: ChatGPT and Academic Integrity : Cluster 2 comprises research studies conducted by authors from Japan, Bangladesh, Hong Kong, Nigeria, Pakistan, UAE, the UK, Vietnam and the Netherlands. The most influential study in this cluster, “Unlocking the power of ChatGPT: A framework for applying Generative AI in education”, was authored by researchers from Hong Kong (Su & Yang, 2023 ). They document that ChatGPT can be used to respond to student inquiries, reducing the time and effort required of educators and allowing them to focus their resources on other activities, such as scholarly investigations. Farrokhnia et al. ( 2023 ) and Yeadon et al. ( 2023 ) state that ChatGPT can write scientific abstracts with fabricated data and essays that can evade detection by reviewers. According to Liebrenz et al. ( 2023 ), ChatGPT tends to produce erroneous and incoherent responses, thereby raising the potential for disseminating inaccurate information in scholarly literature. The higher-order cognitive abilities of ChatGPT are relatively low, especially in areas related to creativity, critical thinking, reasoning, and problem-solving. ChatGPT could reduce students’ motivation to explore topics independently, draw their own conclusions, and solve problems independently (Kasneci et al., 2023 ). Ibrahim et al. ( 2023 ) find that ChatGPT can engage students in their academic pursuits. ChatGPT can enhance the writing abilities of non-native English speakers to allow them to concentrate on higher-order cognitive processes. This technological development allows faculty members to allocate more attention to conceptualisation and writing rather than focusing on the mechanics of grammar and spelling. However, there is a debate among intellectuals regarding the implications of AI for content creation, with some asserting that it detracts from innovative content development. The possibility that ChatGPT threatens academic honesty by facilitating essay plagiarism is being acknowledged. In addition, in the absence of appropriate citations, this textual content may violate copyright regulations. Cotton et al. ( 2023 ) express concerns about the potential impact of ChatGPT on academic integrity and plagiarism. Their work corroborates Dehouche’s ( 2021 ) assertion that students may use ChatGPT to engage in academic dishonesty by submitting essays that are not their original work. According to Cotton et al. ( 2023 ), ChatGPT users have a competitive advantage over non-users and can achieve higher grades on their coursework assignments by utilising the AI-based language tool. They classify ChatGPT as a versatile instrument with the potential to pose a threat to academic integrity, noting that AI essay writing systems are specifically programmed to generate content based on specific parameters or prompts, thereby challenging the discernment between human-authored and machine-generated content. Distinguishing between the academic work produced by students and the content of ChatGPT when evaluating assignments is a significant challenge for faculty. It is recommended that academic staff continually monitor student assignments for academic misconduct infractions, coupled with transparent communication about the potential risks associated with AI-generated content.

Cluster 3: ChatGPT and Students’ Learning : Cluster 3 comprises Malaysia, China and Australia. This cluster mainly includes studies of the role of AI-based models in student learning. Researchers from Australia (Crawford et al., 2023 ; Lim et al., 2023 ; Lawrie, 2023 ; Li et al., 2023 ; Seth et al., 2023 ; Cingillioglu, 2023 ; Skavronskaya, 2023 ; and Johinke, 2023 ) have contributed the most (8 studies) to this cluster and put their weight behind the role of AI and student learning in various disciplines. One of the most influential papers, “Generative AI and the future of education: Ragnarök or reformation? A paradoxical perspective from management educators”, was authored by researchers from both Australia and Malaysia (Lim et al., 2023 ) and reflected on the role of AI in classroom learning and teaching. Rather than banning AI tools, the authors advocate for the productive use of these tools in classrooms to facilitate more engaging student learning. Another Australian study titled, “Leadership is needed for ethical ChatGPT: Character, assessment, and learning using artificial intelligence (AI)” (Crawford et al., 2023 ) highlights AI as an alternative path of learning for students. ChatGPT can promptly evaluate students’ assignments and help them identify areas of weakness. Educators have the option to provide innovative assessments to their students instead of adhering solely to conventional assessments. ChatGPT can augment pedagogical approaches, evaluation structures, and the comprehensive educational milieu by reinforcing the trilateral association among instructors, learners, and technology. The implementation of ChatGPT can provide students with a personalised and interactive learning and research experience facilitated by virtual tutors and customised recommendations. In light of the research in this cluster, the integration of ChatGPT into education should inspire a paradigm shift towards a more dynamic and personalised learning environment. Institutions can explore strategic partnerships with AI researchers to develop context-specific applications of ChatGPT that cater to diverse educational needs, promoting a symbiotic relationship between human instructors, students, and technology for an enriched learning experience.

Cluster 4: ChatGPT and Field-specific Research : This cluster includes research by authors in Asian and European countries (India, Oman, Bulgaria and New Zealand) that has emphasised the potential role of ChatGPT in the medical and tourism industries. Authors from India explored the role of ChatGPT in the medical field (Seetharaman, 2023 ; Subramani et al., 2023 ). Seetharaman ( 2023 ) reports that ChatGPT offers supplementary language assistance to students who are not proficient in English, enabling them to enhance their language proficiency and effectively communicate in English, the principal language of instruction in medical establishments. The ChatGPT platform has the potential to serve as a tool for medical students to replicate patient interactions in a simulated environment, such as accurately obtaining medical histories and documenting symptoms. According to Subramani et al. ( 2023 ), ChatGPT is a highly efficient and user-friendly AI technology that can aid healthcare professionals in various aspects, such as diagnosis, critical decision-making, and devising appropriate treatment plans. ChatGPT has demonstrated impressive performance on medical exams, indicating its potential as a valuable resource for enhancing medical education and assessment (Subramani et al., 2023 ) and can support interdisciplinarity in tourism research (Nautiyal et al., 2023 ). Ivanov and Soliman ( 2023 ) note the potential of ChatGPT to serve as a digital instructor to provide students with enhanced and effective learning experiences and outcomes. Digital instructors can impart knowledge in diverse languages and thus can be used to educate individuals of varying nationalities and backgrounds in the field of tourism. Furthermore, LLM-based chatbots, including ChatGPT, can assess written assignments and provide direction on linguistic proficiency, syntax, and composition, ultimately enhancing students’ scholarly writing proficiency. In exploring the intersection of ChatGPT with medical education, institutions can pioneer innovative approaches by using the platform to create immersive, simulated patient interactions that go beyond language assistance, allowing medical students to practice nuanced skills such as medical history gathering and symptom documentation. Simultaneously, leveraging ChatGPT as a versatile digital instructor offers a unique opportunity to provide cross-cultural and multilingual education, contributing to a more inclusive and globally competent workforce within the tourism industry.

3.4 Challenges of ChatGPT in higher education

In addition to some previously mentioned challenges, such as the potential for plagiarism, the investigation also identified other key challenges in implementing ChatGPT within the context of higher education’s teaching and learning environment. Wu and Yu ( 2023 ) found that the benefits of AI-based ChatGPT are more in higher education as compared to primary and secondary education. The study also reported that the novelty effects of AI chatbots may enhance learning outcomes in brief interventions, but their efficacy diminishes in longer interventions.

First, the implementation of ChatGPT within the educational context engenders learning impediments. In the absence of adequate monitoring and regulation, the technology could lead to human unintelligence and unlearning, but teachers will become more adaptive and create authentic assessments to enhance student learning (Alafnan et al., 2023 ; Lawrie, 2023 ). Second, the technology could be used in a manner that violates students’ privacy. If the model is not adequately secured, it could surreptitiously gather confidential data from students without their explicit awareness or authorisation (Kanseci, 2023). Third, the technology could facilitate discrimination against particular students. If the model is not trained on a dataset that accurately represents the entire student population, it has the potential to create disparities in educational access (Cingillioglu, 2023 ; Lin et al., 2023 ). Fourth, according to Ivanov and Soloman (2023), ChatGPT lacks access to real-time data. Therefore, its responses may be inconsequential, inaccurate, or outdated. The information provided in response to a specific query may also be insufficient. Gao et al. (2022) highlight the need for further investigation of the precision and scholarly authenticity of ChatGPT. Fifth, it may be difficult for ChatGPT to comprehend the context and subtleties of complex academic subjects and answer complex questions (Adetayo, 2023 ; Eysenbach, 2023 ; Neumann et al., 2023 ). The system can misinterpret inquiries, offer inadequate or inaccurate responses, or struggle to comprehend the fundamental purpose behind questions (Clark, 2023 ). In particular, ChatGPT may not have the requisite expertise in highly specialised or advanced subjects such as advanced mathematics or specific sciences. Hence, it may not deliver precise and accurate answers (Neumann et al., 2023 ; Fergus et al., 2023 ). Karaali ( 2023 ) claimed that the primary emphasis in the field of AI is currently directed towards the enhancement of advanced cognitive abilities and mental processes associated with quantitative literacy and quantitative reasoning. However, it is important to acknowledge that fundamental skills such as writing, critical thinking, and numeracy continue to serve as essential foundational components among students. Although AI is making significant progress in fundamental domains, it appears that students are experiencing a decline in performance in the context of fundamental skills. Consequently, NLP-based adaptive learner support and education require further investigation (Bauer et al., 2023 ).

In addressing the challenges of ChatGPT in education, educators need to adapt and develop authentic assessments that mitigate the risk of human unlearning, ensuring that technology enhances, rather than hinders, student learning experiences. Simultaneously, recognising the limitations of ChatGPT in comprehending the nuances of highly specialised subjects underscores the importance of balancing advancements in AI’s cognitive abilities with continued emphasis on fundamental skills like critical thinking, writing, and numeracy, urging a reevaluation of priorities in AI-driven educational research towards comprehensive learner support.

4 Conclusion, implications and agenda for future research

This study identified the most influential articles and top journals and countries in terms of citations and publication productivity related to ChatGPT in higher education, as well as highlighted emerging thematic clusters and geographic clusters in research on the role and challenges of ChatGPT in teaching and learning in higher education institutions. Articles on the topic of ChatGPT in higher education published up to May 2023 were identified by searching the Scopus database. Given the emergent nature of ChatGPT starting in late 2022, all the included articles were published in 2023. Thus, this specific research domain remains relatively unexplored. The findings of this analysis reveal that the United States is the most productive country in terms of research on the role of ChatGPT in higher education, especially relating to academic integrity and research. US researchers also emerged as the most influential in terms of number of citations in the literature. Our findings corroborate those of previous research (Crompton & Burke, 2023 ). However, 60% of the articles in our shortlisted literature emanated from Asian countries.

Four thematic clusters (academic integrity, student engagement, learning environment and research) were identified. Furthermore, the country-based bibliographic analysis revealed that research has focused on student examinations, academic integrity, student learning and field-specific research in medical and tourism education (Nautiyal et al., 2023 ; Subramani et al., 2023 ). Plagiarism is recognised as a major challenge that hinders students’ creativity, innovativeness and originality when using ChatGPT in their academic pursuits. To mitigate the potential drawbacks of using ChatGPT in educational and research settings, proactive measures should be taken to educate students and researchers alike on the nature of plagiarism, its negative impacts and academic integrity (Shoufan, 2023 ; Teixeira, 2023 ) Educators may ask students to provide a written acknowledgement of the authenticity of their assignments and their non-reliance on ChatGPT. Such an acknowledgement would discourage students from utilising ChatGPT in their academic and research endeavours and establish accountability for their academic pursuits. In addition, educators should develop authentic assessments that are ChatGPT-proof.

ChatGPT lacks emotional intelligence and empathy, both of which are crucial in effectively addressing the emotional and psychological dimensions of the learning process (Farrokhnia et al., 2023 ; Neumann et al., 2023 ). Higher education institutions may encounter challenges in using ChatGPT to deliver suitable assistance, comprehension, or direction to students needing emotional or mental health support. The significance of human interaction in learning cannot be overstated. Achieving a balance between using AI and the advantages of human guidance and mentorship is a persistent challenge that requires attention (Neumann et al., 2023 ; Rahman et al., 2023 ). Strzelecki ( 2023 ) observed in his research that behavioural intention and personal innovativeness are the two major determinants behind the adoption of ChatGPT among students.

4.1 Implications

The findings of the present study have numerous important implications. This study provides insight into the current state of ChatGPT in higher education and thus can serve as valuable guidance for academics, practitioners, and policymakers. The study’s findings contribute to the literature by providing new insights into the role of ChatGPT and strategies for mitigating its negative aspects and emphasising its positive attributes.

First, the implementation of AI in education can improve academic performance and student motivation, particularly by facilitating personalised learning. Educational institutions should monitor and regulate students’ use of such technologies proactively. Higher education institutions also ought to prioritise the training of their educators in effectively utilising AI technologies, including ChatGPT. Concurrently, it is imperative for these institutions to equip students with comprehensive academic integrity training, shedding light on the appropriate and inappropriate applications of AI tools like ChatGPT. This includes creating awareness about the potential consequences of utilising these technologies for dishonest practices. Furthermore, educational establishments need to urgently revisit and refine their academic integrity policies to address the evolving landscape shaped by the integration of artificial intelligence tools in various academic facets. This proactive approach will foster a learning environment that embraces technological advancements and upholds the principles of honesty and responsible use. Institutional regulations on accountability and transparency should guide the frameworks that govern the use of AI in the campus environment (Pechenkina, 2023 ; Sun & Hoelscher, 2023 ; Dencik & Sanchez-Monedero, 2022 ).

Second, faculty members must proactively replace traditional coursework with modern alternatives that foster elevated levels of critical thinking among students, as suggested by Zhai ( 2022 ). Educators and learners can augment the academic material produced by ChatGPT with their own insights and information obtained from credible scholarly resources (Emenike & Emenike, 2023 ).

Third, ChatGPT should not be considered a threat to the education sector but a supplementary tool for human instruction that can enhance teaching and learning. It is imperative to acknowledge that the vital role of human educators cannot be replaced (Karaali, 2023 ) Moreover, ChatGPT can potentially enhance the accessibility and inclusivity of higher education. Alternative formats, linguistic support, and individualised explanations can help students who are studying English as a second language, are not native English speakers, or have other unique learning needs. Furthermore, Alnaqbi and Fouda ( 2023 ) highlight the implications of AI in evaluating the teaching style of faculty in higher education by collecting the feedback of students through social media and ChatGPT.

Fourth, the faculty in higher education institutions could address ethical concerns by providing students with explicit and comprehensive guidelines about the prescribed structure of academic assignments (Cotton et al., 2023 ; Gardner & Giordano, 2023 ). This practice can facilitate the production of more cohesive assignments. In addition, teachers can use rubrics to assess assignments and blend automated and manual assessment methodologies to evaluate students’ comprehension of the subject matter (Cotton et al., 2023 ; Shoufan, 2023 ).

In summary, using ChatGPT is recommended for enhancing creativity, refining writing proficiency, and improving research abilities. Nonetheless, it is crucial to emphasise that ChatGPT should not be employed as a substitute for critical thinking and producing original work. While it serves as a valuable tool for augmentation, upholding the integrity of independent thought and authentic content creation in academic endeavours is essential.

4.2 Limitations

The present study acknowledges several limitations. Firstly, the reliance on Scopus as the primary data source for bibliometric analysis may have limitations in capturing the full landscape of relevant literature. Future research may consider incorporating additional databases like Web of Science to ensure a comprehensive assessment. Secondly, due to the English language restriction in the review, potentially relevant studies may have been omitted. Future research could enhance inclusivity by extending its scope to encompass papers written in languages other than English. Thirdly, the current study exclusively focused on journal articles. Expanding the scope to include diverse sources, such as conference proceedings or book chapters, could offer a more comprehensive overview.

Additionally, as a rapidly evolving field, literature published after our inclusion dates need capturing, and future studies should consider adjusting their inclusion criteria to accommodate the dynamic nature of the subject matter. Lastly, the specificity of the bibliometric data search, centred around terms like ChatGPT, AI, higher education, and academic integrity, may have excluded certain relevant articles. Future studies should consider employing more generalised search parameters to encompass synonyms associated with these terms.

4.3 Future scope

The findings of the study suggest new avenues for future research. The effectiveness of evaluation criteria for assessments incorporating ChatGPT-generated text needs to be investigated. Specifically, the appropriate level of ChatGPT-produced text that students may use in academic tasks or assessments has not been established. Research on the ethical implications of using AI tools such as ChatGPT in higher education is also needed. Issues pertaining to data confidentiality, bias, and transparency in algorithms used for decision-making remain to be addressed. Feasible approaches for mitigating the excessive reliance of scholars and learners on ChatGPT or similar AI models are needed. Researchers could also explore the implementation of verification processes that go beyond traditional plagiarism detection methods, accounting for the unique challenges posed by AI systems. Future research in this domain could focus on establishing guidelines and best practices for the integration of AI tools like ChatGPT in academic settings, ensuring a balance between technological innovation and the preservation of academic rigour. Finally, the literature on ChatGPT in higher education has largely focused on the medical and tourism sectors. Future researchers must explore applications of ChatGPT in other disciplines.

Data availability

The datasets generated during and/or analysed during the current study are available from the corresponding author upon reasonable request.

Adetayo, A. J. (2023). ChatGPT and librarians for reference consultations. Internet Reference Services Quarterly , 27 (3), 131–147.

Article   Google Scholar  

AlAfnan, M. A., Dishari, S., Jovic, M., & Lomidze, K. (2023). ChatGPT as an educational tool: Opportunities, challenges, and recommendations for communication, business writing, and composition courses. Journal of Artificial Intelligence and Technology , 3 (2), 60–68. https://doi.org/10.37965/jait.2023.0184 .

Alnaqbi, N. M., & Fouda, W. (2023). Exploring the role of ChatGPT and social media in enhancing student evaluation of teaching style in higher education using Neutrosophic sets. International Journal of Neutrosophic Science , 20 (4), 181–190.

Angelis, L. D., Baglivo, F., Arzilli, G., Privitera, G. P., Ferragina, P., Tozzi, A. E., & Rizzo, C. (2023). ChatGPT and the rise of large language models: The new AI-driven infodemic threat in public health. Frontiers in Public Health , 11 , 1–8.

Archambault, E., Campbell, D., Gingras, Y., & Larivière, V. (2009). Comparing bibliometric statistics obtained from the web of Science and Scopus. Journal of the American Society for Information Science and Technology , 60 (7), 1320–1326.

Bajaj, Y., Kashiramka, S., & Singh, S. (2020). Application of capital structure theories: A systematic review. Journal of Advances in Management Research , 18 (2), 173–199. https://doi.org/10.1108/JAMR-01-2020-001 .

Bauer, E., Greisel, M., Kuznetsov, I., Berndt, M., Kollar, I., Dresel, M., Fischer, M. R., & Fischer, F. (2023). Using natural language processing to support peer-feedback in the age of artificial intelligence: A cross-disciplinary framework and a research agenda. British Journal of Educational Technology , 54 (5), 1222–1245.

Boyack, K. W., & Klavans, R. (2010). Co-citation analysis, bibliographic coupling, and direct citation: Which citation approach represents the research front most accurately? Journal of the American Society for Information Science and Technology , 61 (12), 2389–2404.

Cingillioglu, I. (2023). Detecting AI-generated essays: The ChatGPT challenge. International Journal of Information and Learning Technology , 40 (3), 259–268. https://doi.org/10.1108/IJILT-03-2023-0043 .

Clark, T. M. (2023). Investigating the Use of an Artificial Intelligence Chatbot with General Chemistry exam questions. Journal of Chemical Education , 100 (5), 1905–1916.

Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2023). Chatting and cheating: Ensuring academic integrity in the era of ChatGPT. Innovations in Education and Teaching International , 1–13. https://doi.org/10.1080/14703297.2023.2190148 .

Crawford, J., Cowling, M., & Allen, K. (2023). Leadership is needed for ethical ChatGPT: Character, assessment, and learning using artificial intelligence (AI). Journal of University Teaching & Learning Practice , 20 (3), 1–21. https://doi.org/10.53761/1.20.3.02 .

Crompton, H., & Burke, D. (2023). Artificial intelligence and higher education: The state of the field. International Journal of Educational Technology in Higher Education , 20 , 1–22. https://doi.org/10.1186/s41239-023-00392-8 .

Dehouche, N. (2021). Plagiarism in the age of massive generative pre-trained transformers (GPT-3). Ethics in Science and Environmental Politics , 2 , 17–23. https://doi.org/10.3354/esep00195 .

Dencik, L., & Sanchez-Monedero, J. (2022). Data justice. Internet Policy Review , 11 (1), 1–16. https://doi.org/10.14763/2022.1.1615 .

Donthu, N., Kumar, S., & Pattnaik, D. (2020). Forty-five years of Journal of Business Research : A bibliometric analysis. Journal of Business Research , 109 , 1–14. https://doi.org/10.1016/j.jbusres.2019.10.039 .

Emenike, M., & Emenike, B. (2023). Was this title generated by ChatGPT? Considerations for artificial intelligence text-generation software programs for chemists and chemistry educators. Journal of Chemical Education , 100 (4), 1413–1418. https://doi.org/10.1021/acs.jchemed.3c00063 .

Eysenbach, G. (2023). The role of ChatGPT, generative language models, and artificial intelligence in medical education: A conversation with ChatGPT and a call for papers. JMIR Medical Education , 6 (9), 1–13. https://doi.org/10.2196/46885 .

Farrokhnia, M., Banihashem, K. S., Noroozi, O., & Wals, A. (2023). A SWOT analysis of ChatGPT: Implications for educational practice and research. Innovations in Education and Teaching International , 1–15. https://doi.org/10.1080/14703297.2023.2195846 .

Fergus, S., Botha, M., & Ostovar, M. (2023). Evaluating academic answers generated using ChatGPT. Journal of Chemical Education , 100 (4), 1672–1675. https://doi.org/10.1021/acs.jchemed.3c00087 .

Firat, M. (2023). How chat GPT can transform autodidactic experiences and open education. Department of Distance Education, Open Education Faculty , Anadolu University, 1–6. https://doi.org/10.31219/osf.io/9ge8m .

Gardner, D. E., & Giordano, A. E. (2023). The challenges and value of undergraduate oral exams in the physical chemistry classrooms: A useful tool in the assessment toolbox. Journal of Chemical Education , 100 (5), 1705–1709.

Geerling, W., Mateer, G. D., Wooten, J., & Damodaran, N. (2023). ChatGPT has aced the test of understanding in college economics: Now what? The American Economist , 68 (2), 233–245.

Gilson, A., Safranek, C. W., Huang, T., Socrates, V., Chi, L., Taylor, R. A., & Chartash, D. (2023). How does ChatGPT perform on the United States medical licensing examination? The implications of large language models for medical education and knowledge assessment. JMIR Medical Education , 9 , 1–9. https://doi.org/10.2196/45312 .

Gisev, N., Bell, J. S., & Chen, T. F. (2013). Interrater agreement and interrater reliability: Key concepts, approaches, and applications. Research in Social and Administrative Pharmacy , 9 (3), 330–338.

Gupta, R., Pandey, R., & Sebastian, V. J. (2021). International entrepreneurial orientation (IEO): A bibliometric overview of scholarly research. Journal of Business Research , 125 , 74–88. https://doi.org/10.1016/j.jbusres.2020.12.00 .

Ibrahim, H., Asim, R., & Zaffar, F. (2023). Rethinking homework in the age of artificial intelligence. Intelligent Systems , 38 (2), 24–27.

Google Scholar  

Iskender, A. (2023). Holy or Unholy? Interview with Open AI’s ChatGPT. European Journal of Tourism Research , 34 , 3414. https://doi.org/10.54055/ejtr.v34i.3169 .

Ivanov, S., & Soliman, M. (2023). Game of algorithms: ChatGPT implications for the future of tourism education and research. Journal of Tourism Futures , 9 (2), 214–221.

Jarneving, B. (2007). Bibliographic coupling and its application to research-front and other core documents. Journal of Informetrics , 1 (4), 287–307.

Jeon, J., Lee, S., & Cho, S. (2023). A systematic review of research on speech-recognition chatbots for language learning: Implications for future directions in the era of large language models. Interactive Learning Environment , 1–19. https://doi.org/10.1080/10494820.2023.2204343 .

Johinke, R., Cummings, R., & Di Lauro, F. (2023). Reclaiming the technology of higher education for teaching digital writing in a post-pandemic world. Journal of University Teaching and Learning Practice , 20 (2), 1–18. https://doi.org/10.53761/1.20.02.01 .

Karaali, G. (2023). Artificial Intelligence, Basic skills, and quantitative literacy. Numeracy , 16 (1), 1–22. https://doi.org/10.5038/1936-4660.16.1.1438 .

Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., & Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences , 103 (2023), 102274–102271. https://doi.org/10.1016/j.lindif.2023.102274 .

Lawrie, G. (2023). Establishing a delicate balance in the relationship between artificial intelligence and authenticate assessment in student learning. Chemistry Education Research and Practice , 24 (2), 392–393.

Lee, H. (2023). The rise of ChatGPT: Exploring its potential in medical education. Anatomical Science Education , 1–6. https://doi.org/10.1002/ase.2270 .

Li, Y., Sha, L., Yan, L., Lin, J., Raković, M., Galbraith, K., Lyons, K., Gašević, D., & Chen, G. (2023). Can large language models write reflectively. Computer and Education: Artificial Intelligence , 4 (2023), 100140–100141.

Liebrenz, M., Schleifer, R., Buadze, A., Bhugra, D., & Smith, A. (2023). Generating scholarly content with ChatGPT: Ethical challenges for medical publishing. Lancet Digital Health , 5 (3), e105–106. https://doi.org/10.1016/S2589-7500(23)00019-5 .

Lim, W. M., Gunasekara, A., Pallant, J. L., Pallant, J. I., & Pechenkina, E. (2023). Generative AI and the future of education: Ragnarök or reformation? A paradoxical perspective from management educators. The International Journal of Management Education , 21 (2), 1–13.

Lin, C. C., Huang, A. Y. Q., Stephen, J. H., & Yang (2023). A review of AI-Driven conversational chatbots implementation methodologies and challenges (1999–2022). Sustainability , 15 (5), 4012. https://doi.org/10.3390/su15054012 .

Memarian, B., & Doleck, T. (2023). ChatGPT in education: Methods, potentials and limitations. Computers in Human Behavior: Artificial Humans , 100022 , 1–11.

Mishra, D., Gunasekaran, A., Papadopoulos, T., & Hazen, B. (2017). Green supply chain performance measures: A review and bibliometric analysis. Sustainable Production and Consumption , 10 , 85–99. https://doi.org/10.1016/j.spc.2017.01.003

Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Annals of Internal Medicine , 151 (4), 264–269.

Nautiyal, R., Albrecht, J. N., & Nautiyal, A. (2023). ChatGPT and tourism academia. Annals of Tourism Research , 99 , 1–3.

Neumann, M., Rauschenberger, M., & Schon, E-M. (2023). 2023 IEEE/ACM 5th International Workshop on Software Engineering Education for the Next Generation (SEENG), Melbourne, Australia, 2023, pp. 29–32, https://doi.org/10.1109/SEENG59157.2023.00010 .

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., … & Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ , 372 (71), 1–9. https://doi.org/10.1136/bmj.n71

Palomo, J., Figueroa-Domecq, C., & Laguna, P. (2017). Women, peace and security state-of-art: A bibliometric analysis in social sciences based on SCOPUS database. Scientometrics , 113 (1), 123–148. https://doi.org/10.1007/s11192-017-2484-x .

Patrício, L. D., & Ferreira, J. J. (2020). Blockchain security research: Theorizing through bibliographic-coupling analysis. Journal of Advances in Management Research , 18 (1), 1–35. https://doi.org/10.1108/JAMR-04-2020-0051 .

Paul, J., Lim, W. M., O’Cass, A., Hao, A. W., & Bresciani, S. (2021). Scientific procedures and rationales for systematic literature reviews (SPAR-4-SLR). International Journal of Consumer Studies . https://doi.org/10.1111/ijcs.12695 .

Pechenkina, K. (2023). Artificial intelligence for good? Challenges and possibilities of AI in higher education from a data justice perspective. In L. Czerniewicz, & C. Cronin (Eds.), Higher education for good: Teaching and learning futures (#HE4Good) . Open Book.

Perkins, M. (2023). Academic integrity considerations of AI large Language models in the post-pandemic era: ChatGPT and beyond. Journal of University Teaching & Learning Practice , 20 (2), 1–26. https://doi.org/10.53761/1.20.02.07 .

Rahman, M., Mostafizer, & Yutaka Watanobe. (2023). ChatGPT for Education and Research: Opportunities, threats, and strategies. Applied Sciences , 13 (9), 1–21. https://doi.org/10.3390/app13095783 .

Rana, S. (2023). AI and GPT for Management scholars and practitioners: Guidelines and implications. FIIB Business Review , 12 (1), 7–9.

Seetharaman, R. (2023). Revolutionising medical education: Can ChatGPT boost subjective learning and expression? Journal of Medical System , 47 (61), 1–4. https://doi.org/10.1007/s10916-023-01957-w .

Seth, I., Bulloch, G., & Lee, C. H. A. (2023). Redefining academic integrity, authorship, and innovation: The impact of ChatGPT on surgical research. Annals of Surgical Oncology , 30 , 5284–5285. https://doi.org/10.1245/s10434-023-13642-w .

Shoufan, A. (2023). Exploring students’ perceptions of ChatGPT: Thematic analysis and follow-up survey. IEEE Access , 11, 38805–38818, 2023, https://doi.org/10.1109/ACCESS.2023.3268224 .

Singh, V. K., Singh, P., Karmakar, M., Leta, J., & Mayr, P. (2021). The journal coverage of web of Science, Scopus and dimensions: A comparative analysis. Scientometrics , 126 , 5113–5142.

Skavronskaya, L., Hadinejad, A., & Cotterell, D. (2023). Reversing the threat of artificial intelligence to opportunity: A discussion of ChatGPT in tourism education. Journal of Teaching in Travel & Tourism , 23 , 1–6. https://doi.org/10.1080/15313220.2023.2196658 .

Strzelecki, A. (2023). To use or not to use ChatGPT in higher education? A study of students’ acceptance and use of technology. Interactive Learning Environments , 1–15. https://doi.org/10.1080/10494820.2023.2209881 .

Su, J., & Yang, W. (2023). Unlocking the power of ChatGPT: A framework for applying generative AI in education. ECNU Review of Education , 6 (3), 355–366.

Subramani, M., Jallel, I., & Mohan, S. K. (2023). Evaluating the performance of ChatGPT in medical physiology university examination of phase I MBBS. Advances in Physical Education , 47 , 270–271.

Sun, G. H., & Hoelscher, S. H. (2023). The ChatGPT Storm and what Faculty can do. Nurse Educator , 48 , 119–124.

Tang, G. (2023a). Academic journals cannot simply require authors to declare that they used ChatGPT. Irish Journal of Medical Science , 192 (6), 3195–3196.

Tang, G. (2023b). Letter to editor: Academic journals should clarify the proportion of NLP-generated content in papers. Accountability in Research , 1–3. https://doi.org/10.1080/08989621.2023.2180359 .

Teixeira da Silva, J. A. (2023). How are authors’ contributions verified in the ICMJE model? Plant Cell Reports , 42 , 1529–1153. https://doi.org/10.1007/s00299-023-03022-9 .

Thorp, H. H. (2023). ChatGPT is fun, but not an author. Science , 379 (6630), 313–313.

Tlili, A., Shehata, B., Adarkwah, M., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023). What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments , 10 (15), 1–24.

Van Eck, N., & Waltman, L. (2010). Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics , 84 (2), 523–538.

Vijaya, V., & Mathur, H. P. (2023). A decade of donation-based crowdfunding: A bibliometric analysis using the SCOPUS Database. Purushartha , 15 (2), 32–51.

Vogels, E. A. (2023). A majority of Americans have heard of ChatGPT, but few have tried it themselves, https://www.pewresearch.org/short-reads/2023/05/24/a-majority-of-americans-have-heard-of-chatgpt-but-few-have-tried-it-themselves/ .

Wu, R., & Yu, Z. (2023). Do AI Chatbots improve students learning outcome? Evidence from a meta analysis. British Journal of Educational Technology . https://doi.org/10.1111/bjet.13334 .

Wu, T. Y., He, S. Z., Liu, J. P., Sun, S. Q., Liu, K., Han, Q. L., & Tang, Y. (2023). A brief overview of ChatGPT: The history, status quo and potential future development. IEEE/CAA Journal of Automatica Sinica , 10 (5), 1122–1136. https://doi.org/10.1109/JAS.2023.123618 .

Yan, D. (2023). Impact of ChatGPT on learners in a L2 writing practicum: An exploratory investigation. Education and Information Technologies , 1–25. https://doi.org/10.1007/s10639-023-11742-4 .

Yeadon, W., Inyang, O-O., Mizouri, A., Peach, A., & Testrow, C. P. (May 2023). The death of the short-term physics essay in the coming AI revolution. Physics Education , 58 , 1–13.

Zhai, X. (2022). ChatGPT user experience: Implications for education. SSRN , 4312418 , 1–18.

Download references

Open Access funding enabled and organized by CAUL and its Member Institutions

Author information

Authors and affiliations.

University Business School, Maharaja Ranjit Singh Punjab Technical University, Punjab, India

Pritpal Singh Bhullar

Department of Financial Planning and Tax, School of Accounting, Information Systems and Supply Chain, RMIT University, Melbourne, Australia

Mahesh Joshi

School of Engineering and Technology, CML‑NET & CREATE Research Centres, Central Queensland University, Queensland, Australia

Ritesh Chugh

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Ritesh Chugh .

Ethics declarations

Conflict of interest.

No Conflict of Interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Bhullar, P.S., Joshi, M. & Chugh, R. ChatGPT in higher education - a synthesis of the literature and a future research agenda. Educ Inf Technol (2024). https://doi.org/10.1007/s10639-024-12723-x

Download citation

Received : 04 October 2023

Accepted : 16 April 2024

Published : 02 May 2024

DOI : https://doi.org/10.1007/s10639-024-12723-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Artificial Intelligence
• Generative AI
• Higher education
• Academic integrity
• Systematic review
• Bibliometric analysis
• Find a journal
• Publish with us
• Track your research

Have a thesis expert improve your writing

Check your thesis for plagiarism in 10 minutes, generate your apa citations for free.

• Knowledge Base

Using AI tools

• How to Write a Paper with ChatGPT | Tips & Examples

How to Write a Paper with ChatGPT | Tips & Examples

Published on 17 August 2023 by Koen Driessen .

A good research paper demonstrates your academic writing skills and knowledge of your chosen topic.

Your research paper should be based on in-depth independent research. However, generative AI tools like ChatGPT can be effectively used throughout the research process to:

• Brainstorm research questions
• Develop a methodology
• Create an outline
• Find sources
• Summarise and paraphrase text
• Provide feedback

Instantly correct all language mistakes in your text

Be assured that you'll submit flawless writing. Upload your document to correct all your mistakes.

Table of contents

Brainstorming research questions, developing a methodology, creating an outline, finding sources, summarising or paraphrasing text, receiving feedback, other interesting articles, frequently asked questions.

You can use ChatGPT to help formulate research questions , brainstorm potential thesis statements , or narrow down a broad subject area to a specific topic that interests you.

However, make sure that the outputs make sense and meet the requirements of your assignment. You can adapt these outputs manually, or use further prompts to ensure they meet your needs. For more information, see our tips for creating effective ChatGPT prompts .

• Generate three possible research questions for a research paper on the following topic: “The role of virtual reality in enhancing training and skill development in various industries.”
• “What are the cognitive and skill development benefits of virtual reality-based training compared to traditional training methods across different industries?”
• “How does the level of immersion in virtual reality training impact knowledge retention and skill transfer in fields such as medicine, aviation, and manufacturing?”
• “What are the key factors that influence the successful integration and adoption of virtual reality technology for training purposes in diverse industries, and what challenges hinder its widespread implementation?”

The only proofreading tool specialized in correcting academic writing

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Correct my document today

Your paper should clearly indicate the data collection and analysis methods you used in your research.

You can use ChatGPT to help decide what kind of research you will conduct (e.g., qualitative or quantitative ), how you will analyse the data, and how you will mitigate potential research bias .

Bear in mind that you should be able to rationalize your chosen methodology . For example, you should ensure that it is either a standard methodology in the field or can be justified.

• Suggest three quantitative research methods appropriate to the following question: “What are the cognitive and skill development benefits of virtual reality-based training compared to traditional training methods in the field of healthcare?”
• Experimental Study: Conduct a randomized controlled trial (RCT) where healthcare professionals are randomly assigned to either a virtual reality-based training group or a traditional training group. Both groups would receive training in specific healthcare tasks, and their cognitive and skill development outcomes would be measured and compared. Performance metrics, such as accuracy, speed, and efficiency, could be quantitatively assessed to determine the benefits of virtual reality-based training over traditional methods.
• Survey Research: Design a comprehensive survey targeting healthcare professionals who have undergone both virtual reality-based training and traditional training. The survey could include Likert-scale questions and structured response options to gauge their perceived cognitive and skill development benefits from each training method. Additionally, open-ended questions could be included to gather qualitative insights on specific strengths and weaknesses of each training approach.
• Task Performance Analysis: Implement a task-based assessment that simulates real-life healthcare scenarios, where participants are evaluated on their ability to handle critical situations. Participants would go through both virtual reality-based training and traditional training before being assessed in the simulated scenarios. Their performance in the assessment could be objectively measured using predefined criteria, allowing for a quantitative comparison of skill development between the two training methods.

A clear outline can help structure your paper, organise your arguments effectively, and make your research more efficient.

You can use generative AI tools like ChatGPT to formulate a potential outline to help guide your research. You can be as specific as you like, giving an overview of your research objectives , main arguments, and any evidence you intend to include in your paper.

However, ChatGPT outputs sometimes contain factual inaccuracies. It’s important to check the headings and subheadings suggested in the output and adapt them to meet your needs.

• Create an outline for an academic research paper with the following research question:“What are the cognitive and skill development benefits of virtual reality-based training compared to traditional training methods in the field of healthcare?” The paper will be about 10 pages long.

B. Existing studies on cognitive and skill development benefits of virtual reality training

C. Comparison of virtual reality-based training with traditional training methods

D. Gaps in the literature and research limitations

III. Theoretical Framework

A. Theoretical basis for cognitive and skill development in training

B. Application of cognitive theories to virtual reality-based training

C. The role of immersive learning environments in skill acquisition

D. Conceptual framework for the study

IV. Research Methodology

A. Research design and approach (e.g., quasi-experimental, pre-post intervention)

B. Selection of participants and sample size calculation

C. Description of the virtual reality-based training and traditional training programs

D. Instruments and measures for assessing cognitive and skill development

E. Data collection procedures and ethical considerations

V. Data Analysis

A. Overview of statistical methods used for data analysis

B. Treatment of data and data cleaning

C. Quantitative analysis of cognitive and skill development outcomes

D. Presentation of findings with charts and tables

VI. Results

A. Presentation of the results of the cognitive and skill development assessments

B. Comparison of outcomes between virtual reality-based training and traditional training

C. Statistical significance and effect size analysis

D. Interpretation of the results and implications

VII. Discussion

A. Interpretation of findings in the context of the research question

B. Comparison of results with existing literature

C. Explanation of how virtual reality-based training impacts cognitive processes and skill development

D. Limitations of the study and areas for future research

VIII. Practical Implications

A. Practical applications of virtual reality-based training in healthcare settings

B. Recommendations for integrating virtual reality training in healthcare education

C. Benefits of virtual reality for continuous professional development in healthcare

IX. Conclusion

A. Summary of key findings

B. Reiteration of the research question and hypothesis

C. Contributions of the study to the field of healthcare training

D. Concluding remarks

X. References

You can use ChatGPT to help find sources relevant to your research. However, it’s important to be aware of the limitations of ChatGPT (for example, it occasionally cites nonexistent sources).

Instead of asking ChatGPT for specific source recommendations, try asking it for recommendations on the types of sources relevant to your research topic. You can also use it to generate a list of relevant keywords to use in your literature search.

• What types of sources could I use to write a research paper on the following research question? “What are the cognitive and skill development benefits of virtual reality-based training compared to traditional training methods in the field of healthcare?”
• Academic Journals: Peer-reviewed journals are valuable sources of scholarly research. Look for articles in medical education, cognitive psychology, virtual reality applications in healthcare, and technology-enhanced learning.
• Research Studies: Seek out published research studies that specifically investigate the impact of virtual reality-based training on cognitive and skill development in healthcare professionals.
• Systematic Reviews and Meta-Analyses: These studies provide comprehensive overviews of existing research on a specific topic, making them a valuable source for summarizing and analyzing the overall evidence.
• Books and Book Chapters: Academic books and book chapters authored by experts in the fields of medical education, virtual reality, cognitive science, or instructional design can provide in-depth insights and theoretical foundations.
• Conference Proceedings: Proceedings from relevant conferences and symposiums may present the latest findings and advancements in virtual reality-based training and healthcare education.
• Educational Institution Websites: Universities and medical schools often publish research papers, case studies, and whitepapers related to medical education and virtual reality training.
• Industry Reports: Reports and whitepapers from healthcare organizations or technology companies exploring the benefits of virtual reality-based training in the healthcare sector.
• Government and Non-profit Organisations: Look for reports and publications from government agencies and non-profit organisations that focus on medical education and healthcare training.
• Online Databases: Utilize academic databases such as PubMed, Google Scholar, IEEE Xplore, and ScienceDirect to search for peer-reviewed research articles and conference papers.
• Expert Interviews: If possible, conduct interviews with experts in medical education, cognitive science, and virtual reality to gain valuable insights and perspectives.
• Surveys and Polls: Refer to reputable surveys and polls related to virtual reality-based training in the healthcare industry for additional statistical data.

You can use ChatGPT to paraphrase or summarise text. This can help you to condense sources to their most important points and explore new ways of expressing your ideas.

Alternatively you can use the more specialised tools featured on Scribbr’s AI writing resources page, which are designed specifically for these purposes and will give a smoother user experience.

• Provide a short summary of the following text:“One of the most profound applications of VR in healthcare is its use in surgical training. Traditionally, surgical training has relied on a combination of didactic lectures, observation of procedures, and hands-on experience in the operating room. However, these methods may not provide the ideal environment for novices to develop the necessary dexterity and spatial understanding essential for complex surgeries. VR overcomes these limitations by offering immersive, true-to-life simulations that allow surgeons to navigate through intricate anatomical structures, practice incisions, and learn to manage unexpected challenges that may arise during surgeries. This hands-on experience in a risk-free environment not only enhances technical skills but also builds confidence and reduces the learning curve for new surgical techniques.”
• Virtual Reality (VR) has a profound impact on healthcare, particularly in surgical training, as it provides immersive simulations that help surgeons develop dexterity, spatial understanding, and confidence for complex surgeries in a risk-free environment, ultimately enhancing their technical skills and reducing the learning curve.

When you’ve finished writing your research paper, you can use ChatGPT to receive feedback. You can be as specific as you like, selecting particular aspects the output should focus on (e.g., tone, clarity of structure, appropriateness of evidence to support your arguments).

You can also use ChatGPT to check grammar, spelling, and punctuation. However, it’s not designed for this purpose and occasionally misses errors. We recommend using a more specialised tool like Scribbr’s free grammar checker . Or, for more comprehensive feedback, Scribbr’s proofreading and editing service .

• Feedback the above text with regard to appropriateness of evidence and coherence of structure.
• The research paper demonstrates a well-structured and coherent presentation of the cognitive and skill development benefits of VR-based surgical training. The introduction effectively sets the context and rationale, while the literature review incorporates relevant historical and empirical evidence. The theoretical framework strengthens the foundation for the analysis.The methodology section could provide more details on the research design and study selection criteria. However, the sections on cognitive and skill development benefits present compelling evidence, supported by appropriate sub-sections.The exploration of improved patient outcomes lacks concrete examples but effectively outlines potential benefits. The section on challenges and future directions raises relevant points, but suggestions for solutions could enhance practicality.Overall, the paper combines theoretical, empirical, and practical evidence, offering a well-rounded approach. It establishes the significance of VR-based surgical training and concludes with a concise summary of key findings.

If you want more tips on using AI tools , understanding plagiarism , and citing sources , make sure to check out some of our other articles with explanations, examples, and formats.

• Citing ChatGPT
• Best grammar checker
• Best paraphrasing tool
• ChatGPT in your studies
• Is ChatGPT trustworthy?
• Types of plagiarism
• Self-plagiarism
• Avoiding plagiarism
• Academic integrity
• Best plagiarism checker

Citing sources

• Citation styles
• In-text citation
• Citation examples
• Annotated bibliography

Yes, you can use ChatGPT to summarise text . This can help you understand complex information more easily, summarise the central argument of your own paper, or clarify your research question.

You can also use Scribbr’s free text summariser , which is designed specifically for this purpose.

Yes, you can use ChatGPT to paraphrase text to help you express your ideas more clearly, explore different ways of phrasing your arguments, and avoid repetition.

However, it’s not specifically designed for this purpose. We recommend using a specialised tool like Scribbr’s free paraphrasing tool , which will provide a smoother user experience.

No, having ChatGPT write your college essay can negatively impact your application in numerous ways. ChatGPT outputs are unoriginal and lack personal insight.

Furthermore, Passing off AI-generated text as your own work is considered academically dishonest . AI detectors may be used to detect this offense, and it’s highly unlikely that any university will accept you if you are caught submitting an AI-generated admission essay.

However, you can use ChatGPT to help write your college essay during the preparation and revision stages (e.g., for brainstorming ideas and generating feedback).

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

Driessen, K. (2023, August 17). How to Write a Paper with ChatGPT | Tips & Examples. Scribbr. Retrieved 6 May 2024, from https://www.scribbr.co.uk/using-ai-tools/chatgpt-paper/

Is this article helpful?

Koen Driessen

Other students also liked, how to write good chatgpt prompts, chatgpt citations | formats & examples, ethical implications of chatgpt, still have questions.

• Do Not Sell My Personal Info

•  ⋅ 

How To Use ChatGPT For Keyword Research

ChatGPT for keyword research can be a powerful assistant. Understand any topic for keyword research in minutes with these prompts.

Anyone not using ChatGPT for keyword research is missing a trick.

You can save time and understand an entire topic in seconds instead of hours.

In this article, I outline my most effective ChatGPT prompts for keyword research and teach you how I put them together so that you, too, can take, edit, and enhance them even further.

But before we jump into the prompts, I want to emphasize that you shouldn’t replace keyword research tools or disregard traditional keyword research methods .

ChatGPT can make mistakes. It can even create new keywords if you give it the right prompt. For example, I asked it to provide me with a unique keyword for the topic “SEO” that had never been searched before.

“ Interstellar Internet SEO : Optimizing content for the theoretical concept of an interstellar internet, considering the challenges of space-time and interplanetary communication delays.”

Although I want to jump into my LinkedIn profile and update my title to “Interstellar Internet SEO Consultant,” unfortunately, no one has searched that (and they probably never will)!

You must not blindly rely on the data you get back from ChatGPT.

What you can rely on ChatGPT for is the topic ideation stage of keyword research and inspiration.

ChatGPT is a large language model trained with massive amounts of data to accurately predict what word will come next in a sentence. However, it does not know how to do keyword research yet.

Instead, think of ChatGPT as having an expert on any topic armed with the information if you ask it the right question.

In this guide, that is exactly what I aim to teach you how to do – the most essential prompts you need to know when performing topical keyword research .

Best ChatGPT Keyword Research Prompts

The following ChatGPT keyword research prompts can be used on any niche, even a topic to which you are brand new.

For this demonstration, let’s use the topic of “ SEO ” to demonstrate these prompts.

Generating Keyword Ideas Based On A Topic

What are the {x} most popular sub-topics related to {topic}.

The first prompt is to give you an idea of the niche.

As shown above, ChatGPT did a great job understanding and breaking down SEO into three pillars: on-page , off-page & technical .

The key to the following prompt is to take one of the topics ChatGPT has given and query the sub-topics.

What Are The {X} Most Popular Sub-topics Related To {Sub-topic}?

For this example, let’s query, “What are the most popular sub-topics related to keyword research?”

Having done keyword research for over 10 years, I would expect it to output information related to keyword research metrics, the types of keywords, and intent .

Let’s see.

Again, right on the money.

To get the keywords you want without having ChatGPT describe each answer, use the prompt “list without description.”

Here is an example of that.

List Without Description The Top {X} Most Popular Keywords For The Topic Of {X}

You can even branch these keywords out further into their long-tail .

Example prompt: List Without Description The Top {X} Most Popular Long-tail Keywords For The Topic “{X}”

List Without Description The Top Semantically Related Keywords And Entities For The Topic {X}

You can even ask ChatGPT what any topic’s semantically related keywords and entities are!

Tip: The Onion Method Of Prompting ChatGPT

When you are happy with a series of prompts, add them all to one prompt. For example, so far in this article, we have asked ChatGPT the following:

• What are the four most popular sub-topics related to SEO?
• What are the four most popular sub-topics related to keyword research
• List without description the top five most popular keywords for “keyword intent”?
• List without description the top five most popular long-tail keywords for the topic “keyword intent types”?
• List without description the top semantically related keywords and entities for the topic “types of keyword intent in SEO.”

Combine all five into one prompt by telling ChatGPT to perform a series of steps. Example:

“Perform the following steps in a consecutive order Step 1, Step 2, Step 3, Step 4, and Step 5”

“Perform the following steps in a consecutive order Step 1, Step 2, Step 3, Step 4 and Step 5. Step 1 – Generate an answer for the 3 most popular sub-topics related to {Topic}?. Step 2 – Generate 3 of the most popular sub-topics related to each answer. Step 3 – Take those answers and list without description their top 3 most popular keywords. Step 4 – For the answers given of their most popular keywords, provide 3 long-tail keywords. Step 5 – for each long-tail keyword offered in the response, a list without descriptions 3 of their top semantically related keywords and entities.”

Generating Keyword Ideas Based On A Question

Taking the steps approach from above, we can get ChatGPT to help streamline getting keyword ideas based on a question. For example, let’s ask, “ What is SEO? ”

“Perform the following steps in a consecutive order Step 1, Step 2, Step 3, and Step 4. Step 1 Generate 10 questions about “{Question}”?. Step 2 – Generate 5 more questions about “{Question}” that do not repeat the above. Step 3 – Generate 5 more questions about “{Question}” that do not repeat the above. Step 4 – Based on the above Steps 1,2,3 suggest a final list of questions avoiding duplicates or semantically similar questions.”

Generating Keyword Ideas Using ChatGPT Based On The Alphabet Soup Method

One of my favorite methods, manually, without even using a keyword research tool, is to generate keyword research ideas from Google autocomplete , going from A to Z.

You can also do this using ChatGPT.

Example prompt:

“give me popular keywords that includes the keyword “SEO”, and the next letter of the word starts with a”

Tip : Using the onion prompting method above, we can combine all this in one prompt.

“Give me five popular keywords that include “SEO” in the word, and the following letter starts with a. Once the answer has been done, move on to giving five more popular keywords that include “SEO” for each letter of the alphabet b to z.”

Generating Keyword Ideas Based On User Personas

When it comes to keyword research, understanding user personas is essential for understanding your target audience and keeping your keyword research focused and targeted. ChatGPT may help you get an initial understanding of customer personas.

“For the topic of “{Topic}” list 10 keywords each for the different types of user personas”

You could even go a step further and ask for questions based on those topics that those specific user personas may be searching for:

As well as get the keywords to target based on those questions:

“For each question listed above for each persona, list the keywords, as well as the long-tail keywords to target, and put them in a table”

Generating Keyword Ideas Using ChatGPT Based On Searcher Intent And User Personas

Understanding the keywords your target persona may be searching is the first step to effective keyword research. The next step is to understand the search intent behind those keywords and which content format may work best.

For example, a business owner who is new to SEO or has just heard about it may be searching for “what is SEO.”

However, if they are further down the funnel and in the navigational stage, they may search for “top SEO firms.”

You can query ChatGPT to inspire you here based on any topic and your target user persona.

SEO Example:

“For the topic of “{Topic}” list 10 keywords each for the different types of searcher intent that a {Target Persona} would be searching for”

ChatGPT For Keyword Research Admin

Here is how you can best use ChatGPT for keyword research admin tasks.

Using ChatGPT As A Keyword Categorization Tool

One of the use cases for using ChatGPT is for keyword categorization .

In the past, I would have had to devise spreadsheet formulas to categorize keywords or even spend hours filtering and manually categorizing keywords.

ChatGPT can be a great companion for running a short version of this for you.

Let’s say you have done keyword research in a keyword research tool, have a list of keywords, and want to categorize them.

You could use the following prompt:

“Filter the below list of keywords into categories, target persona, searcher intent, search volume and add information to a six-column table: List of keywords – [LIST OF KEYWORDS], Keyword Search Volume [SEARCH VOLUMES] and Keyword Difficulties [KEYWORD DIFFICUTIES].”

Tip : Add keyword metrics from the keyword research tools, as using the search volumes that a ChatGPT prompt may give you will be wildly inaccurate at best.

Using ChatGPT For Keyword Clustering

Another of ChatGPT’s use cases for keyword research is to help you cluster . Many keywords have the same intent, and by grouping related keywords, you may find that one piece of content can often target multiple keywords at once.

However, be careful not to rely only on LLM data for clustering. What ChatGPT may cluster as a similar keyword, the SERP or the user may not agree with. But it is a good starting point.

The big downside of using ChatGPT for keyword clustering is actually the amount of keyword data you can cluster based on the memory limits.

So, you may find a keyword clustering tool or script that is better for large keyword clustering tasks. But for small amounts of keywords, ChatGPT is actually quite good.

A great use small keyword clustering use case using ChatGPT is for grouping People Also Ask (PAA) questions.

Use the following prompt to group keywords based on their semantic relationships. For example:

“Organize the following keywords into groups based on their semantic relationships, and give a short name to each group: [LIST OF PAA], create a two-column table where each keyword sits on its own row.

Using Chat GPT For Keyword Expansion By Patterns

One of my favorite methods of doing keyword research is pattern spotting.

Most seed keywords have a variable that can expand your target keywords.

Here are a few examples of patterns:

1. Question Patterns

(who, what, where, why, how, are, can, do, does, will)

“Generate [X] keywords for the topic “[Topic]” that contain any or all of the following “who, what, where, why, how, are, can, do, does, will”

2. Comparison Patterns

“Generate 50 keywords for the topic “{Topic}” that contain any or all of the following “for, vs, alternative, best, top, review”

3. Brand Patterns

Another one of my favorite modifiers is a keyword by brand.

We are probably all familiar with the most popular SEO brands; however, if you aren’t, you could ask your AI friend to do the heavy lifting.

“For the top {Topic} brands what are the top “vs” keywords”

4. Search Intent Patterns

One of the most common search intent patterns is “best.”

When someone is searching for a “best {topic}” keyword, they are generally searching for a comprehensive list or guide that highlights the top options, products, or services within that specific topic, along with their features, benefits, and potential drawbacks, to make an informed decision.

“For the topic of “[Topic]” what are the 20 top keywords that include “best”

Again, this guide to keyword research using ChatGPT has emphasized the ease of generating keyword research ideas by utilizing ChatGPT throughout the process.

Keyword Research Using ChatGPT Vs. Keyword Research Tools

Free vs. paid keyword research tools.

Like keyword research tools, ChatGPT has free and paid options.

However, one of the most significant drawbacks of using ChatGPT for keyword research alone is the absence of SEO metrics to help you make smarter decisions.

To improve accuracy, you could take the results it gives you and verify them with your classic keyword research tool – or vice versa, as shown above, uploading accurate data into the tool and then prompting.

However, you must consider how long it takes to type and fine-tune your prompt to get your desired data versus using the filters within popular keyword research tools.

For example, if we use a popular keyword research tool using filters, you could have all of the “best” queries with all of their SEO metrics:

And unlike ChatGPT, generally, there is no token limit; you can extract several hundred, if not thousands, of keywords at a time.

As I have mentioned multiple times throughout this piece, you cannot blindly trust the data or SEO metrics it may attempt to provide you with.

The key is to validate the keyword research with a keyword research tool.

ChatGPT For International SEO Keyword Research

ChatGPT can be a terrific multilingual keyword research assistant.

For example, if you wanted to research keywords in a foreign language such as French. You could ask ChatGPT to translate your English keywords;

But don’t worry, there is a workaround: If you have access to a competitor keyword research tool, you can see what webpage is ranking for that query – and then identify the top keyword for that page based on the ChatGPT translated keywords that do have search volume.

Or, if you don’t have access to a paid keyword research tool, you could always take the top-performing result, extract the page copy, and then ask ChatGPT what the primary keyword for the page is.

Key Takeaway

ChatGPT can be an expert on any topic and an invaluable keyword research tool. However, it is another tool to add to your toolbox when doing keyword research; it does not replace traditional keyword research tools.

As shown throughout this tutorial, from making up keywords at the beginning to inaccuracies around data and translations, ChatGPT can make mistakes when used for keyword research.

You cannot blindly trust the data you get back from ChatGPT.

However, it can offer a shortcut to understanding any topic for which you need to do keyword research and, as a result, save you countless hours.

But the key is how you prompt.

The prompts I shared with you above will help you understand a topic in minutes instead of hours and allow you to better seed keywords using keyword research tools.

It can even replace mundane keyword clustering tasks that you used to do with formulas in spreadsheets or generate ideas based on keywords you give it.

Paired with traditional keyword research tools, ChatGPT for keyword research can be a powerful tool in your arsenal.

More resources:

• Tools Of The Trade: When To Use ChatGPT And When To Use Google 
• What Is ChatGPT & How Can You Use It?
• 8 ChatGPT Alternatives You Can Try In 2024

Featured Image: Tatiana Shepeleva/Shutterstock

Dan Smullen, a News SEO Consultant from Dublin, Ireland has an MBA with a specialism in Marketing from the prestigious ...

Subscribe To Our Newsletter.

Conquer your day with daily search marketing news.

• Starting a Business
• Growing a Business
• Small Business Guide
• Business News
• Science & Technology
• Money & Finance
• For Subscribers
• Write for Entrepreneur
• Entrepreneur Store
• United States
• Asia Pacific
• Middle East
• South Africa

Copyright © 2024 Entrepreneur Media, LLC All rights reserved. Entrepreneur® and its related marks are registered trademarks of Entrepreneur Media LLC

These 4 Words Make It Obvious You Used AI to Write a Paper, According to New Research Scientists are increasingly using ChatGPT and other AI bots to write studies.

By Sherin Shibu • May 3, 2024

Key Takeaways

• Four words have exploded in popularity in academic writing since ChatGPT launched: realm, intricate, showcasing, and pivotal.
• A Stanford study tracked the occurrence of these words over time and found that the pressure to publish quickly may have made AI writing options appealing.

AI has infiltrated scientific research papers — and a handful of words give away that scientists may have had some writing help from ChatGPT.

A new Stanford University study published in arXiv suggests that since OpenAI's ChatGPT came on the scene in November 2022 , researchers have steadily increased how often they use AI to help with academic writing.

In the first large-scale review of how AI has impacted scholarly writing, the Stanford researchers analyzed nearly a million papers published in arXiv, bioRxiv, and Nature from January 2020 to February 2024.

They looked for certain words that AI tends to overuse, words that exploded in popularity since ChatGPT launched: realm, intricate, showcasing, and pivotal.

Related: A New AI Chatbot Is Revolutionizing Business School Curriculum and Accreditation

Their findings, released in April, revealed a 6.3% to 17.5% growth in the use of AI over time.

The fastest growth was in the computer science department, where abstracts and introductions with common AI-used words rose to 17.5% and 15.3% respectively by February 2024.

The rising popularity of four common words used more by AI than humans in arXiv computer science abstracts. Credit: Stanford University paper titled "Mapping the Increasing Use of LLMs in Scientific Papers"

The researchers suggested that computer science might have grown the most quickly because academics in that department might have been more familiar with ChatGPT and have had better access to AI models.

Related: This One Word Is a Giveaway That You Used ChatGPT to Write an Email, According to an Expert

It's not just the Stanford study: Dr. Jeremy Nguyen , a senior researcher and lecturer at Swinburne Business School in Australia, shared findings specific to medical papers that showed a possible increase in AI writing.

Nguyen searched all PubMed articles published in the past 34 years for another popular word used by ChatGPT: " delve ."

He found a remarkable uptick in research articles that used the word, suggesting that AI had been used to help write those pieces.

Are medical studies being written with ChatGPT? Well, we all know ChatGPT overuses the word "delve". Look below at how often the word 'delve' is used in papers on PubMed (2023 was the first full year of ChatGPT). pic.twitter.com/iNxZfFLkxL — Jeremy Nguyen ✍? ? (@JeremyNguyenPhD) March 30, 2024

Scientists face increasing pressure to publish , especially because researchers are ranked based on how many times their articles are cited. In certain disciplines like AI research, that pressure can be more intense.

"The fast-paced nature of [AI] research and the associated pressure to publish quickly may incentivize the use of [AI] writing assistance," the Stanford researchers stated.

Related: AI Is Changing How Businesses Recruit for Open Roles — and How Candidates Are Gaming the System

Entrepreneur Staff

News Reporter

Want to be an Entrepreneur Leadership Network contributor? Apply now to join.

Editor's Pick Red Arrow

• This Mother and Daughter Were 'Kind of Fringe Weirdos' When They Started an Uncommon Business in Their Garage. Now They're in Major Retailers — and Victoria Beckham Is a Fan.
• Lock A Leadership Shortage Is Coming. Here's What Needs to Happen to Prevent It.
• Lock The Author of 'Million Dollar Weekend' Says This Is the Only Difference Between You and the Many 'Very, Very Dumb People' Making a Lot of Money
• What the NLRB Appeal of the Expanded Joint Employer Rule Judgment Means for Your Business
• Lock 12 Books That Self-Made Millionaires Swear By
• The Sweet Side Hustle She Started in an Old CVS Made $800,000 in One Year. Now She's Repeating the Success With Her Daughter — and They've Already Exceeded 8 Figures.

Most Popular Red Arrow

Ask co-founder of netflix marc randolph anything: how to watch.

How to watch the new live streaming episode of 'Ask Marc' on May 9th at 2 PM ET.

How to Know It's Time to Add an HR Department

HR activities at startups are often reactive in nature instead of proactive. That leaves gaps in the system.

Employers Say They Want to Hire Candidates With AI Skills, But Employees Are Still Sneaking AI Tool Use in the Office

A new joint report from LinkedIn and its parent company Microsoft revealed the contradictory state of AI at work.

63 Small Business Ideas to Start in 2024

We put together a list of the best, most profitable small business ideas for entrepreneurs to pursue in 2024.

Building an Agile Remote Team Is No Easy Feat — But It's About to Get a Whole Lot Easier Thanks to This Transformative Tool.

The virtual reality of the metaverse promises to put a radical new spin on returning to the office.

What I Learned From Spending $5.9 Million on Marketing Last Year

Road-tested tips to 6X your revenue per lead, double your social media leads and increase sales conversations. I know because I lived it!

Successfully copied link

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 01 May 2024

A critical assessment of using ChatGPT for extracting structured data from clinical notes

• Jingwei Huang   ORCID: orcid.org/0000-0003-2155-6107 1 ,
• Donghan M. Yang 1 ,
• Ruichen Rong 1 ,
• Kuroush Nezafati   ORCID: orcid.org/0000-0002-6785-7362 1 ,
• Colin Treager 1 ,
• Zhikai Chi   ORCID: orcid.org/0000-0002-3601-3351 2 ,
• Shidan Wang   ORCID: orcid.org/0000-0002-0001-3261 1 ,
• Xian Cheng 1 ,
• Yujia Guo 1 ,
• Laura J. Klesse 3 ,
• Guanghua Xiao 1 ,
• Eric D. Peterson 4 ,
• Xiaowei Zhan 1 &
• Yang Xie   ORCID: orcid.org/0000-0001-9456-1762 1  

npj Digital Medicine volume  7 , Article number:  106 ( 2024 ) Cite this article

3231 Accesses

69 Altmetric

Metrics details

• Non-small-cell lung cancer

Existing natural language processing (NLP) methods to convert free-text clinical notes into structured data often require problem-specific annotations and model training. This study aims to evaluate ChatGPT’s capacity to extract information from free-text medical notes efficiently and comprehensively. We developed a large language model (LLM)-based workflow, utilizing systems engineering methodology and spiral “prompt engineering” process, leveraging OpenAI’s API for batch querying ChatGPT. We evaluated the effectiveness of this method using a dataset of more than 1000 lung cancer pathology reports and a dataset of 191 pediatric osteosarcoma pathology reports, comparing the ChatGPT-3.5 (gpt-3.5-turbo-16k) outputs with expert-curated structured data. ChatGPT-3.5 demonstrated the ability to extract pathological classifications with an overall accuracy of 89%, in lung cancer dataset, outperforming the performance of two traditional NLP methods. The performance is influenced by the design of the instructive prompt. Our case analysis shows that most misclassifications were due to the lack of highly specialized pathology terminology, and erroneous interpretation of TNM staging rules. Reproducibility shows the relatively stable performance of ChatGPT-3.5 over time. In pediatric osteosarcoma dataset, ChatGPT-3.5 accurately classified both grades and margin status with accuracy of 98.6% and 100% respectively. Our study shows the feasibility of using ChatGPT to process large volumes of clinical notes for structured information extraction without requiring extensive task-specific human annotation and model training. The results underscore the potential role of LLMs in transforming unstructured healthcare data into structured formats, thereby supporting research and aiding clinical decision-making.

Similar content being viewed by others

Highly accurate protein structure prediction with AlphaFold

scGPT: toward building a foundation model for single-cell multi-omics using generative AI

Causal machine learning for predicting treatment outcomes

Introduction.

Large Language Models (LLMs) 1 , 2 , 3 , 4 , 5 , 6 , such as Generative Pre-trained Transformer (GPT) models represented by ChatGPT, are being utilized for diverse applications across various sectors. In the healthcare industry, early applications of LLMs are being used to facilitate patient-clinician communication 7 , 8 . To date, few studies have examined the potential of LLMs in reading and interpreting clinical notes, turning unstructured texts into structured, analyzable data.

Traditionally, the automated extraction of structured data elements from medical notes has relied on medical natural language processing (NLP) using rule-based or machine-learning approaches or a combination of both 9 , 10 . Machine learning methods 11 , 12 , 13 , 14 , particularly deep learning, typically employ neural networks and the first generation of transformer-based large language models (e.g., BERT). Medical domain knowledge needs to be integrated into model designs to enhance performance. However, a significant obstacle to developing these traditional medical NLP algorithms is the limited existence of human-annotated datasets and the costs associated with new human annotation 15 . Despite meticulous ground-truth labeling, the relatively small corpus sizes often result in models with poor generalizability or make evaluations of generalizability impossible. For decades, conventional artificial intelligence (AI) systems (symbolic and neural networks) have suffered from a lack of general knowledge and commonsense reasoning. LLMs, like GPT, offer a promising alternative, potentially using commonsense reasoning and broad general knowledge to facilitate language processing.

ChatGPT is the application interface of the GPT model family. This study explores an approach to using ChatGPT to extract structured data elements from unstructured clinical notes. In this study, we selected lung cancer pathology reports as the corpus for extracting detailed diagnosis information for lung cancer. To accomplish this, we developed and improved a prompt engineering process. We then evaluated the effectiveness of this method by comparing the ChatGPT output with expert-curated structured data and used case studies to provide insights into how ChatGPT read and interpreted notes and why it made mistakes in some cases.

Data and endpoints

The primary objective of this study was to develop an algorithm and assess the capabilities of ChatGPT in processing and interpreting a large volume of free-text clinical notes. To evaluate this, we utilized unstructured lung cancer pathology notes, which provide diagnostic information essential for developing treatment plans and play vital roles in clinical and translational research. We accessed a total of 1026 lung cancer pathology reports from two web portals: the Cancer Digital Slide Archive (CDSA data) ( https://cancer.digitalslidearchive.org/ ) and The Cancer Genome Atlas (TCGA data) ( https://cBioPortal.org ). These platforms serve as public data repositories for de-identified patient information, facilitating cancer research. The CDSA dataset was utilized as the “training” data for prompt development, while the TCGA dataset, after removing the overlapping cases with CDSA, served as the test data for evaluating the ChatGPT model performance.

From all the downloaded 99 pathology reports from CDSA for the training data, we excluded 21 invalid reports due to near-empty content, poor scanning quality, or missing report forms. Seventy-eight valid pathology reports were included as the training data to optimize the prompt. To evaluate the model performance, 1024 pathology reports were downloaded from cBioPortal. Among them, 97 overlapped with the training data and were excluded from the evaluation. We further excluded 153 invalid reports due to near-empty content, poor scanning quality, or missing report forms. The invalid reports were preserved to evaluate ChatGPT’s handling of irregular inputs separately, and were not included in the testing data for accuracy performance assessment. As a result, 774 valid pathology reports were included as the testing data for performance evaluation. These valid reports still contain typos, missing words, random characters, incomplete contents, and other quality issues challenging human reading. The corresponding numbers of reports used at each step of the process are detailed in Fig. 1 .

Exclusions are accounted for due to reasons such as empty reports, poor scanning quality, and other factors, including reports of stage IV or unknown conditions.

The specific task of this study was to identify tumor staging and histology types which are important for clinical care and research from pathology reports. The TNM staging system 16 , outlining the primary tumor features (T), regional lymph node involvement (N), and distant metastases (M), is commonly used to define the disease extent, assign prognosis, and guide lung cancer treatment. The American Joint Committee on Cancer (AJCC) has periodically released various editions 16 of TNM classification/staging for lung cancers based on recommendations from extensive database analyses. Following the AJCC guideline, individual pathologic T, N, and M stage components can be summarized into an overall pathologic staging score of Stage I, II, III, or IV. For this project, we instructed ChatGPT to use the AJCC 7 th edition Cancer Staging Manual 17 as the reference for staging lung cancer cases. As the lung cancer cases in our dataset are predominantly non-metastatic, the pathologic metastasis (pM) stage was not extracted. The data elements we chose to extract and evaluate for this study are pathologic primary tumor (pT) and pathologic lymph node (pN) stage components, overall pathologic tumor stage, and histology type.

Overall Performance

Using the training data in the CDSA dataset ( n  = 78), we experimented and improved prompts iteratively, and the final prompt is presented in Fig. 2 . The overall performance of the ChatGPT (gpt-3.5-turbo-16k model) is evaluated in the TCGA dataset ( n  = 774), and the results are summarized in Table 1 . The accuracy of primary tumor features (pT), regional lymph node involvement (pN), overall tumor stage, and histological diagnosis are 0.87, 0.91, 0.76, and 0.99, respectively. The average accuracy of all attributes is 0.89. The coverage rates for pT, pN, overall stage and histological diagnosis are 0.97, 0.94, 0.94 and 0.96, respectively. Further details of the accuracy evaluation, F1, Kappa, recall, and precision for each attribute are summarized as confusion matrices in Fig. 3 .

Final prompt for information extraction and estimation from pathology reports.

For meaningful evaluation, the cases with uncertain values, such as “Not Available”, “Not Specified”, “Cannot be determined”, “Unknown”, et al. in reference and prediction have been removed. a Primary tumor features (pT), b regional lymph node involvement (pN), c overall tumor stage, and d histological diagnosis.

Inference and Interpretation

To understand how ChatGPT reads and makes inferences from pathology reports, we demonstrated a case study using a typical pathology report in this cohort (TCGA-98-A53A) in Fig. 4a . The left panel shows part of the original pathology report, and the right panel shows the ChatGPT output with estimated pT, pN, overall stage, and histology diagnosis. For each estimate, ChatGPT gives the confidence level and the corresponding evidence it used for the estimation. In this case, ChatGPT correctly extracted information related to tumor size, tumor features, lymph node involvement, and histology information and used the AJCC staging guidelines to estimate tumor stage correctly. In addition, the confidence level, evidence interpretation, and case summary align well with the report and pathologists’ evaluations. For example, the evidence for the pT category was described as “The pathology report states that the tumor is > 3 cm and < 5 cm in greatest dimension, surrounded by lung or visceral pleura.” The evidence for tumor stage was described as “Based on the estimated pT category (T2a) and pN category (N0), the tumor stage is determined to be Stage IB according to AJCC7 criteria.” It shows that ChatGPT extracted relevant information from the note and correctly inferred the pT category based on the AJCC guideline (Supplementary Fig. 1 ) and the extracted information.

a TCGA-98-A53A. An example of a scanned pathological report (left panel) and ChatGPT output and interpretation (right panel). All estimations and support evidence are consistent with the pathologist’s evaluations. b The GPT model correctly inferred pT as T2a based on the tumor’s size and involvement according to AJCC guidelines.

In another more complex case, TCGA-50-6590 (Fig. 4b ), ChatGPT correctly inferred pT as T2a based on both the tumor’s size and location according to AJCC guidelines. Case TCGA-44-2656 demonstrates a more challenging scenario (Supplementary Fig. 2 ), where the report only contains some factual data without specifying pT, pN, and tumor stage. However, ChatGPT was able to infer the correct classifications based on the reported facts and provide proper supporting evidence.

Error analysis

To understand the types and potential reasons for misclassifications, we performed a detailed error analysis by looking into individual attributes and cases where ChatGPT made mistakes, the results of which are summarized below.

Primary tumor feature (pT) classification

In total, 768 cases with valid reports and reference values in the testing data were used to evaluate the classification performance of pT. Among them, 15 cases were reported with unknown or empty output by ChatGPT, making the coverage rate 0.97. For the remaining 753 cases, 12.6% of pT was misclassified. Among these misclassification cases, the majority were T1 misclassified as T2 (67 out of 753 or 8.9%) or T3 misclassified as T2 (12 out of 753, or 1.6%).

In most cases, ChatGPT extracted the correct tumor size information but used an incorrect rule to distinguish pT categories. For example, in the case TCGA-22-4609 (Fig. 5a ), ChatGPT stated, “Based on the tumor size of 2.0 cm, it falls within the range of T2 category according to AJCC 7th edition for lung carcinoma staging manual.” However, according to the AJCC 7 th edition staging guidelines for lung cancer, if the tumor is more than 2 cm but less than 3 cm in greatest dimension and does not invade nearby structures, pT should be classified as T1b. Therefore, ChatGPT correctly extracted the maximum tumor dimension of 2 cm but incorrectly interpreted this as meeting the criteria for classification as T2. Similarly, for case TCGA-85-A4JB, ChatGPT incorrectly claimed, “Based on the tumor size of 10 cm, the estimated pT category is T2 according to AJCC 7th edition for lung carcinoma staging manual.” According to the AJCC 7 th edition staging guidelines, a tumor more than 7 cm in greatest dimension should be classified as T3.

a TCGA-22-4609 illustrates a typical case where the GPT model uses a false rule, which is incorrect by AJCC guideline. b Case TCGA-39-5028 shows a complex case where there exist two tumors and the GPT model only capture one of them. c Case TCGA-39-5016 reveals a case where the GPT model made a mistake for getting confused with domain terminology.

Another challenging situation arose when multiple tumor nodules were identified within the lung. In the case of TCGA-39-5028 (Fig. 5b ), two separate tumor nodules were identified: one in the right upper lobe measuring 2.1 cm in greatest dimension and one in the right lower lobe measuring 6.6 cm in greatest dimension. According to the AJCC 7 th edition guidelines, the presence of separate tumor nodules in a different ipsilateral lobe results in a classification of T4. However, ChatGPT classified this case as T2a, stating, “The pathology report states the tumor’s greatest diameter as 2.1 cm”. This classification would be appropriated if the right upper lobe nodule were a single isolated tumor. However, ChatGPT failed to consider the presence of the second, larger nodule in the right lower lobe when determining the pT classification.

Regional lymph node involvement (pN)

The classification performance of pN was evaluated using 753 cases with valid reports and reference values in the testing data. Among them, 27 cases were reported with unknown or empty output by ChatGPT, making the coverage rate 0.94. For the remaining 726 cases, 8.5% of pN was misclassified. Most of these misclassification cases were N1 misclassified as N2 (32 cases). The AJCC 7th edition staging guidelines use the anatomic locations of positive lymph nodes to determine N1 vs. N2. However, most of the misclassification cases were caused by ChatGPT interpreting the number of positive nodes rather than the locations of the positive nodes. One such example is the case TCGA-85-6798. The report states, “Lymph nodes: 2/16 positive for metastasis (Hilar 2/16)”. Positive hilar lymph nodes correspond to N1 classification according to AJCC 7th edition guidelines. However, ChatGPT misclassifies this case as N2, stating, “The pathology report states that 2 out of 16 lymph nodes are positive for metastasis. Based on this information, the pN category can be estimated as N2 according to AJCC 7th edition for lung carcinoma staging manual.” This interpretation is incorrect, as the number of positive lymph nodes is not part of the criteria used to determine pN status according to AJCC 7th edition guidelines. The model misinterpreted pN2 predictions in 22 cases due to similar false assertions.

In some cases, the ChatGPT model made classification mistakes by misunderstanding the locations’ terminology. Figure 5c shows a case (TCGA-39-5016) where the ChatGPT model recognized that “6/9 peribronchial lymph nodes involved, “ corresponding with classification as N1, but ChatGPT misclassified this case as N2. By AJCC 7th edition guidelines, N2 is defined as “Metastasis in ipsilateral mediastinal and/or subcarinal lymph node(s)”. The ChatGPT model did not fully understand that terminology and made misclassifications.

Pathology tumor stage

The overall tumor stage classification performance was evaluated using 744 cases with valid reports and reference values as stage I, II and III in the testing data. Among them, 18 cases were reported as unknown or empty output by ChatGPT making the coverage rate as 0.94. For the remaining 726 cases, 23.6% of the overall stage was misclassified. Since the overall stage depends on individual pT and pN stages, the mistakes could come from misclassification of pT or pN (error propagation) or applying incorrect inference rules to determine the overall stage from pT and pN (incorrect rules). Looking into the 56 cases where ChatGPT misclassified stage II as stage III, 22 cases were due to error propagation, and 34 were due to incorrect rules. Figure 6a shows an example of error propagation (TCGA-MP-A4TK). ChatGPT misclassified the pT stage from T2a to T3, and then this mistake led to the incorrect classification of stage IIA to stage IIIA. Figure 6b illustrates a case (TCGA-49-4505) where ChatGPT made correct estimation of pT and pN but made false prediction about tumor stage by using a false rule. Among the 34 cases affected by incorrect rules, ChatGPT mistakenly inferred tumor stage as stage III for 26 cases where pT is T3 and pN is N0, respectively. For example, for case TCGA-55-7994, ChatGPT provided the evidence as “Based on the estimated pT category (T3) and pN category (N0), the tumor stage is determined to be Stage IIIA according to AJCC7 criteria”. According to AJCC7, tumors with T3 and N0 should be classified as stage IIB. Similarly, error analysis for other tumor stages shows that misclassifications come from both error propagation and applying false rules.

a Case TCGA-MP-A4TK: An example of typical errors GPT made in the experiments, i.e. GPT took false rule and further led to faulty propagation. b Case TCGA-49-4505: The GPT model made false estimation of Stage IIIA with a false rule, although it made correct inference with T2b and N1.

Histological diagnosis

The classification performance of histology diagnosis was evaluated using 762 cases with valid reports and reference values in the testing data. Among them, 17 cases were reported as either unknown or empty output by ChatGPT, making the coverage rate 0.96. For the remaining 745 cases, 6 ( < 1%) of histology types were misclassified. Among the mistakes that ChatGPT made for histology diagnosis, ChatGPT misclassified 3 of them as “other” type and 3 cases of actual “other” type (neither adenocarcinomas nor squamous cell carcinomas) as 2 adenocarcinomas and 1 squamous cell carcinoma. In TCGA-22-5485, two tumors exist: one squamous cell carcinoma and another adenocarcinoma, which should be classified as the ‘other’ type. However, ChatGPT only identified and extracted information for one tumor. In the case TCGA-33-AASB, which is the “other” type of histology, ChatGPT captured the key information and gave it as evidence: “The pathology report states the histologic diagnosis as infiltrating poorly differentiated non-small cell carcinoma with both squamous and glandular features”. However, it mistakenly estimated this case as “adenocarcinoma”. In another case (TCGA-86-8668) of adenocarcinoma, ChatGPT again captured key information and stated as evidence, “The pathology report states the histologic diagnosis as Bronchiolo-alveolar carcinoma, mucinous” but could not tell it is a subtype of adenocarcinoma. Both cases reveal that ChatGPT still has limitations in the specific domain knowledge in lung cancer pathology and the capability of correcting understanding its terminology.

Analyzing irregularities

The initial model evaluation and prompt-response review uncovered irregular scenarios: the original pathology reports may be blank, poorly scanned, or simply missing report forms. We reviewed how ChatGPT responded to these anomalies. First, when a report was blank, the prompt contained only the instruction part. ChatGPT failed to recognize this situation in most cases and inappropriately generated a fabricated case. Our experiments showed that, with the temperature set at 0 for blank reports, ChatGPT converged to a consistent, hallucinated response. Second, for nearly blank reports with a few random characters and poorly scanned reports, ChatGPT consistently converged to the same response with increased variance as noise increased. In some cases, ChatGPT responded appropriately to all required attributes but with unknown values for missing information. Last, among the 15 missing report forms in a small dataset, ChatGPT responded “unknown” as expected in only 5 cases, with the remaining 10 still converging to the hallucinated response.

Reproducibility evaluation

Since ChatGPT models (even with the same version) evolve over time, it is important to evaluate the stability and reproducibility of ChatGPT. For this purpose, we conducted experiments with the same model (“gpt-3.5-turbo-0301”), the same data, prompt, and settings (e.g., temperature = 0) twice in early April and the middle of May of 2023. The rate of equivalence between ChatGPT estimations in April and May on key attributes of interest (pT, pN, tumor stage, and histological diagnosis) is 0.913. The mean absolute error between certainty degrees in the two experiments is 0.051. Considering the evolutionary nature of ChatGPT models, we regard an output difference to a certain extent as reasonable and the overall ChatGPT 3.5 model as stable.

Comparison with other NLP methods

In order to have a clear perspective on how ChatGPT’s performance stands relative to established methods, we conducted a comparative analysis of the results generated by ChatGPT with two established methods: a keyword search algorithm and a deep learning-based Named Entity Recognition (NER) method.

Data selection and annotation

Since the keyword search and NER methods do not support zero-shot learning and require human annotations on the entity level, we carefully annotated our dataset for these traditional NLP methods. We used the same training and testing datasets as in the prompt engineering for ChatGPT. The training dataset underwent meticulous annotation by experienced medical professionals, adhering to the AJCC7 standards. This annotation process involved identifying and highlighting all relevant entities and text spans related to stage, histology, pN, and pT attributes. The detailed annotation process for the 78 cases required a few weeks of full-time work from medical professionals.

Keyword search algorithm using wordpiece tokenizer

For the keyword search algorithm, we employed the WordPiece tokenizer to segment words into subwords. We compiled an annotated entity dictionary from the training dataset. To assess the performance of this method, we calculated span similarities between the extracted spans in the validation and testing datasets and the entries in the dictionary.

Named Entity Recognition (NER) classification algorithm

For the NER classification algorithm, we designed a multi-label span classification model. This model utilized the pre-trained Bio_ClinicalBERT as its backbone. To adapt it for multi-label classification, we introduced an additional linear layer. The model underwent fine-tuning for 1000 epochs using the stochastic gradient descent (SGD) optimizer. The model exhibiting the highest overall F1 score on the validation dataset was selected as the final model for further evaluation in the testing dataset.

Performance evaluation

We evaluated the performance of both the keyword search and NER methods on the testing dataset. We summarized the predicted entities/spans and their corresponding labels. In cases where multiple related entities were identified for a specific category, we selected the most severe entities as the final prediction. Moreover, we inferred the stage information for corpora lacking explicit staging information by aggregating details from pN, pT, and diagnosis, aligning with the AJCC7 protocol. The overall predictions for stage, diagnosis, pN, and pT were compared against the ground truth table to gauge the accuracy and effectiveness of our methods. The results (Supplementary Table S1 ) show that the ChatGPT outperforms WordPiece tokenizer and NER Classifier. The average accuracy for ChatGPT, WordPiece tokenizer, and NER Classifier are 0.89, 0.51, and 0.76, respectively.

Prompt engineering process and results

Prompt design is a heuristic search process with many elements to consider, thus having a significantly large design space. We conducted many experiments to explore better prompts. Here, we share a few typical prompts and the performance of these prompts in the training data set to demonstrate our prompt engineering process.

Output format

The most straightforward prompt without special design would be: “read the pathology report and answer what are pT, pN, tumor stage, and histological diagnosis”. However, this simple prompt would make ChatGPT produce unstructured answers varying in format, terminology, and granularity across the large number of pathology reports. For example, ChatGPT may output pT as “T2” or “pT2NOMx”, and it outputs histological diagnosis as “Multifocal invasive moderately differentiated non-keratinizing squamous cell carcinoma”. The free-text answers will require a significant human workload to clean and process the output from ChatGPT. To solve this problem, we used a multiple choice answer format to force ChatGPT to pick standardized values for some attributes. For example, for pT, ChatGPT could only provide the following outputs: “T0, Tis, T1, T1a, T1b, T2, T2a, T2b, T3, T4, TX, Unknown”. For the histologic diagnosis, ChatGPT could provide output in one of these categories: Lung Adenocarcinoma, Lung Squamous Cell Carcinoma, Other, Unknown. In addition, we added the instruction, “Please make sure to output the whole set of answers together as a single JSON file, and don’t output anything beyond the required JSON file,” to emphasize the requirement for the output format. These requests in the prompt make the downstream analysis of ChatGPT output much more efficient. In order to know the certainty degree of ChatGPT’s estimate and the evidence, we asked ChatGPT to provide the following 4 outputs for each attribute/variable: extracted value as stated in the pathology report, estimated value based on AJCC 7th edition for lung carcinoma staging manual, the certainty degree of the estimation, and the supporting evidence for the estimation. The classification accuracy of this prompt with multiple choice output format (prompt v1) in our training data could achieve 0.854.

Evidence-based inference

One of the major concerns for LLM is that the results from the model are not supported by any evidence, especially when there is not enough information for specific questions. In order to reduce this problem, we emphasize the use of evidence for inference in the prompt by adding this instruction to ChatGPT: “Please ensure to make valid inferences for attribute estimation based on evidence. If there is no available evidence provided to make an estimation, please answer the value as “Unknown.” In addition, we asked ChatGPT to “Include “comment” as the last key of the JSON file.” After adding these two instructions (prompt v2), the performance of the classification in the training data increased to 0.865.

Chain of thought prompting by asking intermediate questions

Although tumor size is not a primary interest for diagnosis and clinical research, it plays a critical role in classifying the pT stage. We hypothesize that if ChatGPT pays closer attention to tumor size, it will have better classification performance. Therefore, we added an instruction in the prompt (prompt v3) to ask ChatGPT to estimate: “tumor size max_dimension: [<the greatest dimension of tumor in Centimeters (cm)>, ‘Unknown’]” as one of the attributes. After this modification, the performance of the classification in the training data increased to 0.90.

Providing examples

Providing examples is an effective way for humans to learn, and it should have similar effects for ChatGPT. We provided a specific example to infer the overall stage based on pT and pN by adding this instruction: “Please estimate the tumor stage category based on your estimated pT category and pN category and use AJCC7 criteria. For example, if pT is estimated as T2a and pN as N0, without information showing distant metastasis, then by AJCC7 criteria, the tumor stage is “Stage IB”.” After this modification (prompt v4), the performance of the classification in the training data increased to 0.936.

Although we can further refine and improve prompts, we decided to use prompt v4 as the final model and apply it to the testing data and get the final classification accuracy of 0.89 in the testing data.

ChatGPT-4 performance

LLM evolves rapidly and OpenAI just released the newest GPT-4 Turbo model (GPT-4-1106-preview) in November 2023. To compare this new model with GPT-3.5-Turbo, we applied this newest GPT model GPT-4-1106 to analyze all the lung cancer pathology notes in the testing data. The classification result and the comparison with the GPT-3.5-Turbo-16k are summarized in Supplementary Table 1 . The results show that GPT-4-turbo performs better in almost every aspect; overall, the GPT-4-turbo model increases performance by over 5%. However, GPT-4-Turbo is much more expensive than GPT-3.5-Turbo. The performance of GPT-3.5-Turbo-16k is still comparable and acceptable. As such, this study mainly focuses on assessing GPT-3.5-Turbo-16k, but highlights the fast development and promise of using LLM to extract structured data from clinical notes.

Analyzing osteosarcoma data

To demonstrate the broader application of this method beyond lung cancer, we collected and analyzed clinical notes from pediatric osteosarcoma patients. Osteosarcoma, the most common type of bone cancer in children and adolescents, has seen no substantial improvement in patient outcomes for the past few decades 18 . Histology grades and margin status are among the most important prognostic factors for osteosarcoma. We collected pathology reports from 191 osteosarcoma cases (approved by UTSW IRB #STU 012018-061). Out of these, 148 cases had histology grade information, and 81 had margin status information; these cases were used to evaluate the performance of the GPT-3.5-Turbo-16K model and our prompt engineering strategy. Final diagnoses on grade and margin were manually reviewed and curated by human experts, and these diagnoses were used to assess ChatGPT’s performance. All notes were de-identified prior to analysis. We applied the same prompt engineering strategy to extract grade and margin information from these osteosarcoma pathology reports. This analysis was conducted on our institution’s private Azure OpenAI platform, using the GPT-3.5-Turbo-16K model (version 0613), the same model used for lung cancer cases. ChatGPT accurately classified both grades (with a 98.6% accuracy rate) and margin status (100% accuracy), as shown in Supplementary Fig. 3 . In addition, Supplementary Fig. 4 details a specific case, illustrating how ChatGPT identifies grades and margin status from osteosarcoma pathology reports.

Since ChatGPT’s release in November 2022, it has spurred many potential innovative applications in healthcare 19 , 20 , 21 , 22 , 23 . To our knowledge, this is among the first reports of an end-to-end data science workflow for prompt engineering, using, and rigorously evaluating ChatGPT in its capacity of batch-processing information extraction tasks on large-scale clinical report data.

The main obstacle to developing traditional medical NLP algorithms is the limited availability of annotated data and the costs for new human annotations. To overcome these hurdles, particularly in integrating problem-specific information and domain knowledge with LLMs’ task-agnostic general knowledge, Augmented Language Models (ALMs) 24 , which incorporate reasoning and external tools for interaction with the environment, are emerging. Research shows that in-context learning (most influentially, few-shot prompting) can complement LLMs with task-specific knowledge to perform downstream tasks effectively 24 , 25 . In-context learning is an approach of training through instruction or light tutorial with a few examples (so called few-shot prompting; well instruction without any example is called 0-shot prompting) rather than fine-tuning or computing-intensive training, which adjusts model weights. This approach has become a dominant method for using LLMs in real-world problem-solving 24 , 25 , 26 . The advent of ALMs promises to revolutionize almost every aspect of human society, including the medical and healthcare domains, altering how we live, work, and communicate. Our study shows the feasibility of using ChatGPT to extract data from free text without extensive task-specific human annotation and model training.

In medical data extraction, our study has demonstrated the advantages of adopting ChatGPT over traditional methods in terms of cost-effectiveness and efficiency. Traditional approaches often require labor-intensive annotation processes that may take weeks and months from medical professionals, while ChatGPT models can be fine-tuned for data extraction within days, significantly reducing the time investment required for implementation. Moreover, our economic analysis revealed the cost savings associated with using ChatGPT, with processing over 900 pathology reports incurring a minimal monetary cost (less than $10 using GPT 3.5 Turbo and less than $30 using GPT-4 Turbo). This finding underscores the potential benefits of incorporating ChatGPT into medical data extraction workflows, not only for its time efficiency but also for its cost-effectiveness, making it a compelling option for medical institutions and researchers seeking to streamline their data extraction processes without compromising accuracy or quality.

A critical requirement for effectively utilizing an LLM is crafting a high-quality “prompt” to instruct the LLM, which has led to the emergence of an important methodology referred to as “prompt engineering.” Two fundamental principles guide this process: firstly, the provision of appropriate context, and secondly, delivering clear instructions about subtasks and the requirements for the desired response and how it should be presented. For a single query for one-time use, the user can experiment with and revise the prompt within the conversation session until a satisfactory answer is obtained. However, prompt design can become more complex when handling repetitive tasks over many input data files using the OpenAI API. In these instances, a prompt must be designed according to a given data feed while maintaining the generality and coverage for various input data features. In this study, we found that providing clear guidance on the output format, emphasizing evidence-based inference, providing chain of thought prompting by asking for tumor size information, and providing specific examples are critical in improving the efficiency and accuracy of extracting structured data from the free-text pathology reports. The approach employed in this study effectively leverages the OpenAI API for batch queries of ChatGPT services across a large set of tasks with similar input data structures, including but not limited to pathology reports and EHR.

Our evaluation results show that the ChatGPT (gpt-3.5-turbo-16k) achieved an overall average accuracy of 89% in extracting and estimating lung cancer staging information and histology subtypes compared to pathologist-curated data. This performance is very promising because some scanned pathology reports included in this study contained random characters, missing parts, typos, varied formats, and divergent information sections. ChatGPT also outperformed traditional NLP methods. Our case analysis shows that most misclassifications were due to a lack of knowledge of detailed pathology terminology or very specialized information in the current versions of ChatGPT models, which could be avoided with future model training or fine-tuning with more domain-specific knowledge.

While our experiments reveal ChatGPT’s strengths, they also underscore its limitations and potential risks, the most significant being the occasional “hallucination” phenomenon 27 , 28 , where the generated content is not faithful to the provided source content. For example, the responses to blank or near-blank reports reflect this issue, though these instances can be detected and corrected due to convergence towards an “attractor”.

The phenomenon of ‘hallucination’ in LLMs presents a significant challenge in the field. It is important to consider several key factors to effectively address the challenges and risks associated with ChatGPT’s application in medicine. Since the output of an LLM depends on both the model and the prompt, mitigating hallucination can be achieved through improvements in GPT models and prompting strategies. From a model perspective, model architecture, robust training, and fine-tuning on a diverse and comprehensive medical dataset, emphasizing accurate labeling and classification, can reduce misclassifications. Additionally, enhancing LLMs’ comprehension of medical terminology and guidelines by incorporating feedback from healthcare professionals during training and through Reinforcement Learning from Human Feedback (RLHF) can further diminish hallucinations. Regarding prompt engineering strategies, a crucial method is to prompt the GPT model with a ‘chain of thought’ and request an explanation with the evidence used in the reasoning. Further improvements could include explicitly requesting evidence from input data (e.g., the pathology report) and inference rules (e.g., AJCC rules). Prompting GPT models to respond with ‘Unknown’ when information is insufficient for making assertions, providing relevant context in the prompt, or using ‘embedding’ of relevant text to narrow down the semantic subspace can also be effective. Harnessing hallucination is an ongoing challenge in AI research, with various methods being explored 5 , 27 . For example, a recent study proposed “SelfCheckGPT” approach to fact-check black-box models 29 . Developing real-time error detection mechanisms is crucial for enhancing the reliability and trustworthiness of AI models. More research is needed to evaluate the extent, impacts, and potential solutions of using LLMs in clinical research and care.

When considering using ChatGPT and similar LLMs in healthcare, it’s important to thoughtfully consider the privacy implications. The sensitivity of medical data, governed by rigorous regulations like HIPAA, naturally raises concerns when integrating technologies like LLMs. Although it is a less concern to analyze public available de-identified data, like the lung cancer pathology notes used in this study, careful considerations are needed for secured healthcare data. More secured OpenAI services are offered by OpenAI security portal, claimed to be compliant to multiple regulation standards, and Microsoft Azure OpenAI, claimed could be used in a HIPAA-compliant manner. For example, de-identified Osteosarcoma pathology notes were analyzed by Microsoft Azure OpenAI covered by the Business Associate Agreement in this study. In addition, exploring options such as private versions of these APIs, or even developing LLMs within a secure healthcare IT environment, might offer good alternatives. Moreover, implementing strong data anonymization protocols and conducting regular security checks could further protect patient information. As we navigate these advancements, it’s crucial to continuously reassess and adapt appropriate privacy strategies, ensuring that the integration of AI into healthcare is both beneficial and responsible.

Despite these challenges, this study demonstrates our effective methodology in “prompt engineering”. It presents a general framework for using ChatGPT’s API in batch queries to process large volumes of pathology reports for structured information extraction and estimation. The application of ChatGPT in interpreting clinical notes holds substantial promise in transforming how healthcare professionals and patients utilize these crucial documents. By generating concise, accurate, and comprehensible summaries, ChatGPT could significantly enhance the effectiveness and efficiency of extracting structured information from unstructured clinical texts, ultimately leading to more efficient clinical research and improved patient care.

In conclusion, ChatGPT and other LLMs are powerful tools, not just for pathology report processing but also for the broader digital transformation of healthcare documents. These models can catalyze the utilization of the rich historical archives of medical practice, thereby creating robust resources for future research.

Data processing, workflow, and prompt engineering

The lung cancer data we used for this study are publicly accessible via CDSA ( https://cancer.digitalslidearchive.org/ ) and TCGA ( https://cBioPortal.org ), and they are de-identified data. The institutional review board at the University of Texas Southwestern Medical Center has approved this study where patient consent was waived for using retrospective, de-identified electronic health record data.

We aimed to leverage ChatGPT to extract and estimate structured data from these notes. Figure 7a displays our process. First, scanned pathology reports in PDF format were downloaded from TCGA and CDSA databases. Second, R package pdftools, an optical character recognition tool, was employed to convert scanned PDF files into text format. After this conversion, we identified reports with near-empty content, poor scanning quality, or missing report forms, and those cases were excluded from the study. Third, the OpenAI API was used to analyze the text data and extract structured data elements based on specific prompts. In addition, we extracted case identifiers and metadata items from the TCGA metadata file, which was used to evaluate the model performance.

a Illustration of the use of OpenAI API for batch queries of ChatGPT service, applied to a substantial volume of clinical notes — pathology reports in our study. b A general framework for integrating ChatGPT into real-world applications.

In this study, we implemented a problem-solving framework rooted in data science workflow and systems engineering principles, as depicted in Fig. 7b . An important step is the spiral approach 30 to ‘prompt engineering’, which involves experimenting with subtasks, different phrasings, contexts, format specifications, and example outputs to improve the quality and relevance of the model’s responses. It was an iterative process to achieve the desired results. For the prompt engineering, we first define the objective: to extract information on TNM staging and histology type as structured attributes from the unstructured pathology reports. Second, we assigned specific tasks to ChatGPT, including estimating the targeted attributes, evaluating certainty levels, identifying key evidence of each attribute estimation, and generating a summary as output. The output was compiled into a JSON file. In this process, clinicians were actively formulating questions and evaluating the results.

Our study used the “gpt-3.5-turbo” model, accessible via the OpenAI API. The model incorporates 175 billion parameters and was trained on various public and authorized documents, demonstrating specific Artificial General Intelligence (AGI) capabilities 5 . Each of our queries sent to ChatGPT service is a “text completion” 31 , which can be implemented as a single round chat completion. All LLMs have limited context windows, constraining the input length of a query. Therefore, lengthy pathology reports combined with the prompt and ChatGPT’s response might exceed this limit. We used OpenAI’s “tiktoken” Python library to estimate the token count to ensure compliance. This constraint has been largely relaxed by the newly released GPT models with much larger context windows. We illustrate the pseudocode for batch ChatGPT queries on a large pathology report set in Supplementary Fig. 5 .

Model evaluation

We evaluated the performance of ChatGPT by comparing its output with expert-curated data elements provided in the TCGA structured data using the testing data set. Some staging records in the TCGA structured data needed to be updated; our physicians curated and updated those records. To mimic a real-world setting, we processed all reports regardless of data quality to collect model responses. For performance evaluation, we only used valid reports providing meaningful text and excluded the reports with near-empty content, poor scanning quality, and missing report forms, which were reported as irregular cases. We assessed the classification accuracy, F1, Kappa, recall, and precision for each attribute of interest, including pT, pN, overall stage, and histology types, and presented results as accuracy and confusion matrices. Missing data were excluded from the accuracy evaluation, and the coverage rate was reported for predicted values as ‘unknown’ or empty output.

Reporting summary

Further information on research design is available in the Nature Research Reporting Summary linked to this article.

Data availability

The lung cancer dataset we used for this study is “Pan-Lung Cancer (TCGA, Nat Genet2016)”, ( https://www.cbioportal.org/study/summary?id=nsclc_tcga_broad_2016 ) and the “luad” and “lusc” subsets from CDSA ( https://cancer.digitalslidearchive.org/ ). We have provided a reference regarding how to access the data 32 . We utilized the provided APIs to retrieve clinical information and pathology reports for the LUAD (lung adenocarcinoma) and LUSC (lung squamous cell carcinoma) cohorts. The pediatric data are the EHR data from UTSW clinic services. The data is available from the corresponding author upon reasonable request and IRB approval.

Code availability

All codes used in this paper were developed using APIs from OpenAI. The prompt for the API is available in Fig. 2 . Method-specific code is available from the corresponding author upon request.

Vaswani, A. et al. Attention is all you need. Adv. Neural Info. Processing Syst. 30 , (2017).

Devlin, J. et al. Bert: Pre-training of deep bidirectional transformers for language understanding . arXiv preprint arXiv:1810.04805, 2018.

Radford, A. et al. Improving language understanding by generative pre-training . OpenAI: https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf (2018).

Touvron, H. et al. LLaMA: Open and efficient foundation language models . arXiv preprint arXiv:2302.13971 (2023).

OpenAi, GPT-4 Technical Report . arXiv:2303.08774: https://arxiv.org/pdf/2303.08774.pdf (2023).

Anil, R. et al. Palm 2 technical report . arXiv preprint arXiv:2305.10403 (2023).

Turner, B. E. W. Epic, Microsoft bring GPT-4 to EHRs .

Landi, H. Microsoft’s Nuance integrates OpenAI’s GPT-4 into voice-enabled medical scribe software .

Hao, T. et al. Health Natural Language Processing: Methodology Development and Applications. JMIR Med Inf. 9 , e23898 (2021).

Article   Google Scholar  

Pathak, J., Kho, A. N. & Denny, J. C. Electronic health records-driven phenotyping: challenges, recent advances, and perspectives. J. Am. Med. Inform. Assoc. 20 , e206–e211 (2013).

Article   PubMed   PubMed Central   Google Scholar  

Crichton, G. et al. A neural network multi-task learning approach to biomedical named entity recognition. BMC Bioinforma. 18 , 368 (2017).

Wang, J. et al. Document-Level Biomedical Relation Extraction Using Graph Convolutional Network and Multihead Attention: Algorithm Development and Validation. JMIR Med Inf. 8 , e17638 (2020).

Liu, Y. et al. Roberta: A robustly optimized BERT pretraining approach . arXiv preprint arXiv:1907.11692 (2019).

Rasmy, L. et al. Med-BERT: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction. npj Digit. Med. 4 , 86 (2021).

Wu, H. et al. A survey on clinical natural language processing in the United Kingdom from 2007 to 2022. npj Digit. Med. 5 , 186 (2022).

Amin, M. B. et al. AJCC cancer staging manual . 1024: Springer 2017.

Goldstraw, P. et al. The IASLC Lung Cancer Staging Project: Proposals for the Revision of the TNM Stage Groupings in the Forthcoming (Seventh) Edition of the TNM Classification of Malignant Tumours. J. Thorac. Oncol. 2 , 706–714 (2007).

Article   PubMed   Google Scholar  

Yang, D. M. et al. Osteosarcoma Explorer: A Data Commons With Clinical, Genomic, Protein, and Tissue Imaging Data for Osteosarcoma Research. JCO Clin. Cancer Inform. 7 , e2300104 (2023).

The Lancet Digital, H., ChatGPT: friend or foe? Lancet Digital Health . 5 , e102 (2023).

Nature, Will ChatGPT transform healthcare? Nat. Med. 29 , 505–506 (2023).

Patel, S. B. & Lam, K. ChatGPT: the future of discharge summaries? Lancet Digit. Health 5 , e107–e108 (2023).

Article   CAS   PubMed   Google Scholar  

Ali, S. R. et al. Using ChatGPT to write patient clinic letters. Lancet Digit. Health 5 , e179–e181 (2023).

Howard, A., Hope, W. & Gerada, A. ChatGPT and antimicrobial advice: the end of the consulting infection doctor? Lancet Infect. Dis. 23 , 405–406 (2023).

Mialon, G. et al. Augmented language models: a survey . arXiv preprint arXiv:2302.07842 (2023).

Brown, T. et al. Language Models are Few-Shot Learners . Curran Associates, Inc. (2020).

Wei, J. et al. Chain of thought prompting elicits reasoning in large language models . Adv Neural Inf Processing Syst 35 , 24824–24837 (2022).

Ji, Z. et al. Survey of Hallucination in Natural Language Generation. ACM Comput. Surv. 55 , 1–38 (2023).

Alkaissi, H. & S. I. McFarlane, Artificial Hallucinations in ChatGPT: Implications in Scientific Writing. Cureus , (2023).

Manakul, P. A. Liusie, & M. J. F. Gales, SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models . 2023.

Boehm, B. W. A spiral model of software development and enhancement. Computer 21 , 61–72 (1988).

OpenAi. OpenAI API Documentation . Available from: https://platform.openai.com/docs/guides/text-generation .

Gao, J. et al. Integrative analysis of complex cancer genomics and clinical profiles using the cBioPortal. Sci. Signal. 6 , 1–19 (2013).

Download references

Acknowledgements

This work was partially supported by the National Institutes of Health [P50CA70907, R35GM136375, R01GM140012, R01GM141519, R01DE030656, U01CA249245, and U01AI169298], and the Cancer Prevention and Research Institute of Texas [RP230330 and RP180805].

Author information

Authors and affiliations.

Quantitative Biomedical Research Center, Peter O’Donnell School of Public Health, University of Texas Southwestern Medical Center, 5323 Harry Hines Blvd, Dallas, TX, USA 75390, USA

Jingwei Huang, Donghan M. Yang, Ruichen Rong, Kuroush Nezafati, Colin Treager, Shidan Wang, Xian Cheng, Yujia Guo, Guanghua Xiao, Xiaowei Zhan & Yang Xie

Department of Pathology, University of Texas Southwestern Medical Center, 5323 Harry Hines Blvd, Dallas, TX, USA 75390, USA

Department of Pediatrics, University of Texas Southwestern Medical Center, 5323 Harry Hines Blvd, Dallas, TX, USA 75390, USA

Laura J. Klesse

Department of Internal Medicine, University of Texas Southwestern Medical Center, 5323 Harry Hines Blvd, Dallas, TX, USA 75390, USA

Eric D. Peterson

You can also search for this author in PubMed   Google Scholar

Contributions

J.H., Y.X., X.Z. and G.X. designed the study. X.Z., K.N., C.T. and J.H. prepared, labeled, and curated lung cancer datasets. D.M.Y., X.C., Y.G., L.J.K. prepared, labeled, and curated osteosarcoma datasets. Z.C. provided critical inputs as pathologists. Y.X., G.X., E.P. provided critical inputs for the study. J.H. implemented experiments with ChatGPT. R.R. and K.N. implemented experiments with N.L.P. J.H., Y.X., G.X. and S.W. conducted data analysis. Y.X., G.X., J.H., X.Z., D.M.Y. and R.R. wrote the manuscript. All co-authors read and commented on the manuscript.

Corresponding authors

Correspondence to Xiaowei Zhan or Yang Xie .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplemental figures and tables, reporting summary, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Huang, J., Yang, D.M., Rong, R. et al. A critical assessment of using ChatGPT for extracting structured data from clinical notes. npj Digit. Med. 7 , 106 (2024). https://doi.org/10.1038/s41746-024-01079-8

Download citation

Received : 24 July 2023

Accepted : 14 March 2024

Published : 01 May 2024

DOI : https://doi.org/10.1038/s41746-024-01079-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

This paper is in the following e-collection/theme issue:

Published on 6.5.2024 in Vol 12 (2024)

Assessing the Efficacy of ChatGPT Versus Human Researchers in Identifying Relevant Studies on mHealth Interventions for Improving Medication Adherence in Patients With Ischemic Stroke When Conducting Systematic Reviews: Comparative Analysis

Authors of this article:

Original Paper

• Suebsarn Ruksakulpiwat 1 , RN, MMed, PhD   ; 
• Lalipat Phianhasin 1 , RN, MS, AGPCNP-BC   ; 
• Chitchanok Benjasirisan 1 , RN, MS   ; 
• Kedong Ding 2 , AM   ; 
• Anuoluwapo Ajibade 3 , BS   ; 
• Ayanesh Kumar 4 , MS   ; 
• Cassie Stewart 5  

1 Department of Medical Nursing, Faculty of Nursing, Mahidol University, Bangkok, Thailand

2 Jack, Joseph and Morton Mandel School of Applied Social Sciences, Case Western Reserve University, Cleveland, OH, United States

3 College of Art and Science, Department of Anthropology, Case Western Reserve University, Cleveland, OH, United States

4 School of Medicine, Case Western Reserve University, Cleveland, OH, United States

5 Frances Payne Bolton School of Nursing, Case Western Reserve University, Cleveland, OH, United States

Corresponding Author:

Suebsarn Ruksakulpiwat, RN, MMed, PhD

Department of Medical Nursing, Faculty of Nursing, Mahidol University

2 Wang Lang Road, Siriraj, Bangkok Noi

Bangkok, 10700

Phone: 66 984782692

Email: [email protected]

Background: ChatGPT by OpenAI emerged as a potential tool for researchers, aiding in various aspects of research. One such application was the identification of relevant studies in systematic reviews. However, a comprehensive comparison of the efficacy of relevant study identification between human researchers and ChatGPT has not been conducted.

Objective: This study aims to compare the efficacy of ChatGPT and human researchers in identifying relevant studies on medication adherence improvement using mobile health interventions in patients with ischemic stroke during systematic reviews.

Methods: This study used the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Four electronic databases, including CINAHL Plus with Full Text, Web of Science, PubMed, and MEDLINE, were searched to identify articles published from inception until 2023 using search terms based on MeSH (Medical Subject Headings) terms generated by human researchers versus ChatGPT. The authors independently screened the titles, abstracts, and full text of the studies identified through separate searches conducted by human researchers and ChatGPT. The comparison encompassed several aspects, including the ability to retrieve relevant studies, accuracy, efficiency, limitations, and challenges associated with each method.

Results: A total of 6 articles identified through search terms generated by human researchers were included in the final analysis, of which 4 (67%) reported improvements in medication adherence after the intervention. However, 33% (2/6) of the included studies did not clearly state whether medication adherence improved after the intervention. A total of 10 studies were included based on search terms generated by ChatGPT, of which 6 (60%) overlapped with studies identified by human researchers. Regarding the impact of mobile health interventions on medication adherence, most included studies (8/10, 80%) based on search terms generated by ChatGPT reported improvements in medication adherence after the intervention. However, 20% (2/10) of the studies did not clearly state whether medication adherence improved after the intervention. The precision in accurately identifying relevant studies was higher in human researchers (0.86) than in ChatGPT (0.77). This is consistent with the percentage of relevance, where human researchers (9.8%) demonstrated a higher percentage of relevance than ChatGPT (3%). However, when considering the time required for both humans and ChatGPT to identify relevant studies, ChatGPT substantially outperformed human researchers as it took less time to identify relevant studies.

Conclusions: Our comparative analysis highlighted the strengths and limitations of both approaches. Ultimately, the choice between human researchers and ChatGPT depends on the specific requirements and objectives of each review, but the collaborative synergy of both approaches holds the potential to advance evidence-based research and decision-making in the health care field.

Introduction

Artificial intelligence (AI) is the field of computer science that studies and develops systems that can perform tasks, typically requiring human intelligence, such as reasoning, learning, decision-making, natural language processing (NLP), computer vision, and speech recognition [ 1 ]. AI is a rapidly evolving field with applications in various domains, for example, health care, education, business, and entertainment [ 2 ]. One of the subfields of AI is NLP, which deals with analyzing and generating natural language texts [ 3 ]. Chatbots, a type of NLP system, can interact with humans using natural language, either through text or speech. Chatbots can be used for various purposes, including customer service, entertainment, education, and information retrieval [ 3 ]. However, developing chatbots that can engage in natural and coherent conversations with humans is a challenging task that requires advanced NLP techniques and large-scale data.

One of the recent advances in NLP is the development of GPT models, which are neural network models that can generate natural language texts based on a given input or context [ 4 ]. GPT models are trained on large corpora of text from various sources, such as books, websites, news articles, and social media posts [ 4 ]. GPT models have been used to create chatbots that can generate realistic and diverse responses to human queries or messages [ 4 ]. Although GPT models have been developed by various research groups and companies (ie, OpenAI, Google, Facebook, and Microsoft), the first one was introduced by OpenAI in 2019 [ 5 ]. Since then, ChatGPT has been improved and refined by researchers and developers, who have applied it to various tasks and scenarios, such as customer service, education, entertainment, and social media [ 5 ]. ChatGPT models aim to provide engaging, informative, and coherent dialogues with users across different domains and tasks [ 4 ].

ChatGPT has been applied in the medical field in various ways. For instance, in medical practice, it has the ability to help streamline the clinical workflow, enhance diagnostics, and predict disease risk and outcome [ 6 ]. For medical education, ChatGPT can be useful in tailoring education and enabling powerful self-learning [ 6 ]. In terms of medical research, a previous study reported that ChatGPT has the potential to advance understanding, identify new research questions, and improve data analysis and interpretation [ 7 ]. In addition, ChatGPT extends to involve in writing articles through improvement in language and communication of result findings [ 6 ]. In particular, in the literature review process, which is time and effort consuming, ChatGPT has a promising advantage because of its potential ability to analyze large amounts of data, particularly in scientific articles [ 8 ]. Furthermore, ChatGPT was reported to have the potential to generate effective Boolean queries for systematic review literature searches [ 9 ].

Although ChatGPT has several advantages in medical research, it has limitations that could impact the quality of research, particularly in the literature review and search strategies processes. Citation inaccuracies, insufficient references, and references to nonexistent sources were reported as current problems [ 6 ]. Moreover, ChatGPT has a limited knowledge period based on the data sets used in ChatGPT training, which limits the reliability of the updated source of the literature review [ 6 ]. In a previous study, researchers were advised to consider the potential for incorrect MeSH (Medical Subject Headings) terms and the varying effectiveness of search queries with multiple requests when devising search strategies for a systematic review [ 9 ]. However, ChatGPT has a high potential to be used in medical research in the future. Therefore, it is imperative to explore and develop to improve and use it effectively.

Despite the significant benefits and limitations of using ChatGPT, the evaluation of the quality and performance of ChatGPT models in the review process remains unclear. Therefore, this study aims to compare the efficacy of ChatGPT and human researchers in identifying relevant health-related studies, such as research on medication adherence improvement using mobile health (mHealth) interventions in patients with ischemic stroke. The review will use systematic methods to search, select, appraise, and synthesize to address the following questions: (1) How does ChatGPT’s performance compare to that of human researchers in terms of accuracy in identifying relevant studies? (2) What challenges and limitations arise from using ChatGPT versus human researchers for identifying relevant studies in systematic reviews? (3) What are the implications of using ChatGPT to enhance the efficiency of systematic reviews? The results of this review will provide crucial insights into the potential of ChatGPT as an innovative tool for conducting systematic reviews.

This study aims to compare the efficacy of using ChatGPT and human researchers in identifying relevant studies on medication adherence improvement using mHealth interventions in patients with ischemic stroke during systematic reviews.

Identify Relevant Studies

In this study, we used the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) [ 10 ] guidelines to identify the relevant studies. Overall, 4 electronic databases, including CINAHL Plus with Full Text, Web of Science, PubMed, and MEDLINE, were searched to identify articles published from inception until 2023 on using mHealth interventions for improving medication adherence in patients with ischemic stroke. We used search terms based on MeSH using Boolean phrases generated by human researchers and ChatGPT version 3.5 to identify relevant studies. The reference lists of the included studies, generated by human researchers and ChatGPT, were separately stored and screened in EndNote (EndNote X7 reference management software package). A PRISMA flow diagram was created to present the results of the search and screening process.

Study Selection

The authors independently screened the titles and abstracts of the studies identified through separate searches conducted by human researchers and ChatGPT to determine their relevance. Subsequently, the full text of the selected articles was also assessed to ensure they met the predetermined inclusion criteria. A consistent set of inclusion criteria was applied to ensure that only studies relevant to the review’s objective were included. In contrast, the same exclusion criteria were used to eliminate literature unrelated to the review ( Textbox 1 ).

Inclusion criteria

• Studies that aimed to use mobile health interventions for improving medication adherence
• Studies that primarily included adults with ischemic stroke or transient ischemic attack (TIA) aged ≥18 years (if the study included other stroke types, such as hemorrhagic stroke, it is acceptable, but the study population must include adults with ischemic stroke or TIA)
• Studies in English
• Studies that were published from inception until 2023

Exclusion criteria

• Studies that included children or adolescents aged <18 years
• Conference proceedings, abstracts, review articles, protocols, dissertations, letters to the editor, brief reports, or statement papers
• Studies that involved animal samples

Data Extraction

A separate summary table for data extraction is presented in Multimedia Appendix 1 [ 11 - 20 ], consisting of the following data for each study: reference, year, country, study design, sample size, target population, intervention and objective, and main findings. This table will be used to compare the included studies obtained through the Identify relevant studies phase conducted by human researchers versus ChatGPT. The primary outcome of interest is medication adherence among patients with ischemic stroke. Medication adherence can be measured using various methods, such as drug level measurement, pill count, electronic databases, self-report questionnaires, and electronic monitoring systems [ 21 ]. The findings from studies that aimed to use mHealth interventions for improving medication adherence but did not measure medical adherence directly will be evaluated based on how they operationalized medication adherence according to their study design.

Data Analysis

In this study, we will assess the accuracy of both human researchers and ChatGPT in identifying relevant studies from electronic databases by measuring precision. Precision is a performance metric that measures the accuracy of a model’s positive predictions. It focuses on the proportion of correctly identified positive instances (true positives) out of all the cases that the model predicted as positive (true positives+false positives) [ 22 ]. Precision is calculated using the following formula: precision=true positives/(true positives+false positives).

A high precision value close to 1 indicates that the model has a low rate of false positives. This means that when the model predicts an instance as positive, it will likely be correct. In contrast, a low precision value close to 0 indicates that the model has a high rate of false positives. This means that when the model predicts an instance as positive, it often needs to be corrected [ 22 ]. In the context of this study, precision will help evaluate the ability of both human researchers and ChatGPT to accurately identify relevant studies from electronic databases during the systematic review process. We will compare their precision scores to determine which approach yields a higher proportion of true positives and a lower rate of false positives.

In addition, as the human researcher will still need to conduct the screening, eligibility, and inclusion phases, we will also calculate the percentage of relevance using the formula ([true positives/total studies identified from the search]×100). This approach will be chosen to ensure a fair assessment, as relying solely on a formula based on true and false positives (precision) might only reflect human variability and accuracy during the screening, eligibility, and inclusion phases.

Ethical Considerations

This study considers nonhuman research according to the “Self-Assessment form whether an activity is human subject research which requires ethical approval” recommended by Mahidol University Central Institutional Review Board. Therefore, ethics approval from the research ethics committee was not required.

Search Term

Human researcher.

In the search phase, we used search terms based on MeSH using Boolean operators. The searched topic was related to using mHealth interventions for improving medication adherence in patients with ischemic stroke: (Ischemic Stroke* OR Cryptogenic Ischemic Stroke* OR Cryptogenic Stroke* OR Cryptogenic Embolism Stroke* OR Wake up Stroke* OR Acute Ischemic Stroke* OR Embolic Stroke* OR Cardioembolic Stroke* OR Cardioembolic Stroke* OR Thrombotic Stroke* OR Acute Thrombotic Stroke* OR Lacunar Stroke* OR Lacunar Syndrome* OR Lacunar Infarction* OR Lacunar Infarct*) AND (Medication Adherence OR Medication Nonadherence OR Medication Noncompliance OR Medication Persistence OR Medication Compliance OR Medication Non-Compliance) AND (Tele-Referral* OR Virtual Medicine OR Tele Intensive Care OR Tele ICU OR Mobile Health OR mHealth OR Telehealth OR eHealth OR Remote Consultation OR Teleconsultation* OR Telenursing OR Telepathology OR Teleradiology OR Telerehabilitation* OR Remote Rehabilitation* OR Virtual Rehabilitation*).

To compare with the search by human researchers, we asked ChatGPT [ 23 ] on June 23, 2023, at 1:30 PM EST to provide a search term for conducting a systematic review of the same topic as follows: “Hello ChatGPT, we are researchers and currently conduct a systematic review titled: Using m-health interventions for improving medication adherence in ischemic stroke patients. Can you provide Medical Subject Headings (MeSH) search terms and combine them using Boolean operators for a search process?” The following search terms resulted from ChatGPT, which we used in the search phase and then compared the results with those from human researchers: (Mobile Applications OR Cell Phone OR Smartphone OR Telemedicine OR Text Messaging OR Internet) AND (Medication Adherence OR Patient Compliance OR Medication Systems, Intelligent) AND (Stroke OR Ischemic Attack, Transient OR Cerebrovascular Disorders). The search term (generated by human researchers and ChatGPT) was adjusted according to the database searching requirement before searching, but the original keyword was not changed.

Search Results

We compared the ability of humans and ChatGPT to retrieve all relevant studies. A higher recall indicates a better ability to capture all the relevant literature. Figure 1 shows the flowchart diagram of the selection of included studies based on search terms generated by human researchers. An initial literature search yielded 61 articles, including 30 from PubMed and MEDLINE, 21 from Web of Science, and 10 from CINAHL Plus Full Text. No additional records were found through other sources. After deduplication (n=7 studies), the researchers screened 54 studies, of which 47 (87%) were excluded based on the inclusion and exclusion criteria following the title and abstract screening phase. This left 7 articles for full-text screening, during which 1 article was excluded as it did not include any mHealth-related intervention. Therefore, 6 articles were included in the final analysis. It should be noted that human researchers conducted the identification, screening, eligibility, and inclusion phases.

Figure 2 shows the flowchart diagram of the selection of included studies based on search terms generated by ChatGPT. An initial literature search yielded 334 articles, including 146 from PubMed and MEDLINE, 130 from Web of Science, and 58 from CINAHL Plus Full Text. No additional records were found through other sources. After deduplication (n=104 studies), the researchers screened 230 studies, of which 217 (94.3%) were excluded based on the inclusion and exclusion criteria following the title and abstract screening phase. Of the 13 articles that underwent full-text screening, 3 studies were excluded because the intervention was irrelevant (n=1, 33%), the publication was not in English (n=1, 33%), and it was a letter to the editor (n=1, 33%). Finally, 10 articles were included in the final analysis. It should be noted that ChatGPT has been used only in the identification phase. The human researcher conducted the screening, eligibility, and inclusion phases.

Description of the Included Studies

Studies included from human searches.

The analysis included 6 studies obtained from the human search ( Multimedia Appendix 1 ). Most of these studies (3/6, 50%) were published in 2020. Among the countries where the studies were conducted, 50% (3/6) were from China, whereas 17% (1/6) of the studies each originated from Belgium, the Republic of Korea, and Sweden. In terms of study design, of the 6 studies, 3 (50%) were cohort studies, 2 (33%) were randomized controlled trials (RCTs), and 1 (17%) was a non-RCT. The sample sizes varied, with 50% (3/6) of the studies having a sample size ranging from 1 to 300 and the other 50% (3/6) of the studies having a sample size of >300. Regarding the impact of mHealth interventions on medication adherence, most included studies (4/6, 67%) reported improvements in medication adherence after the intervention [ 11 - 14 ]. However, in 33% (2/6) of the included studies, it was not clearly stated whether medication adherence improved after the intervention [ 15 , 16 ].

Studies Included From ChatGPT Searches

A total of 10 studies were obtained from the ChatGPT search, of which 6 (60%) studies overlapped with the human searches ( Multimedia Appendix 1 ). Most of these studies (4/10, 40%) were published in 2020. Among the countries where the studies were conducted, 50% (5/10) of the studies were from China, whereas 10% (1/10) of the studies each originated from Belgium, the Republic of Korea, Sweden, the United States, and Pakistan. In terms of study design, most were RCTs (6/10, 60%), with 30% (3/10) of the studies being cohort studies, and 10% (1/10) of the studies being a non-RCT. The sample sizes varied, with 70% (7/10) of the studies having a sample size ranging from 1 to 300 (70%), and the other 30% (3/10) of the studies having a sample size of >300. Regarding the impact of mHealth interventions on medication adherence, most included studies (8/10, 80%) reported improvements in medication adherence after the intervention [ 11 - 14 , 17 - 20 ]. However, in 20% (2/10) of the included studies, it was not clearly stated whether medication adherence improved after the intervention [ 15 , 16 ].

In our study, we used precision as a metric to assess the accuracy of both human researchers and ChatGPT in identifying relevant studies from electronic databases during the systematic review process. By comparing their precision scores, we aimed to determine which approach yielded a higher proportion of true positives (correctly identified relevant studies) and a lower rate of false positives (incorrectly identified irrelevant studies). The precision calculation formula used was as follows: precision=true positives/(true positives+false positives).

Moreover, the human researcher conducted identification, screening, eligibility, and inclusion phases, as illustrated in Figure 1 . In contrast, ChatGPT was used only during the identification phase, and the human researcher conducted the screening, eligibility, and inclusion phases, as depicted in Figure 2 . Therefore, we also calculated the percentage of relevance using the formula ([true positives/total studies identified from the search]×100). This approach was chosen to ensure a fair assessment, as relying solely on a formula based on true and false positives might only reflect human variability and accuracy during the screening, eligibility, and inclusion phases.

For human researchers, the precision in accurately identifying relevant studies from electronic databases was calculated as 6/(6+1)=0.86, where 6 is the number of studies included in the review (true positive) and 1 (false positive) represents the study that was incorrectly identified as relevant for inclusion in the review (did not include any mHealth-related intervention; Figure 1 ). This means that out of the studies deemed relevant by human researchers, 86% (6/7) were indeed appropriate for inclusion in the review, whereas 14% (1/7) were falsely identified as relevant. The percentage of relevance for the human researcher was calculated as follows: (true positives/total studies identified from the search)×100=(6/61)×100=9.8%.

Regarding ChatGPT, its precision in accurately identifying relevant studies from electronic databases was calculated as 10/(10+3)=0.77, where 10 is the number of studies included in the review (true positive) and 3 (false positive) represents the studies that were incorrectly identified as relevant for inclusion in the review (irrelevant intervention, non-English publication, and a letter to the editor; Figure 2 ). This indicates that out of the studies identified by ChatGPT as potentially relevant, 77% (10/13) were indeed relevant and suitable for inclusion in the review, whereas 23% (3/13) were mistakenly identified as relevant. The percentage of relevance for ChatGPT was calculated as follows: (true positives/total studies identified from the search)×100=(10/334)×100=3%.

According to our findings, the precision of human researchers was higher (precision=0.86) compared to ChatGPT (precision=0.77). This is consistent with the percentage of relevance, where human researchers (9.8%) demonstrated a higher percentage of relevance than ChatGPT (3%). These results indicate that human researchers were more effective in identifying relevant studies during the systematic review process. However, it is noteworthy that despite the lower precision and percentage of relevance, ChatGPT’s initial search yielded a significantly larger number of studies (n=334) compared to human researchers (n=61), and ultimately resulted in more studies included in the final analysis (n=10 for ChatGPT vs n=6 for human researchers). This suggests that ChatGPT’s performance was more efficient in terms of study retrieval and inclusion, although there was a 60% overlap in the studies included between both approaches.

As reported in the Accuracy subsection, human researchers demonstrated higher precision in identifying relevant studies compared to ChatGPT. However, the efficiency and ability of ChatGPT to retrieve relevant studies could still hold value in the systematic review process. When considering the time required for both humans and ChatGPT to identify relevant studies, from the beginning (search term generation) to the outcome (identification of relevant studies before screening), our study found that ChatGPT substantially outperformed human researchers. ChatGPT took approximately 10 minutes, whereas human researchers spent an hour in the search term identification process using MeSH and Boolean operators before obtaining the relevant study.

In our study, we used ChatGPT to generate search terms for conducting the systematic review based on our research topic. This substantially reduced the time and effort required for initial study identification. However, it is important to note that ChatGPT’s current capabilities are limited to providing search terms, and human researchers are still required to conduct the screening of titles, abstracts, and full texts of the identified studies, using refined inclusion and exclusion criteria.

Principal Findings

According to our findings, the precision of human researchers was higher compared to ChatGPT, indicating that human researchers were more accurate in identifying relevant studies during the systematic review process. Our findings are congruent with a previous study [ 24 ], which reports inaccuracies of using ChatGPT in research that requires an in-depth understanding of the literature. Likewise, Zhao et al [ 25 ] reported that the factual accuracy of ChatGPT cannot be ensured, although it has massive resources such as Microsoft and Google. In addition, a case study of using ChatGPT to conduct literature searches indicated that ChatGPT does not provide an answer to the queries that researchers ask for [ 26 ].

Despite the lower precision of ChatGPT compared to human search, a previous study reported that ChatGPT has more accurate and comprehensive relevance judgments than all other types of NLP models or techniques [ 27 ]. Moreover, our findings show that ChatGPT’s initial search yielded a significantly larger number of studies compared to human researchers and ultimately resulted in more studies being included in the final analysis despite its lower precision. This suggests that ChatGPT’s performance was more efficient in terms of study retrieval and inclusion, although there was a 60% overlap in the studies included between both approaches. Similarly, a study of ChatGPT's insights on the future of scientific publishing reports it as a valuable resource for initiating discussions [ 28 ]. However, a previous study using ChatGPT for retrieval of clinical, radiological information reported that ChatGPT provided only two-thirds of correct responses to questions [ 29 ].

Regarding the efficiency issues of using ChatGPT in identifying relevant search terms, the results of this study suggest that ChatGPT can be a useful tool for generating search terms for systematic reviews, as it can save time and effort for human researchers and potentially retrieve more relevant studies. The previous study on the use of ChatGPT Boolean query construction and refinement for systematic review showed that ChatGPT can generate queries with high precision [ 9 ]. Therefore, ChatGPT could be a valuable tool, especially for rapid reviews where time is limited and high precision is preferred over high recall [ 9 ].

Some researchers may argue that as ChatGPT has lower precision and may generate irrelevant or inaccurate terms, human researchers still need to carefully screen the studies that ChatGPT identified and verify the quality and validity of the evidence [ 30 ]. ChatGPT should be used with caution and verification and supplemented with other methods and sources to ensure the validity and rigor of the literature search [ 9 ]. Furthermore, ChatGPT’s performance may vary depending on the research topic, data availability, and input quality. Thus, future studies are needed to evaluate ChatGPT’s generalizability and reliability across different domains and contexts.

Using ChatGPT to generate search terms for systematic reviews raises some ethical questions regarding the quality and validity of the research process. Although ChatGPT may offer some advantages in terms of efficiency and comprehensiveness, it may also introduce some biases and errors that could affect the reliability and reproducibility of the systematic reviews. For example, ChatGPT may generate search terms that are irrelevant to the research topic or too broad or narrow, resulting in either missing or including studies that do not meet the inclusion criteria [ 31 ]. Moreover, ChatGPT may generate search terms that are based on its own internal knowledge and information, which may not reflect the current state of the art or the best available evidence in the field [ 31 ]. Therefore, human researchers need to carefully evaluate and validate the search terms generated by ChatGPT and document their rationale and methods for using them. In addition, human researchers need to disclose the use of ChatGPT as a tool for generating search terms and report its strengths and limitations and any potential ethical implications in their systematic review reports [ 31 ]. This would ensure that the systematic review process is transparent, accountable, and trustworthy and that the results are credible and useful for informing decision-making.

As we embark on a comparative analysis between ChatGPT and human researchers in the pursuit of identifying relevant studies within systematic reviews, particularly focused on mHealth interventions for improving medication adherence in patients with ischemic stroke, it becomes evident that several challenges and limitations underscore the intricate nature of this exploration. These challenges offer insight into the complex interplay between cutting-edge technology and the established domain expertise of human researchers, shaping the landscape in which this study unfolds.

First and foremost, the outcomes of our study are intrinsically linked to the performance of ChatGPT, an AI-driven tool that relies on its current capabilities to generate search terms. As an entity in constant evolution, ChatGPT’s performance may undergo shifts over time, potentially influencing the accuracy and efficiency with which it generates relevant search terms. Moreover, replicating the search in subsequent studies is essential due to ChatGPT’s intrinsic unpredictability. The lack of such repetition presents challenges in determining whether the observed phenomenon reflects an inherent trait of the model or is simply a random incident.

This dynamic underscores the need to interpret our findings in the context of the tool’s state during the study period. Within the realm of medical research, the intricate and evolving nature of terminology poses a formidable challenge. Although ChatGPT exhibits language generation prowess, the intricate nuances of medical terminology—constantly adapting and expanding—could potentially pose challenges to its accurate formulation of search terms. The complexity inherent to medical concepts demands a level of contextual understanding that might be challenging for an AI system.

Another pivotal consideration revolves around the potential biases embedded within ChatGPT’s training data. Drawing insights from vast data sets, ChatGPT-generated search terms might inadvertently inherit biases present in the underlying data sources. This potential bias, albeit unintentional, introduces an element of caution when relying solely on AI-generated search terms for systematic reviews. A crucial aspect of our study’s execution pertains to refining search terms. Although ChatGPT serves as a catalyst for initial search term generation, human researchers play a pivotal role in the subsequent validation and fine-tuning of these terms. This collaborative process introduces an additional layer of complexity, as human intervention becomes essential to ensure the relevance and accuracy of the generated search terms. Moreover, the resources available and the access to ChatGPT’s capabilities could introduce variability in the study’s outcomes. Depending on factors such as subscription tiers or institutional resources, the extent of ChatGPT’s contributions and, subsequently, its comparative assessment against human researchers may exhibit nuances that warrant consideration. The study’s defined scope, focused on mHealth interventions for medication adherence improvement in patients with ischemic stroke, provides a specific lens through which insights are garnered. However, this specificity inherently limits the direct transposability of findings to other medical domains or broader systematic review topics. The nuances of different research contexts might yield distinct results. Language and geographic considerations further amplify the complexity. The study predominantly engaged with studies in English, potentially omitting valuable research published in other languages or regions. This limitation underscores the need for meticulous attention to language diversity and inclusion in systematic reviews. Human researcher variability introduces a layer of subjectivity into the study. With multiple researchers contributing to search term generation, variations in expertise and individual approaches could impact the study’s outcomes. The potential for differing interpretations and formulations of search terms necessitates careful management. Publication bias, a well-known challenge in research, extends its influence into our study’s design. Both ChatGPT and human researchers might inadvertently be swayed by publication bias, where certain types of studies are more likely to be published, potentially influencing the pool of studies considered in this review.

External factors beyond the purview of our study could exert unanticipated influence. Variables such as changes in database availability, updates to search algorithms, or shifts in the research landscape might subtly shape the study’s design and outcomes, introducing an element of unpredictability. The study’s designated time frame for data collection and inclusion introduces potential time constraints and selection bias. Studies published after the search period might be inadvertently omitted, potentially impacting the completeness of the review. Although the study provides valuable insights within its specific scope, the generalizability of findings to other systematic review topics or research questions requires cautious interpretation. The intricate interplay between technology and human expertise forms the cornerstone of our study, emphasizing the necessity for a balanced and nuanced approach when leveraging ChatGPT for systematic reviews.

The Implications of Using ChatGPT to Improve the Efficiency of Systematic Reviews

The integration of ChatGPT into the systematic review process for identifying relevant studies on mHealth interventions holds several noteworthy implications for research methodology, efficiency, and the advancement of evidence-based practices. This section explores the key implications that arise from incorporating ChatGPT as a tool to expedite and enhance the systematic review process.

One of the most immediate and impactful implications of using ChatGPT is its ability to significantly expedite the systematic review process. Traditionally, the generation of search terms for identifying relevant articles is a time-intensive task that requires meticulous crafting and refinement by human researchers. ChatGPT’s capacity to swiftly generate search terms offers an innovative solution to this bottleneck, reducing the time invested in this preliminary phase. This acceleration holds the potential to expedite the overall timeline of systematic reviews, enabling researchers to allocate more time to critical appraisal, synthesis, and analysis of selected studies.

The inherent nature of ChatGPT’s language generation capabilities allows a more diverse and expansive range of search terms. By tapping into its capacity to comprehend and generate natural language, researchers can explore a broader spectrum of keyword variations and synonyms. This expanded search scope can lead to the inclusion of studies that might have been overlooked using traditional search methods. As a result, the systematic review process becomes more comprehensive, encompassing a wider array of relevant literature.

ChatGPT’s ability to generate novel and contextually relevant search terms introduces a valuable avenue for exploratory research and hypothesis generation. Researchers can leverage ChatGPT to identify emerging trends, novel terminologies, or unconventional associations that may inform the direction of their systematic reviews. This capacity to extract insights from the vast expanse of existing literature can potentially lead to the formulation of innovative research questions and avenues for investigation.

Although ChatGPT demonstrates remarkable efficiency in generating search terms, its use necessitates a collaborative approach with human researchers. The synergy between ChatGPT’s speed and human researchers’ expertise in refining and validating search terms ensures a balanced and accurate outcome. Human researchers play a pivotal role in critically evaluating the generated search terms, refining them to align with the specific objectives of the review, and subsequently verifying the relevance of the identified articles. This collaborative interplay mitigates the risk of introducing erroneous or irrelevant studies into the review process.

In research environments with limited resources, such as time and personnel, ChatGPT offers a solution to address scalability challenges. Its ability to rapidly generate search terms can prove invaluable in scenarios where timely completion of systematic reviews is imperative. Researchers operating within resource-constrained contexts can leverage ChatGPT to conduct preliminary searches efficiently, thus optimizing the allocation of limited resources to subsequent stages of the review.

In summary, the integration of ChatGPT into the systematic review process introduces a transformative approach to enhancing efficiency and enriching the scope of literature exploration. Although its speed and breadth of search terms hold the promise of expediting the review timeline and uncovering hidden associations, the collaborative involvement of human researchers remains pivotal for ensuring accuracy, relevance, and the meticulous execution of subsequent review stages. The strategic use of ChatGPT in conjunction with traditional research practices paves the way for a new era of evidence synthesis and knowledge advancement in the field of health care interventions.

Conclusions

Our study compares the accuracy and efficacy of human researchers and ChatGPT in providing search terms to identify articles during a systematic review on mHealth interventions for improving medication adherence in patients with ischemic stroke. Although human researchers achieved greater precision, ChatGPT’s search results exhibited lower accuracy. However, ChatGPT excelled in efficacy, taking less time to generate search terms compared to human researchers, who required more time to identify appropriate search terms. In addition, ChatGPT’s search yielded a higher number of articles compared to human researchers. Following exclusions, human researchers were left with 6 articles, and ChatGPT resulted in 10 articles after screening, 6 (60%) of which overlapped with the findings of human researchers. The use of ChatGPT in creating search terms can significantly accelerate the systematic review process, although human researchers are still essential to carry out the selection process and ensure accuracy.

Acknowledgments

The authors would like to express their gratitude to Mahidol University, Thailand, for covering the article publishing charge. During the preparation of this work, the authors used ChatGPT (version 3.5, 2024; OpenAI [ 23 ]) to improve language and readability, which the study group further reviewed and revised.

Conflicts of Interest

None declared.

Summary table.

• Chassignol M, Khoroshavin A, Klimova A, Bilyatdinova A. Artificial Intelligence trends in education: a narrative overview. Procedia Comput Sci. 2018;136:16-24. [ CrossRef ]
• Bohr A, Memarzadeh K. The rise of artificial intelligence in healthcare applications. In: Bohr A, Memarzadeh K, editors. Artificial Intelligence in healthcarelsevier. Washington, DC. Academic Press; 2020;25-60.
• Hapke H, Howard C, Lane H. Natural Language Processing in Action: Understanding, Analyzing, and Generating Text With Python. New York, NY. Manning Publications; 2019.
• Lund BD, Wang T, Mannuru NR, Nie B, Shimray S, Wang Z. ChatGPT and a new academic reality: artificial intelligence-written research papers and the ethics of the large language models in scholarly publishing. J Assoc Inf Sci Technol. Mar 10, 2023;74(5):570-581. [ FREE Full text ] [ CrossRef ]
• Why tech insiders are so excited about ChatGPT, a chatbot that answers questions and writes essays. CNBC. 2022. URL: https:/​/www.​cnbc.com/​2022/​12/​13/​chatgpt-is-a-new-ai-chatbot-that-can-answer-questions-and-write-essays.​html [accessed 2024-04-12]
• Sallam M. ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns. Healthcare (Basel). Mar 19, 2023;11(6):887. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ruksakulpiwat S, Kumar A, Ajibade A. Using ChatGPT in medical research: current status and future directions. J Multidiscip Healthc. 2023;16:1513-1520. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Dahmen J, Kayaalp ME, Ollivier M, Pareek A, Hirschmann MT, Karlsson J, et al. Artificial intelligence bot ChatGPT in medical research: the potential game changer as a double-edged sword. Knee Surg Sports Traumatol Arthrosc. Apr 21, 2023;31(4):1187-1189. [ CrossRef ] [ Medline ]
• Wang S, Scells H, Koopman B, Zuccon G. Can chatgpt write a good boolean query for systematic review literature search? arXiv preprint arXiv. arXiv. Preprint posted online February 3, 2023. 2023. [ CrossRef ]
• Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. Jul 21, 2009;6(7):e1000097. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kim DY, Kwon H, Nam KW, Lee Y, Kwon HM, Chung YS. Remote management of poststroke patients with a smartphone-based management system integrated in clinical care: prospective, nonrandomized, interventional study. J Med Internet Res. Feb 27, 2020;22(2):e15377. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Li DM, Lu XY, Yang PF, Zheng J, Hu HH, Zhou Y, et al. Coordinated patient care via mobile phone-based telemedicine in secondary stroke prevention: a propensity score-matched cohort study. J Nurs Care Qual. 2023;38(3):E42-E49. [ CrossRef ] [ Medline ]
• Yan LL, Gong E, Gu W, Turner EL, Gallis JA, Zhou Y, et al. Effectiveness of a primary care-based integrated mobile health intervention for stroke management in rural China (SINEMA): a cluster-randomized controlled trial. PLoS Med. Apr 28, 2021;18(4):e1003582. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Zhang Y, Fan D, Ji H, Qiao S, Li X. Treatment adherence and secondary prevention of ischemic stroke among discharged patients using mobile phone- and WeChat-based improvement services: cohort study. JMIR Mhealth Uhealth. Apr 15, 2020;8(4):e16496. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kamoen O, Maqueda V, Yperzeele L, Pottel H, Cras P, Vanhooren G, et al. Stroke coach: a pilot study of a personal digital coaching program for patients after ischemic stroke. Acta Neurol Belg. Feb 07, 2020;120(1):91-97. [ CrossRef ] [ Medline ]
• Ögren J, Irewall A, Söderström L, Mooe T. Long-term, telephone-based follow-up after stroke and TIA improves risk factors: 36-month results from the randomized controlled NAILED stroke risk factor trial. BMC Neurol. Sep 21, 2018;18(1):153. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Labovitz DL, Shafner L, Reyes Gil M, Virmani D, Hanina A. Using artificial intelligence to reduce the risk of nonadherence in patients on anticoagulation therapy. Stroke. May 2017;48(5):1416-1419. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kamal AK, Shaikh Q, Pasha O, Azam I, Islam M, Memon AA, et al. A randomized controlled behavioral intervention trial to improve medication adherence in adult stroke patients with prescription tailored short messaging service (SMS)-SMS4Stroke study. BMC Neurol. Oct 21, 2015;15:212. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Wang MY, Shen MJ, Wan LH, Mo MM, Wu Z, Li LL, et al. Effects of a comprehensive reminder system based on the health belief model for patients who have had a stroke on health behaviors, blood pressure, disability, and recurrence from baseline to 6 months: a randomized controlled trial. J Cardiovasc Nurs. 2020;35(2):156-164. [ CrossRef ] [ Medline ]
• Wan L, Zhang X, You L, Ruan H, Chen S. The efficacy of a comprehensive reminder system to improve health behaviors and blood pressure control in hypertensive Ischemic stroke patients: a randomized controlled trial. J Cardiovasc Nurs. 2018;33(6):509-517. [ CrossRef ] [ Medline ]
• Anghel LA, Farcas AM, Oprean RN. An overview of the common methods used to measure treatment adherence. Med Pharm Rep. Apr 22, 2019;92(2):117-122. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Powers DM. Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation. arXiv. Preprint posted online October 11, 2020. 2020. [ FREE Full text ]
• ChatGPT (3.5). OpenAI. URL: https://chat.openai.com [accessed 2024-04-05]
• van Dis EA, Bollen J, Zuidema W, van Rooij R, Bockting CL. ChatGPT: five priorities for research. Nature. Feb 03, 2023;614(7947):224-226. [ CrossRef ] [ Medline ]
• Zhao R, Li X, Chia YK, Ding B, Bing L. Can ChatGPT-like generative models guarantee factual accuracy? On the mistakes of new generation search engines. arXiv. Preprint posted online March 3, 2023. 2023. [ FREE Full text ]
• McGee RW. Using ChatGPT to conduct literature searches: a case study. J Bus Ethics. 2023;95(2):165-178. [ CrossRef ]
• Sun W, Yan L, Ma X, Ren P, Yin D, Ren Z. Is ChatGPT good at search? Investigating large language models as re-ranking agents. arXiv. Preprint posted online April 19, 2023. 2023. [ FREE Full text ] [ CrossRef ]
• Hill-Yardin EL, Hutchinson MR, Laycock R, Spencer SJ. A Chat(GPT) about the future of scientific publishing. Brain Behav Immun. May 2023;110:152-154. [ CrossRef ] [ Medline ]
• Wagner MW, Ertl-Wagner BB. Accuracy of information and references using ChatGPT-3 for retrieval of clinical radiological information. Can Assoc Radiol J. Feb 20, 2024;75(1):69-73. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bozkurt A, Karadeniz A, Baneres D, Guerrero-Roldán AE, Rodríguez ME. Artificial intelligence and reflections from educational landscape: a review of ai studies in half a century. Sustainability. Jan 15, 2021;13(2):800. [ CrossRef ]
• Khlaif ZN. Ethical concerns about using AI-generated text in scientific research. SSRN Journal. Preprint posted online March 20, 2023. 2023. [ FREE Full text ] [ CrossRef ]

Abbreviations

Edited by L Buis; submitted 17.08.23; peer-reviewed by S Tran, L Zhu; comments to author 10.02.24; revised version received 11.02.24; accepted 27.03.24; published 06.05.24.

©Suebsarn Ruksakulpiwat, Lalipat Phianhasin, Chitchanok Benjasirisan, Kedong Ding, Anuoluwapo Ajibade, Ayanesh Kumar, Cassie Stewart. Originally published in JMIR mHealth and uHealth (https://mhealth.jmir.org), 06.05.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on https://mhealth.jmir.org/, as well as this copyright and license information must be included.

How to use ChatGPT

O penAI unleashed an  artificial intelligence  (AI) revolution when the company rolled out  ChatGPT  for public use in late 2022. Since then, ChatGPT, a chatbot powered by OpenAI's large language models (LLMs), has dominated headlines and preoccupied the minds of executives running  Twitter ,  Google ,  Amazon , Microsoft , and  Meta , inspiring them to create their own generative AI projects.

After launching in November 2022, ChatGPT had gained 100 million users by January 2023, becoming  the fastest-growing 'app' of all time , surpassing  TikTok -- until Threads dethroned it in July 2023 . 

Also: 5 ways AI can help you study for finals - for free

ChatGPT is so popular and high-performing that some have wondered whether the chatbot could replace   programmers ,  writers , and doctors . Others have considered how ChatGPT could  revolutionize different industries .

ZDNET's ChatGPT guides

ChatGPT users have come up with creative ideas for using the chatbot, from  asking questions in search of funny answers  to  correcting a bug in code . Across all these areas, one thing is abundantly clear: this AI tool is remarkable not due to any particular innovations  but rather because it's accessible and easy to use. 

Here are some of ZDNET's guides on different ways to use ChatGPT:

How to use ChatGPT for coding

• How to use ChatGPT to write code
• How ChatGPT can rewrite and improve your existing code  
• How to use ChatGPT to create an app
• How I used ChatGPT to write a custom JavaScript bookmarklet
• I used ChatGPT to write the same routine in 12 top programming languages. Here's how it did
• Okay, so ChatGPT just debugged my code. For real
• I'm using ChatGPT to help me fix code faster, but at what cost?
• I asked ChatGPT to write a WordPress plugin I needed. It did it in less than 5 minutes    

How to use ChatGPT for work

• How to use ChatGPT to make charts and tables
• How to use ChatGPT to write Excel formulas
• ChatGPT Plus can mine your corporate data for powerful insights. Here's how
• How to use ChatGPT to build your resume
• How to use ChatGPT to write a cover letter (and why you should)
• How I used ChatGPT and AI art tools to launch my Etsy business fast

How to use ChatGPT for school

• Need an AI essay writer? Here's how ChatGPT (and other chatbots) can help
• How to use ChatGPT to do research for papers, presentations, studies, and more
• How to make ChatGPT provide sources and citations
• How to save a ChatGPT conversation to revisit later
• How to use ChatGPT to summarize a book, article, or research paper
• How your kids can use ChatGPT safely, according to a mom

Useful ChatGPT tips

• Here's how to create your own custom chatbots using ChatGPT
• How to use ChatGPT Plus
• You can have voice chats with ChatGPT now. Here's how
• ChatGPT now lets you create and share links to your chatbot conversations
• How to write better ChatGPT prompts for the best generative AI results
• How (and why) to subscribe to ChatGPT Plus
• Can AI detectors save us from ChatGPT? I tried 5 online tools to find out
• How to use ChatGPT to plan a vacation
• 6 helpful ways to use ChatGPT's Custom Instructions
• Six skills you need to become an AI prompt engineer
• 7 advanced ChatGPT prompt-writing tips you need to know
• How to block OpenAI's new AI-training web crawler from ingesting your data
• How to use ChatGPT as a Siri shortcut on your iPhone or iPad

What are some good prompts for ChatGPT?

ChatGPT's responses to prompts are good enough that the technology can be an essential tool for content generation, from  writing essays  to  summarizing a book  for you.

The  better the prompt , the better the response you'll get. Here are examples of prompts you could start with:

• How does a computer store and process information?
• Analyze this code and tell me how to fix it: [Paste the code].
• Write a poem about migraines in Walt Whitman's style.
• What is the difference between a virus and a bacterium?
• Write a sick note for my child who has to miss school.
• Write a song/poem about [insert topic here] (try adding multiple details). 
• Give it a list of ingredients from your pantry and ask it to write a recipe with them.
• Ask it to summarize ideas or concepts.
• Request a packing list for a three-day trip to the beach.

Your imagination is the limit. Have fun with different ChatGPT prompts. For example, ZDNET's David Gewirtz asked the AI chatbot to  write a WordPress plugin  and used it to  help him fix code faster . He also requested ChatGPT to  write a Star Trek script  and  started a business  using the technology and other AI tools.

Also:  4 ways generative AI can stimulate the creator economy

Others have used the tool to  write malware . ChatGPT is an AI assistant programmed to reject inappropriate requests and doesn't generate unsafe content, so it may push back if you give it certain potentially unethical requests.

Can I use ChatGPT without a login?

OpenAI now allows users to access the free version of ChatGPT, powered by GPT-3.5, without logging in, though you must create an account to access your chat history. To access GPT-4, you need an account and a ChatGPT Plus subscription.

Also: ChatGPT vs. Microsoft Copilot vs. Gemini: Which is the best AI chatbot?

If you'd rather access GPT-4 for free and without logging in to a website, you can use Microsoft Copilot, formerly known as Bing Chat , which uses OpenAI's GPT-4, a larger and newer model than the one used in the ChatGPT free tier. You can log in with a Microsoft account for extended conversations.

What is ChatGPT Plus?

ChatGPT Plus is OpenAI's paid subscription to ChatGPT that offers access to GPT-4 with web browsing, data analysis, and image creation capabilities. A paid subscription also gives you priority access to new features and lets you use the AI chatbot during peak times.

Also: Why the future must be BYO AI: Model lock-in deters users and stifles innovation

ChatGPT Plus also allows users to access GPTs, which are customized GPT bots created on specialized data for specific subjects, like travel, cooking, and coding.

Is ChatGPT accurate?

ChatGPT and other AI assistants are prone to misinformation because they're trained on massive amounts of data that humans created. These tools can be biased if the data they use is flawed and can give inaccurate responses, especially regarding world events.

You should challenge any incorrect premises and always fact-check information from ChatGPT and other chatbots.

Can I use ChatGPT on my phone?

OpenAI offers mobile apps for iOS and Android. These apps offer a richer ChatGPT experience, letting you talk with ChatGPT through voice conversations powered by OpenAI's Whisper without requiring a ChatGPT Plus subscription.

Also:  These are my 4 favorite AI chatbot apps for Android

If you don't want to download an app, you can use the AI-based tool in your mobile browser. The steps to use OpenAI's ChatGPT from your mobile browser are the same as on a PC: go to chat.openai.com and start typing. The AI chatbot should work the same way as when you access it from your computer.

• ChatGPT vs. Bing Chat: Which AI chatbot should you use?
• How does ChatGPT work?
• How to get started using ChatGPT