using ai to write essays reddit

Special Features

Vendor voice.

This article is more than 1 year old

University students recruit AI to write essays for them. Now what?

Teachers need to work harder to get students to write and think for themselves.

Feature As word of students using AI to automatically complete essays continues to spread, some lecturers are beginning to rethink how they should teach their pupils to write.

Writing is a difficult task to do well. The best novelists and poets write furiously, dedicating their lives to mastering their craft. The creative process of stringing together words to communicate thoughts is often viewed as something complex, mysterious, and unmistakably human. No wonder people are fascinated by machines that can write too.

Unlike humans, language models don't procrastinate and create content instantly with a little guidance. All you need to do is type a short description, or prompt, instructing the model on what it needs to produce, and it'll generate a text output in seconds. So it should come as no surprise students are now beginning use these tools to complete school work.

Students are the perfect users: They need to write often, in large volumes, and are internet savvy. There are many AI-writing products to choose from that are easy to use and pretty cheap too. All of them lure new users with free trials, promising to make them better writers.

Monthly subscriptions for the most popular platform, Jasper, costs $40 per month to generate 35,000 words. Others, like Writesonic or Sudowrite, are cheaper at $10 per month for 30,000 words. Students who think they can use these products and get away with doing zero work, however, will probably be disappointed.

And then there's ChatGPT ...

Although AI can generate text with perfect spelling, great grammar and syntax, the content often isn't that good beyond a few paragraphs. The writing becomes less coherent over time with no logical train of thought to follow. Language models fail to get their facts right – meaning quotes, dates, and ideas are likely false. Students will have to inspect the writing closely and correct mistakes for their work to be convincing.

Prof: AI-assisted essays 'not good'

Scott Graham, associate professor at the Department of Rhetoric & Writing at the University of Texas at Austin, tasked his pupils with writing a 2,200-word essay about a campus-wide issue using AI. Students were free to lightly edit and format their work with the only rule being that most of the essay had to be automatically generated by software.

In an opinion article on Inside Higher Ed, Graham said the AI-assisted essays were "not good," noting that the best of the bunch would have earned a C or C-minus grade. To score higher, students would have had to rewrite more of the essay using their own words to improve it, or craft increasingly narrower and specific prompts to get back more useful content.

"You're not going to be able to push a button or submit a short prompt and generate a ready-to-go essay," he told The Register .

The limits of machine-written text forces humans to carefully read and edit copy. Some people may consider using these tools as cheating, but Graham believes they can help people get better at writing.

Don't waste all your effort on the first draft....

"I think if students can do well with AI writing, it's not actually all that different from them doing well with their own writing. The main skills I teach and assess mostly happen after the initial drafting," he said.

"I think that's where people become really talented writers; it's in the revision and the editing process. So I'm optimistic about [AI] because I think that it will provide a framework for us to be able to teach that revision and editing better.

"Some students have a lot of trouble sometimes generating that first draft. If all the effort goes into getting them to generate that first draft, and then they hit the deadline, that's what they will submit. They don't get a chance to revise, they don't get a chance to edit. If we can use those systems to speed write the first draft, it might really be helpful," he opined.

Whether students can use these tools to get away with doing less work will depend on the assignment. A biochemistry student claimed on Reddit they got an A when they used an AI model to write "five good and bad things about biotech" in an assignment, Vice reported .

AI is more likely to excel at producing simple, generic text across common templates or styles.

Listicles, informal blog posts, or news articles will be easier to imitate than niche academic papers or literary masterpieces. Teachers will need to be thoughtful about the essay questions they set and make sure students' knowledge are really being tested, if they don't want them to cut corners.

Ask a silly question, you'll get a silly answer

"I do think it's important for us to start thinking about the ways that [AI] is changing writing and how we respond to that in our assignments -- that includes some collaboration with AI," Annette Vee, associate professor of English and director of the Composition Program at the University of Pittsburgh, told us.

"The onus now is on writing teachers to figure out how to get to the same kinds of goals that we've always had about using writing to learn. That includes students engaging with ideas, teaching them how to formulate thoughts, how to communicate clearly or creatively. I think all of those things can be done with AI systems, but they'll be done differently."

The line between using AI as a collaborative tool or a way to cheat, however, is blurry. None of the academics teaching writing who spoke to The Register thought students should be banned from using AI software. "Writing is fundamentally shaped by technology," Vee said.

"Students use spell check and grammar check. If I got a paper where a student didn't use these, it stands out. But it used to be, 50 years ago, writing teachers would complain that students didn't know how to spell so they would teach spelling. Now they don't."

Most teachers, however, told us they would support regulating the use of AI-writing software in education. Anna Mills, who teaches students how to write at a community college in the Bay Area, is part of a small group of academics beginning to rally teachers and professional organizations like the Modern Language Association into thinking about introducing new academic rules.

Critical thinking skills

Mills said she could see why students might be tempted to use AI to write their essays, and simply asking teachers to come up with more compelling assessments is not a convincing solution.

Just $10 to create an AI chatbot of a dead loved one

"We need policies. These tools are already pretty good now, and they're only going to get better. We need clear guidance on what's acceptable use and what's not. Where is the line between using it to automatically generate email responses and something that violates academic integrity?" she asked The Register .

"Writing is just not outputs. Writing and revising is a process that develops our thinking. If you skip that, you're going to be skipping that practice which students need.

"It's too tempting to use it as a crutch, skip the thinking, and skip the frustrating moments of writing. Some of that is part of the process of going deeper and wrestling with ideas. There is a risk of learning loss if students become dependent and don't develop the writing skills they need."

Mills was particularly concerned about AI reducing the need for people to think for themselves, considering language models carry forward biases in their training data. "Companies have decided what to feed it and we don't know. Now, they are being used to generate all sorts of things from novels to academic papers, and they could influence our thoughts or even modify them. That is an immense power, and it's very dangerous."

Lauren Goodlad, professor of English and Comparative Literature at Rutgers University, agreed. If they parrot what AI comes up with, students may end up more likely to associate Muslims with terrorism or mention conspiracy theories, for example.

Computers are alredy interfering and changing the ways we write. Goodlad referred to one incident when Gmail suggested she change the word "importunate" to "impatient" in an email she wrote.

"It's hard to teach students how to use their own writing as a way to develop their critical thinking and as a way to express knowledge. They very badly need the practice of articulating their thoughts in writing and machines can rob them of this. If people really do end up using these things all the way through school, if that were to happen it could be a real loss not just for the writing quality but for the thinking quality of a whole generation," she said.

Rules and regulation

Academic policies tackling AI-assisted writing will be difficult to implement. Opinions are divided on whether sentences generated by machines count as plagiarism or not. There is also the problem of being able to detect writing produced by these tools accurately. Some teachers are alarmed at AI's growing technical capabilities, whilst others believe its overhyped. Some are embracing the technology more than others.

Marc Watkins, lecturer, and Stephen Monroe, chair and assistant professor of writing and rhetoric, are working on building an AI writing pilot programme with the University of Mississippi's Academic Innovations Group. "As teachers, we are experimenting, not panicking," Monroe told The Register .

"We want to empower our students as writers and thinkers. AI will play a role… This is a time of exciting and frenzied development, but educators move more slowly and deliberately… AI will be able to assist writers at every stage, but students and teachers will need tools that are thoughtfully calibrated."

• Human-replacing AI startups reach $1bn unicorn status
• GPT-3 'prompt injection' attack causes bad bot manners
• FauxPilot: It's like GitHub Copilot but doesn't phone home to Microsoft
• AI chatbot trained on posts from web sewer 4chan behaved badly – just like human members

Teachers are getting together and beginning to think about these tools, Watkins added. "Before we have any policy about the use of language models, we need to have sustained conversations with students, faculty, and administration about what this technology means for teaching and learning."

"But academia doesn't move at the pace of Big Tech. We're taking our time and slowly exploring. I don't think faculty need to be frightened. It's possible that these tools will have a positive impact on student learning and advancing equity, so let's approach AI assistants cautiously, but with an open mind."

Regardless of what policies universities may decide to implement in the future, AI presents academia with an opportunity to improve education now. Teachers will need to adapt to the technology if they want to remain relevant, and incentivise students to learn and think on their own with or without assistance from computers. ®

• Machine Learning

Narrower topics

• Deep Learning
• Large Language Model
• Neural Networks
• Tensor Processing Unit

Broader topics

• Self-driving Car

Send us news

Other stories you might like

Big brains divided over training ai with more ai: is model collapse inevitable, ai catholic 'priest' defrocked after recommending gatorade baptism, anthropic goes after iphone fans with claude 3 chatbot app, easing the cloud migration journey.

When AI helps you code, who owns the finished product?

Us senators' ai roadmap aims for $32b in r&d spending, with run:ai acquisition, nvidia aims to manage your ai kubes, google search results polluted by buggy ai-written code frustrate coders, big cloud is still making bank – is this ai adoption, price rises, or what, investment analyst accuses palantir of ai washing, google thinks ai can google better than you can, add ai servers to the list of idevices apple silicon could soon power.

• Advertise with us

Our Websites

• The Next Platform
• Blocks and Files

Your Privacy

• Cookies Policy
• Privacy Policy
• Ts & Cs

Copyright. All rights reserved © 1998–2024

Celebrating 150 years of Harvard Summer School. Learn about our history.

Should I Use ChatGPT to Write My Essays?

Everything high school and college students need to know about using — and not using — ChatGPT for writing essays.

Jessica A. Kent

ChatGPT is one of the most buzzworthy technologies today.

In addition to other generative artificial intelligence (AI) models, it is expected to change the world. In academia, students and professors are preparing for the ways that ChatGPT will shape education, and especially how it will impact a fundamental element of any course: the academic essay.

Students can use ChatGPT to generate full essays based on a few simple prompts. But can AI actually produce high quality work, or is the technology just not there yet to deliver on its promise? Students may also be asking themselves if they should use AI to write their essays for them and what they might be losing out on if they did.

AI is here to stay, and it can either be a help or a hindrance depending on how you use it. Read on to become better informed about what ChatGPT can and can’t do, how to use it responsibly to support your academic assignments, and the benefits of writing your own essays.

What is Generative AI?

Artificial intelligence isn’t a twenty-first century invention. Beginning in the 1950s, data scientists started programming computers to solve problems and understand spoken language. AI’s capabilities grew as computer speeds increased and today we use AI for data analysis, finding patterns, and providing insights on the data it collects.

But why the sudden popularity in recent applications like ChatGPT? This new generation of AI goes further than just data analysis. Instead, generative AI creates new content. It does this by analyzing large amounts of data — GPT-3 was trained on 45 terabytes of data, or a quarter of the Library of Congress — and then generating new content based on the patterns it sees in the original data.

It’s like the predictive text feature on your phone; as you start typing a new message, predictive text makes suggestions of what should come next based on data from past conversations. Similarly, ChatGPT creates new text based on past data. With the right prompts, ChatGPT can write marketing content, code, business forecasts, and even entire academic essays on any subject within seconds.

But is generative AI as revolutionary as people think it is, or is it lacking in real intelligence?

The Drawbacks of Generative AI

It seems simple. You’ve been assigned an essay to write for class. You go to ChatGPT and ask it to write a five-paragraph academic essay on the topic you’ve been assigned. You wait a few seconds and it generates the essay for you!

But ChatGPT is still in its early stages of development, and that essay is likely not as accurate or well-written as you’d expect it to be. Be aware of the drawbacks of having ChatGPT complete your assignments.

It’s not intelligence, it’s statistics

One of the misconceptions about AI is that it has a degree of human intelligence. However, its intelligence is actually statistical analysis, as it can only generate “original” content based on the patterns it sees in already existing data and work.

It “hallucinates”

Generative AI models often provide false information — so much so that there’s a term for it: “AI hallucination.” OpenAI even has a warning on its home screen , saying that “ChatGPT may produce inaccurate information about people, places, or facts.” This may be due to gaps in its data, or because it lacks the ability to verify what it’s generating. 

It doesn’t do research  

If you ask ChatGPT to find and cite sources for you, it will do so, but they could be inaccurate or even made up.

This is because AI doesn’t know how to look for relevant research that can be applied to your thesis. Instead, it generates content based on past content, so if a number of papers cite certain sources, it will generate new content that sounds like it’s a credible source — except it likely may not be.

There are data privacy concerns

When you input your data into a public generative AI model like ChatGPT, where does that data go and who has access to it? 

Prompting ChatGPT with original research should be a cause for concern — especially if you’re inputting study participants’ personal information into the third-party, public application. 

JPMorgan has restricted use of ChatGPT due to privacy concerns, Italy temporarily blocked ChatGPT in March 2023 after a data breach, and Security Intelligence advises that “if [a user’s] notes include sensitive data … it enters the chatbot library. The user no longer has control over the information.”

It is important to be aware of these issues and take steps to ensure that you’re using the technology responsibly and ethically. 

It skirts the plagiarism issue

AI creates content by drawing on a large library of information that’s already been created, but is it plagiarizing? Could there be instances where ChatGPT “borrows” from previous work and places it into your work without citing it? Schools and universities today are wrestling with this question of what’s plagiarism and what’s not when it comes to AI-generated work.

To demonstrate this, one Elon University professor gave his class an assignment: Ask ChatGPT to write an essay for you, and then grade it yourself. 

“Many students expressed shock and dismay upon learning the AI could fabricate bogus information,” he writes, adding that he expected some essays to contain errors, but all of them did. 

His students were disappointed that “major tech companies had pushed out AI technology without ensuring that the general population understands its drawbacks” and were concerned about how many embraced such a flawed tool.

Explore Our High School Programs

How to Use AI as a Tool to Support Your Work

As more students are discovering, generative AI models like ChatGPT just aren’t as advanced or intelligent as they may believe. While AI may be a poor option for writing your essay, it can be a great tool to support your work.

Generate ideas for essays

Have ChatGPT help you come up with ideas for essays. For example, input specific prompts, such as, “Please give me five ideas for essays I can write on topics related to WWII,” or “Please give me five ideas for essays I can write comparing characters in twentieth century novels.” Then, use what it provides as a starting point for your original research.

Generate outlines

You can also use ChatGPT to help you create an outline for an essay. Ask it, “Can you create an outline for a five paragraph essay based on the following topic” and it will create an outline with an introduction, body paragraphs, conclusion, and a suggested thesis statement. Then, you can expand upon the outline with your own research and original thought.

Generate titles for your essays

Titles should draw a reader into your essay, yet they’re often hard to get right. Have ChatGPT help you by prompting it with, “Can you suggest five titles that would be good for a college essay about [topic]?”

The Benefits of Writing Your Essays Yourself

Asking a robot to write your essays for you may seem like an easy way to get ahead in your studies or save some time on assignments. But, outsourcing your work to ChatGPT can negatively impact not just your grades, but your ability to communicate and think critically as well. It’s always the best approach to write your essays yourself.

Create your own ideas

Writing an essay yourself means that you’re developing your own thoughts, opinions, and questions about the subject matter, then testing, proving, and defending those thoughts. 

When you complete school and start your career, projects aren’t simply about getting a good grade or checking a box, but can instead affect the company you’re working for — or even impact society. Being able to think for yourself is necessary to create change and not just cross work off your to-do list.

Building a foundation of original thinking and ideas now will help you carve your unique career path in the future.

Develop your critical thinking and analysis skills

In order to test or examine your opinions or questions about a subject matter, you need to analyze a problem or text, and then use your critical thinking skills to determine the argument you want to make to support your thesis. Critical thinking and analysis skills aren’t just necessary in school — they’re skills you’ll apply throughout your career and your life.

Improve your research skills

Writing your own essays will train you in how to conduct research, including where to find sources, how to determine if they’re credible, and their relevance in supporting or refuting your argument. Knowing how to do research is another key skill required throughout a wide variety of professional fields.

Learn to be a great communicator

Writing an essay involves communicating an idea clearly to your audience, structuring an argument that a reader can follow, and making a conclusion that challenges them to think differently about a subject. Effective and clear communication is necessary in every industry.

Be impacted by what you’re learning about : 

Engaging with the topic, conducting your own research, and developing original arguments allows you to really learn about a subject you may not have encountered before. Maybe a simple essay assignment around a work of literature, historical time period, or scientific study will spark a passion that can lead you to a new major or career.

Resources to Improve Your Essay Writing Skills

While there are many rewards to writing your essays yourself, the act of writing an essay can still be challenging, and the process may come easier for some students than others. But essay writing is a skill that you can hone, and students at Harvard Summer School have access to a number of on-campus and online resources to assist them.

Students can start with the Harvard Summer School Writing Center , where writing tutors can offer you help and guidance on any writing assignment in one-on-one meetings. Tutors can help you strengthen your argument, clarify your ideas, improve the essay’s structure, and lead you through revisions. 

The Harvard libraries are a great place to conduct your research, and its librarians can help you define your essay topic, plan and execute a research strategy, and locate sources. 

Finally, review the “ The Harvard Guide to Using Sources ,” which can guide you on what to cite in your essay and how to do it. Be sure to review the “Tips For Avoiding Plagiarism” on the “ Resources to Support Academic Integrity ” webpage as well to help ensure your success.

Sign up to our mailing list to learn more about Harvard Summer School

The Future of AI in the Classroom

ChatGPT and other generative AI models are here to stay, so it’s worthwhile to learn how you can leverage the technology responsibly and wisely so that it can be a tool to support your academic pursuits. However, nothing can replace the experience and achievement gained from communicating your own ideas and research in your own academic essays.

About the Author

Jessica A. Kent is a freelance writer based in Boston, Mass. and a Harvard Extension School alum. Her digital marketing content has been featured on Fast Company, Forbes, Nasdaq, and other industry websites; her essays and short stories have been featured in North American Review, Emerson Review, Writer’s Bone, and others.

5 Key Qualities of Students Who Succeed at Harvard Summer School (and in College!)

This guide outlines the kinds of students who thrive at Harvard Summer School and what the programs offer in return.

Harvard Division of Continuing Education

The Division of Continuing Education (DCE) at Harvard University is dedicated to bringing rigorous academics and innovative teaching capabilities to those seeking to improve their lives through education. We make Harvard education accessible to lifelong learners from high school to retirement.

• Share full article

Advertisement

Supported by

current events conversation

What Students Are Saying About Learning to Write in the Age of A.I.

Does being able to write still matter when chatbots can do it for us? Teenagers weigh in on an essay from Opinion.

By The Learning Network

With artificial intelligence programs like ChatGPT that can generate prose for us, how much should we care about learning to write — and write well?

In “ Our Semicolons, Ourselves ,” the Opinion contributor Frank Bruni argues that, for a multitude of reasons, communicating effectively is a skill we should still take seriously. “Good writing burnishes your message,” he writes. “It burnishes the messenger, too.”

We asked teenagers what they thought: Does learning to be a good writer still matter in the age of A.I.? Or will the technology someday replace the need for people to learn how to put pen to paper and fingers to keyboard?

Take a look at their conversation below, which explores the benefits of learning to express oneself, the promise and perils of chatbots, and what it means to be a writer.

Thank you to everyone who participated in the conversation on our writing prompts this week, including students from Glenbard North High School in Carol Stream, Ill.; Hinsdale Central High School in Hinsdale, Ill. and New Rochelle High School in New Rochelle, N.Y .

Please note: Student comments have been lightly edited for length, but otherwise appear as they were originally submitted.

Many students agreed with Mr. Bruni that learning to write is important. Some pointed to the practical reasons.

When you write any sort of persuasive essay or analysis essay, you learn to communicate your ideas to your audience. This skill can then be applied to your daily life. Whether it’s talking to your teachers, writing an email to your boss, or sending a text message to your friends, writing and communication is a fundamental ability that is needed to clearly and concisely express yourself. This is something that A.I. cannot help you with.

— Mara F.R., Hinsdale

In order to write, we must first be able to think on our own which allows us to be self-sufficient. With the frequent use of A.I., our minds become reliant on given information rather than us thinking for ourselves. I absolutely believe that learning to be a good writer still matters even in the age of Artificial Intelligence.

— Jordyne, Ellisville

I firmly believe that learning good writing skills develops communication, creativity, and problem-solving skills. A.I. can also be used as a tool; I have used it to ask practice questions, compare my answers, and find different/better ways to express myself. Sure, having my essay written for me in seconds is great, but come time for an interview or presentation later on in my life I’ll lack the confidence and ability to articulate my thoughts if I never learn how.

— CC, San Luis Obispo County

I, being a senior, have just finished my college applications. Throughout the process, I visited several essay help websites, and each one stressed this fact: essay readers want to hear a student’s voice. ChatGPT can write well-structured essays in two minutes, but these essays have no voice. They are formulaic and insipid — they won’t help a student get into UCLA. To have a chance, her essays must be eloquent and compelling. So, at least until AI writing technology improves, a student must put in the work, writing and rewriting until she has produced an essay that tells readers who she is.

— Cole, Central Coast, CA

Others discussed the joy and satisfaction that comes with being able to express oneself.

While AI has its advantages, it can’t replicate the satisfaction and authenticity which comes from writing by yourself. AI uses the existing ideas of others in order to generate a response. However, the response isn’t unique and doesn’t truly represent the idea the way you would. When you write, it causes you to think deeply about a topic and come up with an original idea. You uncover ideas which you wouldn’t have thought of previously and understand a topic for more than its face value. It creates a sense of clarity, in which you can generate your own viewpoint after looking at the different perspectives. Another example is that the feeling of writing something by yourself generates feelings of pleasure and satisfaction. The process of doing research about a topic for hours, to then come up with your own opinion. Or the feeling of having to use a dictionary to understand a word which you don’t know the meaning of. The satisfaction and authenticity or writing by yourself is irreplaceable. Therefore, it is still important to learn to be a good writer.

— Aditya, Hinsdale

You cannot depend on technology to do everything for you. An important factor of writing is expressing yourself and showing creativity. While AI can create a grammatically correct essay, it cannot express how you feel on the subject. Creativity attracts an audience, not being grammatically correct. Learning to write well-written essays without the assistance of AI is a skill that everyone should have.

— Aidan, Ellisville

A few commenters raised ethical concerns around using generators like ChatGPT.

I feel that even with AI, learning how to be a good writer still matters. For example, if you’re writing a college essay or an essay for a class using an AI generated thing, that is plagiarism, which can get you in a lot of trouble because it is against the law to take something that is not yours and try to make it seem like it is your writing. So I believe that learning how to be a good writer still matters a lot because if you want to get into a good college or get good grades, you need to know how to write at least semi-well and make sure the writing is in your own words, not words already generated for you.

— jeo, new york

There are obvious benefits, and I myself have used this software to better understand Calculus problems in a step by step format, or to answer my questions regarding a piece of literature, or time in history. That being said, ethics should be considered, and credit should be given where credit is due; as sources are cited in a traditional paper, so should the use of ChatGPT.

— Ariel, Miami Country Day School

Writing is still an important skill, but maybe not in the same way it has in the past. In an era of improving AI, topics such as grammar and spelling are less important than ever. Google already corrects small grammar mistakes; how long till they can suggest completely restructuring sentences? However, being a good writer is more than just grammar and vocabulary. It’s about collecting your thoughts into a cohesive and thoughtful presentation … If you want to communicate your own ideas, not just a conglomerate of ones on the internet, you’re better off just writing it yourself. That’s not to mention the plethora of issues like AI just making stuff up from time to time. So for now at least, improving your writing is still the best way to share your thoughts.

— Liam, Glenbard West High School

Several students shared how they use A.I. as a resource to aid, rather than replace, their own effort.

I think AI should be a tool for writers. It can help make outlines for writing pieces and it could help solve problems students are stuck on and give them an explanation. However, I think the line should be drawn if students use AI to do the whole entire assignment for them. That’s when it should be considered cheating and not be used.

— Sam, Hinsdale, IL

Sometimes I use A.I. programs such as ChatGPT to help with typing and communication. The results vary, but overall I find it helpful in generating creative ideas, cleaning up language, and speeding up the writing. However, I believe it is important to be careful and filter the results to ensure accuracy and precision. AI tools are valuable aids, but human input and insight are still needed to achieve the desired quality of written communication.

— Zach, New Rochelle High School

As of now, A.I. is not capable of replacing human prose effectively. Just look at the data, the only A.P. tests that ChatGPT did not pass were the ones for English Language and English Literature. This data lays bare a fact that most students refuse to accept: ChatGPT is not able to write a quality essay yet. Now that many schools are loosening restrictions regarding the use of generative A.I., students have two options: either they get back to work or they get a bad grade for their A.I.-generated essay.

On the other hand, there is another alternative that is likely to be the best one yet. A good friend once said, “A.I. software like ChatGPT solves the issue of having a clean sheet of paper”. By nature, humans are terrible at getting anything started. This is the issue that ChatGPT solves. As Bruni asserts, “Writing is thinking, but it’s thinking slowed down — stilled — to a point where dimensions and nuances otherwise invisible to you appear.” This is true, but ChatGPT can help students by creating a rough draft of what those ideas might look like on paper. The endpoint is this: while students are likely to keep needing to become good writers to excel at school, A.I. technology such as ChatGPT and Grammarly will become additional tools that will help students reach even higher levels of literary excellence.

— Francisco, Miami Country Day School

But some thought we might not be far from a future where A.I. can write for us.

I think that AI will eventually replace the need for the average person to write at the level that they do. AI is no different than every other tech advancement we’ve made, which have made tasks like writing easier. Similar concerns could have been raised with the introduction of computers in the classroom, and the loss of people having great handwriting. I don’t think the prospect should be worrying. AI is a tool. Having it write for us will allow us to focus on more important things that AI is not yet capable of.

— zack, Hinsdale Central

AI is becoming wildly accessible and increasingly more competent. The growth of this sector could mean more students find their way to an AI site to look for an answer. I agree that this could spell trouble for student intelligence if passable answers are so readily available. But you might want to consider the students themselves. The majority are hardworking and smart, not just smart about subjects in school, but about how using only AI for their work could end badly. Students will probably not use the newborn tech first hand until it is basically errorless, and that will take some time.

— Beau, Glen Ellyn, IL

Even so, there were students who doubted that technology could ever replace “what it means to be a writer.”

I don’t think AI will fully be able to replace humans, no matter how much time we as a society take to implement it into everyday life, as they are still just a bunch of numbers and code, and the complexity of a human and the intricacies of our emotions, our thoughts, and feelings, along with what makes each of us an individual, someone that matters, proves that humans will never be able to be fully replicated by AI, and that the most emotion-centric jobs, such as writing, and most fields in art, will forever be, or should forever be, dominated by the experiences and emotional complexity of humans.

— Liam, Hinsdale

AI uses data from the internet it gathers and then puts together a paragraph or two, while it may be able to do this faster than any human, it does not have any authenticity. If it is pulling its information from the web where someone has said something similar, the data found may be biased and the AI would not care. Yet some people still insist it’s the future for writing when in reality, AI will probably not come up with an original idea and only use possibly biased data to give to someone so they can just copy it and move on and undermine what it means to be a writer.

— John, Glenbard North HS

I have never personally used ChatGPT as I believe no robot can recreate the creativity or authenticity humans achieve in writing … Even with growing advances in technology, AI can only create with the information it already knows, which takes away the greatest quality writers have: creativity.

— Stella, Glenbard West

In my opinion, learning to be a good writer absolutely still matters in the age of AI. While artificial intelligence can assist with certain aspects of writing, such as grammar and syntax checking, it cannot replace the creativity, critical thinking, and emotional intelligence that we human writers bring to the table. Another reason is that storytelling, persuasion, and the art of crafting a compelling narrative are skills deeply rooted in human intuition and empathy. A good writer can connect with readers on a personal level, inspiring thoughts, feelings, and actions. AI may enhance efficiency, but it cannot replicate the authentic voice and unique perspective that a human writer brings to their work.

— McKenzie, Warrington, PA

Learn more about Current Events Conversation here and find all of our posts in this column .

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 30 October 2023

A large-scale comparison of human-written versus ChatGPT-generated essays

• Steffen Herbold 1 ,
• Annette Hautli-Janisz 1 ,
• Ute Heuer 1 ,
• Zlata Kikteva 1 &
• Alexander Trautsch 1  

Scientific Reports volume  13 , Article number:  18617 ( 2023 ) Cite this article

19k Accesses

15 Citations

94 Altmetric

Metrics details

• Computer science
• Information technology

ChatGPT and similar generative AI models have attracted hundreds of millions of users and have become part of the public discourse. Many believe that such models will disrupt society and lead to significant changes in the education system and information generation. So far, this belief is based on either colloquial evidence or benchmarks from the owners of the models—both lack scientific rigor. We systematically assess the quality of AI-generated content through a large-scale study comparing human-written versus ChatGPT-generated argumentative student essays. We use essays that were rated by a large number of human experts (teachers). We augment the analysis by considering a set of linguistic characteristics of the generated essays. Our results demonstrate that ChatGPT generates essays that are rated higher regarding quality than human-written essays. The writing style of the AI models exhibits linguistic characteristics that are different from those of the human-written essays. Since the technology is readily available, we believe that educators must act immediately. We must re-invent homework and develop teaching concepts that utilize these AI models in the same way as math utilizes the calculator: teach the general concepts first and then use AI tools to free up time for other learning objectives.

Similar content being viewed by others

ChatGPT-3.5 as writing assistance in students’ essays

Perception, performance, and detectability of conversational artificial intelligence across 32 university courses

The model student: GPT-4 performance on graduate biomedical science exams

Introduction.

The massive uptake in the development and deployment of large-scale Natural Language Generation (NLG) systems in recent months has yielded an almost unprecedented worldwide discussion of the future of society. The ChatGPT service which serves as Web front-end to GPT-3.5 1 and GPT-4 was the fastest-growing service in history to break the 100 million user milestone in January and had 1 billion visits by February 2023 2 .

Driven by the upheaval that is particularly anticipated for education 3 and knowledge transfer for future generations, we conduct the first independent, systematic study of AI-generated language content that is typically dealt with in high-school education: argumentative essays, i.e. essays in which students discuss a position on a controversial topic by collecting and reflecting on evidence (e.g. ‘Should students be taught to cooperate or compete?’). Learning to write such essays is a crucial aspect of education, as students learn to systematically assess and reflect on a problem from different perspectives. Understanding the capability of generative AI to perform this task increases our understanding of the skills of the models, as well as of the challenges educators face when it comes to teaching this crucial skill. While there is a multitude of individual examples and anecdotal evidence for the quality of AI-generated content in this genre (e.g. 4 ) this paper is the first to systematically assess the quality of human-written and AI-generated argumentative texts across different versions of ChatGPT 5 . We use a fine-grained essay quality scoring rubric based on content and language mastery and employ a significant pool of domain experts, i.e. high school teachers across disciplines, to perform the evaluation. Using computational linguistic methods and rigorous statistical analysis, we arrive at several key findings:

AI models generate significantly higher-quality argumentative essays than the users of an essay-writing online forum frequented by German high-school students across all criteria in our scoring rubric.

ChatGPT-4 (ChatGPT web interface with the GPT-4 model) significantly outperforms ChatGPT-3 (ChatGPT web interface with the GPT-3.5 default model) with respect to logical structure, language complexity, vocabulary richness and text linking.

Writing styles between humans and generative AI models differ significantly: for instance, the GPT models use more nominalizations and have higher sentence complexity (signaling more complex, ‘scientific’, language), whereas the students make more use of modal and epistemic constructions (which tend to convey speaker attitude).

The linguistic diversity of the NLG models seems to be improving over time: while ChatGPT-3 still has a significantly lower linguistic diversity than humans, ChatGPT-4 has a significantly higher diversity than the students.

Our work goes significantly beyond existing benchmarks. While OpenAI’s technical report on GPT-4 6 presents some benchmarks, their evaluation lacks scientific rigor: it fails to provide vital information like the agreement between raters, does not report on details regarding the criteria for assessment or to what extent and how a statistical analysis was conducted for a larger sample of essays. In contrast, our benchmark provides the first (statistically) rigorous and systematic study of essay quality, paired with a computational linguistic analysis of the language employed by humans and two different versions of ChatGPT, offering a glance at how these NLG models develop over time. While our work is focused on argumentative essays in education, the genre is also relevant beyond education. In general, studying argumentative essays is one important aspect to understand how good generative AI models are at conveying arguments and, consequently, persuasive writing in general.

Related work

Natural language generation.

The recent interest in generative AI models can be largely attributed to the public release of ChatGPT, a public interface in the form of an interactive chat based on the InstructGPT 1 model, more commonly referred to as GPT-3.5. In comparison to the original GPT-3 7 and other similar generative large language models based on the transformer architecture like GPT-J 8 , this model was not trained in a purely self-supervised manner (e.g. through masked language modeling). Instead, a pipeline that involved human-written content was used to fine-tune the model and improve the quality of the outputs to both mitigate biases and safety issues, as well as make the generated text more similar to text written by humans. Such models are referred to as Fine-tuned LAnguage Nets (FLANs). For details on their training, we refer to the literature 9 . Notably, this process was recently reproduced with publicly available models such as Alpaca 10 and Dolly (i.e. the complete models can be downloaded and not just accessed through an API). However, we can only assume that a similar process was used for the training of GPT-4 since the paper by OpenAI does not include any details on model training.

Testing of the language competency of large-scale NLG systems has only recently started. Cai et al. 11 show that ChatGPT reuses sentence structure, accesses the intended meaning of an ambiguous word, and identifies the thematic structure of a verb and its arguments, replicating human language use. Mahowald 12 compares ChatGPT’s acceptability judgments to human judgments on the Article + Adjective + Numeral + Noun construction in English. Dentella et al. 13 show that ChatGPT-3 fails to understand low-frequent grammatical constructions like complex nested hierarchies and self-embeddings. In another recent line of research, the structure of automatically generated language is evaluated. Guo et al. 14 show that in question-answer scenarios, ChatGPT-3 uses different linguistic devices than humans. Zhao et al. 15 show that ChatGPT generates longer and more diverse responses when the user is in an apparently negative emotional state.

Given that we aim to identify certain linguistic characteristics of human-written versus AI-generated content, we also draw on related work in the field of linguistic fingerprinting, which assumes that each human has a unique way of using language to express themselves, i.e. the linguistic means that are employed to communicate thoughts, opinions and ideas differ between humans. That these properties can be identified with computational linguistic means has been showcased across different tasks: the computation of a linguistic fingerprint allows to distinguish authors of literary works 16 , the identification of speaker profiles in large public debates 17 , 18 , 19 , 20 and the provision of data for forensic voice comparison in broadcast debates 21 , 22 . For educational purposes, linguistic features are used to measure essay readability 23 , essay cohesion 24 and language performance scores for essay grading 25 . Integrating linguistic fingerprints also yields performance advantages for classification tasks, for instance in predicting user opinion 26 , 27 and identifying individual users 28 .

Limitations of OpenAIs ChatGPT evaluations

OpenAI published a discussion of the model’s performance of several tasks, including Advanced Placement (AP) classes within the US educational system 6 . The subjects used in performance evaluation are diverse and include arts, history, English literature, calculus, statistics, physics, chemistry, economics, and US politics. While the models achieved good or very good marks in most subjects, they did not perform well in English literature. GPT-3.5 also experienced problems with chemistry, macroeconomics, physics, and statistics. While the overall results are impressive, there are several significant issues: firstly, the conflict of interest of the model’s owners poses a problem for the performance interpretation. Secondly, there are issues with the soundness of the assessment beyond the conflict of interest, which make the generalizability of the results hard to assess with respect to the models’ capability to write essays. Notably, the AP exams combine multiple-choice questions with free-text answers. Only the aggregated scores are publicly available. To the best of our knowledge, neither the generated free-text answers, their overall assessment, nor their assessment given specific criteria from the used judgment rubric are published. Thirdly, while the paper states that 1–2 qualified third-party contractors participated in the rating of the free-text answers, it is unclear how often multiple ratings were generated for the same answer and what was the agreement between them. This lack of information hinders a scientifically sound judgement regarding the capabilities of these models in general, but also specifically for essays. Lastly, the owners of the model conducted their study in a few-shot prompt setting, where they gave the models a very structured template as well as an example of a human-written high-quality essay to guide the generation of the answers. This further fine-tuning of what the models generate could have also influenced the output. The results published by the owners go beyond the AP courses which are directly comparable to our work and also consider other student assessments like Graduate Record Examinations (GREs). However, these evaluations suffer from the same problems with the scientific rigor as the AP classes.

Scientific assessment of ChatGPT

Researchers across the globe are currently assessing the individual capabilities of these models with greater scientific rigor. We note that due to the recency and speed of these developments, the hereafter discussed literature has mostly only been published as pre-prints and has not yet been peer-reviewed. In addition to the above issues concretely related to the assessment of the capabilities to generate student essays, it is also worth noting that there are likely large problems with the trustworthiness of evaluations, because of data contamination, i.e. because the benchmark tasks are part of the training of the model, which enables memorization. For example, Aiyappa et al. 29 find evidence that this is likely the case for benchmark results regarding NLP tasks. This complicates the effort by researchers to assess the capabilities of the models beyond memorization.

Nevertheless, the first assessment results are already available – though mostly focused on ChatGPT-3 and not yet ChatGPT-4. Closest to our work is a study by Yeadon et al. 30 , who also investigate ChatGPT-3 performance when writing essays. They grade essays generated by ChatGPT-3 for five physics questions based on criteria that cover academic content, appreciation of the underlying physics, grasp of subject material, addressing the topic, and writing style. For each question, ten essays were generated and rated independently by five researchers. While the sample size precludes a statistical assessment, the results demonstrate that the AI model is capable of writing high-quality physics essays, but that the quality varies in a manner similar to human-written essays.

Guo et al. 14 create a set of free-text question answering tasks based on data they collected from the internet, e.g. question answering from Reddit. The authors then sample thirty triplets of a question, a human answer, and a ChatGPT-3 generated answer and ask human raters to assess if they can detect which was written by a human, and which was written by an AI. While this approach does not directly assess the quality of the output, it serves as a Turing test 31 designed to evaluate whether humans can distinguish between human- and AI-produced output. The results indicate that humans are in fact able to distinguish between the outputs when presented with a pair of answers. Humans familiar with ChatGPT are also able to identify over 80% of AI-generated answers without seeing a human answer in comparison. However, humans who are not yet familiar with ChatGPT-3 are not capable of identifying AI-written answers about 50% of the time. Moreover, the authors also find that the AI-generated outputs are deemed to be more helpful than the human answers in slightly more than half of the cases. This suggests that the strong results from OpenAI’s own benchmarks regarding the capabilities to generate free-text answers generalize beyond the benchmarks.

There are, however, some indicators that the benchmarks may be overly optimistic in their assessment of the model’s capabilities. For example, Kortemeyer 32 conducts a case study to assess how well ChatGPT-3 would perform in a physics class, simulating the tasks that students need to complete as part of the course: answer multiple-choice questions, do homework assignments, ask questions during a lesson, complete programming exercises, and write exams with free-text questions. Notably, ChatGPT-3 was allowed to interact with the instructor for many of the tasks, allowing for multiple attempts as well as feedback on preliminary solutions. The experiment shows that ChatGPT-3’s performance is in many aspects similar to that of the beginning learners and that the model makes similar mistakes, such as omitting units or simply plugging in results from equations. Overall, the AI would have passed the course with a low score of 1.5 out of 4.0. Similarly, Kung et al. 33 study the performance of ChatGPT-3 in the United States Medical Licensing Exam (USMLE) and find that the model performs at or near the passing threshold. Their assessment is a bit more optimistic than Kortemeyer’s as they state that this level of performance, comprehensible reasoning and valid clinical insights suggest that models such as ChatGPT may potentially assist human learning in clinical decision making.

Frieder et al. 34 evaluate the capabilities of ChatGPT-3 in solving graduate-level mathematical tasks. They find that while ChatGPT-3 seems to have some mathematical understanding, its level is well below that of an average student and in most cases is not sufficient to pass exams. Yuan et al. 35 consider the arithmetic abilities of language models, including ChatGPT-3 and ChatGPT-4. They find that they exhibit the best performance among other currently available language models (incl. Llama 36 , FLAN-T5 37 , and Bloom 38 ). However, the accuracy of basic arithmetic tasks is still only at 83% when considering correctness to the degree of \(10^{-3}\) , i.e. such models are still not capable of functioning reliably as calculators. In a slightly satiric, yet insightful take, Spencer et al. 39 assess how a scientific paper on gamma-ray astrophysics would look like, if it were written largely with the assistance of ChatGPT-3. They find that while the language capabilities are good and the model is capable of generating equations, the arguments are often flawed and the references to scientific literature are full of hallucinations.

The general reasoning skills of the models may also not be at the level expected from the benchmarks. For example, Cherian et al. 40 evaluate how well ChatGPT-3 performs on eleven puzzles that second graders should be able to solve and find that ChatGPT is only able to solve them on average in 36.4% of attempts, whereas the second graders achieve a mean of 60.4%. However, their sample size is very small and the problem was posed as a multiple-choice question answering problem, which cannot be directly compared to the NLG we consider.

Research gap

Within this article, we address an important part of the current research gap regarding the capabilities of ChatGPT (and similar technologies), guided by the following research questions:

RQ1: How good is ChatGPT based on GPT-3 and GPT-4 at writing argumentative student essays?

RQ2: How do AI-generated essays compare to essays written by students?

RQ3: What are linguistic devices that are characteristic of student versus AI-generated content?

We study these aspects with the help of a large group of teaching professionals who systematically assess a large corpus of student essays. To the best of our knowledge, this is the first large-scale, independent scientific assessment of ChatGPT (or similar models) of this kind. Answering these questions is crucial to understanding the impact of ChatGPT on the future of education.

Materials and methods

The essay topics originate from a corpus of argumentative essays in the field of argument mining 41 . Argumentative essays require students to think critically about a topic and use evidence to establish a position on the topic in a concise manner. The corpus features essays for 90 topics from Essay Forum 42 , an active community for providing writing feedback on different kinds of text and is frequented by high-school students to get feedback from native speakers on their essay-writing capabilities. Information about the age of the writers is not available, but the topics indicate that the essays were written in grades 11–13, indicating that the authors were likely at least 16. Topics range from ‘Should students be taught to cooperate or to compete?’ to ‘Will newspapers become a thing of the past?’. In the corpus, each topic features one human-written essay uploaded and discussed in the forum. The students who wrote the essays are not native speakers. The average length of these essays is 19 sentences with 388 tokens (an average of 2.089 characters) and will be termed ‘student essays’ in the remainder of the paper.

For the present study, we use the topics from Stab and Gurevych 41 and prompt ChatGPT with ‘Write an essay with about 200 words on “[ topic ]”’ to receive automatically-generated essays from the ChatGPT-3 and ChatGPT-4 versions from 22 March 2023 (‘ChatGPT-3 essays’, ‘ChatGPT-4 essays’). No additional prompts for getting the responses were used, i.e. the data was created with a basic prompt in a zero-shot scenario. This is in contrast to the benchmarks by OpenAI, who used an engineered prompt in a few-shot scenario to guide the generation of essays. We note that we decided to ask for 200 words because we noticed a tendency to generate essays that are longer than the desired length by ChatGPT. A prompt asking for 300 words typically yielded essays with more than 400 words. Thus, using the shorter length of 200, we prevent a potential advantage for ChatGPT through longer essays, and instead err on the side of brevity. Similar to the evaluations of free-text answers by OpenAI, we did not consider multiple configurations of the model due to the effort required to obtain human judgments. For the same reason, our data is restricted to ChatGPT and does not include other models available at that time, e.g. Alpaca. We use the browser versions of the tools because we consider this to be a more realistic scenario than using the API. Table 1 below shows the core statistics of the resulting dataset. Supplemental material S1 shows examples for essays from the data set.

Annotation study

Study participants.

The participants had registered for a two-hour online training entitled ‘ChatGPT – Challenges and Opportunities’ conducted by the authors of this paper as a means to provide teachers with some of the technological background of NLG systems in general and ChatGPT in particular. Only teachers permanently employed at secondary schools were allowed to register for this training. Focusing on these experts alone allows us to receive meaningful results as those participants have a wide range of experience in assessing students’ writing. A total of 139 teachers registered for the training, 129 of them teach at grammar schools, and only 10 teachers hold a position at other secondary schools. About half of the registered teachers (68 teachers) have been in service for many years and have successfully applied for promotion. For data protection reasons, we do not know the subject combinations of the registered teachers. We only know that a variety of subjects are represented, including languages (English, French and German), religion/ethics, and science. Supplemental material S5 provides some general information regarding German teacher qualifications.

The training began with an online lecture followed by a discussion phase. Teachers were given an overview of language models and basic information on how ChatGPT was developed. After about 45 minutes, the teachers received a both written and oral explanation of the questionnaire at the core of our study (see Supplementary material S3 ) and were informed that they had 30 minutes to finish the study tasks. The explanation included information on how the data was obtained, why we collect the self-assessment, and how we chose the criteria for the rating of the essays, the overall goal of our research, and a walk-through of the questionnaire. Participation in the questionnaire was voluntary and did not affect the awarding of a training certificate. We further informed participants that all data was collected anonymously and that we would have no way of identifying who participated in the questionnaire. We orally informed participants that they consent to the use of the provided ratings for our research by participating in the survey.

Once these instructions were provided orally and in writing, the link to the online form was given to the participants. The online form was running on a local server that did not log any information that could identify the participants (e.g. IP address) to ensure anonymity. As per instructions, consent for participation was given by using the online form. Due to the full anonymity, we could by definition not document who exactly provided the consent. This was implemented as further insurance that non-participation could not possibly affect being awarded the training certificate.

About 20% of the training participants did not take part in the questionnaire study, the remaining participants consented based on the information provided and participated in the rating of essays. After the questionnaire, we continued with an online lecture on the opportunities of using ChatGPT for teaching as well as AI beyond chatbots. The study protocol was reviewed and approved by the Research Ethics Committee of the University of Passau. We further confirm that our study protocol is in accordance with all relevant guidelines.

Questionnaire

The questionnaire consists of three parts: first, a brief self-assessment regarding the English skills of the participants which is based on the Common European Framework of Reference for Languages (CEFR) 43 . We have six levels ranging from ‘comparable to a native speaker’ to ‘some basic skills’ (see supplementary material S3 ). Then each participant was shown six essays. The participants were only shown the generated text and were not provided with information on whether the text was human-written or AI-generated.

The questionnaire covers the seven categories relevant for essay assessment shown below (for details see supplementary material S3 ):

Topic and completeness

Logic and composition

Expressiveness and comprehensiveness

Language mastery

Vocabulary and text linking

Language constructs

These categories are used as guidelines for essay assessment 44 established by the Ministry for Education of Lower Saxony, Germany. For each criterion, a seven-point Likert scale with scores from zero to six is defined, where zero is the worst score (e.g. no relation to the topic) and six is the best score (e.g. addressed the topic to a special degree). The questionnaire included a written description as guidance for the scoring.

After rating each essay, the participants were also asked to self-assess their confidence in the ratings. We used a five-point Likert scale based on the criteria for the self-assessment of peer-review scores from the Association for Computational Linguistics (ACL). Once a participant finished rating the six essays, they were shown a summary of their ratings, as well as the individual ratings for each of their essays and the information on how the essay was generated.

Computational linguistic analysis

In order to further explore and compare the quality of the essays written by students and ChatGPT, we consider the six following linguistic characteristics: lexical diversity, sentence complexity, nominalization, presence of modals, epistemic and discourse markers. Those are motivated by previous work: Weiss et al. 25 observe the correlation between measures of lexical, syntactic and discourse complexities to the essay gradings of German high-school examinations while McNamara et al. 45 explore cohesion (indicated, among other things, by connectives), syntactic complexity and lexical diversity in relation to the essay scoring.

Lexical diversity

We identify vocabulary richness by using a well-established measure of textual, lexical diversity (MTLD) 46 which is often used in the field of automated essay grading 25 , 45 , 47 . It takes into account the number of unique words but unlike the best-known measure of lexical diversity, the type-token ratio (TTR), it is not as sensitive to the difference in the length of the texts. In fact, Koizumi and In’nami 48 find it to be least affected by the differences in the length of the texts compared to some other measures of lexical diversity. This is relevant to us due to the difference in average length between the human-written and ChatGPT-generated essays.

Syntactic complexity

We use two measures in order to evaluate the syntactic complexity of the essays. One is based on the maximum depth of the sentence dependency tree which is produced using the spaCy 3.4.2 dependency parser 49 (‘Syntactic complexity (depth)’). For the second measure, we adopt an approach similar in nature to the one by Weiss et al. 25 who use clause structure to evaluate syntactic complexity. In our case, we count the number of conjuncts, clausal modifiers of nouns, adverbial clause modifiers, clausal complements, clausal subjects, and parataxes (‘Syntactic complexity (clauses)’). The supplementary material in S2 shows the difference between sentence complexity based on two examples from the data.

Nominalization is a common feature of a more scientific style of writing 50 and is used as an additional measure for syntactic complexity. In order to explore this feature, we count occurrences of nouns with suffixes such as ‘-ion’, ‘-ment’, ‘-ance’ and a few others which are known to transform verbs into nouns.

Semantic properties

Both modals and epistemic markers signal the commitment of the writer to their statement. We identify modals using the POS-tagging module provided by spaCy as well as a list of epistemic expressions of modality, such as ‘definitely’ and ‘potentially’, also used in other approaches to identifying semantic properties 51 . For epistemic markers we adopt an empirically-driven approach and utilize the epistemic markers identified in a corpus of dialogical argumentation by Hautli-Janisz et al. 52 . We consider expressions such as ‘I think’, ‘it is believed’ and ‘in my opinion’ to be epistemic.

Discourse properties

Discourse markers can be used to measure the coherence quality of a text. This has been explored by Somasundaran et al. 53 who use discourse markers to evaluate the story-telling aspect of student writing while Nadeem et al. 54 incorporated them in their deep learning-based approach to automated essay scoring. In the present paper, we employ the PDTB list of discourse markers 55 which we adjust to exclude words that are often used for purposes other than indicating discourse relations, such as ‘like’, ‘for’, ‘in’ etc.

Statistical methods

We use a within-subjects design for our study. Each participant was shown six randomly selected essays. Results were submitted to the survey system after each essay was completed, in case participants ran out of time and did not finish scoring all six essays. Cronbach’s \(\alpha\) 56 allows us to determine the inter-rater reliability for the rating criterion and data source (human, ChatGPT-3, ChatGPT-4) in order to understand the reliability of our data not only overall, but also for each data source and rating criterion. We use two-sided Wilcoxon-rank-sum tests 57 to confirm the significance of the differences between the data sources for each criterion. We use the same tests to determine the significance of the linguistic characteristics. This results in three comparisons (human vs. ChatGPT-3, human vs. ChatGPT-4, ChatGPT-3 vs. ChatGPT-4) for each of the seven rating criteria and each of the seven linguistic characteristics, i.e. 42 tests. We use the Holm-Bonferroni method 58 for the correction for multiple tests to achieve a family-wise error rate of 0.05. We report the effect size using Cohen’s d 59 . While our data is not perfectly normal, it also does not have severe outliers, so we prefer the clear interpretation of Cohen’s d over the slightly more appropriate, but less accessible non-parametric effect size measures. We report point plots with estimates of the mean scores for each data source and criterion, incl. the 95% confidence interval of these mean values. The confidence intervals are estimated in a non-parametric manner based on bootstrap sampling. We further visualize the distribution for each criterion using violin plots to provide a visual indicator of the spread of the data (see Supplementary material S4 ).

Further, we use the self-assessment of the English skills and confidence in the essay ratings as confounding variables. Through this, we determine if ratings are affected by the language skills or confidence, instead of the actual quality of the essays. We control for the impact of these by measuring Pearson’s correlation coefficient r 60 between the self-assessments and the ratings. We also determine whether the linguistic features are correlated with the ratings as expected. The sentence complexity (both tree depth and dependency clauses), as well as the nominalization, are indicators of the complexity of the language. Similarly, the use of discourse markers should signal a proper logical structure. Finally, a large lexical diversity should be correlated with the ratings for the vocabulary. Same as above, we measure Pearson’s r . We use a two-sided test for the significance based on a \(\beta\) -distribution that models the expected correlations as implemented by scipy 61 . Same as above, we use the Holm-Bonferroni method to account for multiple tests. However, we note that it is likely that all—even tiny—correlations are significant given our amount of data. Consequently, our interpretation of these results focuses on the strength of the correlations.

Our statistical analysis of the data is implemented in Python. We use pandas 1.5.3 and numpy 1.24.2 for the processing of data, pingouin 0.5.3 for the calculation of Cronbach’s \(\alpha\) , scipy 1.10.1 for the Wilcoxon-rank-sum tests Pearson’s r , and seaborn 0.12.2 for the generation of plots, incl. the calculation of error bars that visualize the confidence intervals.

Out of the 111 teachers who completed the questionnaire, 108 rated all six essays, one rated five essays, one rated two essays, and one rated only one essay. This results in 658 ratings for 270 essays (90 topics for each essay type: human-, ChatGPT-3-, ChatGPT-4-generated), with three ratings for 121 essays, two ratings for 144 essays, and one rating for five essays. The inter-rater agreement is consistently excellent ( \(\alpha >0.9\) ), with the exception of language mastery where we have good agreement ( \(\alpha =0.89\) , see Table  2 ). Further, the correlation analysis depicted in supplementary material S4 shows weak positive correlations ( \(r \in 0.11, 0.28]\) ) between the self-assessment for the English skills, respectively the self-assessment for the confidence in ratings and the actual ratings. Overall, this indicates that our ratings are reliable estimates of the actual quality of the essays with a potential small tendency that confidence in ratings and language skills yields better ratings, independent of the data source.

Table  2 and supplementary material S4 characterize the distribution of the ratings for the essays, grouped by the data source. We observe that for all criteria, we have a clear order of the mean values, with students having the worst ratings, ChatGPT-3 in the middle rank, and ChatGPT-4 with the best performance. We further observe that the standard deviations are fairly consistent and slightly larger than one, i.e. the spread is similar for all ratings and essays. This is further supported by the visual analysis of the violin plots.

The statistical analysis of the ratings reported in Table  4 shows that differences between the human-written essays and the ones generated by both ChatGPT models are significant. The effect sizes for human versus ChatGPT-3 essays are between 0.52 and 1.15, i.e. a medium ( \(d \in [0.5,0.8)\) ) to large ( \(d \in [0.8, 1.2)\) ) effect. On the one hand, the smallest effects are observed for the expressiveness and complexity, i.e. when it comes to the overall comprehensiveness and complexity of the sentence structures, the differences between the humans and the ChatGPT-3 model are smallest. On the other hand, the difference in language mastery is larger than all other differences, which indicates that humans are more prone to making mistakes when writing than the NLG models. The magnitude of differences between humans and ChatGPT-4 is larger with effect sizes between 0.88 and 1.43, i.e., a large to very large ( \(d \in [1.2, 2)\) ) effect. Same as for ChatGPT-3, the differences are smallest for expressiveness and complexity and largest for language mastery. Please note that the difference in language mastery between humans and both GPT models does not mean that the humans have low scores for language mastery (M=3.90), but rather that the NLG models have exceptionally high scores (M=5.03 for ChatGPT-3, M=5.25 for ChatGPT-4).

When we consider the differences between the two GPT models, we observe that while ChatGPT-4 has consistently higher mean values for all criteria, only the differences for logic and composition, vocabulary and text linking, and complexity are significant. The effect sizes are between 0.45 and 0.5, i.e. small ( \(d \in [0.2, 0.5)\) ) and medium. Thus, while GPT-4 seems to be an improvement over GPT-3.5 in general, the only clear indicator of this is a better and clearer logical composition and more complex writing with a more diverse vocabulary.

We also observe significant differences in the distribution of linguistic characteristics between all three groups (see Table  3 ). Sentence complexity (depth) is the only category without a significant difference between humans and ChatGPT-3, as well as ChatGPT-3 and ChatGPT-4. There is also no significant difference in the category of discourse markers between humans and ChatGPT-3. The magnitude of the effects varies a lot and is between 0.39 and 1.93, i.e., between small ( \(d \in [0.2, 0.5)\) ) and very large. However, in comparison to the ratings, there is no clear tendency regarding the direction of the differences. For instance, while the ChatGPT models write more complex sentences and use more nominalizations, humans tend to use more modals and epistemic markers instead. The lexical diversity of humans is higher than that of ChatGPT-3 but lower than that of ChatGPT-4. While there is no difference in the use of discourse markers between humans and ChatGPT-3, ChatGPT-4 uses significantly fewer discourse markers.

We detect the expected positive correlations between the complexity ratings and the linguistic markers for sentence complexity ( \(r=0.16\) for depth, \(r=0.19\) for clauses) and nominalizations ( \(r=0.22\) ). However, we observe a negative correlation between the logic ratings and the discourse markers ( \(r=-0.14\) ), which counters our intuition that more frequent use of discourse indicators makes a text more logically coherent. However, this is in line with previous work: McNamara et al. 45 also find no indication that the use of cohesion indices such as discourse connectives correlates with high- and low-proficiency essays. Finally, we observe the expected positive correlation between the ratings for the vocabulary and the lexical diversity ( \(r=0.12\) ). All observed correlations are significant. However, we note that the strength of all these correlations is weak and that the significance itself should not be over-interpreted due to the large sample size.

Our results provide clear answers to the first two research questions that consider the quality of the generated essays: ChatGPT performs well at writing argumentative student essays and outperforms the quality of the human-written essays significantly. The ChatGPT-4 model has (at least) a large effect and is on average about one point better than humans on a seven-point Likert scale.

Regarding the third research question, we find that there are significant linguistic differences between humans and AI-generated content. The AI-generated essays are highly structured, which for instance is reflected by the identical beginnings of the concluding sections of all ChatGPT essays (‘In conclusion, [...]’). The initial sentences of each essay are also very similar starting with a general statement using the main concepts of the essay topics. Although this corresponds to the general structure that is sought after for argumentative essays, it is striking to see that the ChatGPT models are so rigid in realizing this, whereas the human-written essays are looser in representing the guideline on the linguistic surface. Moreover, the linguistic fingerprint has the counter-intuitive property that the use of discourse markers is negatively correlated with logical coherence. We believe that this might be due to the rigid structure of the generated essays: instead of using discourse markers, the AI models provide a clear logical structure by separating the different arguments into paragraphs, thereby reducing the need for discourse markers.

Our data also shows that hallucinations are not a problem in the setting of argumentative essay writing: the essay topics are not really about factual correctness, but rather about argumentation and critical reflection on general concepts which seem to be contained within the knowledge of the AI model. The stochastic nature of the language generation is well-suited for this kind of task, as different plausible arguments can be seen as a sampling from all available arguments for a topic. Nevertheless, we need to perform a more systematic study of the argumentative structures in order to better understand the difference in argumentation between human-written and ChatGPT-generated essay content. Moreover, we also cannot rule out that subtle hallucinations may have been overlooked during the ratings. There are also essays with a low rating for the criteria related to factual correctness, indicating that there might be cases where the AI models still have problems, even if they are, on average, better than the students.

One of the issues with evaluations of the recent large-language models is not accounting for the impact of tainted data when benchmarking such models. While it is certainly possible that the essays that were sourced by Stab and Gurevych 41 from the internet were part of the training data of the GPT models, the proprietary nature of the model training means that we cannot confirm this. However, we note that the generated essays did not resemble the corpus of human essays at all. Moreover, the topics of the essays are general in the sense that any human should be able to reason and write about these topics, just by understanding concepts like ‘cooperation’. Consequently, a taint on these general topics, i.e. the fact that they might be present in the data, is not only possible but is actually expected and unproblematic, as it relates to the capability of the models to learn about concepts, rather than the memorization of specific task solutions.

While we did everything to ensure a sound construct and a high validity of our study, there are still certain issues that may affect our conclusions. Most importantly, neither the writers of the essays, nor their raters, were English native speakers. However, the students purposefully used a forum for English writing frequented by native speakers to ensure the language and content quality of their essays. This indicates that the resulting essays are likely above average for non-native speakers, as they went through at least one round of revisions with the help of native speakers. The teachers were informed that part of the training would be in English to prevent registrations from people without English language skills. Moreover, the self-assessment of the language skills was only weakly correlated with the ratings, indicating that the threat to the soundness of our results is low. While we cannot definitively rule out that our results would not be reproducible with other human raters, the high inter-rater agreement indicates that this is unlikely.

However, our reliance on essays written by non-native speakers affects the external validity and the generalizability of our results. It is certainly possible that native speaking students would perform better in the criteria related to language skills, though it is unclear by how much. However, the language skills were particular strengths of the AI models, meaning that while the difference might be smaller, it is still reasonable to conclude that the AI models would have at least comparable performance to humans, but possibly still better performance, just with a smaller gap. While we cannot rule out a difference for the content-related criteria, we also see no strong argument why native speakers should have better arguments than non-native speakers. Thus, while our results might not fully translate to native speakers, we see no reason why aspects regarding the content should not be similar. Further, our results were obtained based on high-school-level essays. Native and non-native speakers with higher education degrees or experts in fields would likely also achieve a better performance, such that the difference in performance between the AI models and humans would likely also be smaller in such a setting.

We further note that the essay topics may not be an unbiased sample. While Stab and Gurevych 41 randomly sampled the essays from the writing feedback section of an essay forum, it is unclear whether the essays posted there are representative of the general population of essay topics. Nevertheless, we believe that the threat is fairly low because our results are consistent and do not seem to be influenced by certain topics. Further, we cannot with certainty conclude how our results generalize beyond ChatGPT-3 and ChatGPT-4 to similar models like Bard ( https://bard.google.com/?hl=en ) Alpaca, and Dolly. Especially the results for linguistic characteristics are hard to predict. However, since—to the best of our knowledge and given the proprietary nature of some of these models—the general approach to how these models work is similar and the trends for essay quality should hold for models with comparable size and training procedures.

Finally, we want to note that the current speed of progress with generative AI is extremely fast and we are studying moving targets: ChatGPT 3.5 and 4 today are already not the same as the models we studied. Due to a lack of transparency regarding the specific incremental changes, we cannot know or predict how this might affect our results.

Our results provide a strong indication that the fear many teaching professionals have is warranted: the way students do homework and teachers assess it needs to change in a world of generative AI models. For non-native speakers, our results show that when students want to maximize their essay grades, they could easily do so by relying on results from AI models like ChatGPT. The very strong performance of the AI models indicates that this might also be the case for native speakers, though the difference in language skills is probably smaller. However, this is not and cannot be the goal of education. Consequently, educators need to change how they approach homework. Instead of just assigning and grading essays, we need to reflect more on the output of AI tools regarding their reasoning and correctness. AI models need to be seen as an integral part of education, but one which requires careful reflection and training of critical thinking skills.

Furthermore, teachers need to adapt strategies for teaching writing skills: as with the use of calculators, it is necessary to critically reflect with the students on when and how to use those tools. For instance, constructivists 62 argue that learning is enhanced by the active design and creation of unique artifacts by students themselves. In the present case this means that, in the long term, educational objectives may need to be adjusted. This is analogous to teaching good arithmetic skills to younger students and then allowing and encouraging students to use calculators freely in later stages of education. Similarly, once a sound level of literacy has been achieved, strongly integrating AI models in lesson plans may no longer run counter to reasonable learning goals.

In terms of shedding light on the quality and structure of AI-generated essays, this paper makes an important contribution by offering an independent, large-scale and statistically sound account of essay quality, comparing human-written and AI-generated texts. By comparing different versions of ChatGPT, we also offer a glance into the development of these models over time in terms of their linguistic properties and the quality they exhibit. Our results show that while the language generated by ChatGPT is considered very good by humans, there are also notable structural differences, e.g. in the use of discourse markers. This demonstrates that an in-depth consideration not only of the capabilities of generative AI models is required (i.e. which tasks can they be used for), but also of the language they generate. For example, if we read many AI-generated texts that use fewer discourse markers, it raises the question if and how this would affect our human use of discourse markers. Understanding how AI-generated texts differ from human-written enables us to look for these differences, to reason about their potential impact, and to study and possibly mitigate this impact.

Data availability

The datasets generated during and/or analysed during the current study are available in the Zenodo repository, https://doi.org/10.5281/zenodo.8343644

Code availability

All materials are available online in form of a replication package that contains the data and the analysis code, https://doi.org/10.5281/zenodo.8343644 .

Ouyang, L. et al. Training language models to follow instructions with human feedback (2022). arXiv:2203.02155 .

Ruby, D. 30+ detailed chatgpt statistics–users & facts (sep 2023). https://www.demandsage.com/chatgpt-statistics/ (2023). Accessed 09 June 2023.

Leahy, S. & Mishra, P. TPACK and the Cambrian explosion of AI. In Society for Information Technology & Teacher Education International Conference , (ed. Langran, E.) 2465–2469 (Association for the Advancement of Computing in Education (AACE), 2023).

Ortiz, S. Need an ai essay writer? here’s how chatgpt (and other chatbots) can help. https://www.zdnet.com/article/how-to-use-chatgpt-to-write-an-essay/ (2023). Accessed 09 June 2023.

Openai chat interface. https://chat.openai.com/ . Accessed 09 June 2023.

OpenAI. Gpt-4 technical report (2023). arXiv:2303.08774 .

Brown, T. B. et al. Language models are few-shot learners (2020). arXiv:2005.14165 .

Wang, B. Mesh-Transformer-JAX: Model-Parallel Implementation of Transformer Language Model with JAX. https://github.com/kingoflolz/mesh-transformer-jax (2021).

Wei, J. et al. Finetuned language models are zero-shot learners. In International Conference on Learning Representations (2022).

Taori, R. et al. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/stanford_alpaca (2023).

Cai, Z. G., Haslett, D. A., Duan, X., Wang, S. & Pickering, M. J. Does chatgpt resemble humans in language use? (2023). arXiv:2303.08014 .

Mahowald, K. A discerning several thousand judgments: Gpt-3 rates the article + adjective + numeral + noun construction (2023). arXiv:2301.12564 .

Dentella, V., Murphy, E., Marcus, G. & Leivada, E. Testing ai performance on less frequent aspects of language reveals insensitivity to underlying meaning (2023). arXiv:2302.12313 .

Guo, B. et al. How close is chatgpt to human experts? comparison corpus, evaluation, and detection (2023). arXiv:2301.07597 .

Zhao, W. et al. Is chatgpt equipped with emotional dialogue capabilities? (2023). arXiv:2304.09582 .

Keim, D. A. & Oelke, D. Literature fingerprinting : A new method for visual literary analysis. In 2007 IEEE Symposium on Visual Analytics Science and Technology , 115–122, https://doi.org/10.1109/VAST.2007.4389004 (IEEE, 2007).

El-Assady, M. et al. Interactive visual analysis of transcribed multi-party discourse. In Proceedings of ACL 2017, System Demonstrations , 49–54 (Association for Computational Linguistics, Vancouver, Canada, 2017).

Mennatallah El-Assady, A. H.-J. & Butt, M. Discourse maps - feature encoding for the analysis of verbatim conversation transcripts. In Visual Analytics for Linguistics , vol. CSLI Lecture Notes, Number 220, 115–147 (Stanford: CSLI Publications, 2020).

Matt Foulis, J. V. & Reed, C. Dialogical fingerprinting of debaters. In Proceedings of COMMA 2020 , 465–466, https://doi.org/10.3233/FAIA200536 (Amsterdam: IOS Press, 2020).

Matt Foulis, J. V. & Reed, C. Interactive visualisation of debater identification and characteristics. In Proceedings of the COMMA workshop on Argument Visualisation, COMMA , 1–7 (2020).

Chatzipanagiotidis, S., Giagkou, M. & Meurers, D. Broad linguistic complexity analysis for Greek readability classification. In Proceedings of the 16th Workshop on Innovative Use of NLP for Building Educational Applications , 48–58 (Association for Computational Linguistics, Online, 2021).

Ajili, M., Bonastre, J.-F., Kahn, J., Rossato, S. & Bernard, G. FABIOLE, a speech database for forensic speaker comparison. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16) , 726–733 (European Language Resources Association (ELRA), Portorož, Slovenia, 2016).

Deutsch, T., Jasbi, M. & Shieber, S. Linguistic features for readability assessment. In Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications , 1–17, https://doi.org/10.18653/v1/2020.bea-1.1 (Association for Computational Linguistics, Seattle, WA, USA \(\rightarrow\) Online, 2020).

Fiacco, J., Jiang, S., Adamson, D. & Rosé, C. Toward automatic discourse parsing of student writing motivated by neural interpretation. In Proceedings of the 17th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2022) , 204–215, https://doi.org/10.18653/v1/2022.bea-1.25 (Association for Computational Linguistics, Seattle, Washington, 2022).

Weiss, Z., Riemenschneider, A., Schröter, P. & Meurers, D. Computationally modeling the impact of task-appropriate language complexity and accuracy on human grading of German essays. In Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications , 30–45, https://doi.org/10.18653/v1/W19-4404 (Association for Computational Linguistics, Florence, Italy, 2019).

Yang, F., Dragut, E. & Mukherjee, A. Predicting personal opinion on future events with fingerprints. In Proceedings of the 28th International Conference on Computational Linguistics , 1802–1807, https://doi.org/10.18653/v1/2020.coling-main.162 (International Committee on Computational Linguistics, Barcelona, Spain (Online), 2020).

Tumarada, K. et al. Opinion prediction with user fingerprinting. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021) , 1423–1431 (INCOMA Ltd., Held Online, 2021).

Rocca, R. & Yarkoni, T. Language as a fingerprint: Self-supervised learning of user encodings using transformers. In Findings of the Association for Computational Linguistics: EMNLP . 1701–1714 (Association for Computational Linguistics, Abu Dhabi, United Arab Emirates, 2022).

Aiyappa, R., An, J., Kwak, H. & Ahn, Y.-Y. Can we trust the evaluation on chatgpt? (2023). arXiv:2303.12767 .

Yeadon, W., Inyang, O.-O., Mizouri, A., Peach, A. & Testrow, C. The death of the short-form physics essay in the coming ai revolution (2022). arXiv:2212.11661 .

TURING, A. M. I.-COMPUTING MACHINERY AND INTELLIGENCE. Mind LIX , 433–460, https://doi.org/10.1093/mind/LIX.236.433 (1950). https://academic.oup.com/mind/article-pdf/LIX/236/433/30123314/lix-236-433.pdf .

Kortemeyer, G. Could an artificial-intelligence agent pass an introductory physics course? (2023). arXiv:2301.12127 .

Kung, T. H. et al. Performance of chatgpt on usmle: Potential for ai-assisted medical education using large language models. PLOS Digital Health 2 , 1–12. https://doi.org/10.1371/journal.pdig.0000198 (2023).

Article   Google Scholar  

Frieder, S. et al. Mathematical capabilities of chatgpt (2023). arXiv:2301.13867 .

Yuan, Z., Yuan, H., Tan, C., Wang, W. & Huang, S. How well do large language models perform in arithmetic tasks? (2023). arXiv:2304.02015 .

Touvron, H. et al. Llama: Open and efficient foundation language models (2023). arXiv:2302.13971 .

Chung, H. W. et al. Scaling instruction-finetuned language models (2022). arXiv:2210.11416 .

Workshop, B. et al. Bloom: A 176b-parameter open-access multilingual language model (2023). arXiv:2211.05100 .

Spencer, S. T., Joshi, V. & Mitchell, A. M. W. Can ai put gamma-ray astrophysicists out of a job? (2023). arXiv:2303.17853 .

Cherian, A., Peng, K.-C., Lohit, S., Smith, K. & Tenenbaum, J. B. Are deep neural networks smarter than second graders? (2023). arXiv:2212.09993 .

Stab, C. & Gurevych, I. Annotating argument components and relations in persuasive essays. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers , 1501–1510 (Dublin City University and Association for Computational Linguistics, Dublin, Ireland, 2014).

Essay forum. https://essayforum.com/ . Last-accessed: 2023-09-07.

Common european framework of reference for languages (cefr). https://www.coe.int/en/web/common-european-framework-reference-languages . Accessed 09 July 2023.

Kmk guidelines for essay assessment. http://www.kmk-format.de/material/Fremdsprachen/5-3-2_Bewertungsskalen_Schreiben.pdf . Accessed 09 July 2023.

McNamara, D. S., Crossley, S. A. & McCarthy, P. M. Linguistic features of writing quality. Writ. Commun. 27 , 57–86 (2010).

McCarthy, P. M. & Jarvis, S. Mtld, vocd-d, and hd-d: A validation study of sophisticated approaches to lexical diversity assessment. Behav. Res. Methods 42 , 381–392 (2010).

Article   PubMed   Google Scholar  

Dasgupta, T., Naskar, A., Dey, L. & Saha, R. Augmenting textual qualitative features in deep convolution recurrent neural network for automatic essay scoring. In Proceedings of the 5th Workshop on Natural Language Processing Techniques for Educational Applications , 93–102 (2018).

Koizumi, R. & In’nami, Y. Effects of text length on lexical diversity measures: Using short texts with less than 200 tokens. System 40 , 554–564 (2012).

spacy industrial-strength natural language processing in python. https://spacy.io/ .

Siskou, W., Friedrich, L., Eckhard, S., Espinoza, I. & Hautli-Janisz, A. Measuring plain language in public service encounters. In Proceedings of the 2nd Workshop on Computational Linguistics for Political Text Analysis (CPSS-2022) (Potsdam, Germany, 2022).

El-Assady, M. & Hautli-Janisz, A. Discourse Maps - Feature Encoding for the Analysis of Verbatim Conversation Transcripts (CSLI lecture notes (CSLI Publications, Center for the Study of Language and Information, 2019).

Hautli-Janisz, A. et al. QT30: A corpus of argument and conflict in broadcast debate. In Proceedings of the Thirteenth Language Resources and Evaluation Conference , 3291–3300 (European Language Resources Association, Marseille, France, 2022).

Somasundaran, S. et al. Towards evaluating narrative quality in student writing. Trans. Assoc. Comput. Linguist. 6 , 91–106 (2018).

Nadeem, F., Nguyen, H., Liu, Y. & Ostendorf, M. Automated essay scoring with discourse-aware neural models. In Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications , 484–493, https://doi.org/10.18653/v1/W19-4450 (Association for Computational Linguistics, Florence, Italy, 2019).

Prasad, R. et al. The Penn Discourse TreeBank 2.0. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08) (European Language Resources Association (ELRA), Marrakech, Morocco, 2008).

Cronbach, L. J. Coefficient alpha and the internal structure of tests. Psychometrika 16 , 297–334. https://doi.org/10.1007/bf02310555 (1951).

Article   MATH   Google Scholar  

Wilcoxon, F. Individual comparisons by ranking methods. Biom. Bull. 1 , 80–83 (1945).

Holm, S. A simple sequentially rejective multiple test procedure. Scand. J. Stat. 6 , 65–70 (1979).

MathSciNet   MATH   Google Scholar  

Cohen, J. Statistical power analysis for the behavioral sciences (Academic press, 2013).

Freedman, D., Pisani, R. & Purves, R. Statistics (international student edition). Pisani, R. Purves, 4th edn. WW Norton & Company, New York (2007).

Scipy documentation. https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.pearsonr.html . Accessed 09 June 2023.

Windschitl, M. Framing constructivism in practice as the negotiation of dilemmas: An analysis of the conceptual, pedagogical, cultural, and political challenges facing teachers. Rev. Educ. Res. 72 , 131–175 (2002).

Download references

Open Access funding enabled and organized by Projekt DEAL.

Author information

Authors and affiliations.

Faculty of Computer Science and Mathematics, University of Passau, Passau, Germany

Steffen Herbold, Annette Hautli-Janisz, Ute Heuer, Zlata Kikteva & Alexander Trautsch

You can also search for this author in PubMed   Google Scholar

Contributions

S.H., A.HJ., and U.H. conceived the experiment; S.H., A.HJ, and Z.K. collected the essays from ChatGPT; U.H. recruited the study participants; S.H., A.HJ., U.H. and A.T. conducted the training session and questionnaire; all authors contributed to the analysis of the results, the writing of the manuscript, and review of the manuscript.

Corresponding author

Correspondence to Steffen Herbold .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary information 1., supplementary information 2., supplementary information 3., supplementary tables., supplementary figures., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Herbold, S., Hautli-Janisz, A., Heuer, U. et al. A large-scale comparison of human-written versus ChatGPT-generated essays. Sci Rep 13 , 18617 (2023). https://doi.org/10.1038/s41598-023-45644-9

Download citation

Received : 01 June 2023

Accepted : 22 October 2023

Published : 30 October 2023

DOI : https://doi.org/10.1038/s41598-023-45644-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Defense against adversarial attacks: robust and efficient compressed optimized neural networks.

• Insaf Kraidia
• Afifa Ghenai
• Samir Brahim Belhaouari

Scientific Reports (2024)

AI-driven translations for kidney transplant equity in Hispanic populations

• Oscar A. Garcia Valencia
• Charat Thongprayoon
• Wisit Cheungpasitporn

How will the state think with ChatGPT? The challenges of generative artificial intelligence for public administrations

• Thomas Cantens

AI & SOCIETY (2024)

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

AI-assisted writing is quietly booming in academic journals. Here’s why that’s OK

Lecturer in Bioethics, Monash University & Honorary fellow, Melbourne Law School, Monash University

Disclosure statement

Julian Koplin does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.

Monash University provides funding as a founding partner of The Conversation AU.

View all partners

If you search Google Scholar for the phrase “ as an AI language model ”, you’ll find plenty of AI research literature and also some rather suspicious results. For example, one paper on agricultural technology says:

As an AI language model, I don’t have direct access to current research articles or studies. However, I can provide you with an overview of some recent trends and advancements …

Obvious gaffes like this aren’t the only signs that researchers are increasingly turning to generative AI tools when writing up their research. A recent study examined the frequency of certain words in academic writing (such as “commendable”, “meticulously” and “intricate”), and found they became far more common after the launch of ChatGPT – so much so that 1% of all journal articles published in 2023 may have contained AI-generated text.

(Why do AI models overuse these words? There is speculation it’s because they are more common in English as spoken in Nigeria, where key elements of model training often occur.)

The aforementioned study also looks at preliminary data from 2024, which indicates that AI writing assistance is only becoming more common. Is this a crisis for modern scholarship, or a boon for academic productivity?

Who should take credit for AI writing?

Many people are worried by the use of AI in academic papers. Indeed, the practice has been described as “ contaminating ” scholarly literature.

Some argue that using AI output amounts to plagiarism. If your ideas are copy-pasted from ChatGPT, it is questionable whether you really deserve credit for them.

But there are important differences between “plagiarising” text authored by humans and text authored by AI. Those who plagiarise humans’ work receive credit for ideas that ought to have gone to the original author.

By contrast, it is debatable whether AI systems like ChatGPT can have ideas, let alone deserve credit for them. An AI tool is more like your phone’s autocomplete function than a human researcher.

The question of bias

Another worry is that AI outputs might be biased in ways that could seep into the scholarly record. Infamously, older language models tended to portray people who are female, black and/or gay in distinctly unflattering ways, compared with people who are male, white and/or straight.

This kind of bias is less pronounced in the current version of ChatGPT.

However, other studies have found a different kind of bias in ChatGPT and other large language models : a tendency to reflect a left-liberal political ideology.

Any such bias could subtly distort scholarly writing produced using these tools.

The hallucination problem

The most serious worry relates to a well-known limitation of generative AI systems: that they often make serious mistakes.

For example, when I asked ChatGPT-4 to generate an ASCII image of a mushroom, it provided me with the following output.

It then confidently told me I could use this image of a “mushroom” for my own purposes.

These kinds of overconfident mistakes have been referred to as “ AI hallucinations ” and “ AI bullshit ”. While it is easy to spot that the above ASCII image looks nothing like a mushroom (and quite a bit like a snail), it may be much harder to identify any mistakes ChatGPT makes when surveying scientific literature or describing the state of a philosophical debate.

Unlike (most) humans, AI systems are fundamentally unconcerned with the truth of what they say. If used carelessly, their hallucinations could corrupt the scholarly record.

Should AI-produced text be banned?

One response to the rise of text generators has been to ban them outright. For example, Science – one of the world’s most influential academic journals – disallows any use of AI-generated text .

I see two problems with this approach.

The first problem is a practical one: current tools for detecting AI-generated text are highly unreliable. This includes the detector created by ChatGPT’s own developers, which was taken offline after it was found to have only a 26% accuracy rate (and a 9% false positive rate ). Humans also make mistakes when assessing whether something was written by AI.

It is also possible to circumvent AI text detectors. Online communities are actively exploring how to prompt ChatGPT in ways that allow the user to evade detection. Human users can also superficially rewrite AI outputs, effectively scrubbing away the traces of AI (like its overuse of the words “commendable”, “meticulously” and “intricate”).

The second problem is that banning generative AI outright prevents us from realising these technologies’ benefits. Used well, generative AI can boost academic productivity by streamlining the writing process. In this way, it could help further human knowledge. Ideally, we should try to reap these benefits while avoiding the problems.

The problem is poor quality control, not AI

The most serious problem with AI is the risk of introducing unnoticed errors, leading to sloppy scholarship. Instead of banning AI, we should try to ensure that mistaken, implausible or biased claims cannot make it onto the academic record.

After all, humans can also produce writing with serious errors, and mechanisms such as peer review often fail to prevent its publication.

We need to get better at ensuring academic papers are free from serious mistakes, regardless of whether these mistakes are caused by careless use of AI or sloppy human scholarship. Not only is this more achievable than policing AI usage, it will improve the standards of academic research as a whole.

This would be (as ChatGPT might say) a commendable and meticulously intricate solution.

• Artificial intelligence (AI)
• Academic journals
• Academic publishing
• Hallucinations
• Scholarly publishing
• Academic writing
• Large language models
• Generative AI

Lecturer / Senior Lecturer - Marketing

Compliance Lead

Assistant Editor - 1 year cadetship

Executive Dean, Faculty of Health

Lecturer/Senior Lecturer, Earth System Science (School of Science)

The (AI) sky isn’t falling

Students using generative AI to write their essays is a problem, but it isn’t a crisis, writes Christopher Hallenbrook. We have the tools to tackle the issue of artificial intelligence

Christopher R. Hallenbrook

• More on this topic

You may also like

Popular resources

.css-1txxx8u{overflow:hidden;max-height:81px;text-indent:0px;} Rather than restrict the use of AI, embrace the challenge

Emotions and learning: what role do emotions play in how and why students learn, leveraging llms to assess soft skills in lifelong learning, how hard can it be testing ai detection tools, a diy guide to starting your own journal.

In January, the literary world was rocked by the news that novelist Rie Qudan had used ChatGPT to write 5 per cent of her novel that won Japan’s prestigious Akutagawa Prize. The consternation over this revelation mirrored the conversations that have been taking place in academia since ChatGPT was launched in late 2022. Discussions and academic essays since that time have consistently spoken of a new wave of cheating on campus, one we are powerless to prevent. 

While this reaction is understandable, I disagree with it. Students using AI to write their essays is a problem, but it isn’t a crisis. We have the tools to tackle the issue.

AI is easy to spot

In most cases AI writing can be easily recognised. If you ask multipart questions, as I do, ChatGPT defaults to using section headings for each component. When I grade a paper that has six section headings in a three- to five-page paper (something I have experienced), I see a red flag. ChatGPT’s vocabulary reinforces this impression. Its word choice does not align with how most undergraduates write. I’ve never seen a student call Publius a “collective pseudonym” in a paper about The Federalist Papers , but ChatGPT frequently does. AI is quick to discuss the “ethical foundations of governance”, “intrinsic equilibrium” and other terms that are rare in undergraduate writing if you haven’t used the terms in class. Certainly, some students do use such vocabulary. 

One must be careful and know one’s students. In-class discussions and short response papers can help you get a feel for how your students talk and write. Worst-case scenario, a one-to-one discussion of the paper with the student goes a long way. I’ve asked students to explain what they meant by a certain term. The answer “I don’t know” tells you what you need to know about whether or not they used AI. 

• Resource collection: AI transformers like ChatGPT are here, so what next?
• Rather than restrict the use of AI, let’s embrace the challenge it offers
• AI did not disturb assessment – it just made our mistakes visible

Even when you can’t identify AI writing so readily, you will likely fail the paper on its merits anyway. I’ve found ChatGPT will frequently engage with the topic but will write around the question. The answer is related to what I asked about but doesn’t answer my question. By missing the question, making its points in brief and not using the textual evidence that I instruct students to include (but I don’t put that instruction in the question itself), ChatGPT produces an essay that omits the most essential elements that I grade on. So even if I miss that the essay was AI generated, I’m still going to give it a poor grade.

The summary is ‘dead and buried’

Careful consideration and structuring of essay prompts also reduce the risk of students getting AI-written work past you. A simple summary of concepts is easy for ChatGPT. Even deep questions of political theory have enough written on them for ChatGPT to rapidly produce a quality summary. Summaries were never the most pedagogically sound take-home essay assignment; now they are dead and buried. 

Creativity in how we ask students to analyse and apply concepts makes it much harder for ChatGPT to answer our questions. When I was an undergraduate student, my mentor framed all his questions as “in what manner and to what extent” can something be said to be true. That framework invites nuance, forces students to define their terms and can be used to create less-written-about topics. 

Similarly, when responding to prompts asking about theories of democratic representation, ChatGPT can effectively summarise the beliefs of Publius, the anti-federalist Brutus or Malcolm X on the nature of representation, but it struggles to answer: “Can Professor Hallenbrook properly represent Carson? Why or why not? Draw on the ideas of thinkers we have read in class to justify your answer.” In fact, it doesn’t always recognise that by “Carson”, I am referring to the city where I teach, not a person. By not specifying which thinkers, ChatGPT has to pick its own and in my practice runs with this prompt, it used almost exclusively thinkers I had not taught in my American political thought class.

Ask ChatGPT first, then set the essay topic

I select my phrasing after putting different versions of the question through ChatGPT. Running your prompt through ChatGPT before you assign it will both let you know if you’ve successfully created a question that the generative AI will struggle with and give you a feel for the tells in its approach that will let you know if a student tries to use it. I’d recommend running the prompt multiple times to see different versions of an AI answer and make note of the tells. It is a touch more prep time but totally worth it. After all, we should be continually re-examining our prompts anyway.

So, yes, ChatGPT is a potential problem. But it is not insurmountable. As with plagiarism, some uses may escape our detection. But through attention to detail and careful design of our assignments, we can make it harder for students to use ChatGPT to write their papers effectively and easier to spot it when they do.

Christopher R. Hallenbrook is assistant professor of political science and chair of the general education committee at California State University, Dominguez Hills.

If you would like advice and insight from academics and university staff delivered direct to your inbox each week, sign up for the Campus newsletter .

Rather than restrict the use of AI, embrace the challenge

Let’s think about assessments and ai in a different way, how students’ genai skills affect assignment instructions, how not to land a job in academia, contextual learning: linking learning to the real world, three steps to unearth the hidden curriculum of networking.

Register for free

and unlock a host of features on the THE site

Should universities be worried about the increasing capabilities of AI?

If a piece of writing was 49 per cent written by AI, with the remaining 51 per cent written by a human, is this considered original work? Image:  Unsplash/ Danial Igdery

.chakra .wef-1c7l3mo{-webkit-transition:all 0.15s ease-out;transition:all 0.15s ease-out;cursor:pointer;-webkit-text-decoration:none;text-decoration:none;outline:none;color:inherit;}.chakra .wef-1c7l3mo:hover,.chakra .wef-1c7l3mo[data-hover]{-webkit-text-decoration:underline;text-decoration:underline;}.chakra .wef-1c7l3mo:focus,.chakra .wef-1c7l3mo[data-focus]{box-shadow:0 0 0 3px rgba(168,203,251,0.5);} Sarah Elaine Eaton

Michael mindzak.

.chakra .wef-9dduvl{margin-top:16px;margin-bottom:16px;line-height:1.388;font-size:1.25rem;}@media screen and (min-width:56.5rem){.chakra .wef-9dduvl{font-size:1.125rem;}} Explore and monitor how .chakra .wef-15eoq1r{margin-top:16px;margin-bottom:16px;line-height:1.388;font-size:1.25rem;color:#F7DB5E;}@media screen and (min-width:56.5rem){.chakra .wef-15eoq1r{font-size:1.125rem;}} Education, Gender and Work is affecting economies, industries and global issues

.chakra .wef-1nk5u5d{margin-top:16px;margin-bottom:16px;line-height:1.388;color:#2846F8;font-size:1.25rem;}@media screen and (min-width:56.5rem){.chakra .wef-1nk5u5d{font-size:1.125rem;}} Get involved with our crowdsourced digital platform to deliver impact at scale

Stay up to date:, education, gender and work.

• The use of technology in academic writing is already widespread, with teachers and students using AI-based tools to support the work they are doing.
• However, as AI becomes increasingly advanced, institutions need to properly define what can be defined as AI-assistance and what is plagiarism or cheating, writes an academic.
• For example, if a piece of writing was 49% written by AI, with the remaining 51% written by a human, is this considered original work?

The dramatic rise of online learning during the COVID-19 pandemic has spotlit concerns about the role of technology in exam surveillance — and also in student cheating .

Some universities have reported more cheating during the pandemic, and such concerns are unfolding in a climate where technologies that allow for the automation of writing continue to improve.

Over the past two years, the ability of artificial intelligence to generate writing has leapt forward significantly , particularly with the development of what’s known as the language generator GPT-3. With this, companies such as Google , Microsoft and NVIDIA can now produce “human-like” text .

AI-generated writing has raised the stakes of how universities and schools will gauge what constitutes academic misconduct, such as plagiarism . As scholars with an interest in academic integrity and the intersections of work, society and educators’ labour, we believe that educators and parents should be, at the very least, paying close attention to these significant developments .

AI & academic writing

The use of technology in academic writing is already widespread. For example, many universities already use text-based plagiarism detectors like Turnitin , while students might use Grammarly , a cloud-based writing assistant. Examples of writing support include automatic text generation, extraction, prediction, mining, form-filling, paraphrasing , translation and transcription.

Advancements in AI technology have led to new tools, products and services being offered to writers to improve content and efficiency . As these improve, soon entire articles or essays might be generated and written entirely by artificial intelligence . In schools, the implications of such developments will undoubtedly shape the future of learning, writing and teaching.

Misconduct concerns already widespread

Research has revealed that concerns over academic misconduct are already widespread across institutions higher education in Canada and internationally.

In Canada, there is little data regarding the rates of misconduct. Research published in 2006 based on data from mostly undergraduate students at 11 higher education institutions found 53 per cent reported having engaged in one or more instances of serious cheating on written work, which was defined as copying material without footnoting, copying material almost word for word, submitting work done by someone else, fabricating or falsifying a bibliography, submitting a paper they either bought or got from someone else for free.

Academic misconduct is in all likelihood under-reported across Canadian higher education institutions .

There are different types of violations of academic integrity, including plagiarism , contract cheating (where students hire other people to write their papers) and exam cheating, among others .

Unfortunately, with technology, students can use their ingenuity and entrepreneurialism to cheat. These concerns are also applicable to faculty members, academics and writers in other fields, bringing new concerns surrounding academic integrity and AI such as:

• If a piece of writing was 49 per cent written by AI, with the remaining 51 per cent written by a human, is this considered original work?
• What if an essay was 100 per cent written by AI, but a student did some of the coding themselves?
• What qualifies as “AI assistance” as opposed to “academic cheating”?
• Do the same rules apply to students as they would to academics and researchers?

We are asking these questions in our own research , and we know that in the face of all this, educators will be required to consider how writing can be effectively assessed or evaluated as these technologies improve.

Augmenting or diminishing integrity?

At the moment, little guidance, policy or oversight is available regarding technology, AI and academic integrity for teachers and educational leaders.

Over the past year, COVID-19 has pushed more students towards online learning — a sphere where teachers may become less familiar with their own students and thus, potentially, their writing.

While it remains impossible to predict the future of these technologies and their implications in education, we can attempt to discern some of the larger trends and trajectories that will impact teaching, learning and research.

Have you read?

Professor robot – why ai could soon be teaching in university classrooms, how digital technology is changing the university lecture, this is how university students can emerge from the pandemic stronger, technology & automation in education.

A key concern moving forward is the apparent movement towards the increased automation of education where educational technology companies offer commodities such as writing tools as proposed solutions for the various “problems” within education.

An example of this is automated assessment of student work, such as automated grading of student writing . Numerous commercial products already exist for automated grading, though the ethics of these technologies are yet to be fully explored by scholars and educators.

Overall, the traditional landscape surrounding academic integrity and authorship is being rapidly reshaped by technological developments. Such technological developments also spark concerns about a shift of professional control away from educators and ever-increasing new expectations of digital literacy in precarious working environments .

These complexities, concerns and questions will require further thought and discussion. Educational stakeholders at all levels will be required to respond and rethink definitions as well as values surrounding plagiarism, originality, academic ethics and academic labour in the very near future.

Don't miss any update on this topic

Create a free account and access your personalized content collection with our latest publications and analyses.

License and Republishing

World Economic Forum articles may be republished in accordance with the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International Public License, and in accordance with our Terms of Use.

The views expressed in this article are those of the author alone and not the World Economic Forum.

Related topics:

The agenda .chakra .wef-n7bacu{margin-top:16px;margin-bottom:16px;line-height:1.388;font-weight:400;} weekly.

A weekly update of the most important issues driving the global agenda

.chakra .wef-1dtnjt5{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-flex-wrap:wrap;-ms-flex-wrap:wrap;flex-wrap:wrap;} More on Emerging Technologies .chakra .wef-17xejub{-webkit-flex:1;-ms-flex:1;flex:1;justify-self:stretch;-webkit-align-self:stretch;-ms-flex-item-align:stretch;align-self:stretch;} .chakra .wef-nr1rr4{display:-webkit-inline-box;display:-webkit-inline-flex;display:-ms-inline-flexbox;display:inline-flex;white-space:normal;vertical-align:middle;text-transform:uppercase;font-size:0.75rem;border-radius:0.25rem;font-weight:700;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;line-height:1.2;-webkit-letter-spacing:1.25px;-moz-letter-spacing:1.25px;-ms-letter-spacing:1.25px;letter-spacing:1.25px;background:none;padding:0px;color:#B3B3B3;-webkit-box-decoration-break:clone;box-decoration-break:clone;-webkit-box-decoration-break:clone;}@media screen and (min-width:37.5rem){.chakra .wef-nr1rr4{font-size:0.875rem;}}@media screen and (min-width:56.5rem){.chakra .wef-nr1rr4{font-size:1rem;}} See all

Solar storms hit tech equipment, and other technology news you need to know

Sebastian Buckup

May 17, 2024

Generative AI is trained on just a few of the world’s 7,000 languages. Here’s why that’s a problem – and what’s being done about it

Madeleine North

Critical minerals demand has doubled in the past five years – here are some solutions to the supply crunch

Emma Charlton

May 16, 2024

6 ways satellites are helping to monitor our changing planet from space

Andrea Willige

How can GenAI be optimized for people and processes?

May 15, 2024

This is how AI can empower women and achieve gender equality, according to the founder of Girls Who Code and Moms First

Kate Whiting

May 14, 2024

8 Ways to Create AI-Proof Writing Prompts

C reating 100 percent AI-proof writing prompts can often be impossible but that doesn’t mean there aren’t strategies that can limit the efficacy of AI work. These techniques can also help ensure more of the writing submitted in your classroom is human-generated. 

I started seeing a big uptick in AI-generated work submitted in my classes over the last year and that has continued. As a result, I’ve gotten much better at recognizing AI work , but I’ve also gotten better at creating writing prompts that are less AI-friendly. 

Essentially, I like to use the public health Swiss cheese analogy when thinking about AI prevention: All these strategies on their own have holes but when you layer the cheese together, you create a barrier that’s hard to get through. 

The eight strategies here may not prevent students from submitting AI work, but I find these can incentivize human writing and make sure that any work submitted via AI will not really meet the requirements of the assignment. 

1. Writing AI-Proof Prompts: Put Your Prompt Into Popular AI tools such as ChatGPT, Copilot, and Bard 

Putting your writing prompt into an AI tools will give you an immediate idea of how most AI tools will handle your prompt. If the various AI chatbots do a good, or at least adequate, job immediately, it might be wise to tweak the prompt. 

One of my classes asks students to write about a prized possession. When you put this prompt into an AI chatbot, it frequently returns an essay about a family member's finely crafted watch. Obviously, I now watch out for any essays about watches. 

2. Forbid Cliché Use

Probably the quickest and easiest way to cut back on some AI use is to come down hard on cliché use in writing assignments. AI tools are essentially cliché machines, so banning these can prevent a lot of AI use. 

Equally as important, this practice will help your students become better writers. As any good writer knows, clichés should be avoided like the plague. 

3. Incorporate Recent Events

The free version of ChatGPT only has access to events up to 2022. While there are plugins to allow it to search the internet and other internet-capable AI tools, some students won’t get further than ChatGPT. 

More importantly, in my experience, all AI tools struggle to incorporate recent events as effectively as historic ones. So connecting class material and assignments to events such as a recent State of Union speech or the Academy Awards will make any AI writing use less effective. 

4. Require Quotes

AI tools can incorporate direct quotations but most are not very good at doing so. The quotes used tend to be very short and not as well-placed within essays. 

Asking an AI tool for recent quotes also can be particularly problematic for today’s robot writers. For instance, I asked Microsoft's Copilot to summarize the recent Academy Awards using quotes, and specifically asked it to quote from Oppenheimer's director Christopher Nolan’s acceptance speech. It quoted something Nolan had previously said instead. Copilot also quoted from Wes Anderson’s acceptance speech, an obvious error since Anderson wasn’t at the awards .  

5. Make Assignments Personal

Having students reflect on material in their own lives can be a good way to prevent AI writing. In-person teachers can get to know their students well enough to know when these types of personal details are fabricated. 

I teach online but still find it easier to tell when a more personalized prompt was written by AI. For example, one student submitted a paper about how much she loved skateboarding that was so non-specific it screamed AI written. Another submitted a post about a pair of sneakers that was also clearly written by a "sole-less" AI (I could tell because of the clichés and other reasons). 

6. Make Primary or Scholarly Sources Mandatory

Requiring sources that are not easily accessible on the internet can stop AI writing in its tracks. I like to have students find historic newspapers for certain assignments. The AI tools I am familiar with can’t incorporate these. 

For instance, I asked Copilot to compare coverage of the first Academy Awards in the media to the most recent awards show and to include quotes from historic newspaper coverage. The comparison was not well done and there were no quotes from historical newspaper coverage. 

AI tools also struggle to incorporate journal articles. Encouraging your students to include these types of sources ensures the work they produce is deeper than something that can be revealed by a quick Google search, which not only makes it harder for AI to write but also can raise the overall quality.  

7. Require Interviews, Field Trips, Etc. 

Building on primary and scholarly sources, you can have your students conduct interviews or go on field trips to historic sites, museums, etc. 

AI is still, thankfully, incapable of engaging in these types of behavior. This requires too much work for every assignment but it is the most effective way to truly ensure your work is human- not computer-written. 

If you’re still worried about AI use, you can even go a step further by asking your students to include photos of them with their interview subjects or from the field trips. Yes, AI art generators are getting better as well, but remember the Swiss cheese analogy? Every layer of prevention can help. 

8. Have Students Write During Class

As I said to start, none of the methods discussed are foolproof. Many ways around these safeguards already exist and there will be more ways to bypass these in the future. So if you’re really, really worried about AI use you may want to choose what I call the “nuclear option.” If you teach in person you can require students to write essays in person. 

This approach definitely works for preventing AI and is okay for short pieces, but for longer pieces, it has a lot of downsides. I would have trouble writing a long piece in this setting and imagine many students will as well. Additionally, this requirement could create an accusatory class atmosphere that is more focused on preventing AI use than actually teaching. It’s also not practical for online teaching. 

That all being said, given how common AI writing has become in education, I understand why some teachers will turn to this method. Hopefully, suggestions 1-7 will work but if AI-generated papers are still out of hand in your classroom, this is a blunt-force method that can work temporarily. 

Good luck and may your assignments be free of AI writing! 

• 7 Ways To Detect AI Writing Without Technology
• Best Free AI Detection Sites
• My Student Was Submitting AI Papers. Here's What I Did

The Pros and Cons of AI in Special Education

• Share article

Special education teachers fill out mountains of paperwork, customize lessons for students with a wide range of learning differences, and attend hours of bureaucratic meetings.

It’s easy to see why it would be tempting to outsource parts of that job to a robot.

While there may never be a special educator version of “Star Wars”’ protocol droid C-3PO, generative artificial tools—including ChatGPT and others developed with the large language models created by its founder, Open AI—can help special education teachers perform parts of their job more efficiently, allowing them to spend more time with their students, experts and educators say.

But those shortcuts come with plenty of cautions, they add.

Teachers need to review artificial intelligence’s suggestions carefully to ensure that they are right for specific students. Student data—including diagnoses of learning differences or cognitive disorders—need to be kept private.

Even special educators who have embraced the technology urge to proceed with care.

“I’m concerned about how AI is being presented right now to educators, that it’s this magical tool,” said Julie Tarasi, who teaches special education at Lakeview Middle School in the Park Hill school district near Kansas City, Mo. She recently completed a course in AI sponsored by the International Society for Technology in Education. “And I don’t think that the AI literacy aspect of it is necessarily being [shared] to the magnitude that it should be with teachers.”

Park Hill is cautiously experimenting with AI’s potential as a paperwork partner for educators and an assistive technology for some students in special education.

The district is on the vanguard. Only about 1 in 6 principals and district leaders—16 percent—said their schools or districts were piloting AI tools or using them in a limited manner with students in special education, according to a nationally representative EdWeek Research Center survey conducted in March and April.

AI tools may work best for teachers who already have a deep understanding of what works for students in special education, and of the tech itself, said Amanda Morin, a member of the advisory board for the learner-variability project at Digital Promise, a nonprofit organization that works on equity and technology issues in schools.

“If you feel really confident in your special education knowledge and experience and you have explored AI [in depth], I think those two can combine in a way that can really accelerate the way you serve students,” Morin said.

But “if you are a novice at either, it’s not going to serve your students well because you don’t know what you don’t know yet,” she added. “You may not even know if the tool is giving you a good answer.”

Here are some of the areas where Park Hill educators and other school and district leaders see AI’s promise for special education—and what caveats to look out for:

Promise: Reducing the paperwork burden.

Some special education teachers spend as many as eight hours a week writing student-behavior plans, progress reports, and other documentation.

“Inevitably, we’re gonna get stuck, we’re gonna struggle to word things,” Tarasi said. AI can be great for busting through writer’s block or finding a clearer, more objective way to describe a student’s behavior, she said.

What’s more, tools such as Magic School—an AI platform created for K-12 education—can help special education teachers craft the student learning goals that must be included in an individualized education program, or IEP.

“I can say ‘I need a reading goal to teach vowels and consonants to a student,’ and it will generate a goal,” said Tara Bachmann, Park Hill’s assistive-technology facilitator. “You can put the criteria you want in, but it makes it measurable, then my teachers can go in and insert the specifics about the student” without involving AI, Bachmann said.

These workarounds can cut the process of writing an IEP by up to 30 minutes, Bachmann said—giving teachers more time with students.

AI can also come to the rescue when a teacher needs to craft a polite, professional email to a parent after a stress-inducing encounter with their child.

Some Park Hill special education teachers use “Goblin,” a free tool aimed at helping neurodivergent people organize tasks, to take the “spice” out of those messages, Tarasi said.

A teacher could write “the most emotionally charged email. Then you hit a button called ‘formalize.’ And it makes it like incredibly professional,” Bachmann said. “Our teachers like it because they have a way to release the emotion but still communicate the message to the families.”

Caveat: Don’t share personally identifiable student information. Don’t blindly embrace AI’s suggestions.

Teachers must be extremely careful about privacy issues when using AI tools to write documents—from IEPs to emails—that contain sensitive student information, Tarasi said.

“If you wouldn’t put it on a billboard outside of the school, you should not be putting it into any sort of AI,” Tarasi said. “There’s no sense of guaranteed privacy.”

Tarasi advises her colleagues to “absolutely not put in names” when using generative AI to craft documents, she said. While including students’ approximate grade level may be OK in certain circumstances, inputting their exact age or mentioning a unique diagnosis is a no-no.

To be sure, if the information teachers put into AI is too vague, educators might not get accurate suggestions for their reports. That requires a balance.

“You need to be specific without being, without being pinpoint,” Tarasi said.

Caveat: AI works best for teachers who already understand special education

Another caution: Although AI tools can help teachers craft a report or customize a general education lesson for students in special education, teachers need to already have a deep understanding of their students to know whether to adopt its recommendations.

Relying solely on AI tools for lesson planning or writing reports “takes the individualized out of individualized education,” Morin said. “Because what [the technology] is doing is spitting out things that come up a lot” as opposed to carefully considering what’s best for a specific student, like a good teacher can.

Educators can tweak their prompts—the questions they ask AI—to get better, more specific advice, she added.

“A seasoned special educator would be able to say ‘So I have a student with ADHD, and they’re fidgety’ and get more individualized recommendations,” Morin said.

Promise: Making lessons more accessible.

Ensuring students in special education master the same course content as their peers can require teachers to spend hours simplifying the language of a text to an appropriate reading level.

Generative AI tools can accomplish that same task—often called “leveling a text"—in just minutes, said Josh Clark, the leader of the Landmark School , a private school in Massachusetts serving children with dyslexia and other language-based learning differences.

“If you have a class of 30 kids in 9th grade, and they’re all reading about photosynthesis, then for one particular child, you can customize [the] reading level without calling them out and without anybody else knowing and without you, the teacher, spending hours,” Clark said. “I think that’s a super powerful way of allowing kids to access information they may not be able to otherwise.”

Similarly, in Park Hill, Bachmann has used Canva—a design tool with a version specifically geared toward K-12 schools and therefore age-appropriate for many students—to help a student with cerebral palsy create the same kind of black-and-white art his classmates were making.

Kristen Ponce, the district’s speech and language pathologist, has used Canva to provide visuals for students in special education as they work to be more specific in their communication.

Case-in-point: One of Ponce’s students loves to learn about animals, but he has a very clear idea of what he’s looking for, she said. If the student just says “bear,” Canva will pull up a picture of, for instance, a brown grizzly. But the student may have been thinking of a polar bear.

That gives Ponce the opportunity to tell him, “We need to use more words to explain what you’re trying to say here,” she said. “We were able to move from ‘bear’ to ‘white bear on ice.’”

Caveat: It’s not always appropriate to use AI as an accessibility tool.

Not every AI tool can be used with every student. For instance, there are age restrictions for tools like ChatGPT, which isn’t for children under 13 or those under 18 without parent permission, Bachmann said. (ChatGPT does not independently verify a user’s age.)

“I caution my staff about introducing it to children who are too young and remembering that and that we try to focus on what therapists and teachers can do collectively to make life easier for [students],” she said.

“Accessibility is great,” she said. But when a teacher is thinking about “unleashing a child freely on AI, there is caution to it.”

Promise: Using AI tools to help students in special education communicate.

Park Hill is just beginning to use AI tools to help students in special education express their ideas.

One recent example: A student with a traumatic brain injury that affected her language abilities made thank you cards for several of her teachers using Canva.

“She was able to generate personal messages to people like the school nurses,” Bachmann said. “To her physical therapist who has taken her to all kinds of events outside in the community. She said, ‘You are my favorite therapist.’ She got very personal.”

There may be similar opportunities for AI to help students in special education write more effectively.

Some students with learning and thinking differences have trouble organizing their thoughts or getting their point across.

“When we ask a child to write, we’re actually asking them to do a whole lot of tasks at once,” Clark said. Aspects of writing that might seem relatively simple to a traditional learner—word retrieval, grammar, punctuation, spelling—can be a real roadblock for some students in special education, he said.

“It’s a huge distraction,” Clark said. The student may “have great ideas, but they have difficulty coming through.”

Caveat: Students may miss out on the critical-thinking skills writing builds.

Having students with language-processing differences use AI tools to better express themselves holds potential, but if it is not done carefully, students may miss developing key skills, said Digital Promise’s Morin.

AI “can be a really positive adaptive tool, but I think you have to be really structured about how you’re doing it,” she said.

ChatGPT or a similar tool may be able to help a student with dyslexia or a similar learning difference “create better writing, which I think is different than writing better,” Morin said.

Since it’s likely that students will be able to use those tools in the professional world, it makes sense that they begin using them in school, she said.

But the tools available now may not adequately explain the rationale behind the changes they make to a student’s work or help students express themselves more clearly in the future.

“The process is just as important as the outcome, especially with kids who learn differently, right?” Morin said. “Your process matters.”

Clark agreed on the need for moving cautiously. His own school is trying what he described as “isolated experiments” in using AI to help students with language-processing differences express themselves better.

The school is concentrating, for now, on older students preparing to enter college. Presumably, many will be able to use AI to complete some postsecondary assignments. “How do we make sure it’s an equal playing field?” Clark said.

A version of this article appeared in the May 22, 2024 edition of Education Week as The Pros and Cons of AI in Special Education

Sign Up for EdWeek Update

Edweek top school jobs.

Sign Up & Sign In

Android Police

What is textfx ai tools, textfx ai tools can be helpful for any creative writer.

If you're a creative writer, you've had writer's block more times than you care to admit. The TextFX Project is an artificial intelligence (AI) tool created by musician and producer Lupe Fiasco and Google Lab Sessions. It's a suite of writing tools made for rappers, writers, and wordsmiths. If you're ready to give writer's block a vacation, open a new browser window on your favorite Chromebook , PC, or mobile device and check it out.

What's the tech behind TextFX?

The TextFX Project is powered by Google's large language model, PaLM 2 (Pathways Language Model 2). Google put a lot of effort into scaling up large language models and has been a leader in that area since the early days of AI language models. Operating massive datasets and using advanced neural networks developed by Google Research have given Google an edge.

PaLM 2 can handle multiple language tasks simultaneously. This multitasking ability is part of what Google calls "pathways," a strategic approach to handling complex, large-scale AI model training. It uses a modular approach, where different components of the process can be specialized but still interact effectively with each other.

Top 8 best AI note-taking tools

Textfx has ten ai tools.

All the TextFX tools are offered for free on its website . When you visit the site, you're greeted with a split screen with its logo and a scrolling list of the tools. With names like Fuse, Unexpect, and Unfold , it can be confusing. The interesting thing about TextFX is that many of the tools are based on the techniques Lupe Fiasco developed in his approach to working with words creatively.

Lupo describes his approach as "word explosions." Taking a word and exploding the possibilities inherent in that word. One of the tools is Explode , which takes a single word and breaks it into similar-sounding phrases. You can peek under the hood by looking for the info icon under the tool names. Explode shows examples like taking the word "stabilize" and breaking it into "stable eyes."

What TextFX tools are available, and what can you use them for?

The rest of the tools take a similar approach to the Explode TextFX tool. You enter a word, thing, concept, topic, or scene, and the possibilities associated with that word unfold.

Some tools do exactly what they are named, and any wordsmith will be familiar with them. The Simile TextFX tool takes a thing or concept and outputs several possible similes. Take the word "waiting," and you get things like, "waiting for the train to arrive was like watching a pot of water boil." The Alliteration and Acronym tools in this AI toolset do the thing they are named for.

While Explode breaks a word into similar-sounding phrases, Chain gives you a series of semantically related items. Take a moment to think about what other words are related to the word "guitarist." A guitarist has an instrument that they play music with, possibly in a band . Maybe that guitarist is on tour on a bus , getting ready to play on stage . You get the idea. It's helpful when you know the subject you're writing about but want detailed ways to describe it or something related to it that you can elaborate on.

Unfold takes a word and pairs it with another word that's commonly found next to it. For example, when you type "guitarist," you get outputs like lead guitarist, rhythm guitarist, guitarist prodigy, and rock guitarist. Like most AI, you can get bad responses from these tools. We don't think "guitarist cat" is that common. But who knows? The internet loves cats.

Creating a detailed scene with TextFX is easy

The Scene tool can be helpful for any writer. A lot of writer's block comes when you can visualize or imagine something that's happening but can't describe it. "Watching a movie" is a simple scene. But what does watching a movie feel like? What sounds are around you? What can you see?

Scene outputs things that are familiar to most people and can viscerally draw you in. The feel of plush seats and sticky floors, the taste of cold soda and candy, the smell of fresh popcorn wafting through the theater, the darkness of the theater, and the sight of the bright lights illuminating the screen.

There's also the Unexpect tool. Enter a scene, and you get an unexpected and imaginative result. Want to make a scene where the character is "eating a doughnut" a little stranger? How about, your character is "eating a doughnut with chopsticks"? Or maybe they get a nasty surprise while "eating a doughnut filled with mashed potatoes"?

Explore topics from different perspectives with TextFX

The last few tools in TextFX help you explore a topic or concept in depth. POV is a tool that takes a topic and creates a list of what someone might think about that topic. If you want to explore "succulents," the output is in the form of "succulents are the..."

If you're familiar with succulents, you'll understand some of the points of view. Succulents are low-maintenance plants that thrive in dry environments, like cacti, jade plants, and dragon trees. TextFX outputs the common thoughts that "succulents are the perfect plants for people who don't know how to take care of plants" and "succulents are the perfect plants for people who don't have a green thumb."

On the other hand, some people may think that "succulents are the perfect plants for people who are too lazy to take care of real plants." Or have the hot take that "succulents are the hipster's pet rock." There aren't any detailed points of view, but it provides additional perspective to what you already think about a topic.

The Fuse tool is a bit more interesting. You can take any two things, and it outputs some commonality or intersection between them. It struggles when the fusion of the two things is a stretch, like succulents and guitarists. One output made a little sense and said, "Both succulents and guitarists can be associated with the idea of patience — succulents with their ability to thrive in harsh conditions, and guitarists with their ability to practice for hours on end to perfect their craft."

With so many AI tools focusing on recreating what artists produce , TextFX takes a refreshingly different approach. Instead of creating art similar to an artist or music similar to a musician, TextFX recreates a creative process.

One of the developers said that they thought this project would output writing similar to Lupe Fiasco's work. However, Lupe said the tools would be more helpful if they helped with his process. Now, we get a chance to create in a way that's similar to how Lupe creates. Let's hope the next generation of AI tools trends in this direction, making the lives of creatives easier so that they have more time to be creative.