online assignment plagiarism checker project using python

DEV Community

Posted on Oct 12, 2020 • Updated on May 22, 2022

How to detect plagiarism in text using Python

In this tutorial , we're going to learn how to Make a Plagiarism Detector in Python using machine learning techniques such as word2vec and cosine similarity in just a few lines of code.

Once finished our plagiarism detector will be capable of loading a student’s assignment from files and then compute the similarity to determine if students copied each other.

Requirements

To be able to follow through this tutorial you need to have scikit-learn installed on your machine.

Installation

How do we analyze text.

We all know that computers can only understand 0s and 1s , and for us to perform some computation on textual data we need a way to convert the text into numbers .

Word embedding

The process of converting the textual data into an array of numbers is generally known as word embedding .

The vectorization of textual data to vectors is not a random process instead it follows certain algorithms resulting in words being represented as a position in space. we going to use scikit-learn built-in features to do this.

How do we detect similarity in documents?

Here we gonna use the basic concept of vector , dot product to determine how closely two texts are similar by computing the value of cosine similarity between vectors representations of student’s text assignments.

Also, you need to have sample text documents on the student’s assignments which we gonna use in testing our model.

The text files need to be in the same directory with your script with an extension of .txt , If you wanna use sample textfiles I used for this tutorial download here

The project directory should look like this

Let's now build our Plagiarism detector

• Let’s first import all necessary modules

we gonna use OS Module in loading paths of textfiles and then TfidfVectorizer to perform word embedding on our textual data and cosine similarity to compute the plagiarism .

• Reading all text files using List Comprehension

We are going to use concepts of a list comprehension to load all the path textfiles on our project directory as shown below.

• Lambda function to Vectorize & Compute Similarity

We need to create two lambda functions , one to convert the text to arrays of numbers and the other one to compute the similarity between them.

• Vectorize the Textual Data

adding the below two lines to vectorize the loaded student files.

• Creating a Function to Compute Similarity

Below is the main function of our script responsible for managing the whole process of computing the similarity among students.

When you compile down all the above concepts, you get the below full scripts ready to ** to detect plagiarism** among student's assignments.

Once you run the above app.py the out will look as shown below

Congratulations you have just made your own Plagiarism Detector in Python, Now share it with your fellow peers, press Tweet now to share it.

In case of any comment, suggestion, or difficulties drop it in the comment box below and I will get back to you ASAP.

The original article can be found at kalebujordan.dev

Kalebu / Plagiarism-checker-Python

A python project for checking plagiarism of documents based on cosine similarity, plagiarism-checker-python.

This repo consists of a source code of a python script to detect plagiarism in textual document using cosine similarity

How is it done?

You might be wondering on how plagiarism detection on textual data is done, well it aint that complicated as you may think.

We all all know that computer are good at numbers, so in order to compute the simlilarity between on two text documents, the textual raw data is transformed into vectors => arrays of numbers and then from that we are going to use a basic knowledge vector to compute the the similarity between them.

This repo consist of a basic example on how to do that.

Getting started

To get started with the code on this repo, you need to either clone or download this repo into your machine just as shown below;

Dependencies

Before you begin playing with the…

Top comments (0)

Templates let you quickly answer FAQs or store snippets for re-use.

Are you sure you want to hide this comment? It will become hidden in your post, but will still be visible via the comment's permalink .

Hide child comments as well

For further actions, you may consider blocking this person and/or reporting abuse

HanDiffuser: Text-to-Image Generation With Realistic Hand Appearances

Mike Young - Apr 23

PySpark： missing value

ChelseaLiu0822 - Apr 18

Computer Vision Meetup: Anomaly Detection with Anomalib and FiftyOne

Jimmy Guerrero - May 10

Leveraging Python's Built-In Decorator for Improved Performance

SK RAJIBUL - Apr 17

We're a place where coders share, stay up-to-date and grow their careers.

How to Build a Plagiarism Detector Using Python

Build your own copy checker tool and learn about the Difflib module’s powerful capabilities.

As digital content has grown in popularity, it’s become more important than ever to protect it from copying and misuse. A plagiarism detection tool can help teachers evaluate students' work, institutions check research papers, and writers detect theft of their intellectual property.

Building a plagiarism tool can help you understand sequence matching, file operations, and user interfaces. You’ll also explore natural language processing (NLP) techniques to enhance your application.

The Tkinter and Difflib Module

To build a plagiarism detector, you’ll use Tkinter and the Difflib module. Tkinter is a simple, cross-platform, library that you can use to create graphical user interfaces quickly.

The Difflib module is part of the Python standard library that provides classes and functions for comparing sequences like strings, lists, and files. With it, you can build programs like a text auto-corrector, a simplified version control system , or a text summarization tool.

You can find the entire source code building a plagiarism detector using Python in this GitHub repository .

Import the required modules. Define a method, load_file_or_display_contents() that takes entry and text_widget as arguments. This method will load a text file and display its contents in a text widget.

Use the get() method to extract the file path. If the user has not entered anything, use the askopenfilename() method to open a file dialog window to select the desired file for plagiarism check. If the user selects the file path, clear the previous entry, if any, from the start to the end and insert the path they selected.

Open the file in read mode and store the contents in the text variable. Clear the contents of the text_widget and insert the text you extracted earlier.

Define a method, compare_text() that you will use to compare two pieces of text and calculate their similarity percentage. Use Difflib's SequenceMatcher() class to compare sequences and determine similarity. Set the custom comparison function to None to use the default comparison, and pass the text that you want to compare.

Use the ratio method to get the similarity in a floating-point format that you can use to calculate the similarity percentage. Use the get_opcodes() method to retrieve a set of operations that you can use to highlight similar portions of text and return it along with the similarity percentage.

Define a method, show_similarity() . Use the get() method to extract the text from both the text boxes and pass them into the compare_text() function. Clear the contents of the textbox that will display the result and insert the similarity percentage. Remove the "same" tag from the previous highlighting (if any).

The get_opcode() method returns five tuples: the opcode string, the start index of the first sequence, the end index of the first sequence, the start index of the second sequence, and the end index of the second sequence.

The opcode string can be one of four possible values: replace, delete, insert, and equal. You will get replace when a portion of the text in both sequences is different, and someone replaced one portion with another. You will get delete when a portion of the text exists in the first sequence but not the second.

You get insert when a portion of the text is absent in the first sequence but present in the second. You get equal when the portions of the text are the same. Store all these values in appropriate variables. If the opcode string is equal , add the same tag to the text sequence.

Initialize the Tkinter root window. Set the title of the window and define a frame inside it. Organize the frame with appropriate padding in both directions. Define two labels to display Text 1 and Text 2 . Set the parent element it should reside in and the text it should display.

Define three textboxes, two for the texts you want to compare and one to display the result. Declare the parent element, the width, and the height, and set the wrap option to tk.WORD to ensure that the program wraps the words at the nearest boundary and does not break any word in between.

Define three buttons, two to load the files and one for comparison. Define the parent element, the text it should display, and the function it should execute when clicked. Create two entry widgets to input the file path and define the parent element along with its width.

Organize all these elements in rows and columns using the grid manager. Use pack to organize the compare_button and the text_textbox_diff . Add appropriate padding where necessary.

Highlight the text marked as same with a yellow background and red font color.

The mainloop() function tells Python to run the Tkinter event loop and listen for events until you close the window.

Put it all together and run the code to detect plagiarism.

Example Output of the Plagiarism Detector

When you run the program, it displays a window. On hitting the Load File 1 button, a file dialog opens and asks you to choose a file. On choosing a file, the program displays the contents inside the first text box. On entering the path and hitting Load File 2 , the program displays the contents in the second text box. On hitting the Compare button, you get the similarity as 100%, and it highlights the entire text for 100% similarity.

If you add another line to one of the textboxes and hit Compare , the program highlights the similar portion and leaves out the rest.

If there is little to no similarity, the program highlights some letters or words, but the similarity percentage is pretty low.

Using NLP for Plagiarism Detection

While Difflib is a powerful method for text comparison, it is sensitive to minor changes, has limited context understanding, and is often ineffective for large texts. You should consider exploring Natural Language Processing as it can perform semantic analysis of the text, extract meaningful features, and has contextual understanding.

Moreover, you can train your model for different languages and optimize it for efficiency. A few of the techniques that you can use for plagiarism detection include Jaccard similarity, cosine similarity, word embeddings, latent sequence analysis, and sequence-to-sequence models.

• Manage Email Subscriptions
• How to Post to DZone
• Article Submission Guidelines
• Manage My Drafts

Data Engineering: Work with DBs? Build data pipelines? Or maybe you're exploring AI-driven data capabilities? We want to hear your insights.

Modern API Management : Dive into APIs’ growing influence across domains, prevalent paradigms, microservices, the role AI plays, and more.

Programming in Python: Dive into the Python ecosystem to learn about popular libraries, tools, modules, and more.

PostgreSQL: Learn about the open-source RDBMS' advanced capabilities, core components, common commands and functions, and general DBA tasks.

• How I Converted Regular RDBMS Into Vector Database To Store Embeddings
• How to Build a Full-Stack App With Next.js, Prisma, Postgres, and Fastify
• 3 GPT-3 Tools for Developers, Software and DevOps Engineers, and SREs
• How to Build a Concurrent Chat App With Go and WebSockets
• Monte Carlo Simulations at Scale: Stock Price Prediction
• Snowflake Data Time Travel
• Advanced Linux Troubleshooting Techniques for Site Reliability Engineers
• How To Integrate ChatGPT (OpenAI) With Kubernetes
• Data Engineering

Build a Plagiarism Checker Using Machine Learning

Using machine learning, we can build our own plagiarism checker that searches a vast database for stolen content. in this article, we’ll do exactly that..

Join the DZone community and get the full member experience.

Plagiarism is rampant on the internet and in the classroom. With so much content out there, it’s sometimes hard to know when something has been plagiarized. Authors writing blog posts may want to check if someone has stolen their work and posted it elsewhere. Teachers may want to check students’ papers against other scholarly articles for copied work. News outlets may want to check if a content farm has stolen their news articles and claimed the content as its own.

So, how do we guard against plagiarism? Wouldn’t it be nice if we could have software do the heavy lifting for us? Using machine learning, we can build our own plagiarism checker that searches a vast database for stolen content. In this article, we’ll do exactly that.

We’ll build a Python Flask app that uses  Pinecone —a similarity search service—to find possibly plagiarized content.

Demo App Overview

Let’s take a look at the demo app we’ll be building today. Below, you can see a brief animation of the app in action.

The UI features a simple textarea input in which the user can paste the text from an article. When the user clicks the Submit button, this input is used to query a database of articles. Results and their match scores are then displayed to the user. To help reduce the amount of noise, the app also includes a slider input in which the user can specify a similarity threshold to only show extremely strong matches.

As you can see, when original content is used as the search input, the match scores for possibly plagiarized articles are relatively low. However, if we were to copy and paste the text from one of the articles in our database, the results for the plagiarized article come back with a 99.99% match!

So, how did we do it?

In building the app, we start with a dataset of news articles from Kaggle. This dataset contains 143,000 news articles from 15 major publications, but we're just using the first 20,000. (The full dataset that this one is derived from contains over two million articles!)

Next, we clean up the dataset by renaming a couple columns and dropping a few unnecessary ones. Then, we run the articles through an embedding model to create vector embeddings —that's metadata for machine learning algorithms to determine similarities between various inputs. We use the Average Word Embeddings Model . Finally, we insert these vector embeddings into a vector database managed by Pinecone.

With the vector embeddings added to the database and indexed, we’re ready to start finding similar content. When users submit their article text as input, a request is made to an API endpoint that uses Pinecone’s SDK to query the index of vector embeddings. The endpoint returns 10 similar articles that were possibly plagiarized and displays them in the app’s UI. That’s it! Simple enough, right?

If you’d like to try it out for yourself, you can find the code for this app on GitHub . The README contains instructions for how to run the app locally on your own machine.

Demo App Code Walkthrough

We’ve gone through the inner workings of the app, but how did we actually build it? As noted earlier, this is a Python Flask app that utilizes the Pinecone SDK. The HTML uses a template file, and the rest of the frontend is built using static CSS and JS assets. To keep things simple, all of the backend code is found in the app.py file, which we've reproduced in full below:

Let's go over the important parts of the app.py file so that we understand it.

On lines 1-14, we import our app's dependencies. Our app relies on the following:

dotenv for reading environment variables from the .env file

flask for the web application setup

json for working with JSON

os also for getting environment variables

pandas for working with the dataset

pinecone for working with the Pinecone SDK

re for working with regular expressions (RegEx)

requests for making API requests to download our dataset

statistics for some handy stats methods

sentence_transformers for our embedding model

swifter for working with the pandas dataframe

On line 16, we provide some boilerplate code to tell Flask the name of our app.

On lines 18-20, we define some constants that will be used in the app. These include the name of our Pinecone index, the file name of the dataset, and the number of rows to read from the CSV file.

On lines 22-25, our initialize_pinecone method gets our API key from the .env file and uses it to initialize Pinecone.

On lines 27-29, our delete_existing_pinecone_index method searches our Pinecone instance for indexes with the same name as the one we're using ("plagiarism-checker"). If an existing index is found, we delete it.

On lines 31-35, our create_pinecone_index method creates a new index using the name we chose ("plagiarism-checker"), the "cosine" proximity metric, and only one shard.

On lines 37-40, our create_model method uses the sentence_transformers library to work with the Average Word Embeddings Model. We’ll encode our vector embeddings using this model later.

On lines 62-68, our process_file method reads the CSV file and then calls the prepare_data and   upload_items methods on it. Those two methods are described next.

On lines 42-56, our prepare_data method adjusts the dataset by renaming the first “id” column and dropping the “date” column. It then combines the article title with the article content into a single field. We’ll use this combined field when creating the vector embeddings.

On lines 58-60, our upload_items method creates a vector embedding for each article by encoding it using our model. Then, we insert the vector embeddings into the Pinecone index.

On lines 70-74, our map_titles and map_publications methods create some dictionaries of the titles and publication names to make it easier to find articles by their IDs later.

Each of the methods we've described so far is called on lines 95-101 when the backend app is started. This work prepares us for the final step of actually querying the Pinecone index based on user input.

On lines 103-113, we define two routes for our app: one for the home page and one for the API endpoint. The home page serves up the index.html template file along with the JS and CSS assets, and the API endpoint provides the search functionality for querying the Pinecone index.

Finally, on lines 76-93, our query_pinecone method takes the user's article content input, converts it into a vector embedding, and then queries the Pinecone index to find similar articles. This method is called when the /api/search endpoint is hit, which occurs any time the user submits a new search query.

For the visual learners out there, here’s a diagram outlining how the app works:

Example Scenarios

So, putting this all together, what does the user experience look like? Let’s look at three scenarios: original content, an exact copy of plagiarized content, and “patch written” content.

When original content is submitted, the app responds with some possibly related articles, but the match scores are quite low. This is a good sign, as the content is not plagiarized, so we would expect low match scores.

When an exact copy of plagiarized content is submitted, the app responds with a nearly perfect match score for a single article. That’s because the content is identical. Nice find, plagiarism checker!

Now, for the third scenario, we should define what we mean by “patch written” content. Patch writing is a form of plagiarism in which someone copies and pastes stolen content but then attempts to mask the fact that they’ve plagiarized the work by changing some of the words here and there. If a sentence from the original article says, “He was overjoyed to find his lost dog,” someone might patch write the content to instead say, “He was happy to retrieve his missing dog.” This is somewhat different from paraphrasing because the main sentence structure of the content often stays the same throughout the entire plagiarized article.

Here’s the fun part: Our plagiarism checker does really well in identifying “patch written” content too! If you were to copy and paste one of the articles in the database and then change some words here and there, and maybe even delete a few sentences or paragraphs, the match score will still come back as a nearly perfect match! When I attempted this with a copied and pasted article that had a 99.99% match score, the “patch written” content still returned a 99.88% match score after my revisions!

Not too shabby! Our plagiarism checker looks like it’s working well.

Conclusion and Next Steps

We've now created a simple Python app to solve a real-world problem. Imitation may be the highest form of flattery, but no one likes having their work stolen. In a growing world of content, a plagiarism checker like this would be highly useful to authors and teachers alike.

This demo app does have some limitations, as it is just a demo after all. The database of articles loaded into our index only contains 20,000 articles from 15 major news publications. However, there are millions or even billions of articles and blog posts out there. A plagiarism checker like this is only useful if it is checking your input against all the places where your work may have been plagiarized. This app would be better if our index had more articles in it and if we were continuously adding to it.

Regardless, at this point we’ve demonstrated a solid proof of concept. Pinecone, as a managed similarity search service, did the heavy lifting for us when it came to the machine learning aspect. With it, we were able to build a useful application that utilizes natural language processing and semantic search fairly easily, and now we have peace of mind knowing our work isn’t being plagiarized.

Opinions expressed by DZone contributors are their own.

Partner Resources

• About DZone
• Send feedback
• Community research
• Advertise with DZone

CONTRIBUTE ON DZONE

• Become a Contributor
• Core Program
• Visit the Writers' Zone
• Terms of Service
• Privacy Policy
• 3343 Perimeter Hill Drive
• Nashville, TN 37211
• [email protected]

Let's be friends:

• Python Basics
• Interview Questions
• Python Quiz
• Popular Packages
• Python Projects
• Practice Python
• AI With Python
• Learn Python3
• Python Automation
• Python Web Dev
• DSA with Python
• Python OOPs
• Dictionaries

Simple Plagiarism Detector in Python

• Deleting Duplicate Files Using Python
• Python | Prefix extraction depending on size
• Measuring the Document Similarity in Python
• Python | Pandas Index.identical()
• How to Avoid Plagiarism While Using ChatGPT
• Using Set() in Python Pangram Checking
• 5 Best AI Tools for Plagiarism Detection 2024
• Python | Pandas TimedeltaIndex.identical
• Python | Pandas Index.copy()
• Python | Find dictionary matching value in list
• Python | Unique pairs in list
• Python - Detect loop in Dictionaries
• Text detection using Python
• Python | Test list element similarity
• Censor bad words in Python using better-profanity
• Plagiarism Detection using Python
• PyPhisher - Simple Python Tool for Phishing
• What is Plagiarism in Educational Context ?
• Top 10 Online Plagiarism Checker [FREE] Tools

In this article, we are going to make a simple plagiarism detector in Python .

What is Plagiarism?

Plagiarism is simply called cheating. When one person copies the work or idea of another person and uses that in their work by their name it is called plagiarism. For example, if someone writing an article on geeksforgeeks and he/she copied the content from another site or resource it is said to be plagiarized content.

Difflib Module

In Python, there are various built-in modules used for making different tasks easy and difflib module is one of them. This module provides different functions and classes by using which we can compare the data sets. In this article, we are going to use SequenceMatcher() function class of this module.

SequenceMatcher()

This function is available in difflib module in Python which is used to compare any two strings or files. Using this function we are going to determine the amount plagiarism in a string or file by comparing them with each other.

Syntax: SequenceMatcher(isjunk=None, x, y) Parameter: isjunk: Optional argument isjunk must be None. x, y: string variable or filename.

Example 1: Detecting Plagiarism in a string.

In this example, we are going to compare two strings to detect the plagiarism using SequenceMatcher() function. For that, we are storing two different strings in different variables and passing them as an argument in SequenceMatcher() function after converting the matched sequence into a ratio using ratio() function and then display the final result by converting it into an integer.

Example 2: Detecting Plagiarized Content of Text Files

In this example, we are going to detect plagiarized content by comparing two text files. For that we use file handling in Python to read text files after that comparing them to detect the plagiarism as we have done in the first example. 

Step 1: Create Two Text Files

First, we have to create two text files so we can check Plagiarized content from both files.

Step 2: Creating Plagiarism Detection in Python for Text Files

In this step what we do is open a text file and store the content of that file in variables file1 and file2 after that comparing them using SequenceMatcher() same as in the first example.

Please Login to comment...

Similar reads.

Improve your Coding Skills with Practice

What kind of Experience do you want to share?

Online Assignment Plagiarism Checker Project using Data Mining

Download project document/synopsis.

Plagiarism is defined as to take or theft some work and present it has one’s own work. This grammar and plagiarism checker system is used to analyze the plagiarism data. Plagiarism affects the education quality of the students and thereby reduce the economic status of the country. Plagiarism is done by paraphrased works and the similarities between keywords and verbatim overlaps, change of sentences from one form to other form, which could be identified using WordNet etc. This plagiarism detector measures the similar text that matches and detects plagiarism. Internet has changed the student’s life and also has changed their learning style. It allows the students to get deeper in the approach towards learning and making their task easier. Many methods are employed in detecting plagiarism. Usually plagiarism detection is done using text mining method. In this plagiarism checker software, user can register with their basic registration details and create a valid login id and password. By using login id and password, students can login into their personal accounts. After that students can upload assignment file, which will further divide into content and reference link. This web application will process the content, visit each reference link, and scan the content of that webpage to match the original content. Also, students can view the history of their previous documents. Students can also check the grammar mistakes on the content.

• The system designed to detect similarity among text documents calculates content similarity among specified documents.
• Checking for code also, that is structure-oriented detection.
• Implements WordNet, this will detect synonyms as well.
• Applying efficient string-matching algorithm, which will further reduce the time and increase the efficiency.
• This system is simple and easy to access.
• System will give accurate results based on the content provided.
• This system will generate results in very less time.

Disadvantages

• Active internet connection required.
• If user uploads an incorrect document, then the result won’t be accurate.

Related Posts

Android file finder and sorting.

File finder and sorting is a system developed for android phones that helps user in finding the…

A Commodity Search System For Online Shopping Using Web Mining

Download Project Document/Synopsis With the popularity of Internet and e-commerce, the number of shopping websites has rapidly…

Leave a Comment Cancel Reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

Navigation Menu

Search code, repositories, users, issues, pull requests..., provide feedback.

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly.

To see all available qualifiers, see our documentation .

• Notifications

A plagiarism-detector web application to detect similarity between students' assignment submissions which are in Python programming language

parshva45/Plagiarism-Detector

Folders and files, repository files navigation, plagiarism detector.

A web application developed for the detection of plagiarism in two submissions of code files.

Team Members:

• Praveen Kumar Singh
• Parshva Shah
• Namrata Bilurkar

In order to run this project on your local machine, follow the steps below.

• Open the terminal
• Perform $git clone https://github.com/parshva45/Plagiarism-Detector at the location where you want the project.
• $mvn spring-boot:run

Project URL: Plagiariam Detector Course staff Username: coursestaff Password: password

Student UserName : user1 Student password : user1

Final Presentations :

• Presentation Video : Youtube-Presentation
• Demo Video : Youtube-Demo-Link
• System Setup Video : Youtube-System-Setup-Link

Contributors 2

• Python 7.6%
• JavaScript 2.5%