AI Detection tools (also known as AI writing detectors or AI content detectors) can identify when a text was partly or wholly created by artificial intelligence (AI) tools such as ChatGPT. AI detectors can be used to determine when a piece of writing is likely to have been produced by AI.
This is helpful, for instance, to teachers who want to verify that their students are writing their own work or moderators trying to eliminate fake product reviews and other spam content. However, these tools are still new and experimental and are generally regarded as unreliable. Below, we describe how they function, how trustworthy they are, and how they are applied.
“If You want to Remove AI Detection and Bypass AI Detectors Use Undetectable AI: It can do it in one click“
What Is AI Detection?
AI detection is a term that describes the process of determining whether a human or an artificial intelligence (AI) system wrote a piece of text. It uses classifiers trained on huge datasets of texts written by humans and AI systems on various topics. These classifiers use machine learning algorithms and natural language processing techniques to examine the text and give a confidence score that shows how probable it is that an AI system wrote the text.
Why Is AI Text Detection Important?
AI text detection is essential to ensure that information is trustworthy, and it’s vital in areas like search engine optimization (SEO), academia, and law. AI writing software tools are certainly useful and a must-have to be competitive. But they are also notoriously unreliable. So Google, schools, and clients want to ensure you aren’t just publishing unedited content without using your brain. Think about how bad it would be if people were allowed to:
- Without verifying facts, write about topics related to Your Money or Your Life (YMYL). • Publish journal articles where “peer-reviewed” has no meaning.
- Give generic AI-generated legal advice. There would be no more trust. You must use these tools because people usually need help spot the difference.
Now that we’ve covered why detection is important and what it entails – we’ll explain how it works behind the scenes. Tools that “predict” AI content are mainly based on analyzing the context to the left of the next word. Imagine the sentence, “The best part of my day is when I wake up for ___.”
Work is the most likely word in this example based on the 117 million data points the GPT-2 language model was trained on. The AI model will recall all of its training data and then find and analyze patterns in the context of the word set.
It might be known, for example, that the word “day” is often used before the words “best” and “part”. Based on these contexts, the algorithm will then estimate the probability of each word being the next predicted word. Based on training data, the word work had a 41% chance of happening (the highest probability among other words), so it predicted it.
Perplexity & Burstiness
Some other terms you might encounter when using an AI detector are perplexity and burstiness. Perplexity gauges the uncertainty of a text. In simpler terms, it gauges how frequently a piece of text might puzzle or ‘perplex’ a reader. AI models aim for low perplexity, trying to create content that reads fluently and logically. Human writing often has higher perplexity, with more inventive language choices and occasional mistakes. Burstiness, conversely, assesses the variation in sentence structure and length.
Temperature
A key thing to know when working with AI-generated text is the concept of temperature. Temperature probability is a measure of the randomness of predictions. If the temperature is low, a model will likely output the most accurate text, but it will be dull as it has a smaller range of variation. If the temperature probability is high, the generated text will be more varied, but there is also a higher risk of the model making grammar mistakes or uttering nonsense.
Consumer-facing AI text generation tools like Jasper and ChatGPT are on the safe side. Although ChatGPT responses have more variations than Jasper’s, they are still fairly consistent models. If you’re using an online tool to help write content, you’re working with pre-trained models (which generally use cautious temperature probabilities to avoid mass errors)
So, after doing this for a single sentence, continue with the rest of your text. If a sampled piece of text always chooses the most expected word throughout paragraphs, you’re almost surely working with artificially created text. Think about it personally – the best writers often use complex language and explain things in unexpected, inventive ways. Artificial writing doesn’t.
As language models become increasingly sophisticated, predicting AI based on the context of words will get much harder. The more data in a set, the more variation in generations.
But for now, you could use this pattern to examine large chunks of text. It’s very simple: How well can an AI model predictively recreate a given text example?
How reliable are AI detectors?
In our experience, AI detectors usually work well, especially with longer texts, but can easily fail if the AI output is made less predictable, modified or rephrased after being generated. Detectors can easily mistake human-written text as AI-generated if it happens to match the criteria (low perplexity and burstiness). Our research into the best AI detectors shows that no tool can offer complete accuracy; the highest accuracy we found was 84% in a premium tool or 68% in the best free tool.
These tools give a helpful indication of how probable it is that a text was AI-generated, but we recommend against using them as proof on their own. As language models evolve, detection tools will likely always have to compete with them.
Even the more confident providers usually concede that their tools can’t be used as conclusive evidence that a text is AI-generated, and universities don’t trust them much. Note: The tactics people might use to make AI writing more undetectable can make the text look dubious or unsuitable for its intended purpose.
For example, adding spelling mistakes or irrational word choices to a text will make it less likely to be detected by an AI detector. However, a text full of spelling mistakes and irrational word choices will not be graded poorly as academic writing.
AI detectors vs. plagiarism checkers
AI detectors and plagiarism checkers are both tools that universities may use to prevent academic dishonesty, but they have different methods and goals:
- AI detectors attempt to identify text that seems like it was created by an AI writing tool. They do this by evaluating specific features of the text (perplexity and burstiness)—not by matching it to a database.
- Plagiarism checkers attempt to identify text that is duplicated from another source. They do this by matching the text to a large database of previously published sources, student theses, and so on, and finding similarities—not by evaluating specific features of the text.
However, we’ve discovered that plagiarism checkers mark parts of AI-generated texts as plagiarism. This is because AI writing uses sources that it doesn’t reference. While it often creates original sentences, it may copy sentences directly from existing texts or at least very similar ones.
This is more likely to occur with common or general-knowledge topics and less likely with more specialized topics that have less written about them. Furthermore, as more AI-generated text shows up online, AI writing may become more prone to be marked as plagiarism—simply because other AI-generated texts with similar wording already exist on the same topic.
So, while plagiarism checkers are not meant to act as AI detectors, they may still detect AI writing as partly plagiarized in some cases. But they must be more efficient at identifying AI writing than an AI detector.
What are AI detectors used for?
AI detectors are designed for anyone who wants to verify whether an AI tool might have created a piece of text. Potential users include:
- Educators (teachers and university instructors) who want to ensure that their students’ work is original
- Publishers who want to guarantee that they only publish human-written content
- Recruiters who want to confirm that candidates’ cover letters are their own writing
- Social media moderators and others combating automated misinformation who want to spot AI-generated spam and fake news
- Web content writers who want to publish AI-generated content but are worried that it may rank lower in search engines if it is detected as AI writing
Because of doubts about their reliability, most users are hesitant to fully depend on AI detectors for now, but they are already becoming popular as a sign that a text was AI-generated when the user already had their doubts.
Detecting AI writing manually
Besides using AI detectors, you can also learn to recognize the distinctive features of AI writing yourself. It’s hard to do so reliably—human writing can sometimes look robotic, and AI writing tools are increasingly realistically human—but you can cultivate a good sense for it. The specific criteria that AI detectors use, low perplexity and business, are quite technical, but you can try to detect them manually by looking for text:
- That reads blandly, with little variation in sentence structure or length
- With expected, common word choices and few surprises
You can also use methods that AI detectors don’t by looking for:
- Excessively polite language: Chatbots like ChatGPT are designed to act as helpful assistants, so their language is courteous and formal by default—not casual.
- Hedging language: Look for a lack of strong, original statements and for a tendency to overuse vague hedging phrases: “It’s worth noting that …” “X is widely seen as …” “X is regarded as …” “Some may argue that …”
- Inconsistency in voice: If you know the typical writing style and voice of the person whose writing you’re checking (e.g., a student), you can usually tell when they submit something that reads very differently from how they usually write.
- Unsourced or wrongly cited claims: In the context of academic writing, it’s important to reference your sources. AI writing tools tend not to do this or wrongly (e.g., citing nonexistent or irrelevant sources).
- Logical errors: although increasingly fluent, AI writing may only sometimes be consistent in content. Look for points where the text contradicts itself, makes an unlikely statement, or presents incoherent arguments.
In general, a good way to enhance your skill in detecting text that may be AI-generated is to experiment with some AI writing tools, observe the types of texts they can create, and familiarize yourself with their way of writing.
Best Tools To Detect AI Writing
In addition to using math, there are grammatical and syntactical ways you can help spot if something was written with AI, but you could do that just from reading. So, how do you figure out what percentage chance the context of a word has?
Well, for starters, you could use a few online tools. We wrote a bigger article on how to spot AI content, but depending on what type of writing you’re checking, you could use CopyLeaks (free) or Originality (paid).
If you want to check academic, industry, or professional content (especially in bulk) – look into Copyleaks. It’s free and is better than most other tools out there.
Originality is a more thorough and complete choice if you want to check for marketing or copywriting material created using ChatGPT. Originality lets you check for AI plagiarism and gives you a percentage that it thinks a block of text was written with AI.
Resource:
