Do AI detectors work? What the studies say
In Italy «ai detector» is searched on Google about 90,500 times a month, «ai detector italiano» 3,600 and «rilevatore ia» (AI detector, in Italian) 2,400. These are Google Keyword Planner figures for the last year, and behind many of these searches I imagine a precise scene. A teacher has in front of them an assignment that looks too polished, and is looking for a tool that tells them whether the student wrote it or a chatbot did.
There is, however, another number worth looking at. «ai humanizer», the programs that rewrite a text to make it look human and get past detectors, has 8,100 searches a month. On one side there are people trying to find out, on the other people trying to hide. Students are in the middle, and only one question matters. How far can these tools be trusted when a grade, or an accusation, is at stake?
How a detector reasons
A detector does not find a hidden signature in the text. It estimates a probability, comparing the text with the statistical features typical of language models' writing, for example very predictable words and very regular sentences. The result is a hypothesis, not proof.
The makers say so themselves. Turnitin, a service widely used in universities, writes in its own guide that the AI writing score should not be used as the sole basis for adverse actions against a student. That is why, in my view, before looking at scores it is worth looking at what independent studies say.
What independent studies say
The best-known case is OpenAI's. In January 2023 the company behind ChatGPT released its own text classifier. In its tests it identified only 26% of AI-written texts as «likely AI-written», and mistook 9% of human-written texts for AI. On 20 July 2023 OpenAI withdrew it, explaining that its accuracy was too low.
The same year a group of European researchers led by Debora Weber-Wulff tested 14 tools, including Turnitin, on six types of text. None reached 80% accuracy. The picture changes a lot depending on the text.

Average accuracy of 14 detectors. The range goes from the strictest to the most favourable of the three calculation methods used in the study. Source, Weber-Wulff and colleagues, International Journal for Educational Integrity (2023).
With texts written by people the tools make few mistakes. With AI-generated texts that are then edited by hand, or run through a paraphrasing program, most slip through. The authors' conclusion is clear-cut, the tools tested are neither accurate nor reliable and should not be used in academic settings.
A year later, in 2024, Mike Perkins and colleagues checked how easy they are to get around. On 805 texts, detectors identified unedited AI texts only 39.5% of the time, and with simple techniques such as a few spelling mistakes or sentences of more varied length accuracy fell to 22.1%.
The tools are improving, though. A study from the Vrije Universiteit Brussel published in June 2026, comparing GPTZero, Pangram, Copyleaks and Turnitin on 160 documents, found rare false positives on texts written entirely by people. Detection got worse on mixed texts and «humanised» ones, and these authors too conclude that the tools can give a first signal but should never be used as sole evidence in high-stakes decisions.
Who is most at risk, people writing in a second language
There is one finding that, in my view, every teacher should know. In 2023 a group at Stanford University had seven detectors analyse essays written by non-native speakers for the TOEFL, the test of English as a foreign language, and essays by US eighth-grade students.

Essays written by people, analysed by seven detectors. Source, Liang and colleagues, Patterns (2023).
On the native speakers' essays the detectors were almost perfect. On the non-native speakers' essays, instead, on average 61.3% were mistaken for AI-generated text, and 97.8% were flagged by at least one of the seven tools. The reason lies in the way they reason. People writing in a language they have not mastered use simpler words and more predictable sentences, the very features detectors associate with AI.
The study concerns English, and I found no equivalent data for Italian. In a class with students who learned the language only a few years ago, though, this result should give pause before trusting a score.
A small error, on many assignments
Turnitin states a false positive rate below 1% at document level, for documents with at least 20% AI writing, and around 4% at the level of individual sentences. These look like small numbers, but they have to be multiplied by the number of assignments.
Vanderbilt University, in the United States, did the maths in August 2023. In 2022 it had received about 75,000 papers, so even a 1% error rate would have meant about 750 students wrongly flagged. That is why it switched off Turnitin's detector, writing that it would keep it off for the foreseeable future.
The consequences can end up in court. On 28 January 2026 the Supreme Court of the State of New York for Nassau County annulled the sanction Adelphi University had imposed on a student over an essay to which Turnitin had given an AI score of 100%, and ordered it removed from the student's record. The court criticised the way the university had reached its decision, without ruling on the reliability of detectors.
What about digital watermarks?
There is another route, the watermark. The model's maker inserts an invisible statistical signal into the text, which can then be checked. Google described its system, SynthID, in Nature in 2024 and released it as open source. The authors themselves write, however, that the watermark weakens when the text is edited or paraphrased, and that it only works on models that adopt it.
European rules point in the same direction. Article 50 of the AI Act requires providers of systems that generate text, images, audio and video to mark the content in a machine-readable format, and in June 2026 the European Commission published the final version of a voluntary code of practice on how to do it. It is a useful step, but it does not solve the case of a student who rewrites a generated text in their own words.
What to do in class
The Italian Ministry of Education and Merit's guidelines on introducing AI in schools, adopted by decree 166 of 9 August 2025, say nothing about detectors. They do, however, ask that human involvement remain central and irreplaceable, especially in situations that affect students. Starting from there, and from the studies above, I suggest a few practical habits.
- Never use a score as proof. At most it is a reason to talk to the student, not to sanction them.
- Assess the process, not just the product. Drafts, notes, successive versions and a short conversation about the text say far more than any detector.
- Set the rules beforehand. If it is clear when AI may be used and how its use must be declared, there is less to argue about afterwards.
- Be careful with people writing in a second language, because that is where detectors get it wrong most often.
- Think about students' data. Uploading an assignment to an online service means handing a third party a text that may contain the name, surname and personal information of a minor. If you really need to, remove the personal data first, for example with the Anonimizzatore IA (AI Anonymiser) that we distribute free of charge for macOS and Windows. Data is recognised on your computer and documents stay there, and the result must always be checked. I also wrote about it in the guide on how to anonymise texts.
If you want to receive updates like this one, create your free account on retoria.it (the form is in Italian). If instead you want to work on these topics with your school's teaching staff, starting from real cases, write to us.
Methodological note
I read in full the studies by Weber-Wulff and colleagues (in the July 2023 preprint, whose abstract matches the published one), Liang and colleagues, Perkins and colleagues (March 2024 preprint, the published version refers to six detectors and confirms the 17.4-point drop) and Van Vlasselaer and colleagues, as well as the official pages of OpenAI, Turnitin and Vanderbilt, the ruling in the Adelphi case and the Ministry's guidelines. Some pages block automated reading, and in those cases I consulted the copy kept by the Internet Archive.
In the first chart accuracy is the average of the 14 tools for each type of text. The study calculates it in three different ways, and the range goes from the lowest value to the highest. Unedited AI texts combine the study's two types of generated text.
I do not report the accuracy figures claimed by detector makers, because they have not been independently verified. I found no decisions by Italian courts on this topic. Search volumes come from Google Keyword Planner for Italy, from October 2025 to September 2026. This article is not legal advice.
Sources
- Weber-Wulff D. and colleagues, «Testing of detection tools for AI-generated text», International Journal for Educational Integrity 19, 26 (2023), DOI 10.1007/s40979-023-00146-z, and preprint on arXiv.
- Liang W., Yuksekgonul M., Mao Y., Wu E., Zou J., «GPT detectors are biased against non-native English writers», Patterns 4(7) (2023), DOI 10.1016/j.patter.2023.100779.
- Perkins M. and colleagues, «Simple techniques to bypass GenAI text detectors», International Journal of Educational Technology in Higher Education 21, 53 (2024), DOI 10.1186/s41239-024-00487-w.
- Van Vlasselaer, Van Droogenbroeck, Spruyt, «Who wrote this? Evaluating the reliability of AI detection tools in higher education», International Journal for Educational Integrity 22, 16 (2026), DOI 10.1007/s40979-026-00226-w.
- OpenAI, text classifier page (31 January 2023, with the note on its withdrawal on 20 July 2023).
- Turnitin, article on sentence-level false positives (14 June 2023) and notes on the detection model.
- Vanderbilt University, announcement on disabling Turnitin's detector (16 August 2023).
- Supreme Court of the State of New York, Nassau County, Newby v. Adelphi University, Index No. 615397/25 (28 January 2026).
- Dathathri S. and colleagues, «Scalable watermarking for identifying large language model outputs», Nature 634 (2024), DOI 10.1038/s41586-024-08025-4.
- Regulation (EU) 2024/1689 on artificial intelligence, text on EUR-Lex, Article 50, and European Commission, code of practice on transparency of AI-generated content.
- Italian Ministry of Education and Merit, decree 166 of 9 August 2025 and guidelines on AI in schools.
- Google, Keyword Planner, average monthly search volumes in Italy (October 2025, September 2026), checked on 22 October 2026.