How to determine if a text is written by artificial intelligence? – Salvation
Stéphane Bonvallet, a professor of handicapology at the Lyon faculty, asked his students to work on “The medical approach to disability in Europe.” He questions it when making copies. “It wasn’t copy and paste. But the copies were built the same way. says inside progress. “We found the same grammatical constructions there. The judgment was conducted in the same order, with the same qualities and the same flaws. Finally, they are all illustrated by a personal example from a grandparent…” If not for these copies “not normal”, teacher “I didn’t immediately understand what it was about. Obviously, it wasn’t plagiarism. [de site web]”. One student admits: half of the fourteen graduate students who took the course “ChatGPT used artificial intelligence.”
ChatGPT is a program derived from a “language model” called GPT-3 developed by OpenAI. A language model is derived from a statistical analysis of the distribution of words in pre-existing texts. During the analysis, the machine determines that certain words always precede other words, while others can be inserted: for example, in French, “un”, “le” and “ce” come before “chien”; The adjectives “big”, “small”, “big” are often between these words, and less often after. The power of the artificial intelligence (AI) behind ChatGPT is its ability to extrapolate very long texts in response to questions. The results are surprising, especially when it comes to producing sequences of computer code whose language is highly coded and “logical”, but the performance is quite impressive in everyday French.
In Lyon’s case, it was the homogeneity in the structure of the texts that alerted the teacher. In fact, despite offering a 100-shade answer to the same question, the machine still follows more or less the same recipe, inspired by the thousands of models available online.
Since the experiment was not formally prohibited, the teacher decided to grade the assignment. “From one copy to another it was worth between 10 and 12.5. So I gave 11.75 to seven students using ChatGPT. explains the district daily. Academics are not enough to win the palm. Because so far, for a given topic, this text-based AI is particularly good at synthesizing the most common ideas during learning.
the illusion of precision
A key feature of the language models used by intelligent AIs remains the “illusion of accuracy,” as observed in mid-December. MIT Technology Review, Specializing in AI issues. “The sentences they use seem correct – they use the right kind of words in the right order. But the AI doesn’t know what that means. These models […] have no clue as to what is right or wrong, and they confidently present information as truth, even when it isn’t…”
Several research groups around the world are working on creating software that can detect the stylistic “tics” of artificial intelligence. In a study published in 2020, a team of researchers from the University of Pennsylvania and the Google Brain Lab dedicated to “deep learning” observed that these texts abuse impersonal words and use only a few “rare” words (slang, strong language). …). Texts produced by AIs are also written… without typos.
Plagiarism detection software is already used by universities to identify “debts” from texts submitted online in student productions. Similar devices are being developed to recognize the unique “style” of AIs. Thus, algorithms are trained on detection with sets of texts written by humans and artificial intelligences (related to GPT, GPT-2 or other language models) and labeled as such, respectively. In short, AIs are required to audit AIs. Among these tools, we can mention the GPT-2 Output Detector, which was created within a project in 2019 and is very intuitive to use. “For responsible phased release of GPT-2” Ported by OpenAI. Meanwhile, the GPTrue or False Chrome browser extension developed by the Harvard team in collaboration with IBM, the Giant Language Model Test Room (GLTR), or the CTRL detector developed by software publisher Salesforce. Tools developed and evaluated primarily in the English-language corpus, some particularly effective… as long as the suggested text “completely produced by AI”, note from CheckNews Ganesh Jawahar is an artificial intelligence researcher at the University of British Columbia. This is no longer the case “If the generated text [par l’IA] then edited by students…” including using automated paraphrasing software.
“Cocktail of detection methods”
Called by various experts CheckNews In addition, they believe that the GPT-3 model is far ahead of detection tools. The difference between an unoriginal text written by an uninspired person and an essay signed by ChatGPT is very small… Irene Solaiman, a researcher at Hugging Face and former employee of OpenAI, unsurprisingly confirms to us that “accuracy [des logiciels de détection] decreases with [modèles de langage] stronger”, referring to the evaluation conducted at the end of 2019 in which he participated. However, he says “Pleasantly surprised that the GPT-2 Output Detector still works well enough for ChatGPT” (According to Julien Chaumond, co-founder of Hugging Face, “4.45 million unique users on this detection tool since December 1st”). In early January, Edward Tian, a computer science student at Princeton University, deployed a custom ChatGPT detector called GPTzero, the performance of which has not yet been formally evaluated.
Anyway, “there is no detection method or model [automatisé] It will not be 100% reliable” Judge Irene Solaiman. He is the reason “always recommends using a cocktail of detection methods rather than just one.” For example, note that ChatGPT has been trained extensively on large sets of texts. Therefore, it is wrong when it comes to remembering recent events. The second researcher of the University of British Columbia, Mohammad Abdul-Mageed, gave an interview CheckNews, note that “Because AI doesn’t know these recent events, it can make senseless – ‘Canada won the 2022 World Cup’ type of misjudgment.” Words ending grammatically correct by artificial intelligence can also show a profound ignorance of the physical reality of our world (referring to unimaginable orders of magnitude for distances, durations). Finally, Solaiman points out, because models create texts by extrapolating which terms are more relevant to others. “they can sometimes get stuck in a loop”, creates questionable repetitions and redundancies.
“Breach of Academic Integrity”
Recently, the company OpenAI confirmed that it is working in secret to alter long sequences of AI-generated text to create a signature – and therefore proof – of the text’s origin. For example, this might involve forcing the machine to complete one of ten sentences ending with the same letter, or one of twenty sentences starting with the same letter. Something harmless in appearance, invisible to the reader, but perfectly incriminating.
Access to ChatGPT has been banned from New York schools since early 2023. “Many educational organizations have begun to discuss the use of ChatGPT for student work” Muhammad Abdul-Maged continues. “This includes our own university, other universities such as UBC and the University of Washington. At this stage, universities seem to be trying to take a balanced approach. For example, some universities are encouraging their professors to begin incorporating information about using AI tools to complete or automate assignment responses into their curricula. There is also a positive trend that universities are encouraging teachers to integrate knowledge of these tools into curricula, etc.”
For Irene Solaiman, this seems clear “If the teacher has not agreed to read or grade the essay created with the language model,” looks like a situatione “violation of academic integrity”. If its use is transparent, the tool can find its place in the university: “There is an interesting debate about whether a ‘big language model’ can and should be used as a tool or ‘author’ in publishing.” he explains. Nevertheless, some AI productions will still be forced “to be controlled by man”, and “Will require human approval” – especially those that belong “medical and mental or physical health”. “These are clearly high-risk areas because they directly affect human well-being.” Suleiman insists.
“Critical critical thinking”
“One of the most useful things Judge Muhammad Abdul-Maged, in my opinion, to educate users about the issue of misinformation and to be critical of what we read online. It may be difficult, but it is possible. In any case, even if tools to detect ChatGPT lineages are developed and effective, there will likely be new examples to present. Therefore, this situation should not change in the near future and our critical thinking as humans will remain important.
In review school of letters, Marie-Astrid Clair, a professor of literature at a Parisian institution, offers an interesting insight into the emergence of ChatGPT in the educational sphere. “The existence of such a tool will perhaps reduce the share of homework that our students and, as you know, ladies and gentlemen, often miss in the evening or on the weekend. – your children’s artificial intelligence. Such a machine will perhaps further develop the place given to the spoken word, creativity, brevity. This will perhaps give the French teachers a momentary illusion that, no, the standard has not dropped despite the fewer hours. This may be the case if the student discovers the unique dictionary of synonyms Online from CNRS / CNRTL and then have fun customizing everything. It will also force the learner to concentrate on his own idea expressed in adapted words rather than on the technique…”
