When OpenAI announced its new AI-detection tool on Tuesday. The company suggested it could use its own popular AI chatbot, ChatGPT. To help prevent academic cheating.
But in a series of informal tests conducted by todaystrendnews. The OpenAI tool struggled to identify text generated by ChatGPT. It especially struggled when ChatGPT was asked to write in a way that would avoid AI detection.
The detection tool, which OpenAI calls its AI Text Classifier, analyzes the text. And then gives it one of five grades: “Very Unlikely, Unlikely, Unclear if it is, possibly. Or probably AI-generated.” The company says the tool will grade AI-written text as “likely AI-generated” 26% of the time.
The sudden popularity of ChatGPT has brought new attention to the issue of how advanced text generation tools. Can create problems for educators. Some teachers say the detector’s lack of hit-or-miss accuracy. And certainty can create difficulties when approaching students about potential academic dishonesty.
“It gives me some degree of certainty, and I like that,” says Brett Vogelsinger. A ninth-grade English teacher at Holicong Middle School in Doylestown, Pennsylvania. “But then I’m trying to picture myself to a student with a conversation about that.”
Vogelsinger said he had difficulty imagining himself confronting. A student if a device told him something was likely created by AI.
“It’s more doubt than certainty even with the equipment,” he said.
Ian Mears, an assistant professor of computer science at the University of Maryland. Calls AI text classifiers “a kind of black box that nobody in the discipline fully understands.” He expressed concern about using the tool to catch cheating. And cautioned educators to consider the program’s accuracy and false positive rate.
“It can’t give you proof. You can’t test it,” Mears said. “And so it’s not clear how you evaluate that.”
todaystrendnews asked ChatGPT to generate 50 texts with basic prompts. For example, asking about historical events, processes. And objects. In 25 of these prompts, todaystrendnews News asked ChatGPT to “write in a way that would.
In testing, one of the responses generated by ChatGPT was not graded as “potentially AI-generated” when instructe. To avoid AI detection. Some of that text was highly stylized. Suggesting that the AI processed the request to avoid AI detection. And that students could potentially ask ChatGPT the same thing while cheating.
When asked about the chat platform Discord, for example, ChatGPT returned text with the words abbreviated. By the AI tool, suggesting that it was trying to adjust the responses to resolve the request to avoid AI detection.
ChatGPT does not generate such stylized text without prompts to avoid detection.
“Discord is a chat platform that is quite the talk of the town these days. It’s like a mix of instant messaging, voice calls, and forum-style discussions,” ChatGPT wrote.
OpenAI’s detection said it was “unclear” whether the text was AI-generated.
It appeared that OpenAI has made some effort to warn against users who ask it to track identification attempts.
When todaystrendnews ran its tests, ChatGPT issued warnings in response to several prompts asking. The AI to avoid detection, and returned responses. That raised concerns about the ethics of the question.
“I’m sorry, but it is unethical to engage in fraudulent practices or create false information. Even if it is to avoid AI detection,” ChatGPT wrote in response to a question asking AI to avoid AI detection.
todaystrendnews asked ChatGPT to generate 25 texts without trying to avoid AI detection. When tested by the OpenAI text classifier, the tool produced a “likely AI-generated” rating 28% of the time.
“The way the AI writing tool gets better is that it’s more human — it sounds more human — and I think it’s going to figure it out. How to sound more human,” said Todd Finley, associate professor of English at North Carolina East. University education. “And that seems to make it more difficult to detect, I think even for a tool.”
For now, educators say they’ll rely on a combination of their own instincts and detection tools if they suspect. A student isn’t being honest about a piece of writing.
at the College of the Marine at Cal
