ChatGPT removes the official detection tool, admits that AI text cannot be identified

Source: "Qubit" (ID: QbitAI), author: Mengchen

Without an announcement, OpenAI quietly closed the AI text detection tool, and the page directly 404.

There was no official response to this, and it took a few days before someone found a short description hidden in the blog page of the tool released half a year ago.

...AI detector is no longer available due to low accuracy... We are working on more efficient text detection techniques and are committed to developing audio and visual detection techniques.

Many netizens are dissatisfied with the behavior of CloseAIOpenAI killing a product without issuing an announcement.

But there are many people who agree with turning off the AI detector itself, especially the student group.

Too many teachers believe that this thing is effective, and a large number of wronged students cheat with AI, which has become a witch hunt.

The accuracy rate is almost the same as guessing

How low is the accuracy of this official testing tool?

The data given by OpenAI itself can only correctly identify 26% of AI-generated texts, and wronged 9% of human-written texts.

When it was first released in January, it was summed up by netizens as "almost like guessing."

In addition, some people have done experiments and found that various detection tools on the market will judge that historical texts such as the Bible and the U.S. Constitution may be written by AI. The historical figures who cannot write these contents are time travelers, right?

But there are still many teachers who try to check students' work with various detection methods.

In one of the most famous cases, a professor at Texas A&M University almost judged half of his class to be late.

The latest trend is for teachers to ask students to write assignments in online documents and check edit records, but clever students can find ways around it too.

Finally, in the field of education, some people have suggested that large assignments/course papers may become history, and closed-book exams or oral exams will become mainstream.

Current detection methods can be circumvented

Netizens pointed out that it is contradictory for OpenAI to develop generation and detection tools at the same time.

If one side is doing well, the other side is not doing well, and there may be a conflict of interest.

However, the detection methods developed by third parties are not reliable.

The earliest known as "ChatGPT nemesis" is GPTZero developed by Princeton undergraduate Edward Tian, which uses complexity and changes in length and length of sentences to measure whether an article is generated by AI.

At that time, the GPTZero project was specially created for educators, and teachers could throw the homework of the whole class into it for testing.

But in July, the author admitted that he had given up the direction of detecting students' cheating. It is planned that the next version of GPTZero will no longer detect whether the text is generated by AI, but highlight the most human-like part.

Another detection method that has received a lot of attention is the watermarking method developed by the University of Maryland, which requires large models to hide marks when generating text and use statistical methods to identify them.

But the way to circumvent it is also very simple. For example, someone has developed a simple synonym replacement tool that can destroy statistical features.

Some people even began to wonder why people have to distinguish this.

Just like whether the numbers are calculated by humans or completed by computers, no one has cared for a long time.

Doesn't anyone care whether the speaker's manuscript is written by himself or by the secretary?

Human behavior research, using AI as a subject

The inability to distinguish between AI and human content does not seem to be all bad.

There are already psychological experiments that use AI instead of human subjects to accelerate research.

An article in the Cell sub-journal pointed out that in well-designed experimental scenarios, the responses of ChatGPT were correlated with the responses of about 95% of human participants.

And machine subjects don't tire, allowing scientists to collect data and test theories about human behavior at unprecedented speed.

In a recent opinion piece in the main journal of Science, Igor Grossman, a professor of psychology at the University of Waterloo, believes:

"AI could be a game-changer for social science research, where careful bias management and data fidelity are key."

### Reference link

[1]

[2]

[3]

[4]

[5]

[6]