Can we detect AI automatically?

No. At least not reliably. As evidenced by Weber-Wulff et al and Scarfe P et al, text and digital media generated by AI cannot be accurately or rigorously detected. The outputs from generative AI are non-deterministic and change every time, and so automatically detecting the "fingerprint" of AI is largely impossible (despite what AI detection vendors would have you believe). To make the task even more difficult, many emerging tools now help users to "humanise" AI outputs, which remove the artificial quality of basic AI outputs and give them a more realistic human quality.

Moreover, there are plenty of people who might simply have a prose or design style that appears similar to generative AI platforms. So the risk of a false positive - of a student being wrongly accused of academic misconduct - is high.

Our position

Newcastle University does not recommend or support the use of automatic AI checkers. Our own tests prove they do not work well enough to depend upon, and they provide insufficient detail about how “scores” are generated and what they mean. Low detection rates and a high incidence of false positives in recent research has also confirmed our concerns, and so the AI detection features of our current platforms have been disabled.

Due to GDPR and data security concerns, you should avoid entering student submissions into any third-party AI detection tools yourself (e.g. GPTZero).

Spotting AI use by eye

Expecting markers to identify AI-generated text is also difficult in most scenarios, and this is not an expectation we should place on colleagues. However, we have a shared responsibility to ensure the academic integrity of our assessments, and so we all need to stay vigilant, be aware of obvious red flags, and respond to any suspicions of AI misuse by following the University’s academic misconduct procedures.