A letter to schools regarding AI Detectors

Dear sir/madam,

I would like to clarify a few things regarding the questioning of academic integrity of students 'caught' by AI detectors.  To, effectively, label students as dishonest and not have academic integrity, is something that needs to be addressed for a variety of reasons that I will outline below:

First and foremost, AI plagiarism checkers of any kind do not work effectively. Whether a student use AI or not, these detectors have been proven to inaccurately detect content that is in fact human-generated. Turnitin for example, themselves, have actually admitted that data they provided over the accuracy of their own product in detecting AI content should not be relied upon. Interestingly they will not disclose what the amended accuracy rate is: https://www.turnitin.com/blog/ai-writing-detection-update-from-turnitins-chief-product-officer. Also on this issue, I urge you to read this academic research on Testing of Detection Tools for AI-Generated Text: 

"This paper exposes serious limitations of the state-of-the-art AI-generated text detection tools and their unsuitability for use as evidence of academic misconduct. Our findings do not confirm the claims presented by the systems. They too often present false positives and false negatives."   ( Testing of Detection Tools for AI-Generated Text )

I would also like to share this with you as an example of how a university has communicated with a newsletter announcement that their Teaching Center doesn’t endorse any generative AI detection tools: "the Teaching Center will disable the AI detection tool .... effective immediately"  ( Teaching Center doesn’t endorse any generative AI detection tools )

You should also read some of the academic research by Associate Professor Ethan Mollick, Stefen Bauschard  and others who are worth following in this area (some example posts below):

And this from Joanne Villis  as another good example

Secondly, even if you, your faculty or the school insist on using these unreliable tools for such purposes, sending an email to parents and students, despite their politeness, says a student has cheated, is not an approach that is appropriate or the best option. Even though you may not have known about AI detector deficiencies, you are surely aware that AI use is by no means a binary or easily definable issue. Grammarly uses AI, does that mean the students in your class working with that tool are also plagiarising?  Which then offers the following points of contention:

1. How does the school ensure that tutors, older siblings and parents have zero contribution to the same assessment task? 

2. What exactly does flagged by an AI detector mean?  Does the department or school have a threshold or certainty of the level of AI usage?

3. Do you have a school policy that states us of AI detectors and their thresholds? If there is none then students surely cannot be accused of something out of scope of this policy.

I appreciate that AI is causing major disruption in education and the subjects like English and HSIE are feeling the brunt of this. I recognise that there is an extra workload and pressures on teachers in these disciplines. However, if you are to persist with these ineffective tools and/or in dealing with situations where plagiarism is suspected, perhaps a different approach is needed. Possibly an approach of waiting until you speak to the student and have a conversation  about why you are querying their work, i.e. a sudden change in writing style, language, etc then proceed  to ask them questions that gives the student the chance to demonstrate understanding of what they had written and how they may have used AI to support them if it was even used at all.

Finally,   I do understand that schools are trying to deal with this new age of AI but there are a lot of guidelines stating that this is not the way to deal with it.   There is a new generative AI in schools framework that will help you from ministers that illustrates that accusing students is just unjust. It states that members of the school community that are impacted by AI tools must be actively informed about and have the opportunity to question the use of the outputs of the tools you are using, for example.

I think this is an opportunity for you to embrace the positive use of the new AI tools in school, within given parameters, and use them as an opportunity to rethink the way you assess and guide students to understand your subject but also realise that they can get support from tutors, parents, as well as AI.  Please do not rely on AI tools to ‘detect’ academic integrity. Hopefully I have illustrated their inaccuracy with the research above.  There are some great teachers out there who are talking about the use of AI in History such as Matt Esterman, as well as AI and student agency Dr Nick Jackson.

The links below provide you also with some commentary on these issues:   https://preview.mailerlite.io/preview/282063/emails/85539571521029259 https://www.oneusefulthing.org/p/the-homework-apocalypse

Any letters of concern can cause considerable upset. Generic accusations based on AI detectors are serious and the tools you are using to make such decisions are shown to be highly unreliable. 

As a parent, an ex-teacher and a person who has been involved in education all my career, I will always support the teachers and school. However you should really take the time to embrace this technology and use it as a way to re-engage, enthuse, and even help save some time planning. This technology can be used to inspire, and augment learning and bring new experiences to life as well as get rid of the traditional, mundane and ineffective processes that add to teachers workload.