The recent revelation that a report supporting Australia's teen social media ban may have contained AI-generated hallucinations has sparked intense debate and scrutiny. This incident highlights the potential risks and ethical dilemmas associated with the use of artificial intelligence in research and reporting, particularly in sensitive areas like government policy-making.
The Age Assurance Technology Trial, a $3.48 million initiative by the UK-based Age Check Certification Scheme (ACCS), aimed to test technologies for age verification on social media platforms. The report, which paved the way for Australia's under-16 social media ban, faced criticism for potential AI-generated errors in citations. The Senate inquiry's submission revealed that some citations appeared to be AI-hallucinated, rather than based on real sources.
Initially, ACCS denied using AI, but later admitted to using it to rewrite paragraphs more succinctly. However, further analysis by Guardian Australia identified six references with errors, including non-existent Digital Object Identifiers (DOIs) and incorrect citations. The ACCS's response to these errors was also problematic, as it included incorrect journal articles and citations.
The use of AI in government reports is not without precedent. Deloitte, a consulting firm, faced a similar backlash last year for using AI in a report for the Australian government, resulting in a partial refund. This incident underscores the need for transparency and accountability in AI usage, especially in government-contracted work.
The implications of AI-generated errors in government submissions are far-reaching. False citations can lead to bad decisions and erode public trust in institutions. As Prof. Christian Downie suggests, contractual arrangements and penalties may be necessary to discourage such practices. The incident also raises questions about the role of AI in research and the potential for bias or misinformation.
The ACCS's use of AI, while not for originating research, still raises concerns about the potential for AI to influence the content of reports. The metadata of ChatGPT in the links suggests that AI was involved in the editing process. This highlights the need for rigorous fact-checking and transparency in AI-assisted reporting.
In conclusion, the ACCS's report raises important questions about the ethical use of AI in research and the potential consequences of AI-generated errors. It serves as a reminder that AI, while powerful, must be used responsibly and ethically, especially in contexts where public trust and decision-making are at stake.
