AI Incident Database #807 · 2024-09-25
ChatGPT Reportedly Introduces Errors in Critical Child Protection Court Report
A child protection worker in Victoria, Australia reportedly used ChatGPT to draft a report submitted to the Children's Court. The purportedly AI-generated report contained inaccuracies and downplayed risks to the child, allegedly resulting in a privacy breach when sensitive information was shared with OpenAI.
- Risk class (MIT)
- Compromise of privacy by obtaining, leaking or correctly inferring sensitive information
- Reports
- 9
- Developers and deployers named
- 4
On ShieldMap: OpenAI (named by AIID as a developer or deployer).
Reports
- Australian Information Commissioner Halts GenAI Use for Child Protection Agency as ChatGPT Downplays Risk · medianama.com, 2024-10-01
- AI ban ordered after child protection worker used ChatGPT in Victorian court case · theguardian.com, 2024-09-26
- Victorian child protection worker uses ChatGPT for protection report · cyberdaily.au, 2024-09-26
- Victoria's child protection agency bans AI use after report debacle · newsbytesapp.com, 2024-09-26
- Vic case worker used ChatGPT to draft child protection report · itnews.com.au, 2024-09-25
Title and description from the AI Incident Database (incidentdatabase.ai), snapshot of 2026-11-16, shared under CC BY-SA 4.0. Selected incidents; descriptions shortened; harmed parties and report text left out. The description is AIID's summary of what was reported, not ShieldPi's finding.