AI Incident Database #65 · 2016-12-22
Reinforcement Learning Reward Functions in Video Games
OpenAI published a post about its findings when using Universe, a software for measuring and training AI agents to conduct reinforcement learning experiments, showing that the AI agent did not act in the way intended to complete a videogame.
- Risk class (MIT)
- AI pursuing its own goals in conflict with human goals or values
- Reports
- 1
- Developers and deployers named
- 1
On ShieldMap: OpenAI (named by AIID as a developer or deployer).
Reports
- Faulty Reward Functions in the Wild · blog.openai.com, 2016-12-22
Title and description from the AI Incident Database (incidentdatabase.ai), snapshot of 2026-11-16, shared under CC BY-SA 4.0. Selected incidents; descriptions shortened; harmed parties and report text left out. The description is AIID's summary of what was reported, not ShieldPi's finding.