UTC

Model extraction and training data attacks

Querying a model to copy its behaviour, or to recover data it was trained on.

How it works
  1. 1A model's answers carry information about its weights and training data.
  2. 2Enough queries can train a copy, or surface memorised training text such as personal data or code.
What to watch for
  • High query volume with systematic variation
  • Outputs reproducing training data
How to defend
  • Rate-limit and monitor API use.
  • Keep personal and secret data out of training sets, or remove it first.
  • Watch outputs for memorised data.