SolidGoldMagikarp (plus, prompt generation)

Anomalous tokens: a mysterious failure mode for GPT (which reliably insulted Matthew).

We have found a set of anomalous tokens which result in a previously undocumented failure mode for GPT-2 and GPT-3 models. (The 'instruct' models “are particularly deranged” in this context, as janus has observed.)

Many of these tokens reliably break determinism in the OpenAI GPT-3 playground at temperature 0 (which theoretically shouldn't happen).