Neev Parikh

Potential projects examples include:

• Build an evaluation measuring how good models are at predicting which research directions/approaches are better than others, and how they compare to human predictors
• Try to compute a "Judge" horizon, which aims to understand the trend of models accurately verifying/predicting the score or result of an evaluation transcript. The goal would be to determine if we can plot the discriminator-generator gap and understand if the kinds of tasks that model N can verify are the kinds of tasks model N+1 gains the most improvement on.
• Models could likely use additional inference time compute more effectively for a number of current benchmarks. A potential approach is trying to quantify and reduce this gap by applying existing techniques that target specific weaknesses or experiment with new techniques.

​

These project ideas are less well-scoped by me, but I'd be interested and excited if fellows have clear ideas of what to work on here:

• Build better red-team attack policies for monitorability evaluations.
• Models are increasingly trained with synthetically generated data. Does this mean that models will be able to instinctively "underperform" (or sandbag) by behaving like the less-capable model that generated their training data without needing to reason about this?
• Understand if emergent misalignment behaves similarly with RL training versus fine-tuning (e.g. do you see more reward hacking if you RL models in CTF or reverse engineering environments?).
• Try to understand the relationship between compute and time-horizon trends. METR's time horizon measurements use release date on the x-axis instead of effective compute. Could we try and measure how effective compute relates with time-horizon?
• Try and understand how various recently proposed neuralese/latent reasoning architectures compare to the current open-source default reasoning paradigm and estimate how the alignment tax for visible-chain-of-thought has changed over time. This might look like compiling the presented results and analyzing trends but also might involve collecting new data by benchmarking the relevant models.