Neev Parikh

Some examples of projects I'd be excited about are:

​

• Build an evaluation that measures how good models are at predicting which research directions/approaches are better than others, and how they compare to humans

​

• Try and compute a "Judge" horizon, which aims to understand the trend of models accurately verifying/predicting the score or result of an evaluation transcript. Can we plot the discriminator-generator gap and understand if the kinds of tasks that model N can verify are the kinds of tasks model N+1 gains the most improvement on?

​

• Models could likely use more inference time compute more effectively for a number of current benchmarks. Try to quantify and reduce this gap by applying existing techniques that target specific weaknesses or experiment with new techniques.

​

These project ideas are less well-scoped by me but I'd be interested and excited if scholars had clear ideas of what to work on here:

​

• Build better red-team attack policies for monitorability evaluations.

​

• Models are increasingly trained with synthetically generated data. Does this mean that models will be able to instinctively "underperform" (or sandbag) by behaving like the less-capable model that generated their training data without needing to reason about this?

​

• Understand if emergent misalignment behaves similarly when training via reinforcement learning versus fine-tuning (e.g. do you see more reward hacking if you RL models in CTF or reverse engineering environments?).

​

• Try to understand the relationship between compute and time-horizon trends. METR's time horizon measurements use release date on the x-axis instead of effective compute. Could we try and measure how effective compute relates with time-horizon?

​

• Try and understand how various recently proposed neuralese/latent reasoning architectures compare to the current open-source default reasoning paradigm and estimate how the alignment tax for visible-chain-of-thought has changed over time. This might look like compiling the presented results and analyzing trends but also might involve collecting new data by benchmarking the relevant models.