About this role
FAR.AI is a non-profit AI research institute dedicated to ensuring advanced artificial intelligence is safe and beneficial. The institute focuses on facilitating breakthrough AI safety research, advancing the global understanding of AI risks and solutions, and coordinating a global response to these challenges. The organization has established a strong reputation through academic publications and involvement in high-profile AI safety events and collaborations with leading researchers.
The Applied White-Box Methods team is focused on developing, evaluating, and demonstrating methods that use model internals to enhance the safety of AI systems. The team addresses the limitations of current black-box monitoring by focusing on interpretability research that has realistic evaluations on actual applications. They aim to bridge the gap between theoretical exploration and practical deployment by stress-testing white-box methods against real-world tasks.
This role exists to contribute to this effort by developing techniques such as activation explainers, steering, and influence functions. The team seeks to move beyond early-stage curiosity-driven research toward actionable methods that can be deployed at frontier scale, focusing on areas like long-context agentic coding and mitigating failure modes like reward hacking.
What you will do
- Develop and evaluate methods that leverage model internals to improve the safety of AI systems.
- Investigate diverse AI safety applications of white-box methods, including control, evaluation awareness, and shaping training dynamics.
- Stress-test white-box methods on real-world tasks, such as long-context agentic coding, to gather feedback.
- Evaluate proposed methods against established baselines, including strong black-box methods and activation probes.
- Determine the viability of white-box methods for implementation at frontier scales.
- Focus efforts on creating realistic evaluations and defining Goodhart-resistant metrics for research automation.
What you need
- A strong foundation in machine learning and AI research is required.
- Experience with methods that use model internals to understand or intervene on model behavior is necessary.
- Familiarity with interpretability research, including techniques like activation explainers or influence functions, is beneficial.
- The candidate must be able to evaluate methods against established baselines to make informed decisions.
- Experience working on complex, large-scale AI models is important.
- A capacity to focus on practical deployment considerations for safety methods is needed.
Nice to have
- Experience with natural language autoencoders
- Familiarity with singular learning theory
Benefits
- $150K – $250K salary
Skills and tech stack
Questions candidates ask
What is the remote policy?
The posting does not specify the remote policy for this role.
What is the compensation range?
The salary for this role is listed between $150K and $250K.
What is the team like?
The team works on developing, evaluating, and demonstrating methods that leverage model internals to improve AI system safety. They collaborate with renowned researchers and run FAR.Labs.
What is the hiring process like?
The posting does not detail the specific hiring process steps.