About this role
This role addresses the need to develop and lead the research efforts focused on pre-training safety within FAR.AI. The core mission is to ensure advanced AI systems are safe and beneficial by intervening at the source of model creation, rather than attempting post-hoc fixes. This involves tackling capability control—specifically removing harmful functionalities while preserving benign ones to mitigate misuse risks in areas like CBRN and cyber security, and reducing loss-of-control risks.
The Research Lead will be responsible for validating and scaling novel safety approaches. This includes driving the adoption of methods like pre-training data filtering at scale and exploring techniques such as gradient routing and unlearning. A key focus will be scaling methods like Deep Ignorance to handle very large models, requiring close collaboration with the red team to stress-test resulting models.
This position is ideal for an individual who can set the research direction, mentor technical staff, and remain hands-on in both research and implementation. The successful candidate will thrive in an autonomous environment focused on empirically grounded, scalable machine learning safety research, aiming to translate effective research into real-world adoption with developers and governments.
What you will do
- Direct the research and development efforts focused on pre-training safety and shaping model capabilities at their source.
- Lead the investigation into capability control strategies, such as removing harmful capabilities while retaining benign ones.
- Validate techniques like large-scale pre-training data filtering and explore methods such as gradient routing and unlearning.
- Scale existing methods, such as Deep Ignorance, to apply them effectively to models exceeding 100B parameters and 1T tokens.
- Partner with the red team to rigorously stress-test models resulting from safety interventions.
- Analyze the scalability of proposed safety methods when applied to frontier AI systems.
- Build and lead the research team, defining its direction and mentoring technical staff.
What you need
- Demonstrated experience in developing and leading research in AI safety and robustness is required.
- Proficiency in advanced machine learning techniques, particularly those related to model representation and training.
- Experience with adversarial robustness research or mechanistic interpretability techniques would be beneficial.
- Ability to translate promising research into minimum viable demonstrations and work with external stakeholders.
- Strong technical ability to write code and run complex experiments independently.
- Experience in scaling machine learning solutions to handle very large model sizes is a significant asset.
Nice to have
- Experience with data attribution methods
- Familiarity with gradient routing techniques
- Experience with mechanistic interpretability research
Benefits
- Salary range of $290K – $450K
Skills and tech stack
Questions candidates ask
What is the remote policy for this role?
The posting does not explicitly detail the remote work policy for this position.
What is the expected compensation range?
The salary for this Research Lead role ranges from $290K to $450K.
What is the technology stack involved?
The research directions involve methods like data attribution, gradient routing, and techniques for training to actively remove harmful capabilities.
What is the team structure like?
The Research Lead will build and lead the team, set its research direction, and mentor Members of Technical Staff to scale the vision.