About this role
Investigating emerging model failures, defining correct behaviors, and building safeguards to validate frontier AI systems occupies the daily focus of the Model Policy Manager. Operating inside the Safety Systems organization, the group works to ensure that increasingly capable models behave safely and reliably when interacting with tools, data, and external environments.
Bridging the gap between theoretical alignment research and the practical realities of training and deploying frontier models forms the core of the position. Practitioners analyze extended trajectories where models might persist toward wrong objectives, take shortcuts, or circumvent constraints, turning those findings into concrete evaluation criteria and monitoring frameworks.
Success in the group requires an adversarial mindset, technical fluency to work directly with training and evaluation data, and the ability to balance complex trade-offs between safety, utility, and business risk across cross-functional partners.
What you will do
- Identify vulnerabilities that emerge as models interact with tools, data, and external systems to translate them into robust safeguards.
- Develop threat models and empirical frameworks for understanding harmful outcomes from misaligned behavior.
- Turn findings into policy frameworks, evaluation criteria, online measurement tools, and safety guardrails.
- Build human data campaigns and gold sets to ground the measurement and evaluation of emerging behaviors and risks.
- Partner with research, engineering, security, and product teams to balance difficult trade-offs between safety, utility, and business risk.
- Inform deployment decisions, system cards, safeguards reports, and the broader approach to agentic safety.
- Build monitoring approaches that detect regressions and emerging risks after deployment.
What you need
- Bring a strong background in AI agent safety, privacy, security, cybersecurity, or adjacent fields.
- Demonstrate an adversarial mindset to investigate real-world harmful outcomes and technical drivers of misaligned behavior.
- Possess technical fluency to work directly with evaluation and training data to understand patterns and limitations.
- Work hands-on with model data and evaluation results, including inspecting examples and analyzing failure patterns.
- Use empirical evidence to develop and refine safety policies and safeguards.
- Translate complex or ambiguous alignment risks into precise behavioral expectations and measurable evaluation criteria.
- Communicate clearly about complex and uncertain technical risks across research, engineering, security, product, and policy.
Benefits
- Offers Equity
- Relocation support
Questions candidates ask
What is the remote work policy for this role?
The position is based in the San Francisco office using a hybrid model. Employees work three days in the office per week with optional work from home on Thursdays and Fridays.
What is the salary range for this position?
The salary range is between $207K and $335K, and the compensation package also offers equity.
What does the team actually work on day-to-day?
The team investigates emerging model failures, defines the behavior models should exhibit, and develops the data, evaluations, monitoring, and safeguards needed to improve and validate that behavior.