Staff+ Software Engineer, ML Sampling Path
About this role
Engineers on the Safeguards ML Sampling Path team design and operate production services that power safety systems across every platform running Claude. Every single request passes through these backend services on the token generation path, requiring careful management of latency, streaming contracts, and inference engines to ensure reliable performance. Responsibilities include owning end-to-end reliability by establishing SLOs, maintaining error budgets, driving per-token performance, and leading incident response when dependencies fail.
Anthropic created this role to keep p99 latency flat and maintain system uptime as traffic scales and safety checks grow. The team collaborates closely with research and inference groups to balance latency, cost, and reliability trade-offs while setting technical direction for the sampling path. Ideal candidates bring deep expertise in high-QPS distributed systems, a strong background in failure mode management, and the ability to ship complex changes to mission-critical infrastructure safely.
What you will do
- Design, build, and operate backend systems that process every token on the generation path for Claude requests.
- Manage the streaming contract with the API and inference engines under strict latency constraints.
- Own latency and reliability end to end by establishing and maintaining SLOs and error budgets for availability and time-to-first-token.
- Lead incident response and drive postmortem follow-through when production issues occur.
- Ship rapid updates to the hot path using canaried rollouts, error budget gating, and fast rollbacks.
- Chase tail latency and optimize per-token performance to keep costs flat as traffic and model checks increase.
- Set technical direction for the sampling path by leading design reviews and making trade-off decisions with inference and research teams.
What you need
- Demonstrated experience designing, building, and operating high QPS systems at global scale with direct accountability in production.
- Strong foundational knowledge of distributed systems, covering replication, consistency trade-offs, and failure modes.
- Ability to design systems for graceful degradation that handle slow dependencies, dropped streams, and partial deploys predictably.
- Track record of successfully shipping broad or all-encompassing changes to mission-critical systems such as database migrations or rewrites.
- Experience managing incident response, outages, and postmortem-driven remediation.
Nice to have
- 8+ years of industry software engineering experience
- Familiarity with LLM inference systems and transformer-based models
Benefits
- Competitive compensation and benefits
- Optional equity donation matching
- Generous vacation and parental leave
- Flexible working hours
Skills and tech stack
Questions candidates ask
What is the remote or hybrid work policy?
Anthropic maintains a location-based hybrid policy. Currently, the company expects all staff to be in one of the offices at least 25% of the time, though some roles may require more frequent in-office presence.
What is the salary range for this position?
The annual base salary range for this role is $320,000 to $485,000 USD. Compensation corresponds with internal job level requirements.
Does Anthropic provide visa sponsorship?
Yes, the company sponsors visas for candidates. While sponsorship is not guaranteed for every role or applicant, Anthropic makes every reasonable effort and retains an immigration lawyer to assist if an offer is extended.
What does the team actually work on?
The Safeguards ML Sampling Path team builds and operates the production services powering Claude's safety systems. These services process every token on the generation path across all platforms where Claude runs.