Staff + Sr. Software Engineer, Cloud Inference Launch Engineering
About this role
The Cloud Inference team at Anthropic builds the infrastructure that scales and optimizes Claude to serve massive audiences of developers and enterprise companies across AWS, GCP, Azure, and future cloud service providers. Engineers in this role own the end-to-end product presence on each cloud platform, managing everything from API integration and intelligent request routing to inference execution, capacity management, and day-to-day operations.
This position sits specifically within the model and inference launch team, where the focus is on validation pipelines for inference servers and load balancers. Professionals stepping into this group will handle high-leverage infrastructure challenges that directly determine how fast frontier models and features ship to production while reclaiming capacity when compute resources are constrained. Ideal candidates bring significant software engineering experience in large-scale distributed systems and thrive in cross-functional collaboration across internal teams and external cloud partners.
What you will do
- Manage the critical path for frontier model launches by bringing up inference for new model architectures and deploying them to cloud platforms alongside first-party systems.
- Collaborate with core inference teams to integrate new features like structured sampling and prompt caching into production cloud environments.
- Investigate and resolve cross-platform discrepancies related to configuration drift, observability, deployment patterns, and complex bugs at the source.
- Design and maintain CI/CD infrastructure for inference servers and load balancers using shadow traffic, performance baselines, and automated correctness checks.
- Improve validation velocity by making testing faster, more parallel, and cost-effective enough to run on production-grade accelerator pools.
- Analyze observability metrics across multiple providers to detect performance bottlenecks, cost anomalies, and regressions rooted in real-world workloads.
What you need
- Demonstrate a strong interest in large language model serving with a solid background in high-performance, large-scale distributed systems.
- Show a documented track record of building automation or test infrastructure that measurably improves release velocity or system reliability.
- Bring hands-on experience building or operating services on major cloud platforms including AWS, GCP, or Azure.
- Possess familiarity with Kubernetes, infrastructure as code, or container orchestration technologies.
- Exhibit strong cross-functional collaboration skills alongside high autonomy and end-to-end problem ownership.
- Hold a bachelor’s degree in a relevant field or possess an equivalent combination of education, training, and professional experience.
Nice to have
- LLM inference optimization, batching, and caching strategies
- Capacity-constrained scheduling or shared-resource test infrastructure
- Multi-region deployments, request routing, load balancing, and global traffic management
- Experience working with cloud service provider partner teams on networking, security, privacy, and managed services
- Proficiency in Python or Rust
Benefits
- Competitive compensation and benefits
- Optional equity donation matching
- Generous vacation
- Generous parental leave
- Flexible working hours
Skills and tech stack
Questions candidates ask
What is the remote work or office policy?
Anthropic operates on a location-based hybrid policy. Staff are expected to be in one of the offices in San Francisco or Seattle at least 25% of the time, though some roles may require more in-office presence.
What is the salary range for this role?
The annual compensation range for this position is between $320,000 and $485,000 USD.
Does Anthropic provide visa sponsorship?
Yes, the company sponsors visas when possible. If an offer is extended, they make every reasonable effort to secure a visa with the help of a retained immigration lawyer.