Latest AI JobsBrowse Jobs
Anthropic logo

Engineering Manager, Inference Infrastructure

AnthropicSan Francisco, CA | New York City, NY | Seattle, WASoftware Engineering - Infrastructure
Apply on company site
Salary
Not listed
Job type
Full-time
Posted
1 Sept 2026
Status
Open · checked 7d ago

About this role

Anthropic is seeking an Engineering Manager for Inference Infrastructure to lead teams building control planes and managing request paths for large-scale AI models. This permanent position is based out of San Francisco, CA, New York City, NY, or Seattle, WA, and is ideal for an experienced technical leader with a strong distributed systems background.

Role Overview

Every interaction with Claude depends on crucial routing decisions made before requests ever reach a model, determining placement, capacity, and load-balancing. This leadership role oversees the technical group responsible for the control plane of the inference fleet and the entire inference request path. The engineering manager guides professionals working on load-balancing algorithms, demand modeling, quantitative system performance, and latency optimization. Team members include machine learning platform, infrastructure, and distributed-systems engineers who collaborate closely with ML internals and cloud infrastructure teams. Leadership must provide architectural oversight, support professional growth, and ensure the operational health, efficiency, and reliability of the end-to-end request path.

Key Responsibilities

Own the technical roadmap for coordinating the inference fleet, including traffic routing, capacity allocation, cache placement, reaction speed to demand, and synchronization protocols.
Partner closely with product, inference engine, performance, and capacity teams to identify and deliver measurable improvements in throughput, latency, utilization, and cost.
Cultivate quantitative modeling habits by ensuring all claims of system wins are measurable and backed by upfront expected effects.
Establish technical strategies for control plane evolution across heterogeneous hardware, diverse cloud providers, and varied serving surfaces.
Oversee the operational backbone, including on-call rotations, incident response, postmortem reviews, and deployment safety.
Build strong cross-functional relationships across API surfaces, inference engines, and capacity planning teams.
Develop, retain, and hire strong engineering talent while scaling team structure as operational scope grows.

Requirements

Engineering management experience leading teams focused on critical-path production infrastructure at scale.
A deep systems background encompassing load balancing, scheduling, cluster orchestration, autoscaling, cache-coherent distributed state, or high-performance networking.
Demonstrated experience shipping performance or efficiency improvements in large-scale systems with verifiable numerical impact.
Practical background running production infrastructure with real operational stakes involving on-call duties, incident response, and deployment discipline.
A Bachelor’s degree in a relevant field of study, or an equivalent combination of education, training, and experience.
Ability to work under a hybrid policy requiring time in one of the designated offices.

Nice to Have

Five or more years of engineering management experience.
Prior experience with large language model inference serving such as KV caching, continuous batching, request scheduling, and prefill/decode disaggregation.
Background working with cluster schedulers, autoscalers, load balancers, service meshes, or fleet control planes at scale like Kubernetes internals or Borg-style systems.
Experience operating workloads across multiple clouds or partner platforms.
Familiarity with heterogeneous accelerator fleets and hardware differences affecting workload placement.
Experience leading teams at supercomputing or hyperscaler infrastructure scale.

Skills and Tools

Kubernetes
LLM Inference Serving
Distributed Systems
Cluster Orchestration
High-Performance Networking
Load Balancing and Autoscaling

Salary and Benefits

Annual salary range of $405,000 to $625,000 USD.
Competitive compensation and benefits package.
Optional equity donation matching.
Generous vacation and parental leave policies.
Flexible working hours.
Visa sponsorship is available for qualifying candidates.

About the Company

Anthropic is a public benefit corporation focused on building reliable, interpretable, and steerable AI systems that remain safe and beneficial for society. Operating as a single cohesive team, the organization views empirical AI research through an interdisciplinary lens akin to physical sciences. The company offers a collaborative office environment in San Francisco and encourages diverse perspectives to address the profound social and ethical implications of advanced artificial intelligence.

How to Apply

Interested candidates should submit their application directly through the official Anthropic careers page.

Skills and tech stack

KubernetesLLM

Questions candidates ask

What are the location options for this role?

The position is based out of San Francisco, CA, New York City, NY, or Seattle, WA, operating under a hybrid policy requiring at least 25% time in an office.

What is the salary range for the Engineering Manager, Inference Infrastructure?

The annual base salary ranges from $405,000 to $625,000 USD.

Does Anthropic offer visa sponsorship for this position?

Yes, visa sponsorship is available, and the company retains an immigration lawyer to assist qualified candidates who receive an offer.

What level of experience is required?

Candidates must have engineering management experience leading teams on critical-path production infrastructure at scale, with preferred candidates bringing 5 or more years of management experience.

How should I apply for this job?

Candidates should visit the official Anthropic careers page to submit their application safely.

Ready to apply?

The application is on the official Anthropic careers page.

Apply on company site