Staff Software Engineer, Observability & Profiling
About this role
The Observability team at Anthropic builds the monitoring and telemetry infrastructure that every engineer and researcher depends on, ranging from logging pipelines and distributed tracing to profiling and alerting dashboards. Engineers on this team design next-generation observability systems, including high-throughput telemetry pipelines, fleet-wide continuous profiling, and eBPF-based tracing to help colleagues resolve issues across massive GPU, TPU, and Trainium clusters.
This role exists to scale observability platforms as infrastructure grows rapidly and operational data increases in volume and complexity. The person in this position will work on deep, low-overhead visibility into system behavior, building instrumentation libraries, and creating AI-assisted diagnostic tooling to reduce mean time to detection. Ideal candidates bring hands-on experience with large-scale monitoring infrastructure, understand telemetry pipeline tradeoffs, and enjoy collaborating across research, inference, and product teams.
What you will do
- Design and build scalable telemetry ingest and storage pipelines for metrics, logs, traces, and error data across multi-cluster infrastructure.
- Build observability solutions that give engineers deep, low-overhead visibility into system behavior across the fleet.
- Own and evolve core observability platforms by driving migrations and architectural improvements that enhance reliability and reduce cost.
- Build instrumentation libraries, SDKs, and eBPF-based auto-instrumentation to emit high-quality telemetry.
- Reduce mean time to detection and resolution through cross-signal correlation, unified query interfaces, and AI-assisted diagnostic tooling.
- Drive fleet-wide efficiency by turning continuous profiling and utilization telemetry into actionable optimization insights.
- Partner with Research, Inference, Product, and Infrastructure teams to ensure observability solutions meet specific organizational needs.
What you need
- Have hands-on experience building and operating large-scale observability or monitoring infrastructure.
- Demonstrate deep, practical experience with observability signals end to end, from instrumentation to ingest and analysis.
- Understand high-throughput telemetry pipelines and the tradeoffs involved in collecting, storing, and querying operational data at scale.
- Show comfort digging below the application layer into the kernel, the network stack, or the hardware.
- Possess excellent communication skills and enjoy partnering with internal teams to improve operational visibility and incident response.
- Demonstrate enthusiasm for building foundational infrastructure while navigating ambiguous, high-impact technical challenges.
Nice to have
- 10+ years of relevant industry experience, including building and operating large-scale observability or monitoring infrastructure
- Experience building or operating eBPF-based observability in production, such as tracing, profiling, or network visibility
- Experience running continuous profiling at fleet scale, managing overhead budgets, and symbolization
- Kernel- and syscall-level debugging experience and performance engineering craft
- Experience profiling or instrumenting accelerator workloads and operating high-cardinality metrics systems
Benefits
- Competitive compensation
- Optional equity donation matching
- Generous vacation
- Parental leave
- Flexible working hours
Skills and tech stack
Questions candidates ask
What is the remote work or office policy?
Anthropic operates under a location-based hybrid policy. Staff are expected to be in the office at least 25% of the time, though some roles may require more time in-office.
What is the compensation range for this position?
The annual salary range for this role is between £325,000 and £390,000 GBP.
Is visa sponsorship available for this role?
Yes, visa sponsorship is provided. The company retains an immigration lawyer and makes every reasonable effort to secure visas for offered candidates.