Latest AI JobsBrowse Jobs
OpenRouter logo

Site Reliability Engineer, Provider Operations

OpenRouterRemote (US)Engineering
Apply on company site
Salary
Not listed
Job type
Full-time
Work mode
Remote
Experience
Senior · 4+ yrs
Posted
9 Oct 2026
Status
Open · checked 7d ago

About this role

Remote (US) is the location for this Site Reliability Engineer position at OpenRouter, which suits an experienced infrastructure professional looking to manage high-volume LLM provider operations. This role focuses on maintaining the reliability, performance, and operational health of a massive AI routing infrastructure processing billions of daily requests.

Role Overview

OpenRouter seeks its first AI Inference Site Reliability Engineer to join the Provider Operations team and report to the Provider Operations Manager. The position centers on absorbing provider instability, ensuring endpoint availability, and building robust monitoring systems for large language model infrastructure.

Key Responsibilities

Build and own observability metrics for every provider and endpoint, tracking latency, throughput, error rates, uptime, and output correctness while setting and alerting on tier-based SLOs.
Enhance detection mechanisms for degraded endpoints to ensure automated traffic shifting away from problematic routes.
Manage on-call duties for provider incidents, handling triage, mitigation, external communication, postmortems, and follow-up actions.
Convert telemetry data into actionable scorecards and SLO reports for providers, acting as the technical escalation point for misbehaving endpoints.
Develop continuous canaries and evaluations to identify silent regressions such as quantization shifts, broken tool calling, truncated streams, and mismatched reporting.
Create safe, auditable tooling to eliminate manual operational tasks including capacity updates, endpoint disabling, rate-limit adjustments, and deprecations.
Implement load-testing tools for endpoints prior to major launches to handle initial traffic surges.

Requirements

Minimum of 4 years of professional experience in site reliability, production engineering, or infrastructure positions managing high-traffic systems facing customers.
Expertise in observability practices and tooling, including metrics, logs, tracing, actionable alerting, and error budgets.
Software engineering capability with a preference for tool creation over manual runbooks, specifically using TypeScript and Python.
Familiarity with failure modes inherent to distributed systems, such as retries, timeouts, backpressure, partial failures, and noisy neighbors.
Incident management skills combined with strong communication under pressure when dealing with external partners.
Knowledge of or strong willingness to learn LLM inference mechanics, covering prompt caching, streaming, tool calling, latency tradeoffs, and provider API variances.

Nice to Have

Background working at an inference provider, GPU cloud, model lab, API gateway, or CDN organization.
Familiarity with the company technology stack including TypeScript, Cloudflare Workers, Postgres, ClickHouse, GCP, and Vercel.
Previous work in traffic management, load balancing, or routing systems.
Experience building synthetic monitoring or evaluations for machine learning systems.

Skills and Tools

TypeScript
Python
Cloudflare Workers
Postgres
ClickHouse
GCP
Vercel
Observability and monitoring tooling

About the Company

OpenRouter operates an AI routing and infrastructure layer that enables enterprises to access, manage, and optimize large language models across multiple providers without vendor lock-in, capacity limits, or unnecessary expenses. The organization supports advanced AI teams by providing flexibility and scalability as foundational models continue to evolve.

How to Apply

Interested candidates should apply directly on the official OpenRouter careers page.

Skills and tech stack

TypeScriptPythonCloudflare WorkersPostgresClickHouseGCPVercelLLMLLMsDistributed Systems

Questions candidates ask

Where is this job located?

This position is remote within the United States.

Who does this role report to?

The Site Reliability Engineer reports to the Provider Operations Manager.

How many years of experience are required?

Candidates must have 4 or more years of experience in SRE, production engineering, or infrastructure roles managing high-traffic, customer-facing systems.

What programming languages are required?

Proficiency in TypeScript and/or Python is required.

What tools and technologies are part of the tech stack?

The tech stack includes TypeScript, Cloudflare Workers, Postgres, ClickHouse, GCP, and Vercel.

Ready to apply?

The application is on the official OpenRouter careers page.

Apply on company site