Latest AI JobsBrowse Jobs
Sierra logo

Software Engineer, Inference

SierraSan Francisco, CAEngineering
Apply on company site
Salary
$230K – $390K
Job type
Full-time
Experience
Senior
Posted
5 Oct 2026
Status
Open · checked 7d ago

About this role

This role suits an experienced distributed systems engineer looking to build scalable AI inference architecture in San Francisco. Software Engineer, Inference is a full-time position at Sierra focused on developing reliable, low-latency systems that power customer-facing artificial intelligence agents.

Role Overview

Sierra is hiring a Software Engineer for the Inference team in San Francisco. The position centers on designing and operating distributed infrastructure that makes foundational models fast, cost-effective, and reliable at scale.

Key Responsibilities

Partner with frontier labs and providers to secure capacity, training, and inference infrastructure.
Shape inference architecture by designing traffic flow across models, proxy layers, and providers.
Build systems for routing, failover, capacity management, and quota to ensure high availability and low latency.
Run self-hosted models on GPU infrastructure, including container creation, inference engine operations, and compute capacity management.
Optimize performance alongside applied researchers using techniques like speculative decoding and serving-engine tuning.
Manage a hybrid inference stack utilizing both company-managed resources and third-party platforms.
Support the broader model lifecycle by contributing to post-training infrastructure and collaborating with agent runtime teams.

Requirements

Strong background in distributed systems fundamentals and deep systems thinking.
Track record of designing, building, and operating production systems at scale.
Ability to balance tradeoffs involving latency, reliability, capacity, and cost.
Experience taking complex infrastructure ownership from initial architecture through production.
Enthusiasm for applying systems expertise to artificial intelligence infrastructure.

Nice to Have

Experience with machine learning infrastructure, MLOps, or production inference systems.
Background serving large language models or other large models at scale.
Practical knowledge of operating self-hosted inference and GPU infrastructure.
Familiarity with serving frameworks such as vLLM or SGLang.
Background in post-training infrastructure or inference-performance optimization.

Skills and Tools

Distributed systems
GPU infrastructure
vLLM
SGLang
Self-hosted inference engines

Salary and Benefits

Salary range of $230K to $390K with equity offers included.
Flexible, unlimited paid time off.
Medical, dental, vision, life, and disability insurance benefits.
Retirement plans and parental leave options.
Fertility and family building benefits via Carrot.
On-site meals, snacks, coffee, and discretionary benefit stipends.
Free alphorn lessons.

About the Company

Sierra provides a platform for customer-facing AI agents, serving major global brands such as The GAP, Rocket Mortgage, SoFi, Sutter Health, and SoftBank. Co-founded by Bret Taylor and Clay Bavor, the company maintains its primary in-person office in San Francisco alongside growing presences across North America, Europe, and Asia. The corporate culture centers around values including trust, customer obsession, craftsmanship, intensity, and family.

How to Apply

Interested candidates should apply on the official company careers page.

Skills and tech stack

vLLMSGLangGPU infrastructureLLMsMLOpsDistributed Systems

Questions candidates ask

Where is this job located?

This is primarily an in-person role based in San Francisco, CA.

What is the salary and compensation range?

The position offers a salary between $230K and $390K along with equity.

Do I need to be a machine learning researcher to apply?

No, this is a systems-first role focused on distributed infrastructure rather than ML research.

What tools or frameworks are helpful for this role?

Familiarity with inference frameworks like vLLM or SGLang, GPU infrastructure, and ML infrastructure is considered a plus.

What benefits are provided for full-time employees?

Benefits include flexible PTO, medical, dental, vision, life and disability insurance, parental leave, family building benefits through Carrot, meals, discretionary stipends, and equity.

Ready to apply?

The application is on the official Sierra careers page.

Apply on company site