Latest AI JobsBrowse Jobs
B

Product Manager, Inference Platform

BasetenSan FranciscoEPD
Apply on company site
Salary
$235K - $335K USD
Job type
Full-time
Work mode
Remote
Posted
2 Apr 2026
Status
Open · checked 8d ago

About this role

Role Overview

Baseten is seeking a Product Manager, Inference Platform to join their EPD department on a full-time basis, with options for remote work or out of San Francisco. This position is ideal for an experienced product leader with a deep technical background who wants to shape a nascent product function alongside founders and expert engineers. The person in this role will drive the systems that keep machine learning deployments fast, reliable, and economical at massive scale.

Key Responsibilities

Own end-to-end capabilities covering workload scaling, demand-based autoscaling, and placement policies that account for region, compliance, and capacity preferences.
Grant compliance-bound workloads priority access to sensitive capacity and ensure production inference remains reliable by default without dropped traffic during rolling deployments.
Establish region-aware routing strategies featuring active-active multi-region failover and health-aware recovery mechanisms for problematic replicas.
Develop the release engine for safe rollouts, encompassing traffic-shifting for canary, shadow, and A/B tests alongside warm-ups, drains, and probes.
Push the boundaries of cost efficiency, latency, throughput, and uptime while driving a measurable decrease in mean time to resolution via self-serve incident management.
Formulate roadmaps for infrastructure and product teams, taking ownership of capabilities from backend behavior to customer configuration and observability.

Requirements

Minimum of 8 years of product management experience, featuring substantial exposure to distributed systems, infrastructure, or machine learning serving.
Strong technical fluency regarding routing, failover, scaling, and the performance-to-cost frontier that gains the confidence of staff engineers.
Proven history of managing capabilities fully from backend to user interface rather than focusing on a single layer.
Demonstrated ability to guide cross-functional roadmaps and manage complex dependencies.
High comfort level defining an emerging industry category.

Nice to Have

Practical background working with GPU infrastructure.
Familiarity with Kubernetes.
Background using serving platforms like TensorRT-LLM, vLLM, or SGLang.

Skills and Tools

Kubernetes
Spark
LLM
Distributed Systems
Machine Learning Serving

Salary and Benefits

Annual salary range between $235,000 and $335,000 USD.
Competitive total compensation package including significant equity.
Full 100% medical, dental, and vision insurance coverage for employees and dependents within the United States.
Flexible paid time off guidelines, featuring a company-wide winter break closure between Christmas Eve and New Year's Day.
Paid parental leave benefits alongside family-building and fertility stipends via Carrot.
Company-facilitated 401(k) program for U.S. staff.
Direct exposure to innovative machine learning startups providing networking and learning prospects.

About the Company

Baseten delivers mission-critical inference infrastructure for dynamic artificial intelligence organizations like Notion, Cursor, Abridge, OpenEvidence, Gamma, Clay, and Writer. By combining applied AI research, adaptable infrastructure, and developer utilities, the organization helps frontier teams push cutting-edge models into production environments. Backed by key investors including Altimeter Capital, Conviction Partners, and Spark Capital following a recent Series F funding round, Baseten provides an inclusive workplace with equal opportunity employment.

How to Apply

Interested candidates should visit the official Baseten careers page to submit an application for this opportunity.

Skills and tech stack

KubernetesSparkLLM

Questions candidates ask

What is the salary range for the Product Manager, Inference Platform role?

The annual salary is between $235,000 and $335,000 USD.

Where is this job located?

The role is based in San Francisco with remote work options available.

How many years of experience are required?

Candidates must have 8 or more years of experience in product management, focusing on distributed systems, infrastructure, or ML serving.

What department does this role belong to?

This position sits within the EPD department at Baseten.

What benefits are included for U.S. employees?

U.S. employees receive 100% medical, dental, and vision coverage for themselves and dependents, a company-facilitated 401(k), flexible PTO including a winter break, paid parental leave, and fertility stipends through Carrot.

Ready to apply?

The application is on the official Baseten careers page.

Apply on company site