Latest AI JobsBrowse Jobs
B

Engineering Manager - Inference Performance

BasetenSan FranciscoEPD
Apply on company site
Salary
$240K – $270K
Job type
Full-time
Work mode
Remote
Experience
Senior
Posted
7 Oct 2026
Status
Open · checked 7d ago

About this role

Engineering Manager - Inference Performance at Baseten in San Francisco is a hands-on leadership role suited for an experienced technical manager ready to guide a growing team optimizing AI workloads on GPUs. This position involves directing runtime performance initiatives, scaling engineering talent, and ensuring mission-critical infrastructure operates with maximum efficiency.

Role Overview

Baseten is seeking an Engineering Manager to lead a segment of the Inference Performance team based in San Francisco. The role requires a blend of people management and deep technical engagement with GPU architecture, runtimes, and inference engines.

Key Responsibilities

Guide, mentor, and evaluate a team of inference performance engineers through consistent feedback and career development.
Recruit top talent in GPU and inference engineering to scale the runtime division.
Establish technical roadmaps for runtime performance, balancing user demands and platform investments.
Engage directly in technical reviews, profiling, and optimization strategies.
Oversee the production rollout of inference techniques like quantization, speculative decoding, and custom scheduling.
Translate performance improvements into metrics such as token throughput, utilization, latency, and cost reduction.
Collaborate across departments including Infrastructure, Kernels, and Model APIs to coordinate releases.

Requirements

Bachelor's, Master's, or Ph.D. degree in Computer Science, Engineering, Mathematics, or a related field.
Prior experience managing engineers, encompassing hiring, performance reviews, and mentorship.
Background leading or supporting GPU optimization teams across training, inference, or recommendation systems.
Technical expertise in GPU workloads, architecture, and performance constraints.
Working knowledge of machine learning libraries like PyTorch, TensorRT, or TensorRT-LLM.
Demonstrated history of executing complex technical project roadmaps.
Effective written and verbal communication skills for cross-functional alignment.

Nice to Have

Understanding of inference engines such as vLLM, SGLang, or TensorRT-LLM.
Production experience with LLM optimization methods including continuous batching and quantization.
Proficiency with GPU kernels like CUDA, Triton, or CUTLASS.
Background in scaling teams during rapid startup growth.
Previous experience as a hands-on systems or performance engineer.

Skills and Tools

GPUs
PyTorch
TensorRT
TensorRT-LLM
vLLM
SGLang
CUDA
Triton
CUTLASS

Salary and Benefits

Salary range of $240K to $270K with equity offers.
Full 100% medical, dental, and vision insurance coverage for employees and dependents in the U.S.
Flexible paid time off alongside a company-wide winter break.
Paid parental leave.
Fertility and family-building support through Carrot.
U.S. company-facilitated 401(k) plan.

About the Company

Baseten delivers inference infrastructure for AI organizations such as Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma, and Writer. The organization recently secured a $1.5 billion Series F funding round led by Altimeter Capital, Conviction Partners, and Spark Capital.

How to Apply

Interested candidates should apply on the official company careers page.

Skills and tech stack

PyTorchTensorRTTensorRT-LLMvLLMSGLangCUDATritonCUTLASSLLMSparkMachine Learning

Questions candidates ask

Where is the Engineering Manager - Inference Performance role located?

The position is based in San Francisco.

What is the compensation range for this position?

The salary ranges from $240K to $270K, accompanied by equity offerings.

What educational background is required?

Candidates must hold a Bachelor's, Master's, or Ph.D. degree in Computer Science, Engineering, Mathematics, or a related field.

What technical libraries and tools are relevant to the role?

Relevant tools and libraries include PyTorch, TensorRT, TensorRT-LLM, vLLM, SGLang, CUDA, Triton, and CUTLASS.

Are benefits provided for U.S. employees?

Yes, U.S. employees receive 100% coverage for medical, dental, and vision insurance for themselves and dependents, a 401(k), paid parental leave, and a fertility stipend.

Ready to apply?

The application is on the official Baseten careers page.

Apply on company site