Latest AI JobsBrowse Jobs
B

Software Engineer - Model Performance Systems

BasetenSan FranciscoEPD
Apply on company site
Salary
$165K - $330K USD
Job type
Full-time
Work mode
Remote
Posted
7 Jan 2026
Status
Open · checked 7d ago

About this role

Role Overview

The Software Engineer for Model Performance Systems role at Baseten is a specialized, high-impact position operating at the intersection of high-performance computing and Large Language Model engineering. Based in San Francisco with remote flexibility, this position suits technical professionals eager to own performance infrastructure, benchmarking, and optimization tools for cutting-edge AI.

Key Responsibilities

Evaluate, execute, and automate standard LLM quality benchmarks like GSM8K and MMLU, alongside custom performance suites for workloads such as long-context windows, KV cache reuse, and disaggregated serving.
Build and manage internal GPU-enabled development environments similar to GitHub Codespaces to provide the team with optimized, high-performance dev machines for model experimentation.
Contribute to and develop open-source tools including InferenceMAX and genai-bench to streamline model evaluation, analysis, and benchmarking.
Utilize profilers like NVIDIA Nsight Systems, PyTorch Profiler, and py-spy to capture performance profiles, troubleshoot the compute and networking stack, and pinpoint bottlenecks.
Construct real-time dashboards and alerting mechanisms to track system health, runtime performance, and model startup times.
Implement automated performance testing through CI/CD pipelines to prevent regressions and establish release workflow automation for model runtimes.
Create optimization automation tools that map the Pareto frontier to determine optimal configurations balancing latency, cost, and quality for specific workloads.

Requirements

Passion for systems and hardware, with a strong desire to understand GPU memory subsystems, InfiniBand, and cluster data movement.
A dedicated automation mindset focused on scripting repetitive tasks and stress-testing systems to discover breaking points.
Mathematical curiosity regarding the underlying mechanics of Transformers and how they translate into memory requirements and FLOPs.
Strong technical toolkit featuring proficiency in Python, readiness to master the NVIDIA software stack, and excellent communication skills to lead cross-team initiatives.

Nice to Have

Practical familiarity with C++.

Skills and Tools

Python
PyTorch
Spark
C++
LLM
NVIDIA software stack
PyTorch Profiler
NVIDIA Nsight Systems
py-spy

Salary and Benefits

Competitive yearly salary ranging from $165,000 to $330,000 in USD.
Meaningful equity options.
Complete 100% coverage of medical, dental, and vision insurance for employees and their dependents (U.S. only).
Flexible paid time off policy, featuring a company-wide Winter Break with offices closed from Christmas Eve through New Year's Day.
Paid parental leave programs.
Fertility and family-building stipends provided via Carrot.
Company-facilitated 401(k) retirement plan (U.S. only).
Access to diverse machine learning startups offering unique networking and learning opportunities.

About the Company

Baseten powers mission-critical inference for leading AI companies such as Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma, and Writer. By combining applied AI research, flexible infrastructure, and seamless developer tooling, the organization helps frontier AI enterprises move advanced models directly into production. Backed by recent Series F funding led by Altimeter Capital, Conviction Partners, and Spark Capital, Baseten is rapidly expanding its platform.

How to Apply

Interested candidates should visit the official Baseten careers page to submit an application and learn more about open opportunities.

Skills and tech stack

PythonPyTorchSparkC++LLM

Questions candidates ask

What is the salary range for this role?

The annual salary for this position ranges from $165,000 to $330,000 USD.

Is this position remote?

Yes, this role is listed with remote availability based out of San Francisco.

What technical skills are required?

Candidates need strong proficiency in Python, a desire to master the NVIDIA software stack, familiarity with tools like PyTorch, Spark, and LLMs, with C++ experience listed as nice to have.

What benefits are included for U.S. employees?

U.S. employees receive 100% medical, dental, and vision coverage for themselves and dependents, a company-facilitated 401(k), flexible PTO including a winter break, paid parental leave, and fertility stipends.

How do I apply for the Software Engineer position?

Candidates can apply directly through the official Baseten careers page.

Ready to apply?

The application is on the official Baseten careers page.

Apply on company site