Latest AI JobsBrowse Jobs
B

Software Engineer - GPU Networking & Distributed Systems

BasetenSan FranciscoEPD
Apply on company site
Salary
$165K - $330K USD
Job type
Full-time
Work mode
Remote
Posted
23 Feb 2026
Status
Open · checked 7d ago

About this role

Role Overview

Baseten is seeking a Software Engineer for GPU Networking & Distributed Systems to join their team in San Francisco or remotely. This full-time position suits foundational engineers interested in leading GPU networking efforts and shaping the global operating system for distributed, heterogeneous AI hardware.

Key Responsibilities

Integrate RDMA, RoCE, and InfiniBand capabilities directly into the inference stack to bypass traditional TCP/IP limitations.
Implement and tune networking layers for efficient Disaggregated KV Cache Offload and Wide Expert Parallelism across NVLink and InfiniBand.
Work with checkpointing and storage mechanisms to achieve serverless-grade startup speeds under ten seconds for trillion-parameter models.
Characterize and validate networking performance on advanced clusters including H100, H200, B200, B300, and GB200/300 NVL72 architectures.
Design observability tools to visualize packet flow, congestion, and effective bandwidth across GPU interconnects.
Collaborate with communication libraries like NCCL and NVSHMEM, and potentially author custom communication kernels to overlap compute and data transfer.

Requirements

Deep experience with high-performance networking protocols such as InfiniBand and RoCE v2, alongside an understanding of data movement physics.
Fluency in C++ or Python with the capability to connect high-level logic to hardware.
Strong comprehension of the memory hierarchy in modern NVIDIA architectures like H100 and Blackwell.
Willingness to dive deep into source code, write custom bindings, and debug complex NVLink topology issues.
Judgment to determine when to utilize off-the-shelf components versus building custom infrastructure when upstream tools fall short of speed requirements.

Nice to Have

Deep knowledge of NCCL, NVSHMEM, and UCX.
Experience using Rust for systems-level or performance-critical networking code.
Familiarity with GPUDirect Storage or high-performance filesystems like Weka and 3FS.
Working knowledge of TensorRT-LLM, vLLM, or SGLang.
Prior experience running low-level benchmarks to qualify new hardware clusters.

Skills and Tools

Python
Kubernetes
Rust
Spark
C++
LLM
InfiniBand
RoCE v2
RDMA
NCCL
NVSHMEM
UCX

Salary and Benefits

Competitive annual salary ranging from $165,000 to $330,000 in USD.
Meaningful equity opportunities.
100% coverage of medical, dental, and vision insurance for employees and dependents within the United States.
Flexible paid time off policy featuring a company-wide winter break from Christmas Eve to New Year's Day.
Paid parental leave.
Fertility and family-building stipend through Carrot.
Company-facilitated 401(k) plan for U.S. employees.

About the Company

Baseten powers mission-critical inference for dynamic artificial intelligence enterprises like Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma, and Writer. By combining applied AI research, flexible infrastructure, and developer tooling, Baseten helps frontier AI companies bring advanced models into production. The company recently raised a $1.5 billion Series F funding round led by Altimeter Capital, Conviction Partners, and Spark Capital, and is committed to fostering an inclusive, equal opportunity workplace.

How to Apply

Interested candidates should visit the official Baseten careers page to apply for this position.

Skills and tech stack

PythonKubernetesRustSparkC++LLM

Questions candidates ask

What is the salary range for this role?

The salary ranges from $165,000 to $330,000 per year in USD.

Is this position remote?

Yes, this role offers remote flexibility, alongside an office presence option in San Francisco.

What hardware architectures will I work with?

You will work with bleeding-edge clusters including H100, H200, B200, B300, and GB200/300 NVL72 architectures.

What programming languages are required?

Candidates must be fluent in C++ or Python, with Rust considered a strong plus.

What benefits are included for U.S. employees?

U.S. employees receive 100% medical, dental, and vision coverage, a company-facilitated 401(k), flexible PTO, parental leave, and a fertility stipend.

Ready to apply?

The application is on the official Baseten careers page.

Apply on company site