This position suits a systems engineer who enjoys working deep in the Linux stack and building production runtime infrastructure. Based in San Francisco, this full-time role involves managing the container runtime stack for machine learning workloads at Modal.
Role Overview
The Inference Runtime team at Modal owns the container runtime stack that executes training and inference workloads across the fleet. The position focuses on making demanding machine learning workloads start quickly, operate efficiently, and remain securely isolated across various hardware configurations.
Key Responsibilities
Accelerate container startup, checkpoint, and restore processes for large training and inference workloads.
Construct multi-GPU and accelerator-aware snapshotting, encompassing efficient GPU memory handling and RDMA-enabled workloads.
Architect zero-copy and low-copy data paths connecting container memory, storage, filesystems, and the runtime.
Refine large snapshot pipelines utilizing parallel uploads, direct I/O, incremental snapshots, and improved memory handling.
Enhance filesystem and container image performance across remote storage, FUSE, EROFS, overlay filesystems, and page caches.
Scale the sandboxed runtime to accommodate new device capabilities, GPUs, drivers, and profiling tools for NVIDIA and AMD hardware.
Troubleshoot intricate failures related to virtual memory, system calls, process lifecycle, kernel behavior, container isolation, and GPU drivers.
Deploy runtime and kernel updates safely across a diverse fleet using feature flags, observability, scheduling constraints, and compatibility controls.
Drive ambiguous production challenges from initial investigation through to deployment across storage, GPU infrastructure, scheduler, and runtime.
Requirements
Comprehensive Linux systems knowledge covering virtual memory, processes, filesystems, system calls, namespaces, cgroups, scheduling, and signals.
Background in building or troubleshooting container runtimes, sandboxes, the Linux kernel, or comparable low-level infrastructure.
Proficiency in systems programming using Rust, Go, or another systems language, alongside motivation to become productive in both Rust and Go.
Track record of profiling and enhancing systems constrained by memory movement, kernel interactions, I/O, or synchronization.
Ability to debug across multiple abstraction layers, from application code down to the kernel, drivers, and runtimes.
Competence in translating loosely defined production issues into reliable, well-designed systems.
Motivation to take ownership of critical infrastructure and collaborate directly with internal teams and customers.
Nice to Have
Experience with gVisor, runsc, runc, OCI runtimes, seccomp, or checkpoint/restore systems.
Background in Linux kernel development, virtualization, sandboxing, kernel modules, or device proxying.
Familiarity with CUDA, ROCm, GPU drivers, accelerator virtualization, or GPU profiling.
Knowledge of RDMA and high-performance networking.
Exposure to FUSE, EROFS, direct I/O, mmap, page-cache behavior, or storage engines.
Hands-on involvement with large-memory or multi-GPU inference and training systems.
Prior contributions to open-source systems software.
Skills and Tools
Rust
Go
Linux
gVisor
CUDA
ROCm
RDMA
Salary and Benefits
Salary range of $220K to $300K
Equity offerings included
About the Company
Modal operates a platform supporting machine learning inference and training workloads, utilizing a container runtime stack that interacts with Linux, storage, filesystems, and GPU drivers.
How to Apply
Interested candidates should apply on the official company careers page.
