About this role
Managing the firmwide interface to modern AI models involves optimizing large language, vision, multimodal, and embedding workloads from early evaluation to reliable production use. DRW seeks an AI Inference Platform Engineer to build, operate, and scale serving infrastructure across NVIDIA GPU architectures for diverse trading strategies. This practitioner owns the end-to-end inference platform, onboarding new models, evaluating quality and performance, and scheduling multi-tenant workloads while continuously improving latency, throughput, utilization, reliability, and cost.
Candidates who succeed in this environment bring deep GPU literacy, hands-on experience with modern inference runtimes, and a measurement-driven approach to distributed systems engineering. The ideal person thrives with high expectations and autonomy, balancing rapid reprioritization with a commitment to predictable, robust production operations.
What you will do
- Optimize LLM inference performance across modern NVIDIA GPU architectures and inference runtimes.
- Build end-to-end performance profiling and observability to identify bottlenecks from individual GPU kernels through multi-node inference systems.
- Design and optimize KV cache and distributed inference architectures, including caching, routing, memory tiering, and prefill/decode strategies.
- Manage the production serving lifecycle of models, including versioning, compatibility, staging, canarying, promotion, rollback, and retirement.
- Design and operate multi-tenant scheduling and isolation across shared GPU capacity, balancing latency SLOs, throughput, and priority across concurrent workloads.
What you need
- Hands-on experience serving LLMs on NVIDIA GPUs, with familiarity across current and emerging architectures, HBM, Tensor Cores, NVLink/NVSwitch, and compute and memory bottlenecks.
- Deep expertise in at least one modern inference runtime such as TensorRT-LLM, vLLM, or SGLang.
- Practical knowledge of inference optimization techniques including continuous batching, scheduling, chunked prefill, speculative decoding, quantization, CUDA Graphs, and paged attention.
- Strong Linux and systems performance fundamentals, with the ability to diagnose bottlenecks across hardware, drivers, runtimes, networking, and application layers.
- Production experience with model serving infrastructure, including CI/CD, automated testing, observability, and production readiness.
Benefits
- Group medical, pharmacy, dental and vision insurance
- 401k with discretionary employer match
- Short and long-term disability
- Life and AD&D insurance
- Health savings accounts
- Flexible spending accounts
Skills and tech stack
Questions candidates ask
What is the salary range for this role?
The annual base salary range for this position is $200,000 to $250,000 depending on the candidate’s experience, qualifications, and relevant skill set. The position is also eligible for an annual discretionary bonus.
Where is the position located?
The role is headquartered in Chicago.
What technologies and runtimes will I work with?
You will work with modern NVIDIA GPU architectures and have deep expertise in at least one modern inference runtime such as TensorRT-LLM, vLLM, or SGLang. The tech stack also involves GPU performance and observability tooling like Nsight, DCGM, OpenTelemetry, Prometheus, and Grafana.
What employee benefits are offered?
DRW offers a comprehensive suite of employee benefits including group medical, pharmacy, dental and vision insurance, 401k with discretionary employer match, and disability and life insurance. Additional offerings include health savings accounts and flexible spending accounts.