Staff Software Engineer, RL Environments
About this role
Engineers who thrive on building complex systems from the ground up and possess a deep curiosity about reinforcement learning and agentic behavior will find this position at Scale AI immensely rewarding. As part of the Frontier Data organization, the team focuses on creating training and evaluation data that powers leading AI models, with reinforcement learning environments serving as the core of these efforts. This role exists to own the technical foundation for how the company builds, runs, verifies, and delivers RL environments at high throughput.
In this capacity, the engineer will design platform infrastructure like sandboxed execution, versioning, and rollout orchestration, while also diving deep into instrumenting applications and building robust graders. Success requires a rare blend of systems engineering expertise and intuition for model behavior to solve complex challenges in scaled AI training.
What you will do
- Own the technical foundation for how Scale builds, runs, verifies, and delivers reinforcement learning environments at scale.
- Design the platform encompassing sandboxed execution, environment packaging and versioning, rollout orchestration, trajectory capture, and verifier frameworks.
- Build authoring surfaces that empower engineers and domain experts to produce environments without reinventing infrastructure.
- Instrument real applications, design task suites that expose specific capability gaps, and build graders that withstand adversarial optimization.
- Set technical direction across multiple teams while actively writing complex system components.
- Tackling ambiguous, undefined problems end-to-end and driving them to fully shipped production systems.
What you need
- Bring eight or more years of software engineering experience with strong fundamentals in distributed systems, system design, data structures, and algorithms.
- Demonstrate strong Python skills alongside a proven track record of shipping production software and comfort in at least one other stack such as TypeScript, Go, or Rust.
- Possess deep experience with containerization and sandboxed execution tools like Docker, VMs, gVisor, Firecracker, or Kubernetes.
- Have experience building or operating high-throughput backend systems including orchestration, job scheduling, queuing, and large-scale data pipelines.
- Show hands-on experience building with LLMs, including agent loops, tool calling, MCP, or evaluation harnesses.
- Demonstrate the ability to own ambiguous problems end-to-end and communicate technical direction clearly to diverse partners.
Nice to have
- Direct experience building RL environments, agentic benchmarks, or eval harnesses
- Familiarity with post-training methods such as RLHF, RLAIF, RLVR, and GRPO or PPO-family algorithms
- Experience designing verifiable reward signals and defending against reward hacking
- Experience with RL training or serving stacks like verl, TRL, Ray, vLLM, or SGLang
- Experience with high-scale sandbox or code-execution infrastructure and cloud-native environments across AWS, GCP, or Azure
Benefits
- Comprehensive health, dental, and vision coverage
- Retirement benefits
- Learning and development stipend
- Generous paid time off
- Commuter stipend
Skills and tech stack
Questions candidates ask
What is the base salary range for this role?
For pay transparency purposes, the base salary range for this full-time position in the locations of San Francisco, New York, and Seattle is $252,000 to $315,000 USD. Compensation packages may also include equity and benefits determined during the interview process.
What locations are available for this position?
This position is located in San Francisco, CA or New York, NY, with specific location details referenced in the job posting's subtitle.
What benefits are included with the position?
Employees receive comprehensive health, dental, and vision coverage, retirement benefits, a learning and development stipend, generous PTO, and potentially a commuter stipend.
Is there a waiting period if I have applied previously?
Scale AI has a policy requiring a 90-day waiting period before reconsidering candidates for the same role to ensure a fair and thorough evaluation of all applicants.