Member of Technical Staff - Evaluation Infrastructure
About this role
Engineers who thrive in lean, fast-paced environments and enjoy solving complex distributed systems problems will find a strong fit within this group. Operating with a flat organizational structure, the team expects every member to be hands-on, communicate clearly, and take direct ownership of building foundational systems that understand the universe.
This role focuses entirely on developing robust evaluation infrastructure to support next-generation machine learning models. The incoming engineer will design scalable orchestration frameworks, partner closely with modeling groups to maintain signal quality, and eliminate performance bottlenecks across inference and resource management layers.
What you will do
- Design and construct highly reliable and efficient infrastructure capable of running model evaluations at scale.
- Partner directly with modeling and evaluation teams to build internal tooling and maintain signal quality for upcoming architectures.
- Debug and resolve deep performance bottlenecks across the entire evaluation stack.
- Manage asynchronous evaluation orchestration, inference layers, and capacity fleet operations.
What you need
- Demonstrated background in building, debugging, and optimizing large-scale distributed systems.
- Strong technical drive to solve difficult architectural challenges across all layers of the software stack.
- Ability to communicate complex technical details concisely and accurately with teammates.
- Strong prioritization skills and a robust work ethic in a fast-moving, flat organization.
Nice to have
- Experience in LLM inference
- Experience in GPU compute management, workload scheduling, and dynamic resource optimization
- Experience in developing interfaces for comparing models and evals
Benefits
- Equity
- Comprehensive medical, vision, and dental coverage
- Access to a 401(k) retirement plan
- Short and long-term disability insurance
- Life insurance
- Various other discounts and perks
Skills and tech stack
Questions candidates ask
What is the compensation range for this position?
The base salary ranges from $180,000 to $440,000 USD. Base salary is just one part of the total rewards package, which also includes equity and comprehensive benefits.
What kind of team environment should I expect?
The team is small, highly motivated, and focused on engineering excellence. It operates with a flat organizational structure where all employees are expected to be hands-on.
What will I be working on day-to-day?
You will build reliable and efficient infrastructure that runs all evaluations at scale. This includes collaborating with modeling teams and resolving performance bottlenecks in inference and capacity management.