Senior Manager, Engineering - AI Inference
About this role
A problem-solver with a drive for impact and a comfort level operating at scale would find this role suitable. This position is for a Senior Engineering Manager who will guide an engineering team dedicated to improving the performance, cost, and reliability of large language models in production. Crusoe operates as a vertically integrated AI infrastructure company, meaning the work is directly tied to powering ambitious AI workloads by solving the bottleneck of AI compute and energy. The ideal candidate is someone who thrives in an environment where technical optimization directly translates into real customer deployments and measurable business value.
This role requires a unique blend of people management and deep technical expertise. The manager will not only lead the team but will also remain hands-on, engaging in performance profiling and diving into the serving code to implement modern optimization techniques. You will bridge the gap between abstract performance goals and concrete production solutions, working closely with customer engineering teams to transition proof-of-concept work into stable, monitored services.
If you are motivated by tangible results, enjoy navigating ambiguity to find the right technical path, and want to build infrastructure that has a massive real-world impact, this opportunity aligns well with the mission of Crusoe.
What you will do
- Lead an engineering team focused on optimizing large language models to run faster, more cheaply, and more reliably in production environments.
- Perform end-to-end work on the inference stack, including profiling performance and integrating modern optimization techniques into live deployments.
- Design and optimize serving architectures such as prefill and decode disaggregation and request routing strategies.
- Dive into the serving stack, examining frameworks like vLLM and SGLang, and analyzing CUDA kernels to resolve performance issues.
- Adapt and scale optimization methods across various ML models, with a specific focus on large language models.
- Profile and tune deployments against defined targets for latency, throughput, and cost while ensuring dependability under live traffic.
- Collaborate with customer engineering teams to tailor deployments, moving workloads from proof of concept to production services.
What you need
- Possess at least two years of experience directly managing and leading an engineering team within a high-performance or machine learning focused setting.
- Demonstrate strong, hands-on technical experience in software engineering, low-level optimization, or ML infrastructure, maintaining a close connection to code and architecture.
- A Bachelor's, Master's, or Ph.D. in a quantitative field such as Computer Science, Engineering, Mathematics, or a related discipline is required.
- Must have practical experience shipping production code using one or more general-purpose languages, with a strong preference for Python.
- Familiarity with established methods for optimizing Large Language Models for high throughput is necessary for success in this role.
Nice to have
- Experience with CUDA kernels
Benefits
- $250,000 - $300,000 salary
Skills and tech stack
Questions candidates ask
What is the remote policy for this role?
The job is located in San Francisco, CA - US.
What is the compensation range for this position?
The salary range for this role is between $250,000 and $300,000.
What is the primary tech stack used for inference optimization?
The role involves working down into serving frameworks like vLLM and SGLang, and analyzing CUDA kernels underneath.
How is technical direction handled in this role?
The role involves guiding projects from early experiments to production optimizations, establishing performance goals, and partnering with product and engineering leaders on technical strategy and roadmaps.