About this role
Modal needs to solve the challenge of delivering frontier performance for open-source models with high elasticity and developer experience as AI workloads demand a completely new infrastructure layer. The infrastructure team builds an environment that provides instant GPU access, sub-second container starts, and native storage for category-defining companies.
As the Inference Engineering and Product Lead based in San Francisco, this person will direct the LLM inference platform spanning the serving stack, routing infrastructure, internal agentic optimization platform, and user-facing product surface area. The role exists to scale the platform, guide customer engagements with frontier workloads, and translate those insights into a cohesive product roadmap.
A successful candidate combines deep technical capability in systems engineering with product vision and people management skills, balancing coding, architecture design, and team leadership.
What you will do
- Recruit, hire, and mentor a high-performing engineering team through regular coaching and career development.
- Drive technical and product decisions by participating in design reviews, code reviews, and architectural discussions.
- Collaborate with frontier customers to ensure the successful deployment of novel workloads on the platform.
- Translate customer learnings into a robust roadmap for internal optimization and user-facing products.
- Establish strict standards for reliability, ownership, and product excellence from specification through production.
- Partner with business operations and compute strategy teams to shape procurement plans across various accelerators.
- Work alongside go-to-market teams to support product launches, messaging, and deal win rates.
What you need
- Ten or more years of professional industry experience, including at least three years in a leadership capacity.
- Demonstrated track record of building high-performance systems operating at scale.
- Strong foundational background in cloud infrastructure environments.
- Deep working knowledge of low-level operating system foundations such as the Linux kernel, file systems, and containers.
Nice to have
- Experience working with LLM inference in production environments
- Familiarity with underlying concepts such as inference engines, kernels, routing, KV cache management, and speculative decoding
Benefits
- $300K – $350K salary
- Offers Equity
Skills and tech stack
Questions candidates ask
What will I actually be working on in this role?
You will own the direction and execution of Modal's LLM inference platform, spanning the serving stack, routing infrastructure, internal agentic optimization platform, and user-facing product surface area. You will split your time between technical contribution, product shaping, and people management.
What is the compensation package for this position?
The position offers a salary range of $300K to $350K along with equity offerings.
Who are some of the customers and partners I will interact with?
You will work closely with category-defining companies such as Cognition, DoorDash, Ramp, Lovable, and Suno, helping them run novel or frontier workloads successfully.