About this role
Scaling a rapidly growing compute footprint requires solving complex challenges in hardware, distributed systems, and AI research to support frontier model training. The GPU Clusters team at Cohere builds and operates the superclusters that power these foundation models, sitting squarely at the intersection of infrastructure and machine learning.
As an Engineering Manager for GPU Infrastructure, this person will lead a dedicated team of engineers focused on performance, career development, and difficult technical infrastructure problems. Responsibilities include setting the technical roadmap for deployment, operation, and scaling of Kubernetes clusters while partnering closely with researchers and cross-functional stakeholders.
Candidates who thrive in this environment possess a strong background in running large compute fleets and managing distributed engineering or SRE teams. Someone who enjoys deep technical challenges, strategic capacity planning, and hands-on operational excellence will find this role a natural fit.
What you will do
- Hire, mentor, and grow a team of GPU infrastructure engineers by providing performance reviews, career development, and technical guidance on hard infrastructure problems.
- Own the technical roadmap for the fleet regarding how the organization deploys, operates, and scales Kubernetes clusters.
- Partner directly with researchers and ML engineers to ensure the training and inference stack functions effectively on new GPU architectures.
- Collaborate with cross-functional stakeholders such as Capacity, Finance, Legal, Security, and other infrastructure teams on planning, cost, compliance, and shared dependencies.
- Drive operational excellence through observability for GPU utilization and reliability, automation of cluster provisioning, cost optimization, and vendor relationships.
What you need
- Bring experience managing engineering or SRE teams with a focus on technical mentorship, hiring, and growth in remote, distributed settings.
- Possess a background running large Kubernetes compute fleets in production across multi-cloud environments.
- Demonstrate deep knowledge in at least one layer of a GPU training fleet, such as cluster-wide operations, GPU networking, or hardware, with willingness to learn the rest hands-on.
- Show experience handling cost optimization and capacity planning specifically for GPU infrastructure.
- Maintain a track record of partnering with researchers or ML engineers while making data-informed tradeoffs across reliability, cost, and delivery.
Benefits
- Weekly lunch stipend of $75/£75 or local currency equivalent
- Full health and dental benefits, including a separate budget for mental health
- RRSP matching, 401K, and Pension Scheme
- 100% Parental Leave top-up for up to 6 months
- Annual enrichment benefits for arts, culture, fitness, wellness, and education
- 6 weeks of paid vacation
Skills and tech stack
Questions candidates ask
What is the remote policy for this role?
Cohere is remote-friendly, and employees can work from home with a $500 home office stipend. The company also maintains offices in Toronto, London, New York City, San Francisco, Montreal, Paris, Berlin, and Seoul for those who prefer an office environment.
What tools and technologies are used on the team?
The team works heavily with GPU infrastructure and large Kubernetes compute fleets in production multi-cloud environments. Responsibilities also touch upon infrastructure as code and infrastructure monitoring tools.
What kind of growth and development support is available?
Employees receive an education and learning stipend for conferences, courses, and coaching. Management is also actively involved in career development, mentorship, and growing the team.
What perks are provided for remote employees not near an office?
Those not near an office receive a co-working benefit so they can work alongside others in their city. Additionally, remote employees get a budget for traveling to other offices and attending the annual company offsite.