About this role
Engineers who thrive in complex environments and enjoy building high-performance systems for frontier AI will find a natural home on this team. Cohere maintains a dedicated infrastructure group responsible for the foundational tools and compute clusters used to train, evaluate, and serve enterprise AI models. This practitioner will collaborate directly with AI researchers to design, operate, and scale multi-cloud GPU and TPU superclusters optimized for demanding machine learning workloads.
Operating at the intersection of systems engineering and artificial intelligence, the position focuses heavily on stability, scalability, and observability. Responsibilities include deploying Kubernetes-based clusters, fine-tuning infrastructure for cost efficiency and low-latency performance, and building self-service tools that empower researchers to manage training jobs independently. Candidates who combine strong programming abilities in Python and Go with deep knowledge of low-level Linux internals and distributed training frameworks will succeed in addressing the unique scaling challenges presented by modern foundation models.
What you will do
- Deploy and manage Kubernetes-based GPU and TPU superclusters across multiple cloud environments to support high-throughput AI workloads.
- Collaborate with cloud providers to optimize infrastructure performance, reliability, and cost efficiency using technologies like RDMA and NCCL.
- Identify and resolve infrastructure bottlenecks, performance degradation, and system failures to maintain continuous AI and ML workflows.
- Design intuitive interfaces and self-service tools that enable researchers to monitor, debug, and optimize training jobs independently.
- Partner with AI researchers to translate emerging distributed training needs into robust, scalable infrastructure solutions.
- Advocate for observability, automation, and infrastructure-as-code practices across the engineering organization.
- Participate in a 24x7 on-call rotation with appropriate scheduling compensation.
What you need
- Demonstrated expertise in machine learning and high-performance computing infrastructure, including experience with GPU or TPU clusters and distributed training frameworks such as JAX, PyTorch, and TensorFlow.
- Proven ability to deploy, manage, and troubleshoot cloud-native Kubernetes clusters at scale for AI workloads.
- Strong programming skills in Python for machine learning tooling and Go for systems engineering.
- Familiarity with low-level systems concepts including Linux internals, RDMA networking, and performance optimization.
- Experience collaborating directly with AI researchers or machine learning engineers to solve complex infrastructure challenges.
- Self-directed problem-solving abilities focused on identifying bottlenecks and driving impact in a fast-paced setting.
Benefits
- Weekly lunch stipend of $75/£75 or local currency equivalent
- Full health and dental benefits with a separate mental health budget
- RRSP matching, 401K, and Pension Scheme
- 100% Parental Leave top-up for up to 6 months
- Annual enrichment benefits covering arts, fitness, wellness, workspace improvement, and education
- 6 weeks of paid vacation
Skills and tech stack
Questions candidates ask
What is the remote policy for this role?
Cohere operates as a remote-friendly company while maintaining physical offices in Toronto, London, New York City, San Francisco, Montreal, Paris, Berlin, and Seoul. Remote employees receive a co-working benefit to work alongside others locally, along with a home office setup stipend.
What compensation and financial perks are provided?
The salary range for this position is CA$285K to CA$340K with included equity. Additional financial perks include retirement savings options like RRSP matching, 401K, and pension schemes, plus weekly lunch stipends and annual enrichment budgets.
What are the on-call requirements for infrastructure team members?
All infrastructure roles require participating in a 24x7 on-call rotation. Team members are compensated for their participation in the on-call schedule.
What tools and technologies will I be working with?
The tech stack centers around Kubernetes-based GPU and TPU superclusters across multiple clouds. Engineers utilize Python for machine learning tooling, Go for systems engineering, and technologies like RDMA, NCCL, JAX, PyTorch, and TensorFlow.