Latest AI JobsBrowse Jobs
Cohere logo

Senior ML Systems Engineer, Frameworks & Tooling

CohereLondonModeling
Apply on company site
Salary
$250K - $535K CAD
Job type
Full-time
Work mode
Remote
Posted
1 Dec 2025
Status
Open · checked 7d ago

About this role

The Senior ML Systems Engineer, Frameworks & Tooling position at Cohere is a full-time role based in London or available remotely, designed for an experienced professional passionate about large-scale model training. This role suits a skilled systems engineer who wants to optimize infrastructure and connect advanced research concepts to thousands of GPUs.

Role Overview

Cohere seeks a senior engineer to shape, support, and evolve the training framework that drives frontier-scale language models. Operating at the crossroads of high-performance computing, distributed systems, and massive-scale training, the professional in this role will design core components for scalable, fast, and reliable model training. Responsibilities include building high-performance data loading pipelines, implementing system-wide performance profiling, developing monitoring metrics, and establishing fault-tolerant distributed checkpointing systems.

Key Responsibilities

Build and take ownership of the training framework utilized for large-scale language model training.
Formulate distributed training abstractions encompassing pipeline, tensor, and data parallelism alongside memory management and checkpointing strategies.
Enhance stability and training throughput across multi-node clusters featuring hardware like AMD, H200/100, and GB200/300.
Construct and maintain internal tooling for debugging, logging, developer ergonomics, and cluster monitoring.
Partner alongside infrastructure teams to guarantee hardware configurations, container environments, and cluster setups enable high-performance training.
Diagnose and fix performance bottlenecks throughout the machine learning systems stack.
Create robust systems guaranteeing reproducible and debuggable large-scale runs.

Requirements

Extensive engineering background in HPC systems or large-scale distributed training.
Deep familiarity regarding JAX internals, custom kernels, fused ops, or distributed training libraries.
Practical knowledge involving multi-node cluster orchestration tools such as Ray, Kubernetes, or Slurm.
Capability to debug performance challenges spanning networking, CUDA/NCCL, IO, and data pipelines.
Hands-on background utilizing containerization technologies like Docker and Singularity or Apptainer.
Proven history of delivering tools that elevate developer velocity for machine learning teams.
Strong judgment balancing maintainability, research velocity, performance, and complexity.
Exceptional collaboration abilities to partner smoothly with research, infrastructure, and deployment teams.

Nice to Have

Background working with large transformer architectures or training large language models.
Direct contributions to machine learning frameworks such as PyTorch, JAX, Megatron, DeepSpeed, or xFormers.
Acquaintance with serving and evaluation frameworks including custom KV caches, vLLM, or TensorRT-LLM.
Familiarity regarding sharded datasets, data pipeline optimization, or caching strategies.
Experience within low-level systems, profiling, or performance engineering.
Bonus qualification includes publishing a paper at top-tier venues like NeurIPS, ICML, ICLR, ACL, or EMNLP.

Skills and Tools

PyTorch
Kubernetes
Docker
LLMs
JAX
CUDA/NCCL
Distributed Systems
HPC Infrastructure

Salary and Benefits

Competitive compensation package with a salary range of 250,000 to 535,000 CAD per year.
Weekly lunch stipend equivalent to 75 USD or local currency.
Comprehensive health and dental coverage accompanied by a dedicated mental health budget.
Pension scheme, 401K, and RRSP matching programs.
Up to six months of 100% parental leave top-up for either parent.
Annual enrichment benefits covering fitness, wellness, arts, culture, workspace improvement, and a continuous education stipend.
Six weeks of paid vacation totaling 30 working days.
Travel budget to visit other company offices alongside an annual company offsite.
Home office setup stipend of 500 dollars for remote team members.

About the Company

Cohere functions as a security-first enterprise artificial intelligence company building cutting-edge foundation models and end-to-end products to address real-world business challenges. The organization operates globally with a headquarters in Toronto alongside key offices located in London, New York City, San Francisco, Montreal, Paris, Berlin, and Seoul.

How to Apply

Interested candidates should review the official company careers page at cohere.com to submit their application for this opportunity.

Skills and tech stack

PyTorchKubernetesDockerLLM

Questions candidates ask

Where is this job located?

This role can be performed remotely or based out of the London office, as well as several other international office locations.

What is the salary range for this position?

The salary ranges from 250,000 to 535,000 Canadian Dollars per year.

What perks are provided to full-time employees?

Perks include a weekly lunch stipend, full health and dental benefits, RRSP/401K/pension matching, six weeks of paid vacation, parental leave top-up, and learning stipends.

What technical skills are relevant for this role?

Key technical skills include PyTorch, Kubernetes, Docker, large language models, JAX, CUDA, and distributed systems orchestration.

How should I submit my application?

Candidates should apply directly through the official Cohere careers page.

Ready to apply?

The application is on the official Cohere careers page.

Apply on company site