Latest AI JobsBrowse Jobs
Cohere logo

Site Reliability Engineer, Inference Infrastructure

CohereTorontoInference
Apply on company site
Salary
$160K - $260K USD
Job type
Full-time
Work mode
Remote
Posted
12 Jan 2026
Status
Open · checked 7d ago

About this role

Role Overview

The Site Reliability Engineer, Inference Infrastructure position at Cohere in Toronto or remote is a full-time opportunity for professionals with over five years of engineering experience in production environments. This role suits individuals who are passionate about building scalable, high-performance machine learning systems and shaping the next generation of natural language processing platforms.

Key Responsibilities

Build self-service systems to automate the management, deployment, and operation of services, including custom Kubernetes operators for language model deployments.
Automate environment resilience and observability to empower all developers in troubleshooting and resolving issues efficiently.
Take necessary steps to meet defined service level objectives, which includes participating in an on-call rotation.
Forge strong partnerships with internal developers and guide the infrastructure roadmap using direct user feedback.
Contribute to team growth through active code reviews and knowledge sharing initiatives.

Requirements

Bring a minimum of 5 years of engineering experience managing large-scale production infrastructure.
Design large, highly available distributed systems utilizing Kubernetes and GPU workloads.
Demonstrate proficiency with Kubernetes development, production coding, and cluster support.
Work with multi-cloud, hybrid, or on-premises serving environments across GCP, Azure, AWS, and OCI.
Design, deploy, support, and troubleshoot complex Linux-based computing environments.
Manage compute, storage, network resources, and infrastructure costs effectively.
Apply knowledge of distributed systems alongside programming experience in Golang, C++, or similar languages built for high-performance servers.
Understand the computational traits of hardware accelerators like GPUs and TPUs and their impact on inference latency and throughput.

Skills and Tools

AWS
GCP
Azure
Kubernetes
Golang
C++
NLP

Salary and Benefits

Annual salary ranging from $160,000 to $260,000 USD.
Weekly lunch stipend of $75 or local currency equivalent, alongside a daily lunch program for office-based staff.
Comprehensive health and dental benefits, featuring a dedicated mental health budget.
Retirement plans including RRSP matching, 401K, and Pension Schemes.
Full parental leave top-up of up to 6 months for either parent.
Annual enrichment benefits covering wellness, fitness, arts, culture, and workspace improvements.
Education and learning stipend designated for courses, conferences, and coaching.
30 working days of paid vacation per year (6 weeks).
Travel budget for visiting other company offices and attending annual company offsites.
Home office setup stipend of $500 or co-working space benefits for remote workers.

About the Company

Cohere is a security-first enterprise artificial intelligence company creating cutting-edge foundation models and end-to-end products to address real-world business challenges. Headquartered in Toronto with additional major global offices, the organization brings together researchers, designers, and engineers dedicated to driving the widespread enterprise adoption of AI.

How to Apply

Interested candidates should visit the official Cohere careers page to submit an application for this position.

Skills and tech stack

AWSGCPAzureKubernetesGolangC++NLP

Questions candidates ask

What is the salary range for this role?

The position offers an annual salary between $160,000 and $260,000 USD.

Is this position remote?

Yes, the role is remote-friendly, with options to work from Cohere offices in Toronto and other global locations.

How much experience is required?

Candidates must have 5+ years of engineering experience running production infrastructure at a large scale.

What programming languages are preferred?

Experience in Golang, C++, or other languages designed for high-performance scalable servers is required.

What perks and benefits are included for full-time employees?

Perks include health and dental benefits, retirement matching, 6 weeks of paid vacation, parental leave top-ups, learning stipends, and a weekly lunch allowance.

Ready to apply?

The application is on the official Cohere careers page.

Apply on company site