Latest AI JobsBrowse Jobs
Cohere logo

Staff Software Engineer, Inference Infrastructure

CohereSan FranciscoInference
Apply on company site
Salary
$190K - $325K USD
Job type
Full-time
Work mode
Remote
Posted
12 Jan 2026
Status
Open · checked 7d ago

About this role

Role Overview

The Staff Software Engineer, Inference Infrastructure role at Cohere in San Francisco (with remote options) involves building and maintaining high-performance machine learning systems that power advanced natural language processing applications. This position is ideal for experienced engineers who want to contribute to the deployment and operation of large language models through scalable API endpoints.

Key Responsibilities

Develop, deploy, and operate the AI platform that delivers large language models through easy-to-use API endpoints.
Work closely with multiple teams to push optimized NLP models into production within low-latency, high-throughput, and high-availability environments.
Interface directly with customers to build customized deployments tailored to their unique operational needs.
Design, support, and troubleshoot complex Linux-based computing environments.
Handle compute, storage, and network resource allocation alongside cost management.

Requirements

Bring at least 5 years of professional engineering experience operating large-scale production infrastructure.
Demonstrate background in designing large, highly available distributed systems utilizing Kubernetes and managing GPU workloads.
Possess practical knowledge of Kubernetes development, production coding, and cluster support.
Show experience working across multiple cloud providers such as GCP, Azure, AWS, and OCI, as well as hybrid or on-premise serving environments.
Understand the computational traits of hardware accelerators like GPUs, TPUs, or custom chips and their impact on inference latency and throughput.
Demonstrate coding proficiency in Golang, C++, or other languages built for high-performance scalable servers.

Skills and Tools

AWS, GCP, Azure, OCI
Kubernetes
Golang, C++
Linux
NLP models and GPU/TPU accelerators

Salary and Benefits

Annual base salary ranging from $190,000 to $325,000 USD.
Weekly lunch stipend of $75 or local currency equivalent, alongside daily lunch and snacks for those working from an office.
Comprehensive health and dental benefits, including a dedicated mental health budget.
Retirement plans including RRSP matching, 401(k), and pension schemes.
Up to 6 months of 100% parental leave top-up for either parent.
Annual enrichment benefits covering arts and culture, fitness and wellness, quality time, workspace improvement credits, and education and learning stipends.
6 weeks (30 working days) of paid vacation.
Travel budget for remote employees to visit other company offices, plus an annual company offsite and a $500 home office stipend.

About the Company

Cohere is a security-first enterprise artificial intelligence company building cutting-edge foundation models and end-to-end products designed for real-world business challenges. Headquartered in Toronto with global offices including London, New York City, San Francisco, Montreal, Paris, Berlin, and Seoul, the company unites researchers, engineers, and designers passionate about expanding AI capabilities for enterprises.

How to Apply

Visit the official Cohere careers page to submit an application for this position.

Skills and tech stack

AWSGCPAzureKubernetesGolangC++NLP

Questions candidates ask

What is the salary range for the Staff Software Engineer, Inference Infrastructure role?

The annual salary for this position ranges from $190,000 to $325,000 USD.

Is this position remote?

Yes, Cohere is remote-friendly, and this role is based out of San Francisco with remote flexibility.

How many years of experience are required?

Candidates must have 5+ years of engineering experience running production infrastructure at a large scale.

What programming languages are preferred for this role?

Experience in Golang, C++, or other languages designed for high-performance scalable servers is required.

What perks and benefits are offered to full-time employees?

Full-time employees receive health and dental benefits, retirement plans like 401(k) and RRSP matching, 6 weeks of paid vacation, parental leave top-up, learning stipends, home office stipends, and weekly lunch allowances.

Ready to apply?

The application is on the official Cohere careers page.

Apply on company site