Latest AI JobsBrowse Jobs
Cohere logo

Member of Technical Staff, Model Efficiency

CohereNew YorkModeling
Apply on company site
Salary
$250K - $535K CAD
Job type
Full-time
Work mode
Remote
Posted
7 Nov 2025
Status
Open · checked 7d ago

About this role

Role Overview

The Member of Technical Staff, Model Efficiency role at Cohere in New York is a full-time position focused on building reliable machine learning systems and improving large language model inference performance. This role suits experienced software engineers and researchers who want to optimize frontier artificial intelligence models for enterprise production environments.

Key Responsibilities

Work across the inference stack to enhance core performance metrics by analyzing model execution and removing bottlenecks.
Design and implement innovative optimizations to drive down latency and increase throughput.
Partner closely with modeling and systems engineering teams to test, measure, and release performance upgrades.
Develop advanced performance techniques including GPU and CUDA optimizations, kernel-level adjustments, and execution strategies for Mixture of Experts and large-scale architectures.

Requirements

Minimum of five years of professional experience writing production-grade, high-performance code.
Strong proficiency in C++ or Python, with Rust and Go also considered acceptable.
Practical background working with large language models and familiarity with the LLM inference ecosystem such as vLLM and SGLang.
Demonstrated capability to identify and fix performance bottlenecks throughout the model execution stack.
Strong bias for action emphasizing rapid shipping, impact measurement, and continuous iteration.

Nice to Have

Background in GPU programming, CUDA, or low-level systems optimization.
Experience with transformer language modeling covering Mixture of Experts, speculative decoding, and KV-cache optimizations.
Expertise in scaling distributed systems that are performance-critical across computation, search, or storage.

Skills and Tools

Python
Rust
C++
Large Language Models (LLMs)
Inference optimization frameworks (vLLM, SGLang)
GPU and CUDA programming

Salary and Benefits

Annual salary range between 250,000 and 535,000 CAD.
Weekly lunch stipend of $75 or local currency equivalent.
Comprehensive health and dental coverage including a dedicated mental health budget.
Retirement plans including 401K, RRSP matching, and pension scheme.
Fully paid parental leave top-up for up to six months for either parent.
Annual enrichment benefits covering wellness, fitness, arts, culture, and workspace improvements.
Dedicated education and learning stipend for coaching, courses, and conferences.
Six weeks of paid vacation totaling 30 working days.
Travel budget to visit other company offices and attend annual company offsites.
Home office setup stipend of $500 or a co-working benefit for remote employees.

About the Company

Cohere is a leading security-first enterprise artificial intelligence company that builds cutting-edge foundation models and end-to-end products to solve real-world business problems. The organization operates globally with a headquarters in Toronto and major offices in London, New York City, San Francisco, Montreal, Paris, Berlin, and Seoul. Cohere embraces a flexible, remote-friendly environment while strategically distributing teams based on time zones, interests, and expertise to foster collaboration.

How to Apply

Interested candidates should submit their application directly on the official Cohere careers page.

Skills and tech stack

PythonRustC++LLM

Questions candidates ask

What is the salary range for this role?

The annual salary for this position ranges from 250,000 to 535,000 CAD.

Where is this job located?

This position can be based remotely or out of any of Cohere office locations, including New York, with team members primarily concentrated in EST and PST time zones.

What level of experience is required?

Applicants must have at least five years of experience writing high-performance, production-quality code.

What programming languages are expected?

Strong programming skills in C++ or Python are required, while experience with Rust or Go is also welcome.

How much paid vacation do employees receive?

Full-time employees at Cohere receive six weeks of paid vacation, which equals 30 working days.

How should I apply for this position?

You should apply directly through the official Cohere careers page.

Ready to apply?

The application is on the official Cohere careers page.

Apply on company site