Latest AI JobsBrowse Jobs
Cohere logo

Audio Inference Engineer, Model Efficiency

CohereNew YorkModeling
Apply on company site
Salary
$250K - $535K CAD
Job type
Full-time
Work mode
Remote
Posted
7 Nov 2025
Status
Open · checked 7d ago

About this role

Cohere is hiring an Audio Inference Engineer to join the Model Efficiency team in New York or on a remote basis. This full-time role suits an experienced engineer who wants to optimize machine learning systems and improve real-time audio inference serving performance.

Role Overview

The Model Efficiency team is a fast-growing group of researchers and engineers building reliable machine learning systems. The mission focuses on optimizing audio inference serving efficiency through innovative techniques. In this position, professionals advance core audio model serving metrics like latency, throughput, and quality. Team members analyze systems, pinpoint bottlenecks, and deliver creative solutions for audio processing and streaming workloads. Professionals collaborate closely with training and serving infrastructure teams to ensure seamless integration between model development and deployment, prioritizing real-time and streaming audio inference.

Key Responsibilities

Build reliable machine learning systems and optimize audio inference serving efficiency using innovative techniques.
Advance core audio model serving metrics, including latency, throughput, and quality.
Dive deep into systems, identify bottlenecks, and deliver creative solutions for audio processing and streaming workloads.
Collaborate closely with training and serving infrastructure teams to ensure seamless integration between model development and deployment.
Focus heavily on real-time and streaming audio inference workflows.

Requirements

Significant experience developing high-performance audio or machine learning inference systems.
Proficiency with programming languages such as C++ and Python.
Hands-on experience with deep learning models for audio, speech, or language applications.
A bias for action and a strong results-oriented mindset.

Nice to Have

Considerable experience with GPU programming, low-level system optimization, and model parallelization techniques over multiple GPUs.
Experience with duplex real-time streaming architectures.
Familiarity with the internals of machine learning frameworks for audio, such as PyTorch, TensorFlow, or specialized audio libraries.
Experience with inference frameworks like vLLM, SGLang, TensorRT-LLM, or custom distributed inference systems.
Background in sequence modeling, such as transformers for audio or speech, and end-to-end audio pipeline optimization.

Skills and Tools

Python
PyTorch
TensorFlow
C++
LLMs
GPU Programming and Low-Level Optimization
Inference Frameworks (vLLM, SGLang, TensorRT-LLM)

Salary and Benefits

Annual salary ranging from $250,000 to $535,000 CAD.
Weekly lunch stipend of $75/£75 or local currency equivalent.
Full health and dental benefits, featuring a separate mental health budget.
Retirement savings plans including RRSP matching, 401K, and Pension Scheme.
100% parental leave top-up for up to 6 months for either parent.
Annual enrichment benefits covering arts and culture, fitness and wellness, quality time, and a workspace improvement credit.
Education and learning stipend for conferences, courses, and coaching.
30 working days (6 weeks) of paid vacation.
Travel budget for visiting other offices and attending an annual company offsite.
$500 home office stipend or co-working benefit for remote staff.

About the Company

Cohere is a leading security-first enterprise AI company that builds cutting-edge foundation AI models and end-to-end products to solve real-world business problems. The organization trains and deploys frontier models for enterprises building AI systems, believing that this work drives widespread AI adoption. Cohere operates as a global technology company headquartered in Toronto with key offices in London, New York City, San Francisco, Montreal, Paris, Berlin, and Seoul.

How to Apply

Interested candidates should submit their application directly through the official company careers page.

Skills and tech stack

PythonPyTorchTensorFlowC++LLM

Questions candidates ask

What is the salary range for the Audio Inference Engineer role?

The salary for this full-time position ranges from $250,000 to $535,000 CAD per year.

Can this position be done remotely?

Yes, this role can be based remotely or from one of the company's office locations, with preferred locations concentrated in the EST and PST time zones.

What programming languages are required for the job?

Candidates must have proficiency in programming languages such as C++ and Python.

What perks and benefits does Cohere offer?

Full-time employees enjoy full health and dental benefits, a weekly lunch stipend, 6 weeks of paid vacation, a 401K/RRSP matching program, parental leave top-ups, and learning stipends.

How should I apply for this position?

Candidates should apply directly on the official Cohere careers page.

Ready to apply?

The application is on the official Cohere careers page.

Apply on company site