Latest AI JobsBrowse Jobs
Lilt logo

Machine Learning Engineer (Real-Time Speech Translation)

LiltWashington D.C.Engineering
Apply on company site
Salary
$120K – $161,434
Job type
Full-time
Work mode
Remote
Experience
Mid · 3+ yrs
Posted
2 Oct 2026
Status
Open · checked 7d ago

About this role

Machine Learning Engineer (Real-Time Speech Translation) is a full-time backend engineering role based in Washington D.C. that suits professionals with strong Python and ML serving experience looking to own a real-time speech translation backend. Lilt is hiring for this position to build a new live translation product that streams audio inputs to translated outputs.

Role Overview

This position focuses on building the real-time speech translation backend for a new live translation product at Lilt. The professional in this role will own the backend from live audio input to translated output, utilizing Ray Serve on GPU Kubernetes clusters and in-house adaptive machine translation models. Candidates are preferred to be located in Washington, D.C.; Boston, MA; or Indianapolis, IN, with US citizenship and residence in the United States required.

Key Responsibilities

Build and manage services for high-throughput, real-time audio and text streaming, handling signal processing, session lifecycles, and concurrency management.
Integrate and serve streaming speech recognition and machine translation models to operate within strict latency budgets.
Develop logic for model-based confidence scoring, routing segments for human intervention, and broadcasting real-time updates to users.
Architect and scale production machine learning infrastructure on GPU-accelerated Kubernetes clusters with batching, load balancing, and autoscaling.
Establish instrumentation for real-time performance to identify bottlenecks and reduce end-to-end latency metrics.
Define technical contracts and interfaces for audio ingestion and downstream service integrations in partnership with frontend and platform teams.

Requirements

BS or MS in Computer Science or a related field, or equivalent practical experience.
3+ years building production backend or ML serving systems in Python, including async programming (asyncio) skills.
Hands-on experience with real-time streaming transport like WebSocket or gRPC bidirectional streaming, session state, backpressure, and connection lifecycle handling.
Experience serving machine learning models in production on GPUs using Ray Serve, Triton, vLLM, or similar tools, alongside Docker and Kubernetes.
Experience integrating speech or NLP models into production systems, such as streaming ASR with partial hypotheses, endpointing, and VAD.
Latency-engineering mindset with a background in profiling, instrumenting, and optimizing real-time or low-latency systems.
US citizenship and residence in the United States.

Nice to Have

Experience with Ray Serve specifically, including streaming responses and model multiplexing.
Familiarity with simultaneous or incremental machine translation concepts like retranslation, prefix stability, and wait-k policies.
Machine translation quality estimation or other confidence estimation in production.
Message brokers for real-time fan-out and state distribution such as RabbitMQ.
Streaming text-to-speech integration and time-to-first-audio optimization.
WebRTC and SFU concepts, or voice pipeline frameworks like LiveKit Agents or Pipecat.
Handling of CJK and other non-Latin text in natural language processing pipelines.
Observability tooling like Datadog or Prometheus for production ML systems.

Skills and Tools

Python
asyncio
WebSocket
gRPC
Ray Serve
Triton
vLLM
Docker
Kubernetes
RabbitMQ
Datadog
Prometheus

Salary and Benefits

Salary range of $120,000 to $161,434 with multiple ranges.

About the Company

Lilt is an artificial intelligence company on a mission to make global information accessible regardless of language. Founded in 2015 by former Google Translate researchers, the company combines machine translation, large language models, human-in-the-loop expertise, and agentic AI. Lilt is trusted by enterprise clients and organizations including Intel Corporation, Canva, the United States Department of Defense, the United States Air Force, and ASICS, and is backed by Sequoia, Intel Capital, and Redpoint.

How to Apply

Candidates should apply directly on the official Lilt careers page.

Skills and tech stack

PythonasyncioWebSocketgRPCRay ServeTritonvLLMDockerKubernetesRabbitMQDatadogPrometheusNLPMachine Learning

Questions candidates ask

What are the location and citizenship requirements for this job?

Applicants must be US citizens and reside in the United States. Preferred locations include Washington, D.C., Boston, MA, and Indianapolis, IN.

What experience level is required for this role?

The role requires 3+ years of experience building production backend or machine learning serving systems in Python.

What tools and frameworks will I use on the job?

The tech stack includes Python, asyncio, WebSockets, gRPC, Ray Serve, Triton, vLLM, Docker, Kubernetes, RabbitMQ, Datadog, and Prometheus.

What is the salary range for this position?

The salary ranges from $120,000 to $161,434.

What educational background is required?

A BS or MS in Computer Science or a related field, or equivalent practical experience, is required.

Ready to apply?

The application is on the official Lilt careers page.

Apply on company site