Latest AI JobsBrowse Jobs
Inworld AI logo

Staff / Principal Machine Learning Engineer, Serving - USA

Inworld AIMountain View, California, USAML Engineering
Apply on company site
Salary
$270K - $500K USD
Job type
Full-time
Work mode
Remote
Posted
7 Apr 2026
Status
Open · checked 9d ago

About this role

Role Overview

Inworld AI is hiring a Staff or Principal Machine Learning Engineer for Serving to lead real-time AI infrastructure development from Mountain View or via remote arrangements. This technical role suits experienced systems and machine learning professionals who want to optimize large-scale inference and low-latency workloads for millions of daily users.

Key Responsibilities

Take ownership of machine learning models from the research team, containerize them, optimize serving performance, and ensure production reliability.
Build and maintain high-performance, distributed serving infrastructure capable of reliably handling thousands of concurrent connections.
Solve complex latency and throughput challenges to support real-time, high-volume AI applications.
Collaborate in a flat organizational structure with rapid iterations and minimal bureaucracy.

Requirements

Deep understanding of modern serving frameworks and techniques like vLLM or TRT-LLM.
Hands-on background in model acceleration, including quantization, distillation, caching strategies, continuous batching, paged attention, and speculative decoding.
Proficiency in high-performance systems programming languages such as C++, CUDA, Rust, or highly optimized Python.
Experience with distributed systems and scaling tools like Kubernetes, Ray, custom load balancing, and multi-node GPU inference.
Track record of full-cycle ownership for complex backend or machine learning systems.

Skills and Tools

Python
Kubernetes
Rust
C++
CUDA
LLM
Ray
vLLM and TRT-LLM frameworks

Salary and Benefits

Base salary range: $270,000 to $500,000+ per year
Bonus opportunities included
Equity compensation
Comprehensive benefits package
Relocation assistance for candidates joining the Mountain View office

About the Company

Inworld AI operates as a research lab and inference provider specializing in real-time AI for consumer-facing applications, serving over 10 trillion LLM tokens monthly. The company has raised more than $125 million from prominent investors including Lightspeed Venture Partners, Kleiner Perkins, Microsoft’s M12, and Founders Fund. Recognized by CB Insights as a top global AI startup and featured on LinkedIn’s Top 10 Startups in the USA, Inworld powers technology for industry leaders like NVIDIA and Microsoft Xbox.

How to Apply

Interested candidates should visit the official Inworld AI careers page to submit an application.

Skills and tech stack

PythonKubernetesRustC++LLM

Questions candidates ask

What is the salary range for the Staff / Principal ML Engineer, Serving role?

The base salary range spans from $270,000 to $500,000+ per year, complemented by a bonus, equity, and benefits.

Is this position remote?

Yes, this position offers remote flexibility alongside options to work from the company office in Mountain View, California, with relocation assistance available.

What primary programming languages and tools are required?

Candidates should be proficient in C++, CUDA, Rust, or highly optimized Python, alongside experience with frameworks like vLLM, TRT-LLM, Kubernetes, and Ray.

What scale does Inworld AI operate at?

Inworld AI powers applications used by hundreds of millions of users daily and serves over 10 trillion LLM tokens every month.

How should I apply for this job?

You can apply directly through the official Inworld AI company careers page.

Ready to apply?

The application is on the official Inworld AI careers page.

Apply on company site