Latest AI JobsBrowse Jobs
Hume AI logo

Staff Software Engineer - Inference Backends

Hume AIHume AI HQEngineering
Apply on company site
Salary
$170K – $280K
Job type
Full-time
Posted
17 Sept 2026
Status
Open · checked 7d ago

About this role

Hume AI is seeking a Staff Software Engineer to focus on inference backends at the Hume AI HQ location. This role is well-suited for a systems-oriented engineer ready to own the infrastructure that brings trained machine learning models into production environments.

Role Overview

As the first full-time engineer dedicated to inference backends at Hume AI, the successful candidate will manage the entire lifecycle of models moving from checkpoints to live production. Responsibilities span graph export, engine compilation, runtime integration, serving contracts, and ensuring optimal performance and correctness. Working autonomously, the engineer will oversee Hume's internal inference platform, handling routing, load balancing, capacity management, and safe model rollouts while collaborating closely with research scientists, machine learning engineers, and product teams.

Key Responsibilities

Manage the transition from trained model checkpoints to served requests, encompassing graph export, engine compilation, and runtime integration.
Develop and scale the internal inference platform to support multiple models and workloads across various products.
Design and operate infrastructure for load balancing, routing, health checking, autoscaling, and failure handling.
Create reproducible, versioned artifacts and tooling for validation, deployment, promotion, and rollback.
Implement verification gates to detect numerical, behavioral, and performance regressions before production releases.
Design and maintain standardized serving contracts and internal client libraries.
Profile and optimize latency, throughput, memory consumption, dynamic batching, scheduling, and accelerator utilization.
Troubleshoot complex production issues spanning applications, runtimes, containers, networks, drivers, and hardware.
Enhance observability, testability, resilience, and operational safety throughout the inference stack.
Author technical documentation for built systems and APIs.

Requirements

Significant professional background in server-side, infrastructure, distributed, or systems software engineering.
Solid Linux fundamentals with hands-on experience profiling and troubleshooting production environments.
Professional coding experience in at least one systems language such as Rust, C++, Go, or C.
Familiarity with distributed systems principles, including load balancing, capacity planning, observability, and failure recovery.
Practical knowledge of neural-network execution concepts, including computation graphs, tensor shapes, data types, and hardware accelerators.
Proven ability to profile and optimize production workloads.
Working knowledge of Python for model export, integration, and validation workflows.
Strong sense of ownership, independent technical judgment, and effective communication skills.
Capability to utilize AI-assisted coding tools while retaining the ability to independently debug, validate, and explain results.

Nice to Have

Prior experience operating or building shared model-serving platforms.
Familiarity with GPU inference, CUDA, ONNX, TensorRT, PyTorch, Triton, vLLM, or comparable frameworks.
Experience diagnosing numerical correctness challenges like precision loss, numerical drift, or nondeterminism.
Background in developing high-performance SDKs, client libraries, or networked systems.
Exposure to containers, CI/CD pipelines, and software deployment in customer-managed or on-premises environments.
Contributions to open-source projects focused on machine learning infrastructure, systems, or distributed computing.

Skills and Tools

Python
PyTorch
Rust
C++
Linux
Distributed Systems
Neural-Network Execution

Salary and Benefits

Annual Salary: $170,000 to $280,000 (USD, Year)

About the Company

Hume AI is a Series B startup headquartered in New York City dedicated to creating artificial intelligence optimized for human well-being. Recognized as the first company to release speech language models, Hume AI combines groundbreaking scientific research—with publications in journals like Nature—with practical enterprise applications and ethical guidelines supported through The Hume Initiative.

How to Apply

Interested candidates should visit the official Hume AI careers page to submit an application for this position.

Skills and tech stack

PythonPyTorchC++RustMachine LearningCUDADistributed Systems

Questions candidates ask

What is the salary range for the Staff Software Engineer - Inference Backends role?

The annual salary ranges from $170,000 to $280,000 USD.

Where is this position located?

This position is located at Hume AI HQ.

What programming languages are required or mentioned for the role?

The role requires professional experience with at least one systems-oriented language such as Rust, Go, C, or C++, alongside Python for model export and validation workflows.

What are the core responsibilities of this position?

The engineer will own the path from trained model checkpoint to production inference, build and evolve the internal inference platform, manage serving infrastructure, and ensure performance, reliability, and numerical correctness.

Is this a remote position?

No, the job listing indicates that remote is false and it is located at Hume AI HQ.

Ready to apply?

The application is on the official Hume AI careers page.

Apply on company site