Role Overview
The Embedded AI Engineer position at Deepgram is a remote, full-time opportunity based in the USA, focused on optimizing voice AI models for non-NVIDIA accelerators and edge devices. This position is ideal for experienced professionals with a strong background in hardware-software interaction who want to scale advanced speech-to-text and text-to-speech technologies onto constrained hardware.
Key Responsibilities
Write and optimize custom kernels and operators using C, C++, Rust, and platform assembly or intrinsics for embedded SoCs, DSPs, NPUs, and non-NVIDIA accelerators.
Manage target-side optimization by collapsing models onto device execution units via quantization, operator fusion, memory layout, and architecture-specific compilation.
Integrate with vendor NPU and DSP toolchains, adding custom operators as needed when graphs do not map cleanly.
Supply reusable building blocks and runtime components to Applied ML Engineers with clear interfaces and documented constraints.
Develop performance-critical runtime code for RTOS targets, bare-metal environments, and embedded Linux.
Implement per-platform benchmarking and validation protocols to track latency, accuracy, power consumption, memory footprint, and utilization.
Collaborate directly with silicon and platform vendors on low-level performance tuning and SDK integration for new chipsets.
Communicate hardware constraints back to Research and Applied ML teams to simplify future model deployments on restricted targets.
Requirements
Experience delivering production-ready systems on resource-constrained devices, including embedded systems, mobile platforms, or edge AI hardware.
Strong proficiency in C, C++, and Rust for writing performance-critical code in restricted environments.
Practical experience optimizing models for on-device deployment through techniques like quantization, pruning, knowledge distillation, or architecture-specific compilation.
Familiarity with vendor-specific NPU or DSP toolchains and edge inference runtimes such as ONNX Runtime, TensorRT, TFLite, or ExecuTorch.
Solid understanding of hardware-software interactions, including memory hierarchies, CPU, GPU, NPU, and DSP architectures, fixed-point arithmetic, and power management.
Hands-on background working close to the metal with embedded Linux, microcontrollers, edge SoC development, or RTOS environments like FreeRTOS and Zephyr.
Strong communication skills paired with a builder mindset to scope ambiguous optimization challenges and explain trade-offs effectively.
Nice to Have
Background in real-time audio processing on embedded platforms, encompassing DSP pipelines, audio codec optimization, streaming inference, and wake-word listening.
Advanced depth in machine learning optimization methods like mixed-precision inference, neural architecture search, or custom quantization schemes.
Experience in hardware evaluation and benchmarking to systematically compare SoCs and accelerators.
Track record of shipping AI features at scale in consumer electronics on battery-powered devices.
Familiarity with model compilation and optimization toolchains across multiple hardware targets.
Experience implementing secure on-device deployment practices including encrypted model storage, code signing, and safe update mechanisms.
Skills and Tools
Rust
C++
C
Embedded Linux, RTOS (FreeRTOS, Zephyr), and bare-metal environments
ONNX Runtime, TensorRT, TFLite, and ExecuTorch
Vendor NPU and DSP toolchains
Custom kernel and operator development
Model quantization, pruning, and architecture-specific compilation
Salary and Benefits
Annual Salary Range: $219,300 to $274,100 USD
About the Company
Deepgram provides real-time APIs for speech-to-text, text-to-speech, and scalable voice agents, serving over 200,000 developers and 1,300 organizations globally. The organization maintains an AI-first operating rhythm where team members actively experiment with advanced AI tools in daily workflows. Operating at a rapid pace, the company values individuals who embrace change, adapt quickly, and thrive in dynamic environments.
How to Apply
Interested candidates should visit the official Deepgram careers page to submit an application. All official communications from the recruitment team originate exclusively from an @deepgram.com email address.
