About this role
Porting Deepgram speech models to run efficiently on non-NVIDIA accelerators, edge servers, and embedded platforms forms the core of daily responsibilities. Operating within the Partner Platform Engineering team, the engineer sits one layer above the metal, adapting model structures, choosing quantization schemes, and validating accuracy on real target devices while collaborating with Embedded AI Engineers on custom kernels.
Deepgram provides real-time speech-to-text and text-to-speech APIs underpinning the Voice AI economy, serving over 1,300 organizations. The company expects an AI-first mindset where team members rapidly experiment with and integrate AI tools into everyday workflows. Ideal candidates possess a background in shipping models to non-GPU or edge hardware and enjoy bridging the gap between kernel development and model architecture.
What you will do
- Port Deepgram speech models to non-NVIDIA and edge platforms by adapting model structure and parameters to fit target operator sets and runtimes.
- Manage serving-side model decisions for edge targets including quantization, precision choices, graph rewrites, and architecture tweaks.
- Validate ports on real hardware by building accuracy, latency, throughput, and memory benchmarks for each platform.
- Build deployment paths for edge targets encompassing model packaging, conversion pipelines, versioning, and automated delivery.
- Collaborate with Embedded AI Engineers to specify model requirements and integrate custom kernels when standard solutions fall short.
- Partner with platform and silicon vendors to align their expected model formats and operator conventions with Deepgram deployments.
- Communicate edge constraints back to Research and Impeller teams to facilitate easier porting for future models.
What you need
- Hands-on experience deploying machine learning models to edge or non-NVIDIA hardware in production environments.
- Working knowledge of quantization and precision tradeoffs such as INT8, FP16, and mixed precision calibration.
- Experience utilizing at least one edge or vendor inference runtime and its conversion toolchain.
- Ability to modify models by reading and rewriting model graphs, swapping operators, and adjusting architecture parameters.
- Strong proficiency in Python and PyTorch alongside production-quality engineering habits.
- Comfort building automation systems around model conversion and deployment.
- Clear communication skills and a builder mindset to scope and drive ports on unfamiliar platforms.
Nice to have
- Experience with speech, audio, or streaming and real-time models
- Exposure to writing or reading low-level kernels including CUDA, Metal, NEON, or vendor DSP code
- Experience with model security or integrity on deployed devices
- Familiarity with several accelerator families such as Qualcomm, Apple, ARM, Intel, AMD, or custom NPUs
- Track record of building internal tooling that accelerates model porting or deployment
Benefits
- Offers Equity
- Offers Bonus
- 10% Annual Bonus
Skills and tech stack
Questions candidates ask
What is the remote work policy?
The role is designated as USA - Remote.
What is the compensation range for this position?
The salary ranges from $155K to $245K, alongside offered equity, bonus options, and a 10% annual bonus.
What kind of engineering work is expected on a daily basis?
You will adapt Deepgram speech models to run on non-NVIDIA accelerators and edge servers, handle quantization and precision choices, validate benchmarks on real hardware, and build deployment pipelines.
What operating rhythm does the company expect?
Deepgram expects an AI-first mindset where team members actively experiment with AI tools, adapt quickly to rapid changes, and integrate AI into their everyday workflows.