Based in Redwood City, the Software Engineer, AI Infrastructure - LVM Inference & Evaluation role at Ambient.ai involves designing and optimizing infrastructure for real-time intelligence platforms. This position suits an engineer with a strong background in production machine learning systems, inference optimization, and scalable distributed infrastructure.
Role Overview
Ambient.ai is hiring a Software Engineer, AI Infrastructure - LVM Inference & Evaluation in Redwood City. Reporting to Raghu Nallamothu, this position focuses on building the systems required to run deep learning models across terabytes of video data in real time. The engineer will collaborate closely with research scientists and product teams to bring advanced artificial intelligence into production environments.
Key Responsibilities
Design, build, and maintain AI infrastructure for real-time computer vision, LLM, LVM, and multimodal inference workloads.
Build scalable systems for running state-of-the-art models across large volumes of video and sensor data.
Optimize inference performance across latency, throughput, GPU utilization, reliability, and cost.
Develop robust evaluation harnesses and benchmarking systems to measure model quality, system performance, regressions, and production readiness.
Build infrastructure for continuous model evaluation, experimentation, and deployment.
Partner with research scientists to productionize advances in computer vision, LLMs, LVMs, RAG, and multimodal AI.
Improve model-serving architecture, including batching, caching, routing, quantization, model parallelism, and hardware utilization.
Develop data engines and feedback loops for collecting training data, evaluating model behavior, and continuously improving performance.
Create reliable observability, monitoring, and debugging tools for production AI systems.
Help define best practices for deploying, evaluating, and operating AI systems in real-world enterprise environments.
Requirements
2+ years of industry experience building infrastructure, distributed systems, machine learning platforms, or production AI systems.
BS/MS in Computer Science or a related technical field, or equivalent practical experience.
Strong programming background, especially in Python, with solid software engineering fundamentals.
Experience designing and building scalable machine learning infrastructure for training, inference, evaluation, and deployment.
Hands-on experience running deep learning models in production, including LLMs, LVMs, vision-language models, or multimodal models.
Strong understanding of inference optimization techniques, such as batching, caching, quantization, parallelism, memory optimization, GPU utilization, and latency reduction.
Experience with model-serving frameworks or systems such as vLLM or Triton Inference Server.
Experience building evaluation frameworks, test harnesses, benchmarks, regression tests, or model-quality measurement systems.
Strong background in machine learning and deep learning.
Experience designing data engines or pipelines for collecting, managing, and curating training and evaluation data.
Familiarity with integrating advanced AI systems like LLMs, LVMs, RAG pipelines, embedding models, or multimodal models into production applications.
Experience with cloud infrastructure, containers, orchestration, distributed systems, and GPU-based workloads.
Strong collaboration and communication skills to work effectively with research scientists, product teams, and stakeholders.
Proactive problem-solving ability, an ownership mindset, and adaptability to new technologies.
Nice to Have
Experience operating large-scale GPU infrastructure or distributed inference systems.
Experience with CUDA, NCCL, PyTorch, TensorRT, ONNX, or similar ML systems technologies.
Experience with video understanding, real-time computer vision, multimodal AI, or physical-world AI systems.
Experience with model compression, speculative decoding, distillation, pruning, or low-latency serving techniques.
Experience with prompt evaluation, model regression testing, human-in-the-loop evaluation, or automated quality gates.
Familiarity with retrieval-augmented generation, vector databases, embedding models, re-rankers, or search infrastructure.
Experience building internal ML platforms or tools used by researchers and applied ML teams.
Skills and Tools
Python
vLLM
Triton Inference Server
PyTorch
TensorRT
ONNX
CUDA
NCCL
Salary and Benefits
Salary range of $168K to $205K per year
Offers equity
Comprehensive health and welfare package covering Medical, Dental, Vision, Life, EAP, Legal Services, and a 401(k) plan
Flexible time off including Winter Break
Latest tech and company swag
About the Company
Ambient.ai is the category creator and leader in Agentic Physical Security, founded in 2017 and backed by Andreessen Horowitz, Y Combinator, and Allegion Ventures. The company is powered by Ambient Pulsar, a reasoning Vision-Language Model purpose-built for physical security, serving customers such as Cisco, ServiceNow, SentinelOne, TikTok, Bayer, and MoMA.
How to Apply
Apply on the official company careers page.
