The Inference Optimization Engineer role at hedra is based in San Francisco and involves collaborating with the research team to make advanced visual models fast and efficient during inference. This position suits technical professionals with expertise in machine learning systems, GPU computing, and efficient inference.
Role Overview
Inference Optimization Engineers operate at the intersection of research and systems at hedra in San Francisco. The position focuses on taking model architectures and optimizing them to run efficiently on modern hardware. Professionals in this role analyze time and memory consumption, identify opportunities for algorithmic and system-level enhancements, and execute optimizations across model architecture, inference algorithms, runtimes, kernels, and distributed execution environments.
Key Responsibilities
Collaborate directly with research scientists and engineers to make new visual models fast and efficient at inference time.
Profile model architectures and workloads to understand bottlenecks related to compute, memory, communication, and execution.
Develop and implement approaches to improve inference latency, throughput, memory efficiency, and GPU utilization.
Explore algorithmic optimizations including quantization, sparsity, caching, compilation, attention optimizations, and alternative execution strategies.
Build or optimize GPU kernels using CUDA, Triton, or comparable technologies when existing implementations lack required performance.
Optimize model execution across single-GPU, multi-GPU, and multi-node environments.
Reason about the interaction between model architecture and hardware, working alongside researchers when architectural modifications unlock performance improvements.
Investigate communication, memory movement, parallelism, and distributed execution strategies for large visual models.
Build benchmarking, profiling, and performance-regression infrastructure to evaluate new optimization ideas.
Evaluate new inference runtimes, compilers, frameworks, optimization techniques, and accelerator hardware.
Stay informed on advances in efficient inference, GPU programming, model architectures, compilers, and machine learning systems research.
Assist in turning research breakthroughs into models deployed and served efficiently at scale.
Requirements
Demonstrated technical ability in efficient machine learning inference, machine learning systems, GPU computing, or adjacent research via production engineering, open-source contributions, or independent work.
Understanding of how deep learning models execute on hardware, including relationships between compute, memory, communication, and performance.
Experience profiling machine learning workloads, identifying bottlenecks, forming hypotheses, and driving measurable performance improvements.
Programming fundamentals in Python, C++, or another systems-oriented language.
Experience with combinations of PyTorch, CUDA, Triton, TensorRT, vLLM, SGLang, or comparable technologies.
Ability to reason across abstraction layers instead of treating model architecture, framework, runtime, kernel, and hardware boundaries as fixed.
Intuition for performance tradeoffs across latency, throughput, memory, numerical precision, model quality, and complexity.
Comfort working on ambiguous problems where the bottleneck may not be known in advance.
Ability to communicate technical ideas clearly and collaborate closely with research scientists and engineers.
Nice to Have
Experience optimizing large generative, multimodal, vision, or video models.
CUDA, Triton, CUTLASS, or other GPU kernel development experience.
Knowledge of GPU architecture, memory hierarchy, and hardware-aware optimization.
Experience with attention optimization, kernel fusion, memory-efficient execution, or custom operators.
Model compilation or graph optimization experience.
Quantization, sparsity, caching, speculative execution, or other efficient inference techniques.
Experience optimizing diffusion, autoregressive, transformer, or other large generative architectures.
Multi-GPU or multi-node model execution, including tensor, pipeline, sequence, or other forms of parallelism.
Experience optimizing communication or data movement between accelerators.
Experience with profiling tools such as Nsight Systems or Nsight Compute.
Contributions to machine learning systems, inference runtimes, compilers, GPU libraries, or performance-focused open-source projects.
Research or publications in efficient machine learning, machine learning systems, GPU computing, compilers, or related areas.
Skills and Tools
Python
C++
PyTorch
CUDA
Triton
TensorRT
vLLM
SGLang
CUTLASS
Nsight Systems
Nsight Compute
Salary and Benefits
Competitive compensation and equity
401k
Healthcare including Silver PPO Medical, Vision, and Dental
Lunch and snacks provided at the office
About the Company
hedra builds platforms, models, and infrastructure for visual intelligence. The company creates models and systems that push visual intelligence limits, along with infrastructure to make models fast, efficient, reliable, and accessible at scale. Backed by a16z and other leading investors, the organization operates as a small, highly technical team in San Francisco where staff work together in person five days a week.
How to Apply
Candidates should apply on the official company careers page.
