Software Engineer, GPU Performance
About this role
An engineer who enjoys understanding how software uses the hardware beneath it will thrive in this position. HeyGen is building advanced AI applications such as Avatar IV, Photo Avatar, Interactive Avatar, and Video Translation to make visual storytelling accessible to all. The team creates technology that helps people reach, captivate, and inspire audiences through video. This role exists to make the systems behind these experiences faster and more efficient.
The incoming software engineer will work across model execution and inference infrastructure. Daily responsibilities involve using profiling and measurement to improve latency, throughput, and GPU cost. Collaborating closely with AI researchers and infrastructure engineers, the person in this seat will bring critical performance optimizations into production.
What you will do
- Investigate GPU utilization, kernel execution, memory bandwidth, and CPU-GPU data movement using NVIDIA Nsight Systems, Nsight Compute, and PyTorch Profiler.
- Identify bottlenecks across model execution, preprocessing, and inference serving while measuring the impact of each optimization.
- Improve performance through batching, scheduling, memory management, and better overall GPU utilization.
- Develop or integrate high-performance GPU kernels when existing implementations limit performance.
- Build benchmarks and automated checks that catch performance regressions across representative video workloads.
- Collaborate with AI researchers and infrastructure engineers to bring optimizations into production.
- Measure the effect of changes on latency, throughput, cost, and output quality.
What you need
- Demonstrated experience optimizing GPU-based AI workloads or high-performance computing systems.
- Proficiency in Python paired with practical experience using PyTorch or a similar machine learning framework.
- Strong curiosity about GPU hardware, covering memory bandwidth, cache behavior, tensor cores, and data movement between CPU and GPU.
- Experience using profiling tools to connect hardware behavior to application-level bottlenecks and validate improvements.
- Ability to turn performance experiments into reliable production changes and communicate tradeoffs clearly.
Nice to have
- Experience with CUDA, Triton, or C++ GPU programming.
- Experience optimizing video, image, audio, diffusion, or Transformer models.
- Familiarity with multi-GPU inference, GPU interconnects, quantization, or large-scale model serving.
- Experience building performance benchmarks or regression testing infrastructure.
- Prior experience in a fast-paced technology environment.
Benefits
- Competitive salary and benefits package
- Dynamic and inclusive work environment
- Opportunities for professional growth and advancement
- Collaborative culture that values innovation and creativity
- Access to the latest technologies and tools
Skills and tech stack
Questions candidates ask
What tools and technologies will I use in this role?
You will use profiling tools like NVIDIA Nsight Systems, Nsight Compute, and PyTorch Profiler. Development involves Python and PyTorch, with potential work in CUDA, Triton, or C++ GPU programming.
What will I actually be working on day to day?
You will work across model execution and inference infrastructure for AI applications like Avatar IV and Video Translation. Your focus will be improving latency, throughput, and GPU cost through profiling and measurement.
What kind of team will I collaborate with?
You will collaborate closely with AI researchers and infrastructure engineers. Together, you will bring performance optimizations into production environments.
What benefits are provided with the job?
The company offers a competitive salary and benefits package. Employees also receive access to the latest technologies and tools.