Latest AI JobsBrowse Jobs
Sarvam AI logo

ML Engineer (Training Infra), Foundational Models

Sarvam AIBengaluruModels
Apply on company site
Salary
Not listed
Job type
Full-time
Experience
Senior · 3+ yrs
Posted
7 Oct 2026
Status
Open · checked 7d ago

About this role

Based in Bengaluru, the ML Engineer (Training Infra), Foundational Models position at Sarvam involves building and optimizing large-scale distributed training infrastructure for next-generation AI models. This technical systems role suits engineers with a strong background in distributed systems, GPU profiling, and deep learning infrastructure who want to support sovereign AI development in India.

Role Overview

Sarvam is hiring an ML Engineer to take ownership of the foundational model training infrastructure. This position focuses on distributed training at scale, parallelism strategies, throughput optimization, reliability engineering, and custom GPU kernel work.

Key Responsibilities

Build, maintain, and advance the distributed training stack across large GPU clusters
Design and implement data, tensor, pipeline, sequence, and expert parallelism strategies for various model architectures
Profile and optimize end-to-end training throughput covering kernel performance, memory layout, communication overlap, data loading, and checkpointing
Write and tune custom CUDA and Triton GPU kernels to maximize hardware performance
Ensure the reliability of long-running training jobs through fault tolerance, deterministic restarts, and automated detection of silent corruption or slow nodes
Collaborate closely with researchers and data teams to ensure architectural feasibility and prevent pipeline bottlenecks

Requirements

Bachelor or Master degree in Computer Science or a related technical field, or equivalent experience
Minimum of 3 years of experience building machine learning training infrastructure or large-scale distributed systems
Hands-on experience using distributed training frameworks such as DeepSpeed, Megatron-LM, FSDP, or NeMo, alongside practical on-call experience for pretraining runs
Deep working knowledge of GPU architecture, the CUDA programming model, and workload profiling tools like Nsight and PyTorch profiler
Strong familiarity with PyTorch internals, including reading and modifying low-level training code
Prior open-source contributions to training infrastructure ecosystems such as PyTorch, Megatron, DeepSpeed, Triton, vLLM, or NCCL

Nice to Have

Experience developing custom CUDA or Triton kernels with measurable workload performance gains
Experience training models exceeding 10 billion parameters or utilizing clusters of 1000+ GPUs
Familiarity with large-scale cluster orchestration and job schedulers like Kubernetes or Slurm
Experience with mixed precision training formats such as BF16 and FP8, or quantization-aware training
Authorship of papers or technical reports focused on training systems, scaling, or model efficiency

Skills and Tools

PyTorch
CUDA
Triton
Megatron-LM
DeepSpeed
FSDP
NeMo
NCCL
vLLM
Slurm
Kubernetes
Nsight

About the Company

Sarvam develops a full-stack sovereign AI platform for India, covering research, models, infrastructure, and applications with backing from Lightspeed, Peak XV, and Khosla Ventures. The organization partners with major enterprises and public institutions, including Tata Capital, SBI Life, CRED, IDFC, and LIC, to deliver population-scale AI solutions.

How to Apply

Interested candidates should apply on the official Sarvam careers page.

Skills and tech stack

PyTorchCUDATritonMegatron-LMDeepSpeedFSDPNeMoNCCLvLLMSlurmKubernetesNsightDistributed Systems

Questions candidates ask

Where is the job located?

The role is located in Bengaluru.

How many years of experience are required?

Candidates must have at least 3 years of experience building ML training infrastructure or large-scale distributed systems, though exceptional early-career candidates with a strong systems background are also considered.

What frameworks and tools are used in this role?

The tech stack includes PyTorch, CUDA, Triton, Megatron-LM, DeepSpeed, FSDP, NeMo, NCCL, vLLM, Slurm, Kubernetes, and Nsight.

What educational background is expected?

A Bachelor or Master degree in Computer Science or a closely related technical field is required, or equivalent demonstrated experience.

How should I submit my application?

Applications should be submitted directly through the official Sarvam company careers page.

Ready to apply?

The application is on the official Sarvam AI careers page.

Apply on company site