About this role
The Inference Research team at Together AI builds the next generation of efficient, scalable, and reliable serving systems for large foundation models. Operating at the intersection of model architectures, high-performance systems engineering, and hardware optimization, the group co-designs software, algorithms, and models to lower the cost and latency of modern artificial intelligence systems. Research interns dive into distributed inference, compiler-aware optimization, and novel inference-time computation strategies like speculative decoding and phase-aware execution.
This temporary role exists to tackle cross-layer optimizations across models, systems, and hardware, focusing specifically on KV cache design and large-scale serving architectures. Projects unlock performance and scale for foundation models, enabling faster serving, large model deployment for architectures such as Mixture-of-Experts, and reproducible evaluation under realistic workloads. Candidates who thrive in this environment possess strong deep learning fundamentals, practical programming abilities in Python, and an academic focus in computer science or related technical fields.
What you will do
- Design and conduct rigorous experiments to validate research hypotheses regarding model inference and serving systems.
- Communicate plans, ongoing progress, and project results clearly to the broader engineering and research team.
- Document experimental findings through scientific publications and detailed technical blog posts.
- Co-design and implement cross-layer optimizations spanning models, systems, and underlying hardware infrastructure.
What you need
- Currently pursuing the final year of a Bachelor's, Master's, or Ph.D. degree in Computer Science, Electrical Engineering, or a closely related field.
- Demonstrate strong knowledge of core machine learning and deep learning fundamentals.
- Bring practical experience working with deep learning frameworks such as PyTorch or JAX.
- Possess strong programming skills utilizing Python.
- Maintain familiarity with Transformer architectures and recent developments in foundation models.
Nice to have
- Prior research experience in foundation models, efficient machine learning, or ML systems
- Publications at leading conferences in machine learning or systems like MLSys and ICLR
- Experience with CUDA programming for custom kernel development
- Understanding of model optimization techniques and hardware acceleration approaches
- Contributions to open-source machine learning projects
Benefits
- Competitive hourly compensation ranging from $58 to $70
- Housing stipends for the duration of the program
Skills and tech stack
Questions candidates ask
How long does the internship program last?
The internship program runs 12 to 14 weeks, giving participants the opportunity to work alongside industry-leading engineers and researchers across multiple teams.
What are the available cohort dates for the internship?
This cohort's internship dates span either May 17th to August 6th or June 14th to September 3rd.
What is the compensation for this role?
The estimated US hourly rate for this role is $58 to $70 an hour, complemented by competitive compensation, housing stipends, and other benefits.