About this role
Reducing the cost and latency of modern AI systems requires innovative solutions at the intersection of model architectures, high-performance systems engineering, and hardware optimization. The Inference Research team at Together AI builds efficient, scalable serving systems for large foundation models to support open and transparent artificial intelligence.
Research interns tackle distributed inference, compiler-aware optimization, and novel inference-time computation strategies like speculative decoding and phase-aware execution. Individuals who excel here possess strong programming abilities in Python, deep learning fundamentals, and an academic focus in computer science or electrical engineering, ready to contribute to cross-layer optimizations in KV cache design and large-scale serving architectures.
What you will do
- Design and conduct rigorous experiments to validate research hypotheses.
- Communicate project plans, progress, and results effectively to the broader team.
- Document research findings in scientific publications and blog posts.
- Co-design and implement cross-layer optimizations across models, systems, and hardware.
What you need
- Currently pursuing a final year of Bachelor's, Master's, or Ph.D. degree in Computer Science, Electrical Engineering, or a related field.
- Demonstrate strong knowledge of machine learning and deep learning fundamentals.
- Bring practical experience with deep learning frameworks such as PyTorch and JAX.
- Possess strong programming skills in Python.
- Show familiarity with Transformer architectures and recent developments in foundation models.
Nice to have
- Prior research experience in foundation models, efficient machine learning, or ML systems
- Publications at leading conferences in machine learning or systems like MLSys and ICLR
- Experience with CUDA programming for kernel development
- Understanding of model optimization techniques and hardware acceleration approaches
- Contributions to open-source machine learning projects
Benefits
- Competitive compensation ranging from $58 to $70 an hour
- Housing stipends
- Other competitive benefits
Skills and tech stack
Questions candidates ask
What are the dates for the Winter 2027 internship?
The internship program runs for 12 to 14 weeks, with this cohort's dates spanning from January 4th to April 9th.
What is the compensation for this role?
The estimated US hourly rate for this role is $58 to $70 an hour, determined by location, level, role, experience, skills, and job-related knowledge.
What does the Inference Research team focus on?
The team focuses on building efficient, scalable, and reliable serving systems for large foundation models by co-designing software, algorithms, and models to lower cost and latency.
What deliverables are expected during the internship?
Interns are tasked with designing and conducting rigorous experiments, communicating plans and results to the team, and documenting findings in scientific publications and blog posts.