Research Intern, Frontier Speculative Decoding

About the Role

Together AI is building the Inference Platform that powers the world's most advanced generative AI models. This role focuses on translating internal model training research to production-ready deployment. The work involves data-centric development, hyperparameter tuning, and checkpoint evaluation.

The role will involve understanding customer specific needs and fine-tuning models on internal and proprietary data. The goal is to transform general-purpose models into highly performant, specialized tools. You will not be training foundation models from scratch but rather focusing on creating highly efficient, specialized models by working with dedicated GPU clusters.

Key Responsibilities

• Research and prototype new speculator algorithms, experimenting with architectural ideas and data curation to improve accuracy–efficiency tradeoffs in speculative decoding.
• Own a well-scoped project end to end, from data preparation through training and evaluation, with a real path to production.
• Learn how frontier inference systems work in practice, in a fast-paced environment at the cutting edge of generative AI.
• Collaborate directly with customers to understand their needs, and work closely with our core inference and Applied ML research teams to integrate your work into the production platform.
• Present your results to the broader team at the end of your internship, with the chance to influence the direction of our speculator work.
• A culture of deep technical ownership where you are empowered to take on and solve challenging problems.

Requirements & Tech Stack

• Currently pursuing a degree in Computer Science, Electrical Engineering, Information Science, or a related field.
• A genuine love for data curation and processing, with a meticulous attention to detail.
• Demonstrated ability to perform effective hyperparameter searches and understand the trade-offs involved in tuning models for specific tasks.
• Strong attention-to-detail in evaluating model checkpoints to ensure they meet strict quality, performance, and reliability standards.
• Experience with Python and PyTorch.
• Familiarity with SLURM and/or Kubernetes clusters and experience submitting and managing jobs in a high-performance computing environment.
• Familiarity with modern LLMs and generative models.
• Basic understanding of distributed training frameworks (e.g., FSDP, DeepSpeed).
• Publications at leading research conferences or journals (such as NeurIPS, ICML, ICLR, ACL, EMNLP, Interspeech).

Internship Program Details

• Internship duration: 12 to 14 weeks.
• Internship dates: May 17th to August 6th or June 14th to September 3rd.
• Work alongside industry-leading engineers and researchers.