Research Intern, Frontier Speculative Decoding

About the Role

Together AI is building the Inference Platform for advanced generative AI models. This role focuses on the bridge between research and production, translating model training research to production-ready deployment. It involves data-centric development, hyperparameter tuning, and rigorous checkpoint evaluation.

This role will involve understanding customer needs and fine-tuning models on internal and proprietary data. The goal is to transform general-purpose models into highly performant, specialized tools. You will not be training foundation models from scratch but rather focusing on creating efficient, specialized models using dedicated GPU clusters.

Key Responsibilities

• Research and prototype new speculative algorithms, experimenting with architectural ideas and data curation to improve accuracy–efficiency tradeoffs in speculative decoding.
• Own a well-scoped project end to end, from data preparation through training and evaluation, with a real path to production.
• Learn how frontier inference systems work in practice, in a fast-paced environment at the cutting edge of generative AI.
• Collaborate directly with customers to understand their needs, and work closely with core inference and Applied ML research teams to integrate your work into the production platform.
• Present your results to the broader team at the end of your internship, with the chance to influence the direction of our speculative work.
• Take on and solve challenging problems.

Requirements & Tech Stack

• Currently pursuing a degree in Computer Science, Electrical Engineering, Information Science, or a related field.
• A genuine love for data curation and processing, with a meticulous attention to detail.
• Demonstrated ability to perform effective hyperparameter searches and understand model tuning trade-offs.
• Strong attention-to-detail in evaluating model checkpoints for quality, performance, and reliability.
• Experience with Python and PyTorch.
• Familiarity with SLURM and/or Kubernetes clusters and experience managing jobs in a high-performance computing environment.
• Familiarity with modern LLMs and generative models.
• Basic understanding of distributed training frameworks (e.g., FSDP, DeepSpeed).

Internship Program Details

• Internship duration: 12 to 14 weeks.
• Internship dates: January 4th to April 9th.

About Together AI

Together AI, the AI Native Cloud, is purpose-built for AI engineers. It provides high-performance inference, fine-tuning and reinforcement learning for specialized models, and pre-training at massive scale. Trusted by leading companies, Together serves over 400 trillion tokens a month.