Software Development Engineer I

About the Role

The Annapurna Labs team at Amazon Web Services (AWS) builds the AWS Neuron SDK, used to accelerate deep learning and GenAI workloads on AWS Trainium machine learning accelerators. The Neuron Inference organization focuses on optimizing inference performance for various ML models and architectures on AWS's custom ML accelerators, working across the stack from PyTorch to the hardware-software boundary. Engineers in this role develop infrastructure, high-performance kernels for ML functions, and fine-tune compute units for optimal performance. The role involves combining deep hardware knowledge with ML expertise to advance AI acceleration.

This role offers an opportunity to work at the intersection of machine learning, high-performance computing, and distributed architectures.

Key Responsibilities

• Optimize open models to run on Trainium, including optimizing sharding, developing NKI kernels, and ensuring accuracy.
• Collaborate with cross-disciplinary teams (hardware design, firmware, compiler, runtime, ML engineers) to develop software that enables and optimizes custom ML accelerator silicon.
• Write clean, maintainable code in C/C++, Python, or Rust following best practices for systems-level and performance-critical development.
• Participate in code reviews, contribute to technical documentation, and help deploy agentic tools for model enablement on Trainium.
• Demonstrate operational excellence through monitoring, debugging, and resolving issues across hardware-software integration boundaries, including on-call responsibilities.
• Learn emerging technologies in ML accelerators, compiler optimization, computer architecture, and cloud-scale distributed systems.
• Develop, enable, and performance tune a wide variety of LLM model families, including large language models.
• Create, build, and tune distributed inference solutions with AWS Trainium, working with compiler and runtime engineers.
• Work closely with customers on model enablement, providing direct support and optimization expertise.
• Collaborate with open source ecosystems for seamless integration and peak performance.

Requirements & Tech Stack

• Experience with data structure implementation, basic algorithm development, and/or object-oriented design principles.
• Experience with any programming language such as Python, Java, C++.
• Experience with one or more of the following:
• Compiler design or optimization
• Computer architecture
• Hardware simulation or emulation
• ML frameworks (PyTorch, JAX)
• Performance profiling and analysis
• Linux kernel or embedded systems
• AI tools for development productivity
• Cloud platforms (preferably AWS)
• Version control systems
• Strong problem-solving skills with ability to debug across hardware-software boundaries.
• Demonstrated ability to learn and adapt to new technologies quickly.

Qualifications

• Currently has, or is in the process of obtaining, a Bachelor's degree or above in Computer Science, Computer Engineering, Electrical Engineering, or related fields.
• Experience from previous technical internship(s) or demonstrated project experience is preferred.