Research Scientist, ML H-W/S-W Codesign

About the Role

Meta's Reality Labs (RL) is dedicated to creating the future of Virtual Reality (VR), Augmented Reality (AR), and Wearable AI Devices. To meet the demanding compute performance and power efficiency needs of these AI devices, custom silicon is essential. The Reality Labs Silicon team is at the forefront of research and development across computer vision, machine learning, mixed reality, graphics, displays, sensors, and human body mapping. Their objective is to enable personalized on-device AI and seamlessly merge virtual and physical worlds in wearable devices, requiring a holistic approach from transistors to algorithms.

Meta's Research & Development teams are seeking a Research Scientist to pioneer and implement advanced AI model–hardware codesign techniques. This role operates at the convergence of AI models, hardware acceleration, and software systems, addressing significant challenges within Reality Labs. The focus will be on developing innovative methods for compute- and power-efficient training and on-device inference of vision and language models for AR, VR, and edge devices.

Key Responsibilities

• Identify and resolve multi-discipline ML acceleration problems, encompassing algorithms, network design, hardware architecture, multimodal AI, and AR/VR use cases. This may involve creating novel, industry-first approaches.
• Collaborate across hardware and software domains to tackle co-design challenges with fellow Research Scientists.
• Codesign and invent novel ML accelerator and system architecture solutions, facilitating the integration of algorithms and software to leverage these advancements.
• Develop cutting-edge model compression and scalability techniques, including Numerics, pruning, and distillation.
• Optimize models on hardware accelerators to achieve target performance under real-time latency and power constraints.
• Influence partners to adopt recommended solutions through data-driven analysis and clear communication of trade-offs.
• Define use cases and develop methodologies and benchmarks for evaluating diverse approaches.
• Apply in-depth knowledge of how ML acceleration interacts with surrounding systems.
• Stay abreast of the latest research advancements in ML acceleration by attending conferences and interpreting papers.
• Contribute to patents and/or publications in peer-reviewed conferences and journals.

Requirements & Tech Stack

• C/C++ or Python for AI-System infrastructure, AI algorithms, or AI hardware acceleration development.
• Experience with PyTorch, TensorFlow, or similar machine learning toolsets.
• Experience or knowledge of training/inference of Large scale AI models – CV and/or LLMs.
• Experience or knowledge of architecting ML hardware accelerators and systems.
• Experience evaluating alternative system or algorithm designs by analyzing trade-offs in performance, power, and latency to recommend a solution.
• Demonstrated ability to integrate AI tools to optimize/redesign workflows and drive measurable impact (e.g., efficiency gains, quality improvements).
• Demonstrated ongoing AI skill development (e.g., prompt/context engineering, agent orchestration) and staying current with emerging AI technologies.
• Experience or knowledge of on-device algorithm development, including hardware-aware ML models and/or optimizing ML compilers for efficient deployment on AI accelerators.

Qualifications

• Bachelor's degree in Computer Science, Computer Engineering, a relevant technical field, or equivalent practical experience.
• PhD in Electrical Engineering, Computer Science, or a relevant technical field, or equivalent practical experience.
• PhD in Electrical Engineering, Computer Science, or equivalent experience.
• Experience adhering to and implementing responsible, ethical AI practices (e.g., risk assessment, bias mitigation, quality and accuracy reviews).
• Experience working and communicating cross-functionally in a team environment.
• Demonstrated research and engineering experience via an internship, work experience, coding competitions, or widely used contributions in open source repositories (e.g., GitHub).
• Proven track record of achieving significant results, evidenced by grants, fellowships, patents, and publications at leading workshops, journals, or conferences such as ICLR, NeurIPS, CVPR, ACL, ICML, MLSys, ISCA, MICRO, DAC, ASPLOS, etc.