Software Engineer, Multimodal AI, XR

About the Job

Google's software engineers develop next-generation technologies that change how billions of users connect, explore, and interact with information. As a Software Engineer, Multimodal AI, XR, you will work on projects critical to Google's needs, potentially switching teams and projects as the business grows. We seek versatile, leadership-oriented engineers enthusiastic about new problems across the full-stack.

This role focuses on advancing multimodal and conversational AI for next-generation wearable computers. You will conduct development on large language models (LLMs) and agents, particularly in the multimodal domain (vision, audio), for XR devices. This role offers an opportunity to contribute to AI research and shape future computing products, focusing on the convergence of AI and XR. We are building a future where lightweight XR devices pair with helpful AI to augment human intelligence with personalized, conversational, and contextually aware experiences.

Key Responsibilities

• Lead the development and optimization of on-device and hybrid multimodal models for XR devices, utilizing techniques like prompting, few-shot learning, and post-training (e.g., quantization) to enhance performance and robustness under strict power and latency constraints.
• Create comprehensive evaluation plans for hybrid systems, including dataset development and defining KPIs for model accuracy and on-device efficiency.
• Identify, implement, and ship the latest modeling innovations, focusing on hybrid agent architectures, orchestration between edge and cloud, multimodality, tool integrations, and personalization.
• Prove out concepts for on-device AI features through rapid prototyping and iterative development, facilitating team testing in partnership with XR product teams.
• Work closely with research scientists, engineers, and product teams to drive the technical roadmap, foster collaboration, share findings through conference publications, and contribute to product launches.

Requirements & Tech Stack

• Software development in one or more programming languages.
• Experience with one or more of the following: speech/audio, reinforcement learning, ML infrastructure, or specialization in another ML field.
• Experience with relevant ML frameworks such as JAX, TensorFlow, or PyTorch.
• Familiarity with on-device model training and deployment.
• Experience with multimodal learning, large language models (LLMs), or AI agents.
• Experience with prompt engineering, few-shot learning, post-training techniques, and evaluations.

Qualifications

• Bachelor’s degree or equivalent practical experience.
• 2 years of experience with software development in one or more programming languages, or 1 year of experience with an advanced degree.
• 1 year of experience with one or more of the following: speech/audio, reinforcement learning, ML infrastructure, or specialization in another ML field.
• 1 year of experience with relevant ML frameworks such as JAX, TensorFlow, or PyTorch.
• Preferred: Master’s degree or PhD in Computer Science, or related technical fields.
• Preferred: Experience with multimodal learning, large language models (LLMs), or AI agents.
• Preferred: Experience with prompt engineering, few-shot learning, post-training techniques, and evaluations.
• Preferred: Familiarity with on-device model training and deployment.