AI Engineer II

About the Role

Mastercard's Business & Market Insights (B&MI) group delivers data-driven intelligence and AI solutions to help businesses achieve growth and innovation. We transform complex, multi-modal data into agentic systems and generative applications. This role focuses on the Operational Intelligence Program within B&MI, building production-grade agentic AI systems that make extensive tool calls into enterprise services and APIs.

Key Responsibilities

• Architect and build the tool-calling layer for agents, wrapping internal services, data platforms, and external APIs as tools with clear schemas, contracts, and permissions.
• Engineer reliable tool execution at scale, including parallel and chained tool calls, timeouts, retries, idempotency, error recovery, and graceful handling of tool failures or bad model outputs.
• Develop high-performance Python backend services for LLM orchestration, covering async job handling, streaming responses, caching, rate limiting, and distributed workflows supporting high-throughput Gen AI operations.
• Engineer state, memory, and context management subsystems for agent session continuity across long, multi-step, tool-heavy interactions.
• Architect multi-agent and skills-based agent systems using frameworks such as LangGraph, CrewAI, AutoGen, or similar, for autonomous reasoning, tool use, inter-agent coordination, and adaptive decision-making.
• Build observability for agent behavior, including end-to-end tracing of tool calls, measuring tool-selection accuracy, and debugging multi-step agent runs in production.
• Build retrieval systems (RAG and Graph-RAG) using vector databases (e.g., Pinecone, pgvector, OpenSearch) and knowledge graphs (e.g., Neo4j, AWS Neptune).
• Establish LLMOps and MLOps pipelines on Databricks (AWS), covering MLflow, prompt and agent evaluation, model lineage tracking, CI/CD, and monitoring.
• Design multimodal generative AI pipelines unifying text, image, tabular, and graph data using transformer-based and foundation models.
• Guide fine-tuning, prompt engineering, and alignment strategies (e.g., PEFT, LoRA, instruction tuning, RLHF) to adapt foundation models for specialized enterprise use cases.
• Implement Responsible AI practices, including hallucination mitigation, bias detection, output safety guardrails, explainability, and compliance with data ethics standards.
• Evaluate and productionize advances in agentic AI, multimodal models, and frontier foundation models, benchmarking them against enterprise performance and safety requirements.
• Lead, mentor, and grow a team of engineers, and set the engineering standard for Gen AI development across the program.

Requirements & Tech Stack

• Strong track record of building and operating production backend systems at scale, including distributed services, APIs, async processing, data stores, and observability.
• Strong API design and integration experience.
• Excellent Python engineering skills, including async programming, API development (FastAPI or similar), and microservices.
• SQL proficiency.
• Hands-on experience shipping at least one LLM-powered application to production that uses tool/function calling.
• Working understanding of Gen AI engineering patterns, such as prompt design, embeddings, semantic search, and agent memory, or a demonstrated ability to ramp up quickly.
• Experience with at least one cloud platform (AWS preferred; Azure or GCP welcome).
• Experience leading or mentoring engineers.
• Strong communication and stakeholder management skills.

Nice to have:

• Experience with Model Context Protocol (MCP) or building tool servers and connectors for LLM agents.
• Experience evaluating agent performance, such as tool-selection accuracy, task completion rates, or multi-step reasoning reliability.
• Exposure to fine-tuning (LoRA, PEFT, RLHF), multimodal models, knowledge graphs, or MLOps tooling like MLflow, Weights & Biases, SageMaker, or Bedrock.

Qualifications

• Bachelor's or Master's degree in Computer Science, AI/ML, or Engineering, or equivalent practical experience.