IN-Specialist 3, AI-Testing

About the Role

PwC's data and analytics engineering professionals leverage advanced technologies and techniques to design and develop robust data solutions for clients, transforming raw data into actionable insights for informed decision-making and business growth. This role focuses on designing and building data infrastructure and systems to enable efficient data processing and analysis, including developing and implementing data pipelines, data integration, and data transformation solutions.

Key Responsibilities

• Define and implement enterprise-wide AI testing and quality assurance strategies.
• Drive continuous improvement in test automation, quality metrics, and release processes.
• Build and maintain scalable automation frameworks for AI applications and APIs.
• Develop automated validation pipelines for machine learning models.
• Integrate automated testing into CI/CD workflows.
• Validate LLM responses using automated evaluation frameworks.
• Design testing strategies for chatbots, copilots, agents, and RAG applications.
• Evaluate prompt effectiveness and response quality.
• Implement automated prompt testing and output validation.
• Conduct load, stress, scalability, and resilience testing for AI.

Requirements & Tech Stack

Mandatory Skill Sets:

• Playwright
• Selenium
• MLflow
• OpenAI Evaluations
• CI/CD Pipelines

Preferred Skill Sets:

• Experience testing LLMs, RAG architectures, AI Agents, and Copilot.

Required Skills:

• Azure DevOps

Qualifications

Education:

• B.Tech / M.Tech / MBA / MCA
• Degrees/Field of Study required: Bachelor of Engineering

Experience:

• 0-2 years of experience.