AI Evaluator, English

About the Role

Outlier helps the world’s most innovative companies improve their AI models by providing human feedback. This is a freelance, remote opportunity with flexible hours.

Key Responsibilities

• Hold live, scenario-based conversations in English with AI voice models, ranking them on naturalness, usefulness, and clarity.
• Compare pairs of AI-generated images and videos, judging which better follows the prompt.
• Inspect outputs closely, including zooming and replaying, to identify issues like distorted anatomy, flickering or unnatural motion, text errors, and audio-video mismatches.
• Rate outputs based on dimensions such as adherence, quality, and realism.
• Write specific explanations in English detailing observations.

Requirements & Tech Stack

• A computer with a reliable internet connection and headphones.
• A quiet place to talk for projects involving live conversation.
• Your ratings and written explanations must be your own work; AI tools cannot be used for writing.
• Getting started includes a short screening and onboarding, which may include a brief spoken English check.

Qualifications

• Fluent English speaker with strong, clear written communication skills and comfort with natural, everyday conversational speech.
• Pursuing or holding a Bachelor's degree or above.
• Strong attention to detail, with patience to look and listen closely, and to replay or zoom in when something seems off.
• Comfortable thinking on your feet and talking through realistic situations in a natural, conversational way.
• Experience writing short, evidence-based explanations or reviews.
• Comfortable making a clear decision when both options have flaws.