AI Evaluator, English

About the Role

Outlier helps the world’s most innovative companies improve their AI models by providing human feedback. This freelance opportunity is remote and hours are flexible.

Key Responsibilities

• Hold live, scenario-based conversations in English with AI voice models, then rank how natural, useful, and clear each one sounded.
• Compare pairs of AI-generated images and videos and judge which one better follows the prompt.
• Inspect outputs closely, including zooming in and replaying, to spot problems such as distorted anatomy, flickering or unnatural motion, text errors, and audio that doesn't match the video.
• Rate each output across dimensions like adherence, quality, and realism.
• Write specific explanations in English pointing to what was seen or heard.

Requirements & Tech Stack

• A computer with a reliable internet connection and headphones.
• A quiet place to talk for projects involving live conversation.
• Your ratings and written explanations must be your own work; AI tools cannot be used to write them.
• Getting started includes a short screening and onboarding, which may include a brief spoken English check.

Qualifications

• Fluent English speaker with strong, clear written communication.
• Comfortable with natural, everyday conversational speech.
• Pursuing or holding a Bachelor's degree or above.
• Strong attention to detail, with the patience to look and listen closely, and to replay or zoom in when something seems off.
• Comfortable thinking on your feet and talking through realistic situations in a natural, conversational way.
• Experience writing short, evidence-based explanations or reviews.
• Comfortable making a clear decision when both options have flaws.