$30/hr

About the Role

We are seeking experienced AI Product Evaluators, AI QA Testers, Model Evaluation Specialists, and detail-oriented technology professionals to contribute to a remote AI project focused on testing advanced models and agents.

Key Responsibilities

• Evaluate AI models and AI agents using predefined scenarios, instructions, and grading criteria.
• Execute structured tasks designed to test model accuracy, reasoning, responsiveness, reliability, and instruction-following.
• Compare model behavior against expected outcomes and identify incorrect, incomplete, inconsistent, or unexpected responses.
• Document findings, errors, unusual behaviors, and evaluation results in structured written reports.
• Repeat evaluations when required to verify consistency or test behavior across multiple sessions.
• Reset usage limits or testing environments according to project procedures and continue evaluation workflows.
• Record findings and submit evaluation datasets through designated AI training or assessment platforms.
• Follow detailed project instructions precisely and apply the same standards across large volumes of tasks.
• Escalate unclear scenarios, unexpected platform behavior, or ambiguous evaluation cases when necessary.
• Maintain strong accuracy, consistency, and documentation quality across all assigned evaluations.

Requirements & Tech Stack

• Experience using AI products, AI models, AI agents, or similar digital tools.
• Strong ability to follow complex instructions precisely and consistently.
• Exceptional attention to detail and operational accuracy.
• Regular use of AI tools in daily life or professional work.
• Ability to identify errors, inconsistencies, and unexpected model behavior.
• Strong written communication and documentation skills.
• Comfortable completing repetitive or high-volume evaluation tasks.
• Strong time management and self-organization skills.
• Ability to work independently in a remote environment.

Qualifications

• Relevant backgrounds include AI Product Evaluators, AI Evaluators, Model Evaluators, AI Testers, and AI Quality Specialists.
• Also welcome: AI Trainers, Human Data Specialists, AI Data Annotators, RLHF Contributors, and Model Response Reviewers.
• Software quality backgrounds may include QA Testers, Software QA Analysts, Quality Assurance Engineers, Manual Testers, and Product Quality Specialists.
• Product testing backgrounds may include Product Testers, User Acceptance Testers, Digital Product Evaluators, and Application Testers.
• Operations backgrounds may include Operations Specialists, Process Analysts, Workflow Specialists, Data Operations Professionals, and Research Operations Specialists with strong attention to detail.
• Additional relevant backgrounds include Technical Support Professionals, Product Operations Specialists, Research Assistants, Trust and Safety Reviewers, and technically proficient professionals who use AI tools extensively.
• Preferred: Experience with AI model evaluation, AI product testing, RLHF, human data, or response rating.
• Preferred: Experience with software QA, manual testing, user acceptance testing, or digital product evaluation.
• Preferred: Experience reviewing AI-generated text, code, images, research outputs, or agent behavior.
• Preferred: Experience working with annotation platforms, evaluation dashboards, or structured grading systems.
• Preferred: Experience documenting bugs, test results, or operational findings.
• Preferred: Experience participating in AI research, digital product testing, or software quality projects.
• Preferred: Experience with high-volume operational workflows or structured data collection.