Korean Voice AI Evaluator
Alignerr
New York, NYFresher1h agovia LinkedIn
This job at a glance
- Who can apply
- New grads, no experience needed
- Nice to have
- Customer serviceMedical terminology
- Work
- On-site in New York
- Pay
- Not stated by the employer.
- Skill demand
- Customer service5%of 99 similar entry-level jobs posted in the US this week
- Listed
- Oct 29, 2026 · still open
Sign in free to see which of these skills your resume already shows.
About the Role
We are seeking native Korean speakers based in the United States to engage in spoken conversations with an AI healthcare receptionist. You will role-play as a patient, record conversations, and evaluate the AI's performance. This is a remote, contract (task-based) position with flexible commitment.
Key Responsibilities
- Hold 4 to 5 minute spoken conversations in Korean with an AI front desk assistant, role-playing as a fictional patient.
- Use natural trigger phrases to test AI function calls (e.g., checking patient records, booking appointments, verifying insurance).
- Review transcripts and classify the AI's understanding and response accuracy.
- Flag issues such as mispronunciations, cutoffs, interruptions, delays, and incorrect function calls, including timestamps.
- Test AI performance with background noise, short replies, long numbers, emotional moments, and sensitive/unsafe requests.
- Consistently follow detailed labeling guidelines and submit high-quality work.
Requirements & Tech Stack
- Native or near-native Korean speaker with a clear, natural speaking voice.
- Detail-oriented with patience for reviewing recordings and transcripts.
- Comfortable role-playing and sounding authentic.
- Ability to follow multi-step instructions and apply them consistently.
- Reliable and self-directed for independent work.
- Access to a quiet space, a good microphone, a stable internet connection, and a second device for background noise.
- Comfortable using web-based tools.
Qualifications
- Experience in voice AI evaluation, data annotation, or QA testing (Nice to Have).
- Background in healthcare administration, customer service, or call center work (Nice to Have).
- Strong sense of Korean speech levels and politeness (존댓말) in service conversations, and of how Korean numbers are read aloud (Nice to Have).
- Linguistics or language-focused studies (Nice to Have).
- Familiarity with medical terminology in Korean (Nice to Have).
Note: A short unpaid qualification assessment is required for new contributors.