Korean Voice AI Evaluator
Alignerr
New York, NYFresher3h agovia LinkedIn
This job at a glance
- Who can apply
- New grads, no experience needed
- Must know
- Medical terminology
- Nice to have
- Customer service
- Work
- On-site in New York
- Pay
- Not stated by the employer.
- Listed
- Nov 2, 2026 · still open
Sign in free to see which of these skills your resume already shows.
About the Role
Organization: Alignerr
Type: Contract (task-based)
Location: Remote
Commitment: Flexible, work at your own pace
You will hold natural spoken conversations in Korean with an AI healthcare receptionist, then evaluate its performance. You will role-play as a patient using fictional personas, record short conversations, and grade the AI's performance.
What You'll Do
- Hold 4 to 5 minute spoken conversations in Korean with an AI front desk assistant, playing a fictional patient.
- Say natural trigger phrases to test whether the AI makes the correct function calls (e.g., checking a patient record, booking an appointment, checking insurance).
- Review transcripts and classify how accurately the AI understood and responded.
- Flag issues such as mispronunciations, cutoffs, interruptions, delays, and incorrect function calls, including timestamps.
- Test how the AI handles background noise, short replies ("네," "아니요," "괜찮아요"), long numbers, emotional moments, and sensitive or unsafe requests.
- Follow detailed labeling guidelines consistently and submit high-quality work.
Who You Are
- Native or near-native Korean speaker with a clear, natural speaking voice.
- Detail-oriented with patience for reviewing recordings and transcripts closely.
- Comfortable role-playing a patient and sounding authentic.
- Able to follow multi-step instructions and apply them consistently.
- Reliable and self-directed when working independently.
- Access to a quiet space, a good microphone, a stable internet connection, and a second device for playing background noise.
- Comfortable using web-based tools.
Nice to Have
- Experience in voice AI evaluation, data annotation, or QA testing.
- Background in healthcare administration, customer service, or call center work.
- Strong sense of Korean speech levels and politeness (존댓말) in service conversations.
- Understanding of how Korean numbers are read aloud (native vs. Sino-Korean).
- Linguistics or language-focused studies.
- Familiarity with medical terminology in Korean.
Note: New contributors complete a short unpaid qualification assessment. Contributors who pass are added to paid production work.