Korean-Speaking AI Session Annotator: Voice & Video Assistant Evaluator

About the role

We're looking for detail-oriented, Korean-fluent reviewers to help evaluate the real-world performance of a voice-and-camera AI assistant. In this role, you'll watch recorded sessions where a user speaks to an AI assistant while their camera streams their surroundings, and you'll assess whether the assistant's spoken responses actually match what was visible on screen. Your written observations become the primary signal our team uses to identify and prioritize fixes — this is thoughtful, judgment-driven annotation work, not repetitive volume tasks.

This is a remote, flexible-hours opportunity ideal for people who are meticulous, comfortable giving structured critical feedback, and fluent in both spoken and written Korean.

What you'll do

• Review recorded sessions in which a user speaks to an AI assistant while a live camera feed captures their environment
• Carefully compare the assistant's verbal responses against what is actually visible in the video footage
• Score each session against a detailed evaluation rubric covering accuracy, relevance, and helpfulness
• Write clear, specific comments explaining any discrepancies, failures, or notable successes
• Flag high-priority issues that should be addressed first based on severity and frequency
• Maintain consistency and rigor across a batch of sessions, applying rubric criteria uniformly
• Provide feedback that is actionable for engineering and product teams working to improve the assistant

Requirements

• Native or near-native fluency in Korean (spoken and written)
• Strong English comprehension for rubric instructions and written reporting
• Excellent attention to detail and ability to spot subtle mismatches between audio and video content
• Comfort with independent, judgment-based evaluation work
• Reliable access to a computer with audio/video playback capability and a stable internet connection
• Prior experience in content review, QA, translation, localization, or annotation work is a plus
• Familiarity with AI assistants or voice-interface products is a plus but not required

Compensation

Hourly rate of $24–$30, paid for verified review work completed. Flexible, remote, part-time or full-time availability considered.