AI Chat Evaluator: Everyday Task Testing & Feedback

About the role

We're building a global network of sharp, detail-oriented reviewers to help evaluate AI chat assistants on real-world, everyday tasks. This isn't about rating thumbs up or down—it's about simulating genuine use cases, interacting naturally with an AI assistant across multiple turns, and judging the quality of its output against a professional standard. Your feedback directly shapes how these assistants improve.

This is a flexible, remote-friendly role open to people worldwide who already use AI chat tools regularly in their personal or work life and have an eye for what "good" actually looks like.

What you'll do

• Write short, realistic prompts and requests—just like something you'd genuinely ask an assistant for work, research, planning, or daily life.
• Engage in multi-turn conversations: steer the assistant toward what you actually need, answer its clarifying questions, and push back firmly when it gets something wrong or produces a weak answer.
• Evaluate the final output and the overall interaction quality, scoring it against a clear professional standard.
• Write concise, specific notes explaining what worked, what didn't, and why—feedback that's actionable, not vague.
• Maintain consistency and fairness across many evaluation sessions.

Requirements

• Regular, hands-on experience using AI chat assistants (ChatGPT, Claude, Gemini, or similar) for real personal or professional tasks.
• Strong written English with the ability to write clear, natural-sounding prompts and crisp evaluative notes.
• Good judgment and critical thinking—able to spot subtle errors, hallucinations, or unhelpful responses.
• Comfortable working independently on a flexible schedule with minimal oversight.
• Reliable internet connection and a quiet environment for focused evaluation sessions.

**Nice to have:**

• Background in writing, editing, research, QA, or customer support.
• Familiarity with multiple AI assistant platforms and their differing strengths/weaknesses.
• Prior experience in content moderation, data labeling, or AI training/evaluation work.

Compensation

This role pays a flat rate of $30/hour, remote and global. Multiple slots are open, with flexible hours based on availability.