AI Rater Guidelines Writer — Linguist / Instructional Designer for LLM Training

About the role

We're building the next generation of large language models, and we need sharp linguistic and instructional minds to help make our training data rigorous, consistent, and unambiguous. As an AI Rater Guidelines Writer, you'll sit at the intersection of language, pedagogy, and applied AI — translating fuzzy, high-level program requirements into crystal-clear rating instructions that human raters can apply consistently across thousands of examples.

This is precision writing at scale: every ambiguity you remove from a guideline prevents downstream noise in model training data. You'll work across a wide range of domains — finance, retail, insurance, legal, healthcare, and more — so intellectual curiosity and fast domain ramp-up matter as much as writing craft.

What you'll do

• Translate ambiguous program requirements and stakeholder intent into clear, unambiguous rater guidelines and rubrics
• Design instructional materials, examples, and edge-case documentation that help raters apply judgment consistently
• Collaborate with linguists, prompt engineers, and program managers to pressure-test guidelines against real rating scenarios
• Iterate on guidelines based on rater feedback, inter-rater agreement data, and quality audits
• Identify and resolve edge cases, contradictions, and gaps in existing instructions before they propagate into training data
• Adapt tone, structure, and specificity of guidelines to fit varied domains (finance, retail, insurance, legal, and beyond)
• Document decision rationale so guidelines remain maintainable and auditable over time

Requirements

• Strong background in linguistics, instructional design, technical writing, curriculum development, or a related field
• Demonstrated ability to write clear, structured, unambiguous instructions for non-expert audiences
• Comfort working with ambiguous or evolving requirements and turning them into concrete deliverables
• Excellent attention to detail and consistency across large sets of documentation
• Ability to reason about edge cases and anticipate where instructions might be misapplied
• Strong written English communication skills
• Based in the United States and available for remote collaboration

**Nice to have:**

• Prior experience with AI/ML data annotation, rater guidelines, or RLHF-style quality workflows
• Experience writing for multiple subject-matter domains (finance, insurance, retail, legal, etc.)
• Familiarity with LLM training pipelines or human evaluation processes

Compensation

$45–$65 per hour, based on experience. Remote, U.S.-based engagement with flexible scheduling.