Weekday

Weekday helps companies hire engineers who are vouched by other software engineers, enabling passive income for engineers. They offer services like drafting outreach messages, shortlisting candidates, and conducting reference checks. Backed by Y Combin...

Construction & Engineering

Industrials

11-50 (45)

Founded 2020

75 open positions

Links

View All Jobs

AI Rater Guidelines Writer (Linguist / Instructional Designer)

1 week, 6 days ago

United States

Contract

Mid Level

AI (Artificial Intelligence)

Artificial Intelligence and Machine Learning

Generative AI

Apply Now

Weekday

Construction & Engineering

11-50

Founded 2020

View All Jobs 75

Description

Translate complex and ambiguous program requirements into clear rater guidelines.
Design evaluation rubrics and scoring frameworks for consistent decision-making across AI tasks.
Review existing documentation to identify ambiguities, inconsistencies, contradictions, and coverage gaps.
Revise guidelines to improve clarity, usability, and reviewer confidence.
Convert subject-matter requirements from finance, retail, insurance, legal, and sports into structured evaluation instructions.
Collaborate with research teams, program managers, and subject matter experts to maintain consistency across guideline sets.
Develop documentation for edge cases, exceptions, and complex evaluation scenarios.
Minimize reviewer escalation by creating precise and actionable guidance.
Support high-quality human evaluation processes for AI training and model assessment.

Requirements

Minimum 3 years of professional experience in linguistics, instructional design, technical writing, AI content development, or a related field.
Direct experience creating, refining, or maintaining evaluation guidelines, rating rubrics, or reviewer instructions in Generative AI, RLHF, human evaluation, or AI data annotation environments.
Demonstrated ability to work across multiple subject areas and convert domain knowledge into structured documentation.
Proven experience resolving ambiguity and improving written specifications with measurable results.
Strong analytical thinking with exceptional attention to detail.
Outstanding written communication skills with the ability to explain nuanced concepts clearly and consistently.
Demonstrated career progression and professional growth.
Ability to commit reliably to 35+ hours per week during weekdays.
Experience supporting AI model training, evaluation, RLHF, or large-scale annotation programs (preferred).
Familiarity with structured documentation standards, quality assurance methodologies, and guideline governance (preferred).

Benefits

Compensation of $45-$65 per hour.
Fully remote engagement with flexible working arrangements.
Weekly payments based on approved work completed.
Independent contractor engagement.
Opportunity to influence the quality and consistency of AI training data across multiple industries.
Reasonable accommodations available throughout the application and engagement process.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Singapore English | Consumer Electronics Global Brand

OLIVER 1K-5K Media

OLIVER is seeking a Singapore English freelancer to support a global consumer electronics brand with transcreation and selected copywriting deliverables.

Singapore Freelance Junior Content Writer Technical Writer

Copywriting

11 hours, 45 minutes ago

Apply

11 hours, 45 minutes ago

Canadian English | Consumer Electronics Global Brand

OLIVER 1K-5K Media

OLIVER is seeking a Canadian English freelancer to support a global consumer electronics brand by delivering and maintaining high-quality transcreation for key clients.

Canada Freelance Mid Level Content Writer Technical Writer

Copywriting

12 hours ago

Apply

12 hours ago

South Africa English | Consumer Electronics Global Brand

OLIVER 1K-5K Media

OLIVER is seeking a South African English freelancer to support a global consumer electronics brand with transcreation and copy adaptation for key client communications.