JSON-fluent & Rubrics Expert

1 week, 2 days ago
Contract
Mid Level
Data Science and Analytics
Weekday

Weekday

Weekday helps companies hire engineers who are vouched by other software engineers, enabling passive income for engineers. They offer services like drafting outreach messages, shortlisting candidates, and conducting reference checks. Backed by Y Combin...

Construction & Engineering
11-50
Founded 2020

Description

  • Review written task descriptions and evaluation objectives.
  • Interpret grading rubrics and apply them consistently across assignments.
  • Read, parse, and evaluate JSON-formatted AI model outputs against predefined quality criteria.
  • Assign accurate, evidence-based scores and maintain consistency across evaluations.
  • Identify discrepancies, missing information, and rubric violations.
  • Maintain high standards of quality, accuracy, and attention to detail while meeting turnaround expectations.
  • Complete evaluations independently without using AI-assisted tools.

Requirements

  • Strong ability to read and interpret JSON files and structured data.
  • Experience applying detailed rubrics, quality standards, or evaluation frameworks consistently.
  • Excellent analytical thinking and exceptional attention to detail.
  • Comfort reviewing technical or structured documentation and following written guidelines precisely.
  • Previous experience in data annotation, quality assurance, content evaluation, AI model assessment, or similar analytical work is highly preferred.
  • Strong written communication skills and the ability to justify evaluation decisions clearly.
  • Ability to work independently while maintaining accuracy under tight deadlines.
  • Must be based in the United States.
  • Must be legally authorized to work in the United States; visa sponsorship, including H-1B and STEM OPT support, is not available.
  • Requires access to a desktop or laptop computer; Chromebooks are not supported.

Benefits

  • Compensation of $50-$175 per hour.
  • Fully remote work with flexible working hours.
  • Expected commitment of 20–40 hours per week.
  • Weekly payments processed through supported payment platforms.
  • Immediate onboarding with the opportunity to begin work right away.
  • Project duration may be extended based on business needs and individual performance.
  • Reasonable accommodations are available upon request.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Alpheratz Project - Indonesian (Indonesia) Translation Quality Rater

Welo Global Professional Services

Experienced language professionals are needed for an ongoing Indonesian (Indonesia) customer service content project focused on reviewing and refining machine-translated materials for accuracy and naturalness.

21 hours, 34 minutes ago

Alpheratz Project - Indonesian (Indonesia) Translation Quality Reviewer

Welo Global Professional Services

An ongoing language-quality role at a project supporting Indonesian (Indonesia) customer service content, focused on reviewing machine-translated materials and improving linguistic accuracy and consistency.

1 day, 21 hours ago

British Sign Language - Video Quality Specialist Remote (Ireland)

Welo Global Professional Services

Welocalize is hiring a contract video specialist in Ireland to support multilingual content operations for subscription video services using British Sign Language and technical video expertise.

4 days, 21 hours ago

Maps Personalization Quality Reviewer - Japanese (Japan)

Welo Global Professional Services

Freelance Quality Reviewer needed to audit rater work for a personalization feature within a maps/search experience for a Japanese-language project.

6 days, 21 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers