JSON-fluent & Rubrics Expert

4 days, 1 hour ago
Contract
Mid Level
Data Science and Analytics
Weekday

Weekday

Weekday helps companies hire engineers who are vouched by other software engineers, enabling passive income for engineers. They offer services like drafting outreach messages, shortlisting candidates, and conducting reference checks. Backed by Y Combin...

Construction & Engineering
11-50
Founded 2020

Description

  • Review written task descriptions and evaluation objectives.
  • Interpret grading rubrics and apply them consistently across assignments.
  • Read, parse, and evaluate JSON-formatted AI model outputs against predefined quality criteria.
  • Assign accurate, evidence-based scores and maintain consistency across evaluations.
  • Identify discrepancies, missing information, and rubric violations.
  • Maintain high standards of quality, accuracy, and attention to detail while meeting turnaround expectations.
  • Complete evaluations independently without using AI-assisted tools.

Requirements

  • Strong ability to read and interpret JSON files and structured data.
  • Experience applying detailed rubrics, quality standards, or evaluation frameworks consistently.
  • Excellent analytical thinking and exceptional attention to detail.
  • Comfort reviewing technical or structured documentation and following written guidelines precisely.
  • Previous experience in data annotation, quality assurance, content evaluation, AI model assessment, or similar analytical work is highly preferred.
  • Strong written communication skills and the ability to justify evaluation decisions clearly.
  • Ability to work independently while maintaining accuracy under tight deadlines.
  • Must be based in the United States.
  • Must be legally authorized to work in the United States; visa sponsorship, including H-1B and STEM OPT support, is not available.
  • Requires access to a desktop or laptop computer; Chromebooks are not supported.

Benefits

  • Compensation of $50-$175 per hour.
  • Fully remote work with flexible working hours.
  • Expected commitment of 20–40 hours per week.
  • Weekly payments processed through supported payment platforms.
  • Immediate onboarding with the opportunity to begin work right away.
  • Project duration may be extended based on business needs and individual performance.
  • Reasonable accommodations are available upon request.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Maps Personalization Quality Reviewer - Japanese (Japan)

Welo Global Professional Services

Freelance Quality Reviewer needed to audit rater work for a personalization feature within a maps/search experience for a Japanese-language project.

1 day, 1 hour ago

Maps Personalization Quality Reviewer - French (Canada)

Welo Global Professional Services

Freelance Quality Reviewer for a maps/search personalization project, auditing rater work to ensure outputs follow guidelines, are accurate, and include complete evidence.

1 day, 1 hour ago

Senior QA Engineer - (3 months) - Remote - Immediate Hiring

Robusta Technology Group 51-250 Professional Services

RTG and Octopus are seeking a Senior QA Engineer to ensure product quality and reliability across the software development lifecycle through close collaboration with cross-functional teams in a growing engineering environment.

Agile
5 days, 1 hour ago

Freelance Agent Evaluation Engineer

Mindrift.ai: Be the “I” in AI Internet Software & Services

Mindrift is seeking an experienced software developer for a project-based role building and evaluating datasets that test AI coding agents in realistic development environments.

Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
1 week ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers