JSON-fluent & Rubrics Expert

3 weeks, 4 days ago
Contract
Mid Level
Data Science and Analytics
Weekday

Weekday

Weekday helps companies hire engineers who are vouched by other software engineers, enabling passive income for engineers. They offer services like drafting outreach messages, shortlisting candidates, and conducting reference checks. Backed by Y Combin...

Construction & Engineering
11-50
Founded 2020

Description

  • Review written task descriptions and evaluation objectives.
  • Interpret grading rubrics and apply them consistently across assignments.
  • Read, parse, and evaluate JSON-formatted AI model outputs against predefined quality criteria.
  • Assign accurate, evidence-based scores and maintain consistency across evaluations.
  • Identify discrepancies, missing information, and rubric violations.
  • Maintain high standards of quality, accuracy, and attention to detail while meeting turnaround expectations.
  • Complete evaluations independently without using AI-assisted tools.

Requirements

  • Strong ability to read and interpret JSON files and structured data.
  • Experience applying detailed rubrics, quality standards, or evaluation frameworks consistently.
  • Excellent analytical thinking and exceptional attention to detail.
  • Comfort reviewing technical or structured documentation and following written guidelines precisely.
  • Previous experience in data annotation, quality assurance, content evaluation, AI model assessment, or similar analytical work is highly preferred.
  • Strong written communication skills and the ability to justify evaluation decisions clearly.
  • Ability to work independently while maintaining accuracy under tight deadlines.
  • Must be based in the United States.
  • Must be legally authorized to work in the United States; visa sponsorship, including H-1B and STEM OPT support, is not available.
  • Requires access to a desktop or laptop computer; Chromebooks are not supported.

Benefits

  • Compensation of $50-$175 per hour.
  • Fully remote work with flexible working hours.
  • Expected commitment of 20–40 hours per week.
  • Weekly payments processed through supported payment platforms.
  • Immediate onboarding with the opportunity to begin work right away.
  • Project duration may be extended based on business needs and individual performance.
  • Reasonable accommodations are available upon request.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Senior QA Engineer (AI-Assisted Development)

kake 1-10 Internet Software & Services

Senior QA Engineer role focused on testing modern software products, including web applications, APIs, backend systems, and AI-assisted features, to ensure quality, reliability, and performance across the development lifecycle.

C# CI/CD Cypress Java JavaScript Playwright Python Selenium TypeScript
2 days, 8 hours ago

Project Cursa - Robot Manipulation Video Annotator

Welo Global Professional Services

Welo Data is hiring remote annotators to label robot manipulation videos for AI training by producing precise, structured descriptions of actions across synchronized camera views.

LLM
3 days, 8 hours ago

Project Rana - Human Pose Keypoint Annotation QC (Thermal Imagery)

Welo Global Professional Services

Welo Data is hiring a detail-oriented QC Reviewer to review human pose keypoint annotations on thermal images and ensure annotation quality in Label Studio.

Computer Vision
3 days, 8 hours ago

Data Governance & Quality Engineer

NTT DATA 100K+ IT Services

NTT DATA is hiring a remote Data Governance & Quality Engineer to support a U.S. client from LATAM by implementing and operating data governance, catalog, lineage, and quality platforms.

Azure CRM Databricks ERP
4 days, 8 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers