JSON-fluent & Rubrics Expert

2 months, 1 week ago
Contract
Mid Level
Data Science and Analytics
Weekday

Weekday

Weekday helps companies hire engineers who are vouched by other software engineers, enabling passive income for engineers. They offer services like drafting outreach messages, shortlisting candidates, and conducting reference checks. Backed by Y Combin...

Construction & Engineering
11-50
Founded 2020

Description

  • Review written task descriptions and evaluation objectives.
  • Interpret grading rubrics and apply them consistently across assignments.
  • Read, parse, and evaluate JSON-formatted AI model outputs against predefined quality criteria.
  • Assign accurate, evidence-based scores and maintain consistency across evaluations.
  • Identify discrepancies, missing information, and rubric violations.
  • Maintain high standards of quality, accuracy, and attention to detail while meeting turnaround expectations.
  • Complete evaluations independently without using AI-assisted tools.

Requirements

  • Strong ability to read and interpret JSON files and structured data.
  • Experience applying detailed rubrics, quality standards, or evaluation frameworks consistently.
  • Excellent analytical thinking and exceptional attention to detail.
  • Comfort reviewing technical or structured documentation and following written guidelines precisely.
  • Previous experience in data annotation, quality assurance, content evaluation, AI model assessment, or similar analytical work is highly preferred.
  • Strong written communication skills and the ability to justify evaluation decisions clearly.
  • Ability to work independently while maintaining accuracy under tight deadlines.
  • Must be based in the United States.
  • Must be legally authorized to work in the United States; visa sponsorship, including H-1B and STEM OPT support, is not available.
  • Requires access to a desktop or laptop computer; Chromebooks are not supported.

Benefits

  • Compensation of $50-$175 per hour.
  • Fully remote work with flexible working hours.
  • Expected commitment of 20–40 hours per week.
  • Weekly payments processed through supported payment platforms.
  • Immediate onboarding with the opportunity to begin work right away.
  • Project duration may be extended based on business needs and individual performance.
  • Reasonable accommodations are available upon request.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Configuration Engineer

ARCOS 51-250 Internet Software & Services

As a Configuration Engineer, you will configure, test, and deliver Clearion Web and Mobile applications for customer projects while progressively taking ownership of complex implementations.

Agile Confluence Git HTML JavaScript JIRA JSON SQL VS Code XML
3 days, 18 hours ago

Contractor: Senior Quality Engineer

Newsela 251-1K Diversified Consumer Services

Newsela is hiring a Senior Quality Engineer to ensure reliable, high-quality software across front-end applications, cloud services, internal LLMs, and third-party AI tools supporting K–12 classroom learning.

Agile AWS CI/CD Datadog Git GitHub Actions JavaScript Jest JIRA JMeter K6 Locust Mocha New Relic Playwright Python Scrum Selenium TypeScript
5 days, 17 hours ago

Senior QA Analyst (Frontend) – (India)

Welo Global Professional Services

Welo Data is seeking a remote Senior QA Analyst in India to own frontend-to-middleware quality for a multi-step web onboarding workflow and produce a validated state-model evidence package.

1 week, 1 day ago

QA Engineer (Contract)

CourtAvenue 51-250 Media

CourtAvenue is hiring a junior-to-mid-level contract QA Engineer to support its MarTech practice by validating DAM migrations, content platforms, and integrations before client release.

Agile JIRA Scrum
1 week, 3 days ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers