RemoteLeads Logo
Sign Up Sign In

JSON-fluent & Rubrics Expert

1 month, 2 weeks ago
United States
Contract
Mid Level
QA Engineer
Data Science and Analytics
JSON
Apply Now
Weekday

Weekday

Weekday helps companies hire engineers who are vouched by other software engineers, enabling passive income for engineers. They offer services like drafting outreach messages, shortlisting candidates, and conducting reference checks. Backed by Y Combin...

Construction & Engineering
11-50
Founded 2020
View All Jobs 107

Description

  • Review written task descriptions and evaluation objectives.
  • Interpret grading rubrics and apply them consistently across assignments.
  • Read, parse, and evaluate JSON-formatted AI model outputs against predefined quality criteria.
  • Assign accurate, evidence-based scores and maintain consistency across evaluations.
  • Identify discrepancies, missing information, and rubric violations.
  • Maintain high standards of quality, accuracy, and attention to detail while meeting turnaround expectations.
  • Complete evaluations independently without using AI-assisted tools.

Requirements

  • Strong ability to read and interpret JSON files and structured data.
  • Experience applying detailed rubrics, quality standards, or evaluation frameworks consistently.
  • Excellent analytical thinking and exceptional attention to detail.
  • Comfort reviewing technical or structured documentation and following written guidelines precisely.
  • Previous experience in data annotation, quality assurance, content evaluation, AI model assessment, or similar analytical work is highly preferred.
  • Strong written communication skills and the ability to justify evaluation decisions clearly.
  • Ability to work independently while maintaining accuracy under tight deadlines.
  • Must be based in the United States.
  • Must be legally authorized to work in the United States; visa sponsorship, including H-1B and STEM OPT support, is not available.
  • Requires access to a desktop or laptop computer; Chromebooks are not supported.

Benefits

  • Compensation of $50-$175 per hour.
  • Fully remote work with flexible working hours.
  • Expected commitment of 20–40 hours per week.
  • Weekly payments processed through supported payment platforms.
  • Immediate onboarding with the opportunity to begin work right away.
  • Project duration may be extended based on business needs and individual performance.
  • Reasonable accommodations are available upon request.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Mindrift.ai: Be the “I” in AI

Freelance Agent Evaluation Engineer

Mindrift.ai: Be the “I” in AI Internet Software & Services

Mindrift is seeking experienced software developers for a project-based, remote opportunity creating and evaluating realistic coding tasks and tests for AI coding agents used by leading technology companies.

Poland Freelance Senior AI (Artificial Intelligence) QA Engineer
Up to $83k
Cybersecurity Docker FastAPI JavaScript Kafka Machine Learning PostgreSQL Python React Redis TypeScript
14 hours, 48 minutes ago
Deputy

Lead QA Engineer (12 month contract)

Deputy 251-1K Professional Services

Deputy is seeking a Quality Assurance technical leader for a 12-month maternity leave cover to define the organisation’s QA strategy, advance test automation, and maintain high-quality standards across its global workforce management platform.

Australia Contract Lead QA Engineer
Agile AWS Azure CI/CD GCP Go gRPC JavaScript Jenkins OpenAPI PHP SQL Travis CI
2 days, 14 hours ago
Recruitment & Search Agency - Headhunter in the Philippines

Submission Review Analyst - (Accountant/Book Keeper/Financial Analyst) #34250

Recruitment & Search Agency - Headhunter in the Philippines 11-50 Professional Services

Join an Australian SaaS company in the financial viability industry as a Submission Review (Financial) Analyst, reviewing, reconciling, and preparing financial information for business assessments.

Philippines Contract Mid Level Financial Analyst QA Engineer
3 days, 14 hours ago
PadSplit

QA Engineer (Anywhere in Europe)

PadSplit 51-250 Family Services

PadSplit is hiring a QA Engineer to strengthen quality across its Marketplace pod by validating mobile, web, and backend experiences and improving release confidence.

Ukraine Contract Mid Level QA Engineer
$20k-$30k
Android iOS Postman REST API
1 week, 6 days ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers