Subject Matter Experts: ML Model Evaluation (PhD)

3 days, 15 hours ago
Contract
Senior
Artificial Intelligence and Machine Learning
Weekday

Weekday

Weekday helps companies hire engineers who are vouched by other software engineers, enabling passive income for engineers. They offer services like drafting outreach messages, shortlisting candidates, and conducting reference checks. Backed by Y Combin...

Construction & Engineering
11-50
Founded 2020

Description

  • Review machine learning model outputs for research-focused and domain-specific prompts.
  • Evaluate responses for factual accuracy, relevance, completeness, technical validity, and overall quality.
  • Use a structured rating system to score model responses consistently.
  • Write concise, evidence-based justifications for evaluation scores.
  • Identify factual errors, reasoning gaps, inconsistencies, irrelevant content, and areas for improvement.
  • Complete approximately five hours of independent evaluation work asynchronously.
  • Maintain consistent, detail-oriented evaluation standards throughout the project.

Requirements

  • PhD or equivalent advanced subject matter expertise in a relevant academic, technical, or professional discipline.
  • A finished or ongoing paper in NeurIPS, ICML, ICLR, COLM, or RLC.
  • Strong ability to critically assess complex technical, research, and domain-specific information.
  • Excellent analytical, research, critical-thinking, and written communication skills.
  • Ability to identify subtle inaccuracies, inconsistencies, and gaps in AI-generated content.
  • Ability to work independently in an asynchronous environment.
  • Must be located in the US, UK, Canada, EU, Norway, Switzerland, or Iceland.
  • Must be able to start by Friday end of day.
  • H-1B and STEM OPT candidates are not eligible.

Benefits

  • Pay of $150 per hour.
  • Weekly payments based on services rendered.
  • Fully remote work arrangement.
  • Flexible, asynchronous schedule.
  • Independent contractor engagement.
  • Potential for the project to be extended based on project needs and performance.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

AI Voice Recording Contributor - Simplified Chinese

RWS Group 5001+ Internet Software & Services

Join RWS as a freelance AI Voice Recording Contributor to record Simplified Chinese speech remotely, helping evaluate and improve speech and language AI systems.

15 hours, 27 minutes ago

AI Voice Recording Contributor - Simplified Chinese

RWS Group 5001+ Internet Software & Services

Join RWS as a freelance AI Voice Recording Contributor to provide Simplified Chinese voice recordings that help evaluate and improve speech and language AI systems.

15 hours, 27 minutes ago

AI Voice Recording Contributor - Mandarin Chinese (Hong Kong)

RWS Group 5001+ Internet Software & Services

RWS is seeking freelance Mandarin Chinese Voice Recording Contributors to remotely record scripted speech that supports the evaluation and improvement of AI speech and language systems.

15 hours, 27 minutes ago

AI Voice Recording Contributor - Mandarin Chinese (Hong Kong)

RWS Group 5001+ Internet Software & Services

RWS is seeking freelance Mandarin Chinese Voice Recording Contributors to create natural, high-quality recordings that support speech and language AI evaluation and improvement.

15 hours, 27 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers