Freelance Agent Evaluation Engineer

14 hours, 1 minute ago
Freelance
Senior
Artificial Intelligence and Machine Learning
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services

Description

  • Build realistic simulated developer environments containing codebases, infrastructure, tickets, documentation, and conversation history.
  • Design challenging tasks from intermediate environment states, including prompts and solution criteria.
  • Write functional and integration tests that accept valid agent solutions and reject incorrect ones.
  • Review agent-generated solutions and analyze failures.
  • Iterate on tasks and tests based on QA feedback to improve fairness and robustness.
  • Define scenarios that reveal weaknesses in advanced AI coding models.

Requirements

  • 5+ years of software development experience.
  • Proficiency with Python and FastAPI.
  • Experience with JavaScript or TypeScript and React.
  • Experience with Docker, PostgreSQL, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Master’s degree in Computer Science, Software Engineering, Data Science, AI/ML, NLP, Information Systems, or a related field.
  • A bachelor’s degree is accepted with 5 years of relevant experience.
  • Minimum 3 years of professional experience in related domains, particularly QA automation/testing or cybersecurity roles.
  • CV submitted in English with English proficiency indicated.

Benefits

  • Remote, part-time freelance work that can fit around professional or academic commitments.
  • Payment per accepted task, with rates up to the equivalent of $40 per hour.
  • Opportunity to work on advanced AI projects and strengthen your portfolio.
  • Opportunity to influence how future AI models understand and communicate in your area of expertise.
  • Project-based participation with a qualification and paid-task workflow.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Education Expert - Sourcing Funnel (Private)

Weekday 11-50 Construction & Engineering

An education expert will help a client develop and evaluate AI systems by translating professional educational judgment, workflows, and standards into realistic tasks, structured examples, and detailed feedback.

14 hours, 1 minute ago

Finance operations / audit support Evaluator

Weekday 11-50 Construction & Engineering

A client is seeking remote Finance Operations/Audit Support Evaluators to assess AI-generated documents, spreadsheets, and slide decks for accuracy, rigor, and domain quality.

1 day, 14 hours ago

AI Trainer - Fluent Kazakh Speaker - Stuttgart, Germany

Prolific 51-250 Professional Services

Prolific is seeking fluent Kazakh speakers to evaluate AI-generated text for naturalness, authenticity, tone, and cultural fit, helping improve AI language models.

NLP
2 days, 13 hours ago

AI Trainer - Fluent Kazakh Speaker - Leipzig, Germany

Prolific 51-250 Professional Services

Prolific is seeking fluent Kazakh speakers to evaluate AI-generated text for naturalness, authenticity, tone, and cultural accuracy as part of human-data projects that improve language models.

2 days, 13 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers