Freelance Agent Evaluation Engineer

8 hours, 42 minutes ago
Freelance
Senior
Artificial Intelligence and Machine Learning
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services

Description

  • Build realistic simulated developer environments containing codebases, infrastructure, tickets, documentation, and conversations.
  • Design challenging tasks from intermediate environment states, including prompts and solution criteria.
  • Write functional and integration tests that accept valid agent solutions and reject incorrect ones.
  • Review agent-generated solutions, analyze failures, and refine tasks and tests based on QA feedback.
  • Ensure tasks are solvable, realistic, fair, and robust for evaluating AI coding agents.

Requirements

  • 5+ years of software development experience.
  • Proficiency with Python and FastAPI.
  • Experience with JavaScript/TypeScript and React.
  • Experience with Docker, PostgreSQL, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Master’s degree in Computer Science, Software Engineering, Data Science, AI/ML, NLP, Information Systems, or a related field.
  • A bachelor’s degree is accepted with 5 years of relevant experience.
  • At least 3 years of professional experience in related roles or domains, specifically for QA automation/testing or cybersecurity roles.
  • Submit a CV in English and indicate English proficiency.

Benefits

  • Remote, part-time freelance work that can fit around other commitments.
  • Payment per accepted task, with rates up to the equivalent of $50 per hour.
  • Access to advanced AI projects and portfolio-building experience.
  • Opportunity to influence how future AI models perform in software development.
  • Project-based workflow: qualify, join a project, complete tasks, and get paid.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Strategic Operations Manager, AI-Enabled Systems (AI-Fluent) Consultant

Right Side Up 51-250 Professional Services

Right Side Up is seeking freelance operations-minded builders for its talent collective to help client teams replace manual workflows and ad hoc tools with scalable processes, AI-enabled systems, and internal applications.

Supabase Vercel
7 hours, 57 minutes ago

Music Production Expert - Norwegian

Weekday 11-50 Construction & Engineering

A client is seeking a Norwegian Music Production Expert to remotely evaluate AI-generated music for musicality, production quality, vocal performance, genre alignment, and prompt adherence in a research project.

8 hours, 42 minutes ago

Investment analysis / valuation / credit Evaluator

Weekday 11-50 Construction & Engineering

An independent contractor Evaluator will remotely review AI-generated investment analysis, valuation, and credit documents, spreadsheets, and presentations for accuracy, rigor, and domain quality.

1 day, 8 hours ago

Generative Audio Evaluation - French (France)

RWS Group 5K-10K Internet Software & Services

RWS is seeking French-speaking freelance Generative Audio Evaluation Specialists in France to assess AI-generated audio and provide structured feedback for improving a generative audio model.

Generative AI
2 days, 7 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers