RemoteLeads Logo
Sign Up Sign In

Freelance Agent Evaluation Engineer

3 days, 1 hour ago
Mexico, Chile, Colombia, Costa Rica, Peru, Uruguay
Freelance
Senior
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 61

Description

  • Build realistic virtual developer environments with codebase, infrastructure, and contextual materials such as tickets, docs, and conversations.
  • Design coding tasks from intermediate states of those environments and define what counts as a solved outcome.
  • Write tests that verify agent solutions and accurately distinguish correct from incorrect approaches.
  • Review agent solutions, analyze failures, and refine tasks and tests based on QA feedback.
  • Ensure tasks are solvable by an AI agent while remaining challenging for strong models.
  • Iterate on evaluation criteria to make assessments fair and robust.

Requirements

  • 5+ years of software development experience.
  • Experience with Python and FastAPI.
  • Experience with JavaScript/TypeScript and React.
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2+ level.
  • CV must be submitted in English.
  • Project-based participation, not permanent employment.

Benefits

  • Up to $40/hr equivalent compensation, depending on level and pace.
  • Estimated 20 hours of work per task, with flexible scheduling.
  • Choose when and how to work, with no fixed schedule requirement.
  • Payment after completing qualification and project tasks.
  • Project-based opportunity with leading tech companies.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Welo Global

Shape the Future of AI — Maori Talent Hub

Welo Global Professional Services

Welo Data, part of Welocalize, is building a global contributor network for flexible remote AI data projects that help improve the accuracy, safety, and inclusiveness of AI systems.

New Zealand Europe Anywhere Contract Entry Level AI (Artificial Intelligence) Data Annotator
LLM
1 hour, 8 minutes ago
Volga Partners

AI Evaluation & Annotation Reviewer (L3 - Advanced Level) - Portuguese

Volga Partners 51-250 Internet Software & Services

An AI Evaluation & Annotation Reviewer at an AI project company evaluates language data and AI-generated content for Portuguese-language quality in a long-term, client-facing engagement.

Peru Spain Portugal Brazil Contract Entry Level AI (Artificial Intelligence) Data Annotator
$17k-$24k
Machine Learning
1 hour, 8 minutes ago
Volga Partners

AI Evaluation & Annotation Reviewer (L3 - Advanced) - French/Italian/Portuguese

Volga Partners 51-250 Internet Software & Services

AI Evaluation & Annotation Reviewer for a long-term artificial intelligence project, supporting the review and improvement of AI-generated language content.

United States Contract Junior AI (Artificial Intelligence) Data Annotator
$8k-$37k
Machine Learning
1 hour, 8 minutes ago
redbee

Especialista en Automation Testing

redbee 51-250 Internet Software & Services

redbee busca sumar un perfil de automatización de pruebas para un proyecto de transformación digital en una entidad bancaria líder, enfocado en modernizar canales digitales e integrar sistemas core en equipos ágiles.

Peru Colombia Contract Mid Level SDET (Software Development Engineer in Test)
Agile CI/CD
1 hour, 23 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers