RemoteLeads Logo
Sign Up Sign In

Freelance Agent Evaluation Engineer

1 hour, 51 minutes ago
Mexico, Chile, Colombia, Costa Rica, Peru, Uruguay
Freelance
Senior
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 74

Description

  • Build realistic virtual developer environments with codebases, infrastructure, and supporting context.
  • Design challenging tasks from intermediate environment states and define clear success criteria.
  • Write tests that verify AI agent solutions and distinguish valid approaches from incorrect ones.
  • Review agent outputs, analyze failures, and refine tasks and tests based on QA feedback.
  • Ensure tasks are solvable by an AI agent while remaining sufficiently challenging.
  • Iterate on evaluation criteria to make assessments fair and robust.
  • Complete project tasks within deadline requirements and submit accepted work for payment.

Requirements

  • 5+ years of experience in software development.
  • Experience with Python, including FastAPI.
  • Experience with JavaScript/TypeScript, including React.
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Ability to work on a project-based, non-permanent engagement.
  • Willingness to estimate and complete tasks that may take about 20 hours each.

Benefits

  • Up to $40/hr equivalent compensation, depending on level and pace.
  • Flexible schedule with self-directed working hours.
  • Project-based work with tasks estimated at around 20 hours each.
  • Opportunity to work on AI evaluation projects for leading tech companies.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Gramian Consultancy Group

COBOL Developer (Legacy Systems & AI Training)

Gramian Consultancy Group Professional Services

Gramian Consultancy is seeking an experienced COBOL Developer to support AI training by analyzing legacy enterprise applications, documenting business logic, and creating realistic technical scenarios.

Turkey Poland Romania India Argentina Brazil Contract Senior AI (Artificial Intelligence) Software Engineer
COBOL
1 hour, 51 minutes ago
Weekday

Personal Financial Advisor (Private Wealth)

Weekday 11-50 Construction & Engineering

Remote contractor role for a Personal Financial Advisor (Private Wealth) supporting the training and evaluation of AI-powered financial advisory tools for a U.S. client.

United States Contract Mid Level AI (Artificial Intelligence) Financial Analyst
$166k-$229k
1 hour, 51 minutes ago
Toloka

Record Your Daily Routine & Get Paid - AI Training (Remote)

Toloka 251-1K Internet Software & Services

Project-based freelance work on an AI training platform recording first-person videos of everyday household activities to help train AI systems and robots.

Peru Freelance Entry Level AI (Artificial Intelligence)
$12k-$12k
1 hour, 51 minutes ago
Toloka

Record Your Daily Routine & Get Paid - AI Training (Remote)

Toloka 251-1K Internet Software & Services

Project-based freelance work for an AI training platform recording first-person videos of everyday household activities to help train AI systems and robots in real-world interaction.

Honduras Freelance Entry Level AI (Artificial Intelligence)
$12k-$12k
Machine Learning
1 hour, 51 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers