RemoteLeads Logo
Sign Up Sign In

Freelance Agent Evaluation Engineer

3 hours, 24 minutes ago
Israel, Jordan, Kuwait, Qatar, Saudi Arabia, Turkey
Contract
Senior
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 91

Description

  • Build realistic developer environments that include a codebase, infrastructure, and supporting context such as tickets, docs, and conversations.
  • Design tasks from intermediate states of those environments and define what counts as a solved outcome.
  • Write tests that validate agent solutions, accepting all valid approaches and rejecting incorrect ones.
  • Review agent solutions, analyze failures, and refine tasks and tests based on QA feedback.
  • Iterate on evaluation criteria to make tasks fair, robust, and challenging for AI coding agents.
  • Ensure each task is solvable by an AI agent and grounded in realistic developer workflows.

Requirements

  • 5+ years of software development experience.
  • Experience with the core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Submit a CV in English.
  • Participation is project-based, not permanent employment.
  • Ability to complete tasks within an estimated 20 hours per task and meet acceptance criteria by the deadline.
  • Preferred ability to understand where AI models fail in coding tasks and design scenarios that expose those weaknesses.

Benefits

  • Up to $50/hr equivalent compensation, depending on level and pace.
  • Flexible schedule with self-directed work hours.
  • Project-based paid work opportunities.
  • Estimated 20 hours of work per task, depending on complexity.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

FuseGlobal Partners

MES Application Developer (remote)

FuseGlobal Partners 11-50 Internet Software & Services

FuseGlobal is hiring a remote MES Application Developer for a Fortune 100 food and beverage manufacturer to translate business needs into MES solutions and support factory operations.

Anywhere Contract Senior Software Engineer
$150k-$150k
Agile Angular Azure C# C++ CSS DB2 Entity Framework Hibernate HTML Java JavaScript Scrum SQL Server
3 hours, 9 minutes ago
Toloka

Record Your Daily Routine & Get Paid - AI Training (Remote)

Toloka 251-1K Internet Software & Services

Project-based freelance work for an AI training platform where contributors record first-person videos of everyday household activities to help train AI systems and robots.

Bolivia Costa Rica Dominican Republic Uruguay Freelance Entry Level AI (Artificial Intelligence)
3 hours, 9 minutes ago
Toloka

Record Your Daily Routine & Get Paid - AI Training (Remote)

Toloka 251-1K Internet Software & Services

A project-based freelance opportunity on an AI training platform to record first-person videos of everyday household activities for training AI systems and robots.

Argentina Freelance Entry Level AI (Artificial Intelligence)
$12k-$12k
3 hours, 9 minutes ago
Toloka

Record Your Daily Routine & Get Paid - AI Training (Remote)

Toloka 251-1K Internet Software & Services

Project-based freelance work on an AI training platform recording first-person videos of everyday household activities to help train AI systems and robots.

Colombia Freelance Entry Level AI (Artificial Intelligence)
3 hours, 9 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers