RemoteLeads Logo
Sign Up Sign In

Freelance Agent Evaluation Engineer

6 days, 13 hours ago
South Africa
Contract
Senior
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 65

Description

  • Build realistic developer environments that include a virtual company, codebase, infrastructure, and supporting context such as tickets, docs, and conversations.
  • Design tasks from intermediate states of those environments, including the prompt and clear definitions of what counts as a solved task.
  • Write tests that validate agent solutions by accepting all correct approaches and rejecting incorrect ones.
  • Review AI agent solutions, analyze failures, and refine tasks and tests based on QA feedback.
  • Iterate on evaluation criteria until the task set is fair, robust, and able to distinguish strong and weak solutions.
  • Ensure each task is solvable by an AI agent and grounded in realistic development scenarios.
  • Work within a project-based engagement to complete assigned tasks by the required deadline.
  • Collaborate with the evaluation process rather than writing production code from scratch.

Requirements

  • 5+ years of experience in software development.
  • Strong experience with Python, including FastAPI.
  • Strong experience with JavaScript/TypeScript, including React.
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Ability to evaluate AI coding outputs and understand where models commonly fail.
  • Willingness to work on a project-based basis rather than as a permanent employee.

Benefits

  • Up to $40/hr equivalent compensation, depending on level and pace.
  • Flexible schedule with the ability to choose when and how to work.
  • Project tasks estimated at approximately 20 hours each.
  • Payment is tied to completed tasks that meet the acceptance criteria.
  • Opportunity to work on AI evaluation projects for leading tech companies.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Sapsol Technologies

Dynamics Great Plains Developer

Sapsol Technologies 51-250 Internet Software & Services

Microsoft Dynamics GP Developer needed to design, integrate, test, and support custom business applications and reporting solutions for a third-party-connected enterprise environment.

Canada Contract Senior Software Engineer
Azure C# Git Power BI REST API SOAP SQL Server Visual Studio
1 day, 13 hours ago
Welo Global

Hydrus -Voice Contributor — English (US)

Welo Global Professional Services

Welo Data is hiring a freelance native-level English (US) speaker to record conversational customer-service audio for a remote AI voice project.

United States Freelance Entry Level AI (Artificial Intelligence) Customer Support
$100k-$100k
1 day, 13 hours ago
Bounteous

SQL Developer + DBA (AWS & Azure Experience)

Bounteous 1K-5K Internet Software & Services

Bounteous is hiring an experienced SQL Developer + Database Administrator to manage and optimize production database environments across Azure SQL, AWS MySQL, and related applications in a fast-paced digital consultancy setting.

United States Contract Senior Database Administrator Software Engineer
AWS Entity Framework MySQL SQL Server
1 day, 13 hours ago
Crosslake

AI / ML Consultant

Crosslake 251-1K Capital Markets

Crosslake is hiring a US-based remote consultant to help technology and private equity clients evaluate, improve, and implement AI and software-related initiatives across the investment lifecycle.

United States Contract Lead AI (Artificial Intelligence)
Agile Computer Vision Generative AI LLM Machine Learning MLOps NLP Statistics
1 day, 14 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers