RemoteLeads Logo
Sign Up Sign In

Freelance Agent Evaluation Engineer

6 days, 14 hours ago
India
Freelance
Senior
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 65

Description

  • Build realistic developer environments that include a virtual company, codebase, infrastructure, and supporting context such as tickets, documentation, and conversations.
  • Design tasks from intermediate states of these environments, including crafting prompts and defining what counts as a solved task.
  • Ensure each task is solvable by an AI agent and grounded in realistic development scenarios.
  • Write tests that validate agent solutions, accepting all correct approaches and rejecting incorrect ones.
  • Review agent solutions, analyze failures, and iterate on tasks and tests based on QA feedback.
  • Refine evaluation criteria until the tasks are fair, robust, and accurately measure model performance.
  • Assess the quality of coding-agent outputs across multiple possible solution paths.
  • Work within a project-based workflow to complete assigned tasks by deadline.

Requirements

  • 5+ years of experience in software development.
  • Experience with Python, specifically FastAPI.
  • Experience with JavaScript or TypeScript, specifically React.
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Ability to work on project-based assignments rather than permanent employment.
  • CV must be submitted in English.
  • Ability to complete tasks estimated at around 20 hours each and meet acceptance criteria.

Benefits

  • Up to $30/hr equivalent compensation, depending on level and pace.
  • Flexible project-based schedule with the ability to choose when and how to work.
  • Tasks are estimated at about 20 hours each, with no fixed schedule requirement.
  • Payment after completing tasks in the project workflow.
  • Opportunity to work on cutting-edge AI evaluation work for leading tech companies.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Welo Global

Hydrus -Voice Contributor — English (US)

Welo Global Professional Services

Welo Data is hiring a freelance native-level English (US) speaker to record conversational customer-service audio for a remote AI voice project.

United States Freelance Entry Level AI (Artificial Intelligence) Customer Support
$100k-$100k
1 day, 13 hours ago
Crosslake

AI / ML Consultant

Crosslake 251-1K Capital Markets

Crosslake is hiring a US-based remote consultant to help technology and private equity clients evaluate, improve, and implement AI and software-related initiatives across the investment lifecycle.

United States Contract Lead AI (Artificial Intelligence)
Agile Computer Vision Generative AI LLM Machine Learning MLOps NLP Statistics
1 day, 14 hours ago
Welo Global

Hydrus -Voice Contributor — Spanish (Spain)

Welo Global Professional Services

Welo Data is hiring a freelance native-level Spanish (Spain) speaker to record conversational customer-service voice data for a remote conversational AI project.

Spain Freelance Entry Level AI (Artificial Intelligence) Customer Support
$83k-$83k
1 day, 14 hours ago
Gramian Consultancy Group

Chemical Engineering Expert - AI Training

Gramian Consultancy Group Professional Services

Gramian Consultancy is hiring an experienced Chemical Engineering expert for a short-term remote freelance role reviewing AI training tasks for advanced research environments.

United States Mexico Brazil Colombia Egypt Turkey Bangladesh India Indonesia Pakistan Vietnam Ghana Contract Lead AI (Artificial Intelligence)
Python
2 days, 13 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers