Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services

Information Technology

10 open positions

Links

View All Jobs

Senior Python Engineer - AI Coding Agent Evaluation (Freelance)

7 hours, 6 minutes ago

Japan, Singapore, South Korea, New Zealand

Freelance

Lead

AI (Artificial Intelligence)

Software Development

Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript

Apply Now

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services

View All Jobs 10

Description

Build realistic virtual developer environments with codebases, infrastructure, and supporting context such as tickets, docs, and conversations.
Design tasks from intermediate states of those environments and define what counts as a solved outcome.
Write tests that validate agent solutions and distinguish correct approaches from incorrect ones.
Review agent solutions, analyze failures, and refine tasks and tests based on QA feedback.
Iterate until the evaluation setup is fair, robust, and solvable by an AI agent.
Assess real-world developer task difficulty and identify scenarios that expose model weaknesses.
Work within project deadlines and submit tasks that meet acceptance criteria.

Requirements

8+ years of experience in software development.
Experience with Python, including FastAPI.
Experience with JavaScript or TypeScript, including React.
Experience with Docker, Postgres, Kafka, and Redis.
Experience writing functional and integration tests.
English proficiency at B2 level or higher.
Submit your CV in English.
Ability to complete project-based tasks, estimated at 30 hours depending on complexity.
Interest in evaluating AI coding agents and working in a project-based environment.

Benefits

Paid per accepted task.
Compensation can reach the equivalent of up to $200/hr depending on qualification tier and task efficiency.
Flexible project-based work with no fixed schedule; you choose when and how to work.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Generative AI Analyst | Spanish (Mexico)

Welo Global Professional Services

Welo Data is hiring remote freelance Generative AI Analysts in Mexico to review and evaluate AI-related content for Spanish (Mexico) projects.

Mexico Freelance Junior AI (Artificial Intelligence)

$10k-$10k

Generative AI

6 hours, 51 minutes ago

Apply

6 hours, 51 minutes ago

Generative AI Analyst | Spanish (US)

Welo Global Professional Services

Welo Data is hiring a remote freelance Generative AI Analyst - Spanish (US) to review and annotate multilingual, multimodal AI content for a short-term project supporting dataset quality and model training.

United States Freelance Junior AI (Artificial Intelligence) QA Engineer

$31k-$31k

LLM

6 hours, 51 minutes ago

Apply

6 hours, 51 minutes ago

Senior Full Stack Rust Engineer, Contract - US

DEPT® 1K-5K Media

DEPT® is hiring a Full Stack Rust Engineer to build and maintain production Rust systems in a distributed, containerized environment with backend ownership at the core.