Senior Python Engineer - AI Coding Agent Evaluation (Freelance)

1 week, 3 days ago
Freelance
Lead
Software Development
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services

Description

  • Build realistic developer environments that simulate a virtual company with codebase, infrastructure, tickets, docs, and conversation history.
  • Design tasks from intermediate states of these environments and define clear success criteria.
  • Write tests that validate agent solutions, accepting valid approaches and rejecting incorrect ones.
  • Review agent solutions, analyze failures, and refine tasks and tests based on QA feedback.
  • Ensure tasks are solvable by an AI agent and produce fair, robust evaluations.
  • Assess where frontier models fail and create scenarios that meaningfully differentiate solution quality.
  • Collaborate through the project workflow from qualification to task completion and submission.

Requirements

  • 8+ years of experience in software development.
  • Experience with Python, including FastAPI.
  • Experience with JavaScript or TypeScript, including React.
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2+ level.
  • CV must be submitted in English.
  • Ability to work on a project-based, non-permanent basis.
  • Ability to complete tasks that take approximately 30 hours each and meet acceptance criteria by the deadline.

Benefits

  • Up to $150/hr equivalent compensation, depending on level and pace.
  • Flexible schedule with self-directed work timing.
  • Project-based opportunities with leading tech companies.
  • Estimated 30 hours per task, allowing for work at your own pace.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

AI Trainer - Electrical Engineers - CAD/Python Expertise - (Remote Advisory - Norway)

Prolific 51-250 Professional Services

Prolific is recruiting Electrical Engineers for a remote advisory expert network project that helps train AI models using real-world CAD design and validation workflows.

Linux Python
16 hours, 20 minutes ago

AI Training - Open Source CAD Research - Aerospace Engineers - US

Prolific 51-250 Professional Services

Prolific is seeking aerospace engineers for a remote, task-based AI training project that captures real-world CAD design and validation workflows to help train next-generation AI models.

Linux Python
16 hours, 35 minutes ago

AI Trainer - Mechanical Engineers - CAD Expertise - (Remote Advisory - Netherlands)

Prolific 51-250 Professional Services

Prolific is seeking mechanical engineers to join its remote expert network and help train AI models by completing CAD design and validation tasks based on real-world engineering workflows.

Linux Python
16 hours, 35 minutes ago

AI Trainer - Mechanical Engineers - CAD Expertise - (Remote Advisory - Switzerland)

Prolific 51-250 Professional Services

Prolific is seeking mechanical engineers to join a remote advisory talent pool supporting AI research by demonstrating real-world CAD design and validation workflows.

Linux Python
16 hours, 35 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers