RemoteLeads Logo
Sign Up Sign In

Senior Python Engineer - AI Coding Agent Evaluation (Freelance)

1 hour, 42 minutes ago
Indonesia, Malaysia, Philippines, South Africa
Contract
Lead
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 91

Description

  • Build realistic virtual developer environments with codebases, infrastructure, tickets, docs, and conversation history.
  • Design tasks from intermediate environment states, including prompts and clear definitions of what counts as solved.
  • Write tests that validate agent solutions, accepting correct approaches and rejecting incorrect ones.
  • Review agent solutions, analyze failures, and refine tasks and tests based on QA feedback.
  • Iterate until evaluation criteria are fair, robust, and representative of real-world development work.
  • Assess whether tasks are solvable by an AI agent and appropriately challenging for frontier models.

Requirements

  • 8+ years of software development experience.
  • Experience with Python, including FastAPI.
  • Experience with JavaScript or TypeScript, including React.
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Submit CV in English and indicate English proficiency.
  • Ability to complete project-based work and meet task deadlines.
  • Experience working across multiple valid solution approaches (preferred).

Benefits

  • Up to $150/hr equivalent compensation, depending on level and pace.
  • Project-based work with flexible scheduling.
  • Estimated ~30 hours per task, with work completed on your own schedule.
  • Opportunity to work on leading-edge AI evaluation projects for tech companies.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Toloka

Record Your Daily Routine & Get Paid - AI Training (Remote)

Toloka 251-1K Internet Software & Services

Project-based freelance work for an AI training platform where contributors record first-person videos of everyday household activities to help train AI systems and robots.

Bolivia Costa Rica Dominican Republic Uruguay Freelance Entry Level AI (Artificial Intelligence)
1 hour, 27 minutes ago
Toloka

Record Your Daily Routine & Get Paid - AI Training (Remote)

Toloka 251-1K Internet Software & Services

A project-based freelance opportunity on an AI training platform to record first-person videos of everyday household activities for training AI systems and robots.

Argentina Freelance Entry Level AI (Artificial Intelligence)
$12k-$12k
1 hour, 27 minutes ago
Toloka

Record Your Daily Routine & Get Paid - AI Training (Remote)

Toloka 251-1K Internet Software & Services

Project-based freelance work on an AI training platform recording first-person videos of everyday household activities to help train AI systems and robots.

Colombia Freelance Entry Level AI (Artificial Intelligence)
1 hour, 27 minutes ago
Toloka

Record Your Daily Routine & Get Paid - AI Training (Remote)

Toloka 251-1K Internet Software & Services

Project-based freelance work for an AI training platform that pays individuals to record first-person videos of everyday household activities used to train AI systems and robots.

Peru Freelance Entry Level AI (Artificial Intelligence)
$12k-$12k
1 hour, 27 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers