RemoteLeads Logo
Sign Up Sign In

Senior Python Engineer - AI Coding Agent Evaluation (Freelance)

1 day, 3 hours ago
United States
Freelance
Lead
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 91

Description

  • Build realistic developer environments with codebases, infrastructure, and supporting context such as tickets, docs, and conversations.
  • Design tasks from intermediate environment states, including crafting prompts and defining what counts as solved.
  • Write tests to verify agent solutions and ensure they accept valid approaches while rejecting incorrect ones.
  • Review agent solutions, analyze failures, and refine tasks and tests based on QA feedback.
  • Iterate on evaluation criteria to make assessments fair, robust, and challenging for strong AI models.
  • Assess whether tasks are solvable by an AI agent within the simulated environment.
  • Work within project-based assignments rather than permanent employment.
  • Submit completed tasks by the deadline and meet the listed acceptance criteria.

Requirements

  • 8+ years of experience in software development.
  • Experience with the core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Submit your CV in English.
  • Ability to work on project-based tasks and choose your own schedule.
  • Tasks are estimated at about 30 hours each, depending on complexity.
  • Understanding of how to evaluate AI coding agent outputs and failure modes.

Benefits

  • Up to $200/hr equivalent compensation, depending on level and pace.
  • Project-based flexible schedule; you choose when and how to work.
  • Tasks are estimated at around 30 hours each.
  • Work with leading tech companies on AI evaluation projects.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Verve

Senior software Engineer

Verve
Remote
Contract Backend Engineer Software Engineer
Apache API Gateway AWS CI/CD Go Java Node.js Python Scala System Design TypeScript
17 hours, 30 minutes ago
Toloka

Record Your Daily Routine & Get Paid - AI Training (Remote)

Toloka 251-1K Internet Software & Services

Project-based freelance work for an AI training platform where contributors record first-person videos of everyday household activities to help train AI systems and robots.

Bolivia Costa Rica Dominican Republic Uruguay Freelance Entry Level AI (Artificial Intelligence)
1 day, 2 hours ago
Toloka

Record Your Daily Routine & Get Paid - AI Training (Remote)

Toloka 251-1K Internet Software & Services

A project-based freelance opportunity on an AI training platform to record first-person videos of everyday household activities for training AI systems and robots.

Argentina Freelance Entry Level AI (Artificial Intelligence)
$12k-$12k
1 day, 2 hours ago
Toloka

Record Your Daily Routine & Get Paid - AI Training (Remote)

Toloka 251-1K Internet Software & Services

Project-based freelance work on an AI training platform recording first-person videos of everyday household activities to help train AI systems and robots.

Colombia Freelance Entry Level AI (Artificial Intelligence)
1 day, 2 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers