RemoteLeads Logo
Sign Up Sign In

Senior Python Engineer - AI Coding Agent Evaluation (Freelance)

2 days, 14 hours ago
Egypt, Algeria, Morocco, Tunisia
Freelance
Lead
AI (Artificial Intelligence)
Software Development
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 65

Description

  • Build realistic developer environments that include a virtual company codebase, infrastructure, and supporting context such as tickets, docs, and conversations.
  • Design tasks from intermediate environment states, including crafting prompts and defining what counts as a solved solution.
  • Ensure each task is solvable by an AI agent and suitable for evaluating real-world developer work.
  • Write tests that verify agent solutions by accepting valid approaches and rejecting incorrect ones.
  • Review agent outputs, analyze failures, and iterate on tasks and tests based on QA feedback.
  • Refine evaluation criteria until the benchmark is fair, robust, and reliable.
  • Assess model performance on coding tasks and identify scenarios that expose weaknesses in AI solutions.

Requirements

  • 8+ years of experience in software development.
  • Experience with Python, including FastAPI.
  • Experience with JavaScript/TypeScript, including React.
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Submit a CV in English and indicate English proficiency level.
  • Ability to work project-based rather than as permanent employment.

Benefits

  • Up to $150/hr equivalent compensation, depending on level and pace.
  • Tasks estimated at about 30 hours each.
  • Flexible schedule with full control over when and how you work.
  • Project-based work with payment tied to completed tasks.
  • Opportunity to work on leading-edge AI evaluation projects.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Innovecs

Middle/Senior Back-end Engineer (PHP, 3-6 months project assignment)

Innovecs 251-1K Capital Markets

Innovecs is seeking a Middle/Senior Back-end Engineer to support integrations and core platform work for a logistics-focused fulfillment platform on a 3–6 month engagement.

Poland Romania Spain Ukraine Contract Senior Backend Engineer
PHP
14 hours, 1 minute ago
Workana

Senior Python Backend Engineer (AWS, Client-Facing)

Workana 1K-5K Internet Software & Services

Senior Software Engineer at a consulting-focused company building and scaling backend services and client-facing solutions in a fully remote environment.

Mexico Argentina Brazil Colombia Peru Uruguay Contract Senior Backend Engineer Customer Success
AWS CI/CD Django DynamoDB FastAPI Flask Git MySQL PostgreSQL Python REST API SQL
1 day, 13 hours ago
Welo Global

Hydrus -Voice Contributor — English (US)

Welo Global Professional Services

Welo Data is hiring a freelance native-level English (US) speaker to record conversational customer-service audio for a remote AI voice project.

United States Freelance Entry Level AI (Artificial Intelligence) Customer Support
$100k-$100k
1 day, 13 hours ago
Crosslake

AI / ML Consultant

Crosslake 251-1K Capital Markets

Crosslake is hiring a US-based remote consultant to help technology and private equity clients evaluate, improve, and implement AI and software-related initiatives across the investment lifecycle.

United States Contract Lead AI (Artificial Intelligence)
Agile Computer Vision Generative AI LLM Machine Learning MLOps NLP Statistics
1 day, 14 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers