Senior Software Engineer - AI Agent Evaluation

1 week, 1 day ago
Contract
Senior
Software Development
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services

Description

  • Build realistic virtual developer environments with codebase, infrastructure, and supporting context such as tickets, documentation, and conversations.
  • Design tasks from intermediate states of those environments and define clear success criteria for what counts as solved.
  • Write functional and integration tests that validate agent solutions and distinguish correct approaches from incorrect ones.
  • Review agent solutions, analyze failures, and refine tasks and tests based on QA feedback.
  • Iterate on evaluation materials to make the dataset fair, robust, and challenging for advanced AI models.
  • Work within the project-based workflow by completing assigned tasks and submitting them by the deadline.
  • Guide and evaluate the agent's work rather than writing most of the code from scratch.

Requirements

  • 5+ years of experience in software development.
  • Experience with the core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, and Redis.
  • Experience writing tests, including functional and integration tests.
  • English proficiency at B2 level or higher.
  • CV must be submitted in English.
  • Ability to work on a project-based engagement rather than permanent employment.
  • Understanding of how to assess real-world developer tasks and identify failure modes in AI-generated solutions.
  • Ability to complete tasks estimated at around 20 hours each, subject to complexity and deadline requirements.

Benefits

  • Up to $50/hr equivalent compensation, depending on level and pace.
  • Flexible schedule with tasks completed on your own time.
  • Project-based paid work with estimated 20-hour task assignments.
  • Opportunity to work on cutting-edge AI evaluation projects for leading tech companies.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

QA Contractor

FUTURESEARCH Internet Software & Services

FUTURESEARCH is hiring a meticulous quality checker to verify research evaluation tasks used for training and benchmarking frontier AI models.

13 hours, 35 minutes ago

Dynamics Great Plains Developer

Sapsol Technologies 51-250 Internet Software & Services

Microsoft Dynamics GP Developer needed to design, integrate, test, and support custom business applications and reporting solutions for a third-party-connected enterprise environment.

Azure C# Git Power BI REST API SOAP SQL Server Visual Studio
1 day, 13 hours ago

Expert Rater Project - Dutch (Netherlands)

Welo Global Professional Services

Welo Data is hiring a Dutch-speaking Expert Rater in the Netherlands to support rater quality, client test rounds, and AI advertising data evaluation on a short-term freelance project.

1 day, 13 hours ago

SQL Developer + DBA (AWS & Azure Experience)

Bounteous 1K-5K Internet Software & Services

Bounteous is hiring an experienced SQL Developer + Database Administrator to manage and optimize production database environments across Azure SQL, AWS MySQL, and related applications in a fast-paced digital consultancy setting.

AWS Entity Framework MySQL SQL Server
1 day, 13 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers