RemoteLeads Logo
Sign Up Sign In

Freelance Agent Evaluation Engineer

9 hours, 17 minutes ago
South Korea
Freelance
Senior
AI (Artificial Intelligence)
Artificial Intelligence and Machine Learning
Cybersecurity Docker FastAPI JavaScript Kafka Machine Learning PostgreSQL Python React Redis TypeScript
Apply Now
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services
View All Jobs 2

Description

  • Build realistic simulated developer environments with codebases, infrastructure, tickets, documentation, and conversation history.
  • Design challenging coding tasks from intermediate environment states, including prompts and solution criteria.
  • Write functional and integration tests that accept valid agent solutions and reject incorrect ones.
  • Review AI-generated solutions and analyze failures using QA feedback.
  • Iterate on tasks and tests to make evaluations fair, solvable, and robust.
  • Guide and evaluate AI coding agents rather than primarily writing production code from scratch.

Requirements

  • 5+ years of software development experience.
  • Proficiency with Python and FastAPI.
  • Experience with JavaScript or TypeScript and React.
  • Experience with Docker, PostgreSQL, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • Master’s degree in Computer Science, Software Engineering, Data Science, Artificial Intelligence, NLP, Information Systems, or a related field.
  • A bachelor’s degree is accepted with 5 years of relevant experience.
  • At least 3 years of professional experience in related QA automation/testing or cybersecurity roles.
  • Submit a CV in English and indicate English proficiency.

Benefits

  • Remote, part-time freelance work that can fit around other professional or academic commitments.
  • Payment per accepted task, with rates based on qualification tier and efficiency.
  • Earning potential up to the equivalent of $50 per hour.
  • Experience working on advanced AI projects to strengthen your portfolio.
  • Opportunity to influence how future AI models understand and communicate in software development.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Acclaro

English to Turkish MTPE & AI Translation Evaluator (Software Localization)

Acclaro 251-1K Professional Services

Acclaro is seeking a freelance English-to-Turkish MT/LLM evaluator to assess and improve AI-generated software, cloud, SaaS, and IT localization for global technology projects.

United States Canada Latin America Freelance Senior AI (Artificial Intelligence) Linguist
9 hours, 17 minutes ago
Weekday

Music & Lyrics Expert - Norwegian

Weekday 11-50 Construction & Engineering

A client is seeking a Norwegian Music & Lyrics Expert to evaluate AI-generated lyrics for quality, originality, cultural authenticity, and alignment with intended prompts in a fully remote research project.

United States Contract Mid Level AI (Artificial Intelligence)
$87k-$162k
2 days, 9 hours ago
10A Labs

Biosecurity & AI Safety Fellowship

10A Labs Internet Software & Services

10a Labs is hiring Biology Subject Matter Experts for a short-term remote contract supporting biological content review, risk assessment, and AI safety and biosecurity systems.

United States Contract Mid Level AI (Artificial Intelligence)
$166k-$250k
3 days, 8 hours ago
Toloka

AI Trainer - Freelance Data Annotator

Toloka 251-1K Internet Software & Services

Mindrift is seeking freelance AI annotators to review and label data for project-based efforts that test, evaluate, and improve AI systems when projects are available.

Australia Freelance Junior AI (Artificial Intelligence) Data Annotator
Up to $42k
3 days, 9 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers