RemoteLeads Logo
Sign Up Sign In

Senior Software Engineer - LLM Trainer

8 hours, 15 minutes ago
United States
Contract
Senior
AI (Artificial Intelligence)
Software Development
Bash C++ Docker Git Go Java JavaScript Linux Microservices Python SQL TypeScript
Apply Now
kake

kake

Kake offers premier teams of software engineers for leading brands. Curated by people, supported by AI, and powered by positivity, our high-performing engineers seamlessly join your team to scale with your needs.

Internet Software & Services
1-10
View All Jobs 17

Description

  • Create and review coding tasks covering debugging, refactoring, code generation, API usage, testing, performance, security, and edge cases.
  • Write correct, clear, testable reference solutions aligned with task requirements.
  • Evaluate AI-generated code using rubrics for correctness, clarity, security, performance, maintainability, and instruction-following.
  • Compare model responses, select the strongest answer, and provide clear technical justification.
  • Identify bugs, hallucinated APIs, missing edge cases, weak explanations, and poor engineering decisions.
  • Use terminal-based workflows to run tests, debug issues, manage dependencies, and navigate repositories.
  • Follow detailed evaluation guidelines and participate in calibration activities to maintain consistent quality.

Requirements

  • 5+ years of professional software engineering experience in a backend, fullstack, or systems role.
  • Strong proficiency in at least one of Python, JavaScript/TypeScript, Go, Java, C++, or SQL.
  • Hands-on experience with Terminal-Bench and evaluating AI agents on terminal-based software engineering tasks.
  • Experience with Git, command-line tools, and common development workflows.
  • Ability to assess code for functionality, design quality, security, and maintainability.
  • Experience with AI data production, RLHF, data annotation, or LLM evaluation projects.
  • Excellent written and verbal English communication skills.
  • Ability to work independently in a remote, asynchronous, fast-paced environment.
  • Strong attention to detail and ability to follow complex, rubric-based guidelines.
  • Preferred: experience with Python workflows, automated testing, Docker, Linux, Bash, containers, large codebases, open-source projects, backend systems, data engineering, DevOps, infrastructure, or security; PST (GMT-8) timezone overlap required.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Superior Executive And Legal Recruiting

AI Solutions Engineer (Contractor) - Fully Remote

Superior Executive And Legal Recruiting 1-10 Professional Services

Superior Executive & Legal Recruiting is seeking a fully remote AI Solutions Engineer Consultant to help a high-volume legal business assess workflows and implement a firm-wide AI and automation strategy.

United States Contract Lead AI (Artificial Intelligence)
8 hours, 30 minutes ago
Welo Global

Project Lyra Bengali Culture Expert

Welo Global Professional Services

Welo Data is seeking Bengali-speaking freelance language evaluators in Bangladesh or India to assess and improve AI-generated content and create Bengali language datasets remotely.

India Bangladesh Freelance Mid Level AI (Artificial Intelligence)
$31k-$31k
LLM
9 hours, 15 minutes ago
Toloka

Freelance Annotator (English) - AI Trainer

Toloka 251-1K Internet Software & Services

Mindrift seeks freelance AI annotators to review and label data for project-based initiatives that test and improve AI systems, with work available remotely when projects are active.

Philippines Freelance Junior AI (Artificial Intelligence) Data Annotator
Up to $17k
9 hours, 15 minutes ago
Welo Global

Project Lyra Filipino Culture Expert

Welo Global Professional Services

Welo Data is seeking Filipino-speaking freelance language evaluators in the Philippines to assess and improve AI-generated language content through structured evaluation and writing tasks.

Philippines Freelance Mid Level AI (Artificial Intelligence)
$37k-$37k
LLM
9 hours, 15 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers