RemoteLeads Logo
Sign Up Sign In

LLM Red Team Specialist - Failure Modes & Edge Cases

1 week, 2 days ago
United States
Contract
Junior
AI (Artificial Intelligence)
Artificial Intelligence and Machine Learning
Git LLM Machine Learning Python
Apply Now
Weekday

Weekday

Weekday helps companies hire engineers who are vouched by other software engineers, enabling passive income for engineers. They offer services like drafting outreach messages, shortlisting candidates, and conducting reference checks. Backed by Y Combin...

Construction & Engineering
11-50
Founded 2020
View All Jobs 75

Description

  • Investigate frontier AI model performance across coding, machine learning, analytical reasoning, and complex problem-solving tasks.
  • Identify hidden failure modes, edge cases, reasoning errors, and vulnerabilities not captured by standard testing.
  • Design challenging, objective, and reproducible evaluation tasks for AI capability assessment.
  • Document findings with clear technical explanations, supporting evidence, and reproducible methods.
  • Collaborate with benchmark designers and AI researchers to refine tasks, eliminate loopholes, and strengthen grading criteria.
  • Share insights and recommendations with cross-functional teams to improve benchmark coverage and evaluation quality.

Requirements

  • Master's degree, PhD, or equivalent practical experience in a STEM discipline involving research, coding, or advanced data analysis.
  • Minimum 1 year of experience in AI research, research engineering, security research, AI evaluation, or a related technical field.
  • Demonstrated experience identifying vulnerabilities, adversarial behaviors, edge cases, or failure modes in large language models or other machine learning systems.
  • Strong proficiency in Python and Git, with the ability to build custom scripts for experimentation, testing, and analysis.
  • Solid understanding of modern large language models, their strengths, limitations, and evaluation methodologies.
  • Experience with AI benchmarking, model evaluation, adversarial testing, prompt engineering, or dataset creation is highly desirable.
  • Excellent analytical thinking, creativity, and attention to detail, with the ability to solve ambiguous, open-ended problems independently.
  • Outstanding written communication skills for documenting technical findings clearly and accurately.
  • Ability to commit approximately 35 hours per week on a consistent basis.
  • Experience with AI safety, red teaming, adversarial machine learning, or security research is preferred.
  • Background in benchmark design, evaluation framework development, or AI quality assurance is preferred.
  • Experience creating reproducible technical experiments and documenting complex failure analyses is preferred.
  • Familiarity with frontier AI research methodologies and model capability assessments is preferred.
  • Unable to support H1-B or STEM OPT candidates.

Benefits

  • Compensation of $60-$90 per hour.
  • Fully remote work with flexible working hours.
  • Independent contractor engagement.
  • Weekly payments based on approved work completed.
  • Opportunity to work on cutting-edge AI systems with researchers.
  • Project duration may be extended, shortened, or concluded based on project requirements and performance.
  • Reasonable accommodations are available throughout the application and engagement process.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Prolific

AI Trainer - Electrical Engineers - CAD/Python Expertise - (Remote Advisory - Norway)

Prolific 51-250 Professional Services

Prolific is recruiting Electrical Engineers for a remote advisory expert network project that helps train AI models using real-world CAD design and validation workflows.

Norway Contract Mid Level AI (Artificial Intelligence)
$250k-$250k
Linux Python
16 hours, 21 minutes ago
Prolific

AI Training - Open Source CAD Research - Aerospace Engineers - US

Prolific 51-250 Professional Services

Prolific is seeking aerospace engineers for a remote, task-based AI training project that captures real-world CAD design and validation workflows to help train next-generation AI models.

United States Freelance Mid Level AI (Artificial Intelligence)
$0k-$0k
Linux Python
16 hours, 36 minutes ago
Prolific

AI Trainer - Mechanical Engineers - CAD Expertise - (Remote Advisory - Netherlands)

Prolific 51-250 Professional Services

Prolific is seeking mechanical engineers to join its remote expert network and help train AI models by completing CAD design and validation tasks based on real-world engineering workflows.

Netherlands Contract Senior AI (Artificial Intelligence)
$250k-$250k
Linux Python
16 hours, 36 minutes ago
Prolific

AI Trainer - Mechanical Engineers - CAD Expertise - (Remote Advisory - Switzerland)

Prolific 51-250 Professional Services

Prolific is seeking mechanical engineers to join a remote advisory talent pool supporting AI research by demonstrating real-world CAD design and validation workflows.

Switzerland Contract Mid Level AI (Artificial Intelligence)
$25k-$250k
Linux Python
16 hours, 36 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers