About the role
Applied machine-learning practitioners can earn $70–$90 an hour reviewing the ML challenge tasks a frontier AI lab uses to train and evaluate its models. The work is judging methodological rigor: whether experiments are designed soundly, whether model-selection choices hold up, and whether evaluation avoids leakage and metric gaming, then writing rubric-based feedback. Mercor engages auditors as remote independent contractors in the US, on a self-set schedule, with weekly payment through Stripe or Wise. It is an evaluation role, not LLM app building or MLOps.
Responsibilities
- Assess experiment design and model-selection reasoning in applied ML tasks
- Check evaluation methodology for leakage, metric gaming and train/test/CV hygiene
- Reproduce results and test ML claims against the evidence
- Write clear, rubric-based feedback on each task
Requirements
- 3+ years of hands-on applied or experimental ML (experiment design, model selection, tuning, evaluation)
- Strong data-quality rigor: leakage detection and clean train/test/CV splits
- Proficient with PyTorch, TensorFlow, scikit-learn or XGBoost
- Nice to have: Kaggle or benchmark competition experience, applied ML research or publications, prior peer-review or grading work
- US-based; Mercor cannot support H-1B or STEM OPT candidates
What we offer
- $70–$90 an hour as an independent contractor (1099)
- Weekly pay through Stripe or Wise
- Fully remote, on your own schedule
- Project length can be extended or shortened depending on needs and performance