About the role
Mercor is hiring experienced software engineers to audit the repository-level coding tasks a frontier AI lab uses to train and benchmark its models. You check whether each task, reference patch and test harness is correct and reproducible, look for ways a model could game the grading, and write rubric-based feedback. The role is an hourly independent-contractor engagement at $70–$90 an hour, fully remote for US residents, worked on your own schedule and paid weekly through Stripe or Wise.
Responsibilities
- Audit SWE-Bench-style tasks for correctness and reproducibility
- Review reference patches, test runners and Docker isolation
- Spot answer leakage and reward hacking that would let a model game the grader
- Write clear, rubric-based feedback on each task
Requirements
- 3+ years of professional software engineering
- Real open-source contribution or maintainer history (merged PRs, committer or maintainer roles)
- Fluent in Python plus at least one of Java, Go, TypeScript or C++
- Nice to have: SWE-Bench (Verified) familiarity, maintainer history on major Python projects such as Django, scikit-learn or pytest, and prior code-review or grading work
- US-based; Mercor cannot support H-1B or STEM OPT candidates
What we offer
- $70–$90 an hour as an independent contractor (1099)
- Weekly pay through Stripe or Wise
- Fully remote, on your own schedule
- Project length can be extended or shortened depending on needs and performance