AI Coding Assistant Evaluator — Freelance, Remote (Software Developers)

Alignerr
Remote, US (posting also notes a preference for developers in the Chicago, IL area) Contract Nov 1, 2026 ai trainingdata annotationrlhfsoftware developerfreelanceremote1099
$60–$100/hour

About the role

Alignerr is paying experienced software developers to evaluate AI coding assistants on real, personally-owned projects. You bring your own side project or approved open-source repo, use an AI coding assistant to work on it, and get paid to compare and rate the assistant's responses — classic human-preference AI training work, on your own schedule.

Responsibilities

  • Use an AI coding assistant to build features, fix bugs, refactor code, or write docs on your own project
  • At each step, compare two AI-generated responses and choose the stronger one
  • Write a short, clear rationale explaining why one response was better
  • Submit session transcripts and ratings as deliverables each week

Requirements

  • A personally-owned git repository (a side project, personal tool, or an open-source repo you're approved to use) — employer-owned, client, university, startup, or NDA-covered projects are not eligible
  • Comfortable working from the terminal and with git on macOS, Linux, or Windows with WSL2
  • Clear, concise written communication for explaining your evaluations
  • Self-motivated and able to work independently on a flexible schedule
  • Project must not itself be an AI training/eval tool, ML infrastructure, security-exploit tooling, or anything touching other people's personal data

What we offer

  • Free AI assistant usage during paid sessions
  • Weekly payouts
  • Fully remote, flexible scheduling
  • Work on your own project instead of someone else's dataset
$60–$100/hour Apply now