Medical Fellow, AI Model Evaluation (Remote, US) — Human Frontier Collective

Scale AI
Remote — United States (US work authorization required) Contract Oct 26, 2026 AI trainingmedical AIphysician1099 contractorScale AI
Project-based

About the role

Scale AI's Human Frontier Collective is recruiting physicians for a remote fellowship evaluating how advanced AI systems reason about clinical cases. Fellows write difficult clinical scenarios, judge model answers for safety and accuracy, and give structured feedback that shapes how the models think through medical decisions. It is a 1099 independent contractor engagement expected to last about six months, with a possible extension, and you set your own 10–40 hours a week. Scale does not publish a pay rate for the fellowship; it says rates vary by project, scope, skills and location.

Responsibilities

  • Write complex, realistic clinical scenarios that test how AI models reason
  • Evaluate model decisions for clinical safety and accuracy, including edge cases
  • Give structured, example-based feedback that improves the models' reasoning
  • Work on projects with Scale's partner AI labs as you are invited to them
  • Optionally co-author technical reports and research papers with Scale's research team

Requirements

  • MD or DO with board certification; residents, practicing clinicians and physicians with extensive clinical experience are eligible
  • Background in internal medicine, emergency medicine, radiology, surgery, psychiatry or a related specialty
  • Strong clinical reasoning and clear writing
  • Authorized to work in the United States (no visa sponsorship)
  • Prior AI experience is a plus, not required
  • Selection includes a short skills challenge and an interview; applicants who are turned down wait 90 days to reapply

What we offer

  • Fully remote with a flexible 10–40 hour week
  • About six months of work with potential to extend
  • Membership in Scale's interdisciplinary network of researchers and domain experts
  • Chance to be credited on published AI research
Project-based Apply now