LLM Evaluation Specialist – Analytical Reviewer | $70/hr Remote

Position: Generalist Expert

Type: Hourly contract

Compensation: $70 per hour

Location: Remote

Commitment: 10 - 40 hours/week

Role Responsibilities

• Evaluate AI-generated responses
• Provide structured written feedback
• Apply detailed evaluation guidelines to assess response quality
• Contribute to AI research through accurate and consistent evaluations

Requirements

• Strong analytical and written communication skills
• Ability to work independently and follow detailed task guidelines
• Strong critical reading skills with the ability to identify nuance, implicit meaning, and gaps in reasoning
• Ability to write clear, precise, and well-evidenced written rationales beyond surface-level observations
• Consistent and honest judgment with the ability to provide critical assessments when warranted
• Strict attention to detail and accurate application of structured evaluation guidelines
• Ability to work entirely without AI writing tools

Application Process (Takes 20 Mins)

• Upload resume
• Interview (15 min)
• Submit form