Mercor Is Hiring A Conversational Ai Specialist Remote

Join Our Team as an AI Model Evaluator! 🚀

Mercor connects top-tier creative and technical talent with leading AI research labs. Based in San Francisco, we are backed by renowned investors including Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.

Position Details

• Role: AI Model Evaluator
• Type: Full-time or Part-time Contract Work
• Compensation: $41/hour
• Location: Restricted to Israel and USA
• Commitment: Flexible hours

Role Responsibilities 📝

• Evaluate large language model (LLM) responses for their effectiveness in answering user queries.
• Conduct fact-checking using trusted public sources and external tools.
• Generate high-quality human evaluation data by annotating strengths, areas for improvement, and factual inaccuracies of responses.
• Assess reasoning quality, clarity, tone, and completeness of AI responses.
• Ensure model outputs align with expected conversational behaviors and system guidelines.
• Apply consistent annotations following clear taxonomies, benchmarks, and detailed evaluation guidelines.

Qualifications

Must-Have ✅

• Bachelor’s degree
• Native speaker or ILR 5 / C2 fluency in Hebrew
• Extensive experience with large language models (LLMs)
• Excellent writing skills
• Strong attention to detail
• Adaptability across topics, domains, and customer requirements
• Background or experience in domains requiring structured analytical thinking
• Excellent college-level mathematics skills

Preferred 👌

• Experience with RLHF, model evaluation, or data annotation
• Experience in writing or editing high-quality content
• Ability to compare multiple outputs and make fine-grained qualitative judgments
• Familiarity with evaluation rubrics, benchmarks, or quality scoring systems

Compensation & Legal Details 💼

• Hourly contractor position

Application Process (⏱️ 20–30 minutes)

1. Upload your resume
2. Complete an AI interview based on your resume
3. Submit the application form

Resources & Support 💡

For details about the interview process and platform, visit: https://talent.docs.mercor.com/welcome/welcome

If you need assistance, contact us at: support@mercor.com

Note: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this exciting opportunity! 🚀