English
Hebrew
Mercor Is Hiring A Data Annotation Specialist
Mercor connects elite creative and technical talent with leading AI research labs. We are hiring an AI Model Evaluator to join our team, offering flexible work opportunities in a dynamic environment. 🚀
About The Job
Based in San Francisco and backed by prominent investors such as Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey, Mercor specializes in linking top-tier talent with cutting-edge AI research initiatives.
Position Details
- Role: AI Model Evaluator
- Type: Full-time or Part-time Contract Work
- Compensation: $41/hour 💰
- Location: Restricted to Israel and USA 🌎
- Commitment: Flexible hours 🕒
Role Responsibilities
- Evaluate responses generated by large language models (LLMs) regarding their effectiveness in answering user queries. 🧠
- Conduct fact-checking using trusted public sources and external tools. 🔍
- Generate high-quality human evaluation data by annotating response strengths, areas for improvement, and identifying factual inaccuracies. ✍️
- Assess reasoning quality, clarity, tone, and completeness of responses. ✅
- Ensure model responses align with expected conversational behavior and system guidelines. 🤖
- Apply consistent annotations by following established taxonomies, benchmarks, and detailed evaluation guidelines. 📋
Qualifications
Must-Have
- Bachelor’s degree 🎓
- Native Hebrew speaker or ILR 5/primary fluency (C2 on CEFR scale) in Hebrew 🗣️
- Significant experience working with large language models (LLMs) 🗨️
- Excellent writing skills ✍️
- Strong attention to detail 🔎
- Adaptability across various topics, domains, and customer requirements 🌐
- Background or experience in domains requiring structured analytical thinking 📊
- Strong college-level mathematics skills 📐
Preferred
- Prior experience with RLHF, model evaluation, or data annotation work 📝
- Experience in writing or editing high-quality content ✒️
- Experience comparing multiple outputs and making nuanced qualitative judgments 🧾
- Familiarity with evaluation rubrics, benchmarks, or quality scoring systems 📊
Compensation & Legal
This is a hourly contractor position.
Application Process (20–30 mins to complete)
- Upload your resume 📄
- Complete an AI interview based on your resume 🤖
- Submit the application form 📝
Resources & Support
For details about the interview process and platform information, please visit: https://talent.docs.mercor.com/welcome/welcome
If you need any help or support, feel free to contact us at: support@mercor.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this exciting opportunity! ✨