Mercor Alabaster Is Hiring A Bilingual German Generalist Evaluator Expert 28

Mercor is seeking talented native German speakers from Austria, Switzerland, or Germany with exceptional writing skills to contribute to a high-impact AI research project alongside a leading lab. 🧠✍️

About the Role

This opportunity involves authoring German/English prompt-golden answer pairs to help train and evaluate advanced language models. It is a short-term, flexible position ideal for professionals who excel in language mastery, critical thinking, and instructional clarity. You will distill complex concepts into culturally grounded German texts while maintaining technical precision in English.

Core Responsibilities

• Multilingual Prompt Design & Optimization: Craft detailed prompts in German and/or English, embedding multiple constraints and instructions tailored to German-speaking users in Austria, Switzerland, and Germany.
• Define and Document Evaluation Standards: Establish high-level expectations for accurate responses specific to regional consumer contexts, creating comprehensive rubrics that respect linguistic nuance, tone, and cultural conventions.
• Model Testing and Bilingual Grading: Run prompts through models, assess outputs for accuracy, fluency, and cultural fit in German and English, and compare results accordingly.
• Benchmarking & Quality Assurance: Collaborate on quality reviews to ensure prompt tasks and rubrics maintain consistency and reliability across German benchmarks before official deployment.

Minimum Qualifications

• Native-level fluency in German (writing), specific to Austria, Switzerland, or Germany, with strong English reading and writing skills.
• Residency or significant experience in Austria, Switzerland, or Germany, with deep cultural and linguistic familiarity.
• Educational background: BS or BA from a reputable institution (completed or in progress).
• Strong writing and critical thinking skills.
• Ability to work independently and meet deadlines.
• Familiarity with ChatGPT or similar tools for personal decision-making, hobbies, or interests.
• Location: Based in Austria, Switzerland, or Germany, or capable of producing culturally accurate German aligned with these regions.

Preferred Qualifications

• Experience in teaching, research, editing, or academic writing.
• Creating evaluation criteria, rubrics, or grading guidelines.
• Familiarity with large language models (LLMs), prompting, or model evaluation (helpful but not required).

Application & Onboarding Process

1. Complete a short AI-led interview (~15 minutes).
2. If approved, participate in a paid assessment focused on writing and rubric creation.
3. Upon success, receive an invitation to join the project team.

More Details About This Role

• Expect to dedicate at least 20 hours per week.
• The project duration is approximately 2-4 months.
• You will work within a structured environment with clear goals and tools.
• We value diversity and are committed to providing reasonable accommodations, considering all qualified applicants without regard to protected characteristics. 😊