Mercor Alabaster Is Hiring A Bilingual German Generalist Evaluator Expert 21

Mercor is seeking talented native German speakers from Austria, Switzerland, or Germany to contribute to an impactful AI research project. This freelance role involves crafting high-quality prompt and answer pairs to help train and evaluate advanced language models.

About the Role

This opportunity is ideal for professionals with mastery in language, critical thinking skills, and a knack for clear instruction. The role requires a deep understanding of local language nuances, cultural context, and regional conventions, including Austrian German, Swiss Standard German, and Germany-specific language use.

Key Responsibilities

• Multilingual Prompt Design & Optimization: Develop detailed prompts in German and/or English with multiple constraints, ensuring they are natural, relevant, and tailored for users in Austria, Switzerland, and Germany.
• Define and Document Evaluation Standards: Establish high standards for correct responses within regional consumer contexts, creating rubrics that respect linguistic, tonal, and cultural nuances.
• Model Testing & Grading (Bilingual): Test prompts against AI models, assess output accuracy, fluency, and cultural relevance in both German and English, and compare results as needed.
• Benchmarking & Quality Assurance: Collaborate on QA reviews to ensure prompt quality, standards adherence, and reliability across German language benchmarks before their final integration.

Minimum Qualifications

• Native-level fluency in written German, specifically within Austria, Switzerland, or Germany, with strong English reading and writing skills.
• Residence or significant experience in Austria, Switzerland, or Germany, with deep cultural and linguistic familiarity.
• Bachelor’s degree (completed or in progress) from a reputable institution.
• Excellent writing and critical thinking abilities.
• Ability to work independently and meet deadlines.
• Experience with ChatGPT or similar AI tools for personal or professional use.
• Reliable ability to produce culturally accurate, country-specific German content.

Preferred Qualifications

• Experience in teaching, research, editing, or academic writing.
• Experience developing evaluation rubrics, grading criteria, or assessment guidelines.
• Familiarity with large language models (LLMs), prompting, or model evaluation—helpful but not required.

Application & Onboarding Process

Steps include:

1. Complete a brief AI-led interview (approximately 15 minutes).
2. If successful, undertake a paid assessment focusing on writing skills and rubric creation.
3. Upon approval, receive an invitation to join the project team.

Additional Details

• Work Commitment: Expect around 20 hours per week.
• Project Duration: Approximate commitment of 2 to 4 months.
• Work Environment: Structured project with clear goals and dedicated tools.
• Inclusivity: We welcome all qualified applicants, ensuring equal opportunity regardless of protected characteristics, with accommodations available upon request.