Mercor Is Hiring A Bilingual Spanish Generalist Evaluator Expert

🚀 Mercor is looking for talented native Spanish speakers from the United States, Spain, Chile, or Mexico to contribute to a cutting-edge AI research project. If you have exceptional writing skills and a passion for language, this is your chance to make a significant impact in the field of AI language models!

Job Overview

As a freelancer, you will create high-quality prompts and responses in Spanish and English to train and evaluate advanced language models. This role requires cultural awareness, linguistic precision, and attention to detail to ensure the models perform accurately across different Spanish-speaking regions.

Key Responsibilities

• Prompt Development: Craft detailed prompts in Spanish and/or English, incorporating multiple constraints and instructions. Ensure natural phrasing and cultural relevance for users in the United States, Spain, Chile, and Mexico.
• Evaluation Standards: Define and document high-level expectations for correct responses tailored to specific regional contexts. Develop comprehensive rubrics considering linguistic nuances, tone, and cultural conventions.
• Model Testing & Grading: Run prompts through language models, assess outputs for accuracy, fluency, and cultural appropriateness, and compare bilingual results when necessary.
• Quality Assurance & Benchmarking: Collaborate in QA reviews to ensure prompt tasks and rubrics meet rigorous standards, maintaining consistency and reliability before official integration.

Minimum Qualifications

• Native-level fluency in Spanish, specific to the United States, Spain, Chile, or Mexico, with strong English reading and writing skills.
• Cultural & linguistic familiarity—residence or significant in-country experience in one of these regions.
• Educational background: BS or BA from a reputable institution (completed or in progress).
• Writing & critical-thinking skills with the ability to work independently and meet deadlines.
• Experience with ChatGPT or similar AI tools for decision-making, hobbies, or general interests.
• Location: Based in or able to reliably produce culturally accurate Spanish for the specified country.

Preferred Qualifications

• Experience in teaching, research, editing, or academic writing.
• Background in creating evaluation rubrics, grading guidelines, or assessment criteria.
• Familiarity with Large Language Models (LLMs), prompting, or model evaluation (helpful but not mandatory).

Application & Onboarding Process

🎯 To apply, you'll complete an AI-led interview (~15 minutes). If successful, you'll undertake a paid writing and rubric creation assessment. Upon passing, you'll be invited to join the project team.

More Details About This Role

• Workload: Contribute at least 20 hours per week.
• Duration: Approximately 2 to 4 months.
• Environment: Participate in a structured project with clear goals and tools.
• Inclusivity: Mercor values diversity and encourages all qualified applicants to apply. Reasonable accommodations are available upon request.