Turing Is Hiring A Remote Ai Quality Analyst

Join Turing: Pioneering AI Research & Innovation 🌐✨

Explore a career with Turing, the world's leading research accelerator for frontier AI labs, where innovation meets global enterprise solutions.

About Turing

Based in San Francisco, California, Turing is dedicated to advancing cutting-edge AI research and deploying impactful systems that transform industries worldwide.

We support our clients by:

• Accelerating frontier research with high-quality data, advanced training pipelines, and expert AI researchers specializing in coding, reasoning, STEM, multilinguality, multimodality, and agents.
• Helping enterprises turn AI proof of concepts into reliable, proprietary intelligence systems that deliver measurable impact and lasting results.

Role Overview

As an AI Quality Analyst, you will evaluate a new personalization feature for Gemini. Your focus will be on assessing how effectively the model leverages past conversations and activity data (Gmail, Google Search, YouTube) to deliver relevant and helpful responses.

This role calls for a blend of creativity and analytical rigor. You will design prompts based on your personal experiences and analyze responses based on dimensions like Grounding, Integration, and Helpfulness.

Key Qualifications

• Polish Proficiency: Read and write in Polish at a high level, as Polish is the focus language.
• Personal Account Usage: Willing to use your primary Google account and enable personal data sources for genuine evaluation.
• Schedule Flexibility: Full-time availability in your local time zone to support our 24-hour global team.
• Exceptional Analytical Thinking: Ability to evaluate nuanced AI responses and assess personalization quality.
• Creative Prompt Engineering: Designing multi-turn prompts based on personal context.
• Strong Evaluation Skills: Understanding personalization concepts, identifying issues like incorrect inferences or forced connections.
• Meticulous Attention to Detail: Reviewing model responses and spotting subtle nuances in naturalness and overnarration.
• Excellent Written Communication: Crafting clear, concise rationales with specific references to conversation turns.
• Constructive Feedback: Providing detailed annotations and insights.
• Effective Collaboration: Communicating well with team members.
• Independence & Initiative: Self-motivated, capable of working remotely without supervision.
• Technical Setup: Having a reliable desktop/laptop and good internet connection.

Responsibilities

In this role, your daily activities will include:

• Designing and executing multi-turn conversational prompts (1–5 turns) that incorporate your personal data.
• Evaluating model responses to ensure personalization was appropriately applied based on your prompts.
• Checking for Grounding issues—making sure claims about you are supported by evidence.
• Assessing Integration to ensure seamless incorporation of personal data into responses without overnarration.
• Conducting side-by-side (SxS) evaluations of responses to rank helpfulness, usability, and enjoyment.
• Writing transparent rationales for your comparisons, referencing specific conversation turns.
• Extracting and verifying Debug Info to confirm proper utilization of data sources and chat summaries.
• Maintaining data hygiene by deleting evaluation conversations to prevent data contamination.

Education & Experience

• BS/BA degree or equivalent experience in fields like Policy, Law, Ethics, Linguistics, Journalism, Computer Science, or related analytical disciplines.
• Experience in data annotation, AI quality evaluation, content moderation, or similar roles is highly preferred.

Offer Details

• Time Commitment: At least 4 hours per day, up to 40 hours per week, with 4 hours overlapping with PST.
• Engagement Type: Contractor.