Turing Is Hiring An Ai Quality Analyst

🌟 Join Turing, a global leader in AI research acceleration and enterprise AI transformation, where innovation meets impact.

About Turing

Based in San Francisco, California, Turing stands as the world's leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. We support our customers in two key ways:

• Accelerating frontier research with high-quality data, advanced training pipelines, and top AI researchers skilled in coding, reasoning, STEM, multilinguality, multimodality, and agents.
• Applying that expertise to help enterprises transform AI from proof of concept into proprietary intelligence that performs reliably, delivers measurable impacts, and drives lasting results on the P&L.

Explore the overview of this opportunity to understand the necessary skills—including soft skills and software proficiencies—that are relevant to this role.

Role Overview

🧠 As an AI Quality Analyst, you will evaluate a new personalization feature for Gemini. Your task will be to assess how effectively the model uses information from past Gemini conversations, Gmail, Google Search, and YouTube activities to produce more relevant and helpful responses.

This role combines creativity and analytical rigor. You will design prompts inspired by your personal experiences and then evaluate the model’s personalized responses, focusing on dimensions like Grounding, Integration, and Helpfulness.

Key Qualifications

• Polish Proficiency: Ability to read and write fluently in Polish, as it is the project’s focus language.
• Personal Account Usage: Willing to use your primary personal Google account with enabled data sources for genuine testing.
• Schedule Flexibility: Full-time availability aligned with your local time zone to support our 24-hour global team.
• Exceptional Analytical Thinking: Ability to evaluate nuanced AI responses, specifically assessing personalization quality.
• Creative Prompt Engineering: Experience designing multi-turn prompts based on personal context to thoroughly test model capabilities.
• Strong Evaluation Skills: Understanding personalization concepts, with the ability to identify inaccuracies or forced connections.
• Attention to Detail: Ability to spot subtle differences in naturalness and overnarration in Side-by-Side (SxS) responses.
• Excellent Communication: Clear, structured writing to justify model rankings with specific references.
• Constructive Feedback: Ability to provide detailed annotations and suggestions.
• Collaboration Skills: Strong communication and teamwork abilities.
• Independence: Self-motivated with the ability to work remotely without supervision.
• Technical Setup: A reliable desktop or laptop with a good internet connection.

Responsibilities

• Design and execute multi-turn conversational prompts (1-5 turns) incorporating your personal information and experiences.
• Evaluate model responses for appropriate personalization based on your prompts.
• Identify and analyze grounding issues, ensuring claims about you are supported by evidence and free from hallucinations.
• Assess how well personal data is integrated naturally into responses without robotic overnarrating.
• Compare and rank responses side-by-side, determining which is more helpful, natural, and user-friendly.
• Write clear rationales for your rankings, explicitly stating specific conversational turns.
• Extract and verify "Debug Info" to confirm proper utilization of chat summaries and data sources.
• Maintain strict data hygiene by deleting evaluation conversations to avoid polluting future chat histories.

Education & Experience

• BS/BA degree or equivalent experience in related fields such as Policy, Law, Ethics, Linguistics, Journalism, Computer Science, or related analytical disciplines.
• Experience in data annotation, AI quality evaluation, content moderation, or similar roles is highly preferred.

Offer Details

• Work Commitment: Minimum 4 hours per day up to 40 hours weekly, with at least 4 hours overlapping with PST hours.
• Engagement Type: Contract
• Project Duration: 3 months
• Pay Rate: $15 per hour

Evaluation Process

• Shortlisted candidates will receive a Job Interest Form.
• After profile review, an assessment will be shared, to be completed within 24 hours.
• Successful candidates will be contacted to discuss onboarding requirements.

After applying, you'll receive an email with a login link to complete your profile. Know talented professionals? Refer them here and earn rewards from your network. Remote work options are available for this role. 🚀