Turing Is Hiring A Remote Ai Quality Analyst

Join Turing, a leading research accelerator in frontier AI based in San Francisco, and be part of shaping the future of advanced AI systems through a dynamic and innovative role as an AI Quality Analyst.

About Turing 🚀

Based in San Francisco, California, Turing stands as the world’s premier research accelerator for frontier AI labs and a trusted partner for global enterprises deploying cutting-edge AI solutions. We support our clients by accelerating frontier research with high-quality data, advanced training pipelines, and top-tier AI researchers specializing in coding, reasoning, STEM, multilinguality, multimodality, and agent systems. Additionally, we help enterprises transform AI from proof of concept into reliable, impactful proprietary systems that drive significant results on the P&L.

Role Overview 🎯

As an AI Quality Analyst, you will evaluate a new personalization feature for Gemini. Your primary focus will be assessing how effectively the model uses your past conversations and activity across Gmail, Google Search, and YouTube to enhance response relevance and helpfulness. This role uniquely combines creativity and analytical expertise, requiring you to:

• Design prompts from your personal experiences to test the model
• Assess the quality of personalized responses across dimensions like Grounding, Integration, and Helpfulness

Key Qualifications 📝

• Polish Proficiency: fluency in reading and writing Polish, as it is the project's focus language
• Personal Account Usage: willingness to use your primary personal Google account with enabled data sources for authentic testing
• Schedule Flexibility: full-time availability in your local time zone to support a 24-hour global team
• Exceptional Analytical Skills: ability to evaluate nuanced AI responses, especially personalization quality
• Creative Prompt Engineering: experience in designing multi-turn prompts based on personal context
• Strong Evaluation Skills: understanding of personalization concepts, including identifying errors and forced connections
• Meticulous Attention to Detail: ability to spot subtle differences in model responses, including naturalness and verbosity
• Excellent Communication: capability to provide clear, structured rationales with specific references
• Constructive Feedback: proficiency in annotating and suggesting improvements
• Independence & Tech Setup: self-motivated with a reliable internet-connected device

Role Responsibilities 🎉

• Design and execute multi-turn conversational prompts (1–5 turns) leveraging your personal data
• Evaluate whether personalization is effectively integrated and appropriate
• Review responses for Grounding issues, ensuring claims are supported by evidence
• Assess Integration quality to confirm natural data incorporation without robotic overnarration
• Stack-rank model responses side-by-side, judging helpfulness, usability, and engagement
• Write detailed rationales for comparisons, highlighting specific conversation turns
• Verify model utilize "Debug Info" to ensure proper data use
• Maintain data hygiene by deleting evaluation conversations post-assessment

Education & Experience 🎓

• BS/BA degree or equivalent experience in fields like Policy, Law, Ethics, Linguistics, Journalism, Computer Science, or related analytical areas
• Previous experience in data annotation, AI quality evaluation, content moderation, or similar roles is highly preferred

Offer Details 💼

• Work Commitment: At least 4 hours daily, up to 40 hours weekly with a 4-hour overlap with PST
• Type: Contractor
• Duration: 3 months
• Hourly Rate: $15/hour

Evaluation Process 🔍

• Shortlisted candidates will receive a Job Interest Form
• Completion of an assessment within 24 hours post-profile review
• Qualified candidates will be contacted to discuss onboarding requirements

Once you apply, you'll receive an email with a login link to complete your profile. Know talented individuals? Refer them here and earn rewards from your network! 🎉