Turing Is Hiring A Remote Ai Quality Analyst

Join Turing, a global leader in frontier AI research supportive of enterprise deployment, as an AI Quality Analyst evaluating personalized AI interactions to drive impactful insights and improvements.

🌟 About Turing

Based in San Francisco, California, Turing is the world’s premier research accelerator for cutting-edge AI labs and trusted partner for enterprises deploying advanced AI systems. We accelerate frontier research with high-quality data, sophisticated training pipelines, and top AI researchers specializing in coding, reasoning, STEM, multilinguality, multimodality, and agents. We also help enterprises transform AI from proof of concept to proprietary solutions that deliver reliable performance, measurable impact, and lasting results on the P&L.

🔍 Role Overview

As an AI Quality Analyst, you will evaluate personalized features for Gemini, focusing on how effectively the model utilizes your past conversations and activities across Gmail, Google Search, and YouTube to offer relevant responses. This role combines creativity with analytical rigor, requiring you to design prompts based on personal experiences and assess the quality of model responses across dimensions such as Grounding, Integration, and Helpfulness.

📝 Key Qualifications

• Japanese Proficiency: Ability to read and write in Japanese at a high proficiency level, as it is the focus language for this project.
• Personal Account Usage: Willingness to use your primary Google account and enable personal data sources for genuine assessment.
• Schedule Flexibility: Full-time availability in your local time zone to support a 24-hour global team.
• Exceptional Analytical Thinking: Ability to evaluate nuanced AI responses and assess personalization quality.
• Creative Prompt Engineering: Experience in designing multi-turn prompts based on personal context to thoroughly test AI capabilities.
• Strong Evaluation Skills: Knowledge of personalization concepts and ability to identify issues like incorrect personalization, poor inferences, or forced connections.
• Meticulous Attention to Detail: Skill in reviewing responses and spotting subtle differences in naturalness and overnarrating.
• Excellent Communication: Ability to write clear, concise rationales and provide constructive feedback with detailed annotations.
• Independence & Technical Setup: Self-motivated with a reliable internet-connected setup to work remotely.

🎯 Responsibilities

• Design and execute multi-turn conversational prompts (1-5 turns) that incorporate your personal information and experiences.
• Evaluate model responses for alignment with intent, ensuring proper personalization.
• Analyze responses for Grounding issues, verifying claims are supported and avoiding hallucinations.
• Assess the natural integration of personal data into responses without robotic overnarrating.
• Compare two model responses side-by-side (SxS) to determine which is more helpful, user-friendly, and enjoyable.
• Write clear rationales referencing specific turns and issues.
• Extract and verify "Debug Info" to confirm proper data utilization.
• Maintain data hygiene by deleting evaluation conversations post-assessment.

🎓 Education & Experience

• BS/BA degree or equivalent experience in fields such as Policy, Law, Ethics, Linguistics, Journalism, Computer Science, or related analytical disciplines.
• Experience in data annotation, AI quality assessment, content moderation, or related roles is highly preferred.

🕒 Commitment & Engagement

• Minimum of 4 hours per day and 30 hours per week, with at least 4 hours overlapping with Pacific Standard Time (PST).
• Options available for 30 or 40 hours per week.
• Contract role lasting approximately 3 months.

🚀 Next Steps

After applying, you'll receive an email with a login link to complete your profile on our portal. If you know talented individuals for this role, refer them at turing.com/referrals and earn rewards from your network.