Turing Is Hiring An AI Quality Analyst

Join Turing as an AI Quality Analyst and Shape the Future of Personalized AI! 🚀

About Turing:

Turing is a leading research accelerator based in San Francisco, California, specializing in frontier AI labs. We partner with enterprises to deploy advanced AI systems by accelerating research through high-quality data, innovative training methodologies, and expert AI researchers. Our mission is to transform AI concepts into reliable, impactful, and beneficial proprietary intelligence for long-term success.

Role Overview

As an AI Quality Analyst, you will evaluate a new personalization feature for Gemini. Your role involves assessing how effectively the model leverages data from your conversations, emails, Google searches, and YouTube activities to deliver relevant and helpful responses. This position marries creativity with analytical thinking as you craft prompts based on personal experiences and evaluate the quality of personalized AI replies, focusing on grounding, integration, and helpfulness.

Key Qualifications 🌟

• Proficiency in Polish: Strong reading and writing skills, as Polish is the primary language for this project.
• Personal Google Account: Willingness to use an enabled data source for authentic assessments.
• Full-time Availability: Within your local time zone to support our global, 24-hour operations.
• Analytical Skills: Exceptional ability to evaluate nuanced AI responses, especially personalization quality.
• Prompt Crafting Experience: Ability to create creative multi-turn prompts reflecting personal context to challenge AI capabilities.
• Understanding of Personalization: Recognizing incorrect personalization and poor inferences.
• Attention to Detail: Skilled at side-by-side comparison of responses to detect subtle differences.
• Communication Skills: Clear and structured writing for rationales and feedback.
• Constructive Feedback: Providing detailed annotations on model performance.
• Collaboration & Independence: Strong communication skills and self-motivation for remote work.
• Technical Setup: Reliable internet and a desktop or laptop.

Responsibilities 📝

• Design and execute multi-turn conversational prompts (1-5 turns) that encourage the AI to use your personal info.
• Assess model responses based on your initial prompts to verify personalization effectiveness.
• Analyze responses for grounding accuracy, ensuring claims are supported by evidence.
• Evaluate integration quality, confirming seamless incorporation of personal data without robotic elements.
• Compare and rank two model responses side-by-side, determining which is more helpful and engaging.
• Write clear and defensible rationales explaining your comparison judgments, noting strengths and issues.
• Extract and verify debug info to ensure data and chat summaries are correctly utilized.
• Maintain data hygiene by securely deleting evaluation conversations to prevent future contamination.

Education & Experience 🎓

• Bachelor’s degree or equivalent experience in Policy, Law, Ethics, Linguistics, Journalism, Computer Science, or related fields.
• Experience in data annotation, AI quality evaluation, content moderation, or similar roles is highly preferred.

Offer Details 💼

• Minimum commitment of 4 hours per day, up to 40 hours per week, with at least 4 hours overlapping with Pacific Standard Time.
• This is a contractor position, initially for 3 months.
• Rate: $15 per hour.

Evaluation Process 🔍

• Qualified candidates will receive a Job Interest Form.
• Complete a profile review and an assessment within 24 hours.
• Successful candidates will be contacted to discuss pre-onboarding requirements.

After applying, you will get an email with a login link to access the portal and complete your profile. If you know talented individuals who would excel in this role, consider referring them and earning rewards through your network! 🎁