Turing Inc Is Hiring A Remote Ai Quality Analyst

🌟 Join Our Team as an AI Quality Analyst

Evaluate and enhance personalized AI interactions by testing a new feature that leverages your own data and experiences, blending creativity with analytical rigor. This role offers an exciting opportunity to contribute to cutting-edge AI personalization efforts.

Role Overview

As an AI Quality Analyst, you will assess a new personalization feature for Gemini. Your primary responsibility is to evaluate how effectively the model utilizes information from your past conversations, Gmail, Google Search, and YouTube activity to generate relevant and helpful responses. This role requires a unique combination of creativity, analytic skills, and attention to detail.

You will:

• Design prompts from your personal experience perspective.
• Assess the quality of personalized responses based on dimensions like Grounding, Integration, and Helpfulness.

Key Qualifications

• Korean Proficiency: Ability to read and write in Korean with high fluency, as Korean is the focus language for this project. 🇰🇷
• Personal Account Usage: Willingness to use your primary Google account (not a test account) and enable personal data sources for authentic testing. 🛡️
• Schedule Flexibility: Full-time availability aligned with your local time zone to support a 24-hour global operations team. ⏰
• Exceptional Analytical Thinking: Ability to evaluate nuanced AI responses and assess personalization quality. 🤔
• Creative Prompt Engineering: Experience designing multi-turn prompts based on personal context to test the model's capabilities effectively. 🎨
• Strong Evaluation Skills: Understanding of personalization concepts and ability to detect inaccuracies or forced connections. 🔍
• Attention to Detail: Skill in reviewing model responses and identifying subtle differences in quality and naturalness. 👀
• Excellent Communication: Clear, concise writing skills for rationales and feedback. 📝
• Independence & Collaboration: Self-motivated with strong teamwork skills, capable of working remotely. 🌐
• Technical Setup: Reliable desktop/laptop with a stable internet connection. 💻

Role Responsibilities

• Design and run multi-turn conversational prompts (typically 1-5 turns) utilizing your personal information.
• Evaluate model responses for personalization relevance, grounding, and integration quality.
• Identify and document issues such as hallucinations, poor inferences, or overnarrating.
• Compare and rank model responses side-by-side, providing clear rationales referencing specific conversation turns.
• Extract and verify "Debug Info" to ensure proper data source utilization.
• Maintain data hygiene by securely deleting evaluation conversations when needed.

Education & Experience

• BS/BA degree or equivalent experience in fields like Policy, Law, Ethics, Linguistics, Journalism, Computer Science, or related analytical disciplines.
• Experience in data annotation, AI quality evaluation, content moderation, or similar roles is highly preferred. 🎓

Commitments & Engagement Details

• Time Commitment: At least 4 hours daily, minimum 30 hours weekly, with a 4-hour overlap with Pacific Standard Time (PST). 🕓
• Engagement Type: Contractor
• Duration: 3 months
• Workload Options: 30 or 40 hours/week based on availability. ⚙️