Skill Is Hiring An Ai Quality Analyst

Join as an AI Quality Analyst for Personalized Gemini Responses 🚀

Become a vital part of enhancing AI personalization by evaluating how well Gemini integrates your personal data to deliver relevant responses. This role combines creativity, analytical skills, and technical understanding to improve conversational AI quality.

Role Overview

As an AI Quality Analyst, you will evaluate a new personalization feature in Gemini, focusing on how effectively the AI uses your past conversations, Gmail, Google Search, and YouTube activity. Your insights will help refine the model’s ability to provide helpful, natural, and well-grounded responses based on personal data.

Responsibilities 📝

• Design & execute multi-turn prompts: Create conversational prompts (1-5 turns) that test the AI’s use of personal info and experiences.
• Response evaluation: Assess AI replies to see if personalization aligns with your original intent.
• Grounding analysis: Check if claims about you are supported by evidence, avoiding hallucinations or flawed inferences.
• Integration assessment: Ensure personal data is woven seamlessly into responses without sounding robotic or forced.
• Comparative analysis: Side-by-side comparison of model responses to identify the more helpful, natural, and enjoyable one, with clear rationales.
• Debug information verification: Extract and confirm effective utilization of chat summaries and data sources.
• Data hygiene: Delete conversations after evaluation to maintain privacy and data integrity.

Must-Have Qualifications ✅

• Korean Proficiency: High-level reading and writing skills in Korean, essential for this project.
• Analytical Mindset: Ability to evaluate nuanced AI responses and personalization accuracy.
• Creative Prompt Engineering: Experience designing multi-turn prompts based on personal context.
• Evaluation Skills: Strong understanding of personalization quality, including detecting inaccuracies or forced connections.
• Attention to Detail: Spot subtle differences in naturalness and narrative quality in responses.
• Communication Skills: Write clear, structured rationales referencing specific turns.
• Constructive Feedback: Capable of detailed analysis and annotations to improve model responses.
• Collaboration & Independence: Excellent skills to work remotely and independently, while collaborating as needed.
• Technical Setup: Reliable desktop/laptop with good internet connection.
• Educational Background: BS/BA degree or equivalent in Policy, Law, Ethics, Linguistics, Journalism, Computer Science, or related fields.
• Account Usage: Willing to use your personal Google account with personal data sources enabled for authentic testing.
• Work Schedule: Full-time (8 hours/day) with 4 hours overlap in a specific time zone; flexible hours preferred.

Nice-to-Have Qualifications 🌟

• Experience in data annotation, AI evaluation, content moderation, or related roles is highly desirable.

Offer Details 💼

Engagement: Contractor for 1 month

Hours: Minimum 4 hours per day, totaling 30-40 hours/week with a 4-hour overlap in your time zone

Start Date: Immediate

Compensation 💰

The role pays approximately $20 per hour, with variability based on education, experience, certifications, and location.

Evaluation Process 🔍

1. Applicants will be asked to complete a Job Interest Form.
2. Upon profile review, an assessment will be sent, to be completed within 24 hours.
3. Successful candidates will be contacted for further discussions based on assessment results.