Job Description
About Turing:
Based in San Francisco, California, Turing is a leading research accelerator for frontier AI labs and a trusted partner for enterprises deploying advanced AI systems. We help customers accelerate research with high-quality data and expert AI researchers while also assisting in transforming AI from concepts into proprietary, reliable, and impactful intelligence systems.
Role Overview:
As an AI Quality Analyst, you will evaluate an innovative personalization feature for Gemini, assessing how well it utilizes data from your past activities in Gemini, Gmail, Google Search, and YouTube to provide relevant responses. This role combines creativity with analytical skills to design prompts based on personal experiences and evaluate the quality of the model's responses based on various aspects such as Grounding, Integration, and Helpfulness.
Key Qualifications:
• Korean Proficiency: Ability to read and write fluently in Korean, as it is the focus language for this project.
• Personal Account Usage: Ready to use your primary personal Google account for authentic assessments.
• Schedule Flexibility: Full-time availability in your local time zone is essential; we operate a 24-hour global team.
• Exceptional Analytical Thinking: Ability to evaluate nuanced AI responses, especially related to personalization quality.
• Creative Prompt Engineering: Experience in designing engaging, multi-turn prompts based on personal context.
• Strong Evaluation Acumen: Understanding of personalization concepts to identify incorrect personalization or poor inferences.
• Meticulous Attention to Detail: Skill in reviewing model responses and noticing subtle differences in quality.
• Excellent Written Communication: Ability to write concise and clear rationales for model rankings with ******** references.
• Feedback: Capacity to offer constructive feedback and detailed annotations.
• Communication: Strong communication and collaboration skills.
• Independence: Self-motivated with the capability to work independently in a remote setting.
• Technical Setup: Desktop or laptop with a stable internet connection.
Description:
• You will be part of a dynamic team focused on evaluating the quality of personalized AI interactions.
Your day-to-day work will involve:
• Designing and executing multi-turn conversational prompts that engage the AI to utilize your personal data.
• Evaluating model responses based on your intent, ensuring appropriate application of personalization.
• Analyzing responses for Grounding issues, confirming claims with evidence and avoiding flawed inferences.
• Assessing Integration quality to ensure personal data is naturally woven into the responses.
• Hand-on evaluation and ranking of two model responses side-by-side for helpfulness and user experience.
• Writing clear rationales for comparisons, identifying issues or strengths in the conversation.
• Extracting and verifying debug information from the model to ensure proper utilization of data sources.
• Maintaining data hygiene by deleting evaluation conversations to prevent contamination of your future chat history.
Education & Experience:
• Bachelor's degree or equivalent experience in a relevant field (e.g., Policy, Law, Ethics, Linguistics, Journalism, Computer Science, or a related analytical field).
• Experience in data annotation, AI quality evaluation, content moderation, or a similar role preferred.
Offer Details:
• Commitment: Minimum of 30 hours per week, with at least 4 hours overlap with PST (30 hrs/week or 40 hrs/week options available).
• Engagement type: Contractor
• Engagement Length: 3 months
After applying, an email will be sent with a login link to complete your profile.
Based in San Francisco, California, Turing is a leading research accelerator for frontier AI labs and a trusted partner for enterprises deploying advanced AI systems. We help customers accelerate research with high-quality data and expert AI researchers while also assisting in transforming AI from concepts into proprietary, reliable, and impactful intelligence systems.
Role Overview:
As an AI Quality Analyst, you will evaluate an innovative personalization feature for Gemini, assessing how well it utilizes data from your past activities in Gemini, Gmail, Google Search, and YouTube to provide relevant responses. This role combines creativity with analytical skills to design prompts based on personal experiences and evaluate the quality of the model's responses based on various aspects such as Grounding, Integration, and Helpfulness.
Key Qualifications:
• Korean Proficiency: Ability to read and write fluently in Korean, as it is the focus language for this project.
• Personal Account Usage: Ready to use your primary personal Google account for authentic assessments.
• Schedule Flexibility: Full-time availability in your local time zone is essential; we operate a 24-hour global team.
• Exceptional Analytical Thinking: Ability to evaluate nuanced AI responses, especially related to personalization quality.
• Creative Prompt Engineering: Experience in designing engaging, multi-turn prompts based on personal context.
• Strong Evaluation Acumen: Understanding of personalization concepts to identify incorrect personalization or poor inferences.
• Meticulous Attention to Detail: Skill in reviewing model responses and noticing subtle differences in quality.
• Excellent Written Communication: Ability to write concise and clear rationales for model rankings with ******** references.
• Feedback: Capacity to offer constructive feedback and detailed annotations.
• Communication: Strong communication and collaboration skills.
• Independence: Self-motivated with the capability to work independently in a remote setting.
• Technical Setup: Desktop or laptop with a stable internet connection.
Description:
• You will be part of a dynamic team focused on evaluating the quality of personalized AI interactions.
Your day-to-day work will involve:
• Designing and executing multi-turn conversational prompts that engage the AI to utilize your personal data.
• Evaluating model responses based on your intent, ensuring appropriate application of personalization.
• Analyzing responses for Grounding issues, confirming claims with evidence and avoiding flawed inferences.
• Assessing Integration quality to ensure personal data is naturally woven into the responses.
• Hand-on evaluation and ranking of two model responses side-by-side for helpfulness and user experience.
• Writing clear rationales for comparisons, identifying issues or strengths in the conversation.
• Extracting and verifying debug information from the model to ensure proper utilization of data sources.
• Maintaining data hygiene by deleting evaluation conversations to prevent contamination of your future chat history.
Education & Experience:
• Bachelor's degree or equivalent experience in a relevant field (e.g., Policy, Law, Ethics, Linguistics, Journalism, Computer Science, or a related analytical field).
• Experience in data annotation, AI quality evaluation, content moderation, or a similar role preferred.
Offer Details:
• Commitment: Minimum of 30 hours per week, with at least 4 hours overlap with PST (30 hrs/week or 40 hrs/week options available).
• Engagement type: Contractor
• Engagement Length: 3 months
After applying, an email will be sent with a login link to complete your profile.
💡 Quick Summary
Seeking a career-building opportunity? The Remote AI Quality Analyst (Korean Language Specialist) position is now open for candidates interested in the Work from home Jobs sector. This role in Miami offers a professional environment and growth potential.
Requirement Snapshot: Candidates should possess basic communication skills, a proactive attitude, and the ability to work in a team. Experience in Work from home Jobs is a plus.