Job Description
SUMMARY
As a consultant, conduct validation and qualitative evaluation tasks in the medical/public health domain for an AI solution. Responsibilities include verifying domain specific question-answer pairs for accuracy, completeness, and relevance, updating information from specified documents, and programmatically testing the solution.
Location - Remote/Delhi
ABOUT US - https://www.wadhwaniai.org/
Wadhwani AI is a nonprofit institute building and deploying applied AI solutions to solve critical issues in public health, agriculture, education, and urban development in underserved communities in the global south. We collaborate with governments, social sector organizations, academic and research institutions, and domain experts to identify real-world problems, and develop practical AI solutions to tackle these issues with the aim of making a substantial positive impact.
We have over 30+ AI projects supported by leading philanthropies such as Bill & Melinda Gates Foundation, USAID and Google.org. With a team of over 200 professionals, our expertise encompasses AI/ML research and innovation, software engineering, domain knowledge, design and user research.
In the Press:
• Our Founder Donors are among the Top 100 AI Influencers
• G20 India’s Presidency: AI Healthcare, Agriculture, & Education Solutions Showcased Globally.
• Unlocking the potentials of AI in Public Health
• Wadhwani AI Takes an Impact-First Approach to Applying Artificial Intelligence - data.org
• Winner of the H&M Foundation Global Change Award 2026
• Sole Indian Winners of the 201+ Google AI Impact Challenge, and the first in the Asia Pacific to host Google Fellows
ROLES AND RESPONSIBILITIES
1. Validation of QA Pairs for the AI Assistant
• Verify accuracy of information in QA pairs against specified pages and documents.
• Validate and update identified pages, indicate if the answers are not updated from mentioned pages.
• Validate QA pairs for correctness, completeness, and relevance based on document pages and highlighted text.
• Modify existing QA pairs or create new ones as necessary to ensure all document text is covered accurately.
• Highlight text on document as per modifications made.
2. Qualitative Evaluation of AI Assistant
• Creation of dataset for evaluation, ensuring variation in input text
• Programmatic testing of the AI Assistant
• Provide feedback for model responses
• Refer to the training data and support feedback with reasoning as to how the AI Assistant should generate expected answers.
3. Programmatic support in creating evaluation datasets
• Creation of question-answer pairs that require both field level and domain level knowledge
REQUIREMENTS
• Should have or be in the final year of a degree in public health, medical science, or a related field.
• Strong attention to detail to ensure accuracy and completeness of QA pairs and document updates.
• Excellent written and verbal communication skills to document updates and provide clear feedback.
• Clear and concise visual and written documentation skills to effectively communicate system enhancements to the development team.
• Basic knowledge and keen interest to learn more about AI deployments for social impact.
• Experience training or working with front-line healthcare workers is beneficial.
We are committed to promoting diversity and the principle of equal employment opportunity for all our employees and encourage qualified candidates to apply irrespective of religion or belief, ethnic or social background, gender, gender identity, and disability. If you have any questions, please email us at careers@wadhwaniai.org
As a consultant, conduct validation and qualitative evaluation tasks in the medical/public health domain for an AI solution. Responsibilities include verifying domain specific question-answer pairs for accuracy, completeness, and relevance, updating information from specified documents, and programmatically testing the solution.
Location - Remote/Delhi
ABOUT US - https://www.wadhwaniai.org/
Wadhwani AI is a nonprofit institute building and deploying applied AI solutions to solve critical issues in public health, agriculture, education, and urban development in underserved communities in the global south. We collaborate with governments, social sector organizations, academic and research institutions, and domain experts to identify real-world problems, and develop practical AI solutions to tackle these issues with the aim of making a substantial positive impact.
We have over 30+ AI projects supported by leading philanthropies such as Bill & Melinda Gates Foundation, USAID and Google.org. With a team of over 200 professionals, our expertise encompasses AI/ML research and innovation, software engineering, domain knowledge, design and user research.
In the Press:
• Our Founder Donors are among the Top 100 AI Influencers
• G20 India’s Presidency: AI Healthcare, Agriculture, & Education Solutions Showcased Globally.
• Unlocking the potentials of AI in Public Health
• Wadhwani AI Takes an Impact-First Approach to Applying Artificial Intelligence - data.org
• Winner of the H&M Foundation Global Change Award 2026
• Sole Indian Winners of the 201+ Google AI Impact Challenge, and the first in the Asia Pacific to host Google Fellows
ROLES AND RESPONSIBILITIES
1. Validation of QA Pairs for the AI Assistant
• Verify accuracy of information in QA pairs against specified pages and documents.
• Validate and update identified pages, indicate if the answers are not updated from mentioned pages.
• Validate QA pairs for correctness, completeness, and relevance based on document pages and highlighted text.
• Modify existing QA pairs or create new ones as necessary to ensure all document text is covered accurately.
• Highlight text on document as per modifications made.
2. Qualitative Evaluation of AI Assistant
• Creation of dataset for evaluation, ensuring variation in input text
• Programmatic testing of the AI Assistant
• Provide feedback for model responses
• Refer to the training data and support feedback with reasoning as to how the AI Assistant should generate expected answers.
3. Programmatic support in creating evaluation datasets
• Creation of question-answer pairs that require both field level and domain level knowledge
REQUIREMENTS
• Should have or be in the final year of a degree in public health, medical science, or a related field.
• Strong attention to detail to ensure accuracy and completeness of QA pairs and document updates.
• Excellent written and verbal communication skills to document updates and provide clear feedback.
• Clear and concise visual and written documentation skills to effectively communicate system enhancements to the development team.
• Basic knowledge and keen interest to learn more about AI deployments for social impact.
• Experience training or working with front-line healthcare workers is beneficial.
We are committed to promoting diversity and the principle of equal employment opportunity for all our employees and encourage qualified candidates to apply irrespective of religion or belief, ethnic or social background, gender, gender identity, and disability. If you have any questions, please email us at careers@wadhwaniai.org
💡 Quick Summary
Seeking a career-building opportunity? The Medical|Public Health| Consultant position is now open for candidates interested in the Health Jobs sector. This role in Central Delhi offers a professional environment and growth potential.
Requirement Snapshot: Candidates should possess basic communication skills, a proactive attitude, and the ability to work in a team. Experience in Health Jobs is a plus.
