Job Description
Please submit your CV in English and indicate your level of English proficiency.
Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment.
What This Opportunity Involves
While each project involves unique tasks, contributors may:
Create structured test cases that simulate complex human workflows
Define gold-standard behavior and scoring logic to evaluate agent actions
Analyze agent logs, failure modes, and decision paths
Work with code repositories and test frameworks to validate your scenarios
Iterate on prompts, instructions, and test cases to improve clarity and difficulty
Ensure that scenarios are production-ready, easy to run, and reusable
What We Look For
This opportunity is a good fit for software engineers, open to part-time, non-permanent projects. Ideally, contributors will have:
3+ of software development experience with strong Python focus
Experience with Git and code repositories
Comfortable with structured formats like JSON/YAML for scenario description
Understanding core LLM limitations (hallucinations, bias, context limits) and how these affect evaluation design
Familiarity with Docker
English proficiency - B2
How It Works
Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid
Project time expectations
Tasks for this project are estimated to take 6-10 hours to complete, depending on complexity. This is an estimate and not a schedule requirement; you choose when and how to work. Tasks must be submitted by the deadline and meet the listed acceptance criteria to be accepted.
Payment
Paid contributions, with rates up to $80/hour*
Fixed project rate or individual rates, depending on the project
Some projects include incentive payments
Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project
Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment.
What This Opportunity Involves
While each project involves unique tasks, contributors may:
Create structured test cases that simulate complex human workflows
Define gold-standard behavior and scoring logic to evaluate agent actions
Analyze agent logs, failure modes, and decision paths
Work with code repositories and test frameworks to validate your scenarios
Iterate on prompts, instructions, and test cases to improve clarity and difficulty
Ensure that scenarios are production-ready, easy to run, and reusable
What We Look For
This opportunity is a good fit for software engineers, open to part-time, non-permanent projects. Ideally, contributors will have:
3+ of software development experience with strong Python focus
Experience with Git and code repositories
Comfortable with structured formats like JSON/YAML for scenario description
Understanding core LLM limitations (hallucinations, bias, context limits) and how these affect evaluation design
Familiarity with Docker
English proficiency - B2
How It Works
Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid
Project time expectations
Tasks for this project are estimated to take 6-10 hours to complete, depending on complexity. This is an estimate and not a schedule requirement; you choose when and how to work. Tasks must be submitted by the deadline and meet the listed acceptance criteria to be accepted.
Payment
Paid contributions, with rates up to $80/hour*
Fixed project rate or individual rates, depending on the project
Some projects include incentive payments
Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project
💡 Quick Summary
Seeking a career-building opportunity? The Freelance Agent Evaluation Engineer position is now open for candidates interested in the Software Developer Jobs sector. This role in New York City offers a professional environment and growth potential.
Requirement Snapshot: Candidates should possess basic communication skills, a proactive attitude, and the ability to work in a team. Experience in Software Developer Jobs is a plus.
