Job Description
About the Projects:
We are building evaluation and training datasets for Large Language Models (LLMs) to work on realistic software engineering problems. Our approach involves creating verifiable Software Development tasks based on public repository histories in a synthetic manner, with human involvement. We aim to expand dataset coverage across various task types, including programming languages, difficulty levels, and more.
About the Role:
We seek experienced software engineers at the tech lead level who are familiar with high-quality public GitHub repositories. The ideal candidate will have experience working with well-maintained, widely-used repositories with over 500 stars. This role involves hands-on software engineering, including development environment automation, issue triaging, and evaluating test coverage and quality.
Why Work with Us?
You'll be at the forefront of evaluating how LLMs interact with real code, influencing the future of AI-assisted software development. This is a unique opportunity to blend practical software engineering with AI research.
Day-to-Day Tasks:
• Analyze and triage GitHub issues across trending open-source libraries.
• Set up and configure code repositories, including Dockerization and environment setup.
• Evaluate unit test coverage and quality.
• Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.
• Collaborate with researchers to design and identify repositories and issues challenging for LLMs.
• Opportunities to lead junior engineers in collaborative projects.
Required Skills:
• Strong experience with at least one of the following languages: Python, JavaScript, Java, Go, Rust, C/C++, C#, or Ruby.
• Experience working with well-maintained, widely-used repositories with over 500 stars.
• Proficiency with Git, Docker, and basic software pipeline setup.
• Ability to understand and navigate complex codebases.
• Comfortable running, modifying, and testing real-world projects locally.
• Experience contributing to or evaluating open-source projects is a plus.
Nice to Have:
• Previous participation in LLM research or evaluation projects.
• Experience building or testing developer tools or automation agents.
Work Arrangements:
• 20 hours per week commitment with some overlap with PST.
• Contractor assignment with no medical/paid leave.
• 1-month contract duration with an expected start date as next week
We are building evaluation and training datasets for Large Language Models (LLMs) to work on realistic software engineering problems. Our approach involves creating verifiable Software Development tasks based on public repository histories in a synthetic manner, with human involvement. We aim to expand dataset coverage across various task types, including programming languages, difficulty levels, and more.
About the Role:
We seek experienced software engineers at the tech lead level who are familiar with high-quality public GitHub repositories. The ideal candidate will have experience working with well-maintained, widely-used repositories with over 500 stars. This role involves hands-on software engineering, including development environment automation, issue triaging, and evaluating test coverage and quality.
Why Work with Us?
You'll be at the forefront of evaluating how LLMs interact with real code, influencing the future of AI-assisted software development. This is a unique opportunity to blend practical software engineering with AI research.
Day-to-Day Tasks:
• Analyze and triage GitHub issues across trending open-source libraries.
• Set up and configure code repositories, including Dockerization and environment setup.
• Evaluate unit test coverage and quality.
• Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.
• Collaborate with researchers to design and identify repositories and issues challenging for LLMs.
• Opportunities to lead junior engineers in collaborative projects.
Required Skills:
• Strong experience with at least one of the following languages: Python, JavaScript, Java, Go, Rust, C/C++, C#, or Ruby.
• Experience working with well-maintained, widely-used repositories with over 500 stars.
• Proficiency with Git, Docker, and basic software pipeline setup.
• Ability to understand and navigate complex codebases.
• Comfortable running, modifying, and testing real-world projects locally.
• Experience contributing to or evaluating open-source projects is a plus.
Nice to Have:
• Previous participation in LLM research or evaluation projects.
• Experience building or testing developer tools or automation agents.
Work Arrangements:
• 20 hours per week commitment with some overlap with PST.
• Contractor assignment with no medical/paid leave.
• 1-month contract duration with an expected start date as next week
💡 Quick Summary
Seeking a career-building opportunity? The Senior Software Engineer for Real-World Code Analysis position is now open for candidates interested in the Software Developer Jobs sector. This role in Pune offers a professional environment and growth potential.
Requirement Snapshot: Candidates should possess basic communication skills, a proactive attitude, and the ability to work in a team. Experience in Software Developer Jobs is a plus.
