Job Description
About Us
We believe AI will fundamentally transform how people live and work. CentML's mission is to massively reduce the cost of developing and deploying ML models so we can enable anyone to harness the power of AI and everyone to benefit from its potential.
Our founding team is made up of experts in AI, compilers, and ML hardware and has led efforts at companies like Amazon, Google, Microsoft Research, Nvidia, Intel, Qualcomm, and IBM. Our co-founder and CEO, Gennady Pekhimenko, is a world-renowned expert in ML systems who holds multiple academic and industry research awards from Google, Amazon, Facebook, and VMware.
About the Position
We are seeking highly crafted and motivated software engineers to join our team to empower AI practitioners to develop AI models on CentML Platform, productively and affordably. If you have launched multi-node distributed training jobs before and experienced firsthand how painful and cumbersome to get it functional, let alone high-performing, and you wanna be part of the team that derives solutions to address this challenge so that other AI practitioners wouldn’t feel the same pain that you had, please come and join us!
What you’ll do
Design and implement highly efficient distributed training systems for large-scale deep learning models.
Optimize parallelism strategies to improve performance and scalability across hundreds or thousands of GPUs.
Develop low-level systems components and algorithms to maximize throughput and minimize memory and compute bottlenecks.
Productionize the training systems onto CentML Platform.
Collaborate with researchers and engineers to productionize cutting-edge model architectures and training techniques.
Contribute to the design of APIs, abstractions and UX that make it easier to scale models while maintaining usability and flexibility.
Profile, debug, and tune performance at the system, model, and hardware levels.
Participate in design discussions, code reviews, and technical planning to ensure the product aligns with business goals.
Stay up to date with the latest advancements in large-scale model training and help translate research into practical, robust systems.
What you’ll need to be successful
Bachelor’s, Master’s, or PhD’s degree in Computer Science/Engineering, Software Engineering, related field or equivalent working experience.
3+ years of experience in software development, preferably with Python and C++.
Deep understanding of machine learning pipelines and workflows, distributed systems, parallel computing, and high-performance computing principles.
Hands-on experience with large-scale training of deep learning models using frameworks like PyTorch, Megatron Core, DeepSpeed.
Experience optimizing compute, memory, and communication performance in large model training workflows.
Familiarity with GPU programming, CUDA, NCCL, and performance profiling tools.
Solid grasp of deep learning fundamentals, especially as they relate to transformer-based architectures and training dynamics.
Experience working with cloud platforms (AWS, GCP, or Azure) and containerization tools (Docker, Kubernetes).
Ability to work closely with both research and engineering teams, translating evolving needs into robust infrastructure.
Excellent problem-solving skills, with the ability to debug complex systems.
A passion for building high-impact tools that push the boundaries of what’s possible with large-scale AI.
Bonus points if you have
Experience building tools or platforms for ML model training or fine-tuning.
Experience building backends (e.g., using FastAPI) and frontend (e.g., using React).
Experience building and optimizing LLM inference engines (e.g., vLLM, SGLang).
Exposure to DevOps practices, CI/CD pipelines, and infrastructure as code.
Familiarity with MLOps concepts, including model versioning and serving.
Benefits & Perks
An open and inclusive work environment
Employee stock options
Best-in-class medical and dental benefits
Parental Leave top-up
Professional development budget
Flexible vacation time to promote a healthy work-life blend
We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, gender, ****** orientation, age, marital status, veteran status, disability, and any other protected ground of discrimination under applicable human rights legislation.
CentML strives to respect the dignity and independence of people with disabilities and is committed to giving them the same opportunity to succeed as all other employees.
Inclusiveness is core to our culture at CentML, and we strive to ensure you get the most from your interview experience. CentML makes reasonable accommodations for applicants with disabilities. If a reasonable accommodation is needed to participate in the job application or interview process, please reach out to the Talent team.
We believe AI will fundamentally transform how people live and work. CentML's mission is to massively reduce the cost of developing and deploying ML models so we can enable anyone to harness the power of AI and everyone to benefit from its potential.
Our founding team is made up of experts in AI, compilers, and ML hardware and has led efforts at companies like Amazon, Google, Microsoft Research, Nvidia, Intel, Qualcomm, and IBM. Our co-founder and CEO, Gennady Pekhimenko, is a world-renowned expert in ML systems who holds multiple academic and industry research awards from Google, Amazon, Facebook, and VMware.
About the Position
We are seeking highly crafted and motivated software engineers to join our team to empower AI practitioners to develop AI models on CentML Platform, productively and affordably. If you have launched multi-node distributed training jobs before and experienced firsthand how painful and cumbersome to get it functional, let alone high-performing, and you wanna be part of the team that derives solutions to address this challenge so that other AI practitioners wouldn’t feel the same pain that you had, please come and join us!
What you’ll do
Design and implement highly efficient distributed training systems for large-scale deep learning models.
Optimize parallelism strategies to improve performance and scalability across hundreds or thousands of GPUs.
Develop low-level systems components and algorithms to maximize throughput and minimize memory and compute bottlenecks.
Productionize the training systems onto CentML Platform.
Collaborate with researchers and engineers to productionize cutting-edge model architectures and training techniques.
Contribute to the design of APIs, abstractions and UX that make it easier to scale models while maintaining usability and flexibility.
Profile, debug, and tune performance at the system, model, and hardware levels.
Participate in design discussions, code reviews, and technical planning to ensure the product aligns with business goals.
Stay up to date with the latest advancements in large-scale model training and help translate research into practical, robust systems.
What you’ll need to be successful
Bachelor’s, Master’s, or PhD’s degree in Computer Science/Engineering, Software Engineering, related field or equivalent working experience.
3+ years of experience in software development, preferably with Python and C++.
Deep understanding of machine learning pipelines and workflows, distributed systems, parallel computing, and high-performance computing principles.
Hands-on experience with large-scale training of deep learning models using frameworks like PyTorch, Megatron Core, DeepSpeed.
Experience optimizing compute, memory, and communication performance in large model training workflows.
Familiarity with GPU programming, CUDA, NCCL, and performance profiling tools.
Solid grasp of deep learning fundamentals, especially as they relate to transformer-based architectures and training dynamics.
Experience working with cloud platforms (AWS, GCP, or Azure) and containerization tools (Docker, Kubernetes).
Ability to work closely with both research and engineering teams, translating evolving needs into robust infrastructure.
Excellent problem-solving skills, with the ability to debug complex systems.
A passion for building high-impact tools that push the boundaries of what’s possible with large-scale AI.
Bonus points if you have
Experience building tools or platforms for ML model training or fine-tuning.
Experience building backends (e.g., using FastAPI) and frontend (e.g., using React).
Experience building and optimizing LLM inference engines (e.g., vLLM, SGLang).
Exposure to DevOps practices, CI/CD pipelines, and infrastructure as code.
Familiarity with MLOps concepts, including model versioning and serving.
Benefits & Perks
An open and inclusive work environment
Employee stock options
Best-in-class medical and dental benefits
Parental Leave top-up
Professional development budget
Flexible vacation time to promote a healthy work-life blend
We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, gender, ****** orientation, age, marital status, veteran status, disability, and any other protected ground of discrimination under applicable human rights legislation.
CentML strives to respect the dignity and independence of people with disabilities and is committed to giving them the same opportunity to succeed as all other employees.
Inclusiveness is core to our culture at CentML, and we strive to ensure you get the most from your interview experience. CentML makes reasonable accommodations for applicants with disabilities. If a reasonable accommodation is needed to participate in the job application or interview process, please reach out to the Talent team.
💡 Quick Summary
Seeking a career-building opportunity? The Software Engineer - LLM Training position is now open for candidates interested in the IT Engineer & Developer Jobs sector. This role in Toronto offers a professional environment and growth potential.
Requirement Snapshot: Candidates should possess basic communication skills, a proactive attitude, and the ability to work in a team. Experience in IT Engineer & Developer Jobs is a plus.
