Senior Software Engineer

Microsoft’s bold vision of Azure Machine Learning (ML) is to democratize ML and make it available to every enterprise, developer and data scientist. Do you want to join a team entrusted with serving all internal and external ML workloads? We are already serving billions of requests per day on the most cutting-edge scenarios and models across the company. You will be joining the Inference team that is working on the next generation of model serving that includes hosting OAI models like ChatGPT hosting models at Bing and Office scale and many exciting problems on the intersection of AI and Cloud. We are looking for a strong Software Engineer who is passionate about designing and building highly reliable, available platform to support model inferencing at scale. You will also be expected to work on high throughput/low latency scenarios and drive performance optimization capabilities

Responsibilities
• Engage directly with key partners to understand and implement complex inferencing capabilities for state-of-the-art LLMs and Diffusion models
• Work with cutting edge hardware stacks and a fast-moving software stack to deliver best of class inference and optimal cost
• Anticipate, identify, assess, track, and mitigate project risks and issues in a fast-paced start up like environment
• Motivated to build constructive and effective relationships and solve problems collaboratively
• Support production inference for core AI scenarios on one of the largest GPU fleets in the world

Qualifications
• 7+ years of software development experience 
• Strong C/C++ development
• Practical experience working on real-world applications that use CUDA, experience in optimizing CUDA kernels for performance,
• Experience in low-level program behavior, including performance and memory usage, proficiency using profiling tools such as NVIDIA Visual Profiler, nvprof, and NVIDIA Nsight Compute
• Understanding of parallel algorithms for communication between GPUs, familiarity with related libraries and frameworks such as DeepSpeed, PyTorch Distributed, Horovod, Megatron, MSCCL, NCCL
• Knowledge and experience in OSS, Docker, Kubernetes, Python, GOLANG, Rust programming languages  
• Knowledge of LLM/Diffusion model architectures e.g. GPT and Stable Diffusion
• Experience in distributed computing and architecture
• Experience in developing low latency systems
• Experience in developing and operating high scale, reliable online services
• Good communication, collaboration skills and a great team player
• Experience working in a geo-distributed team

#AIPLATFORM#

Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, color, family or medical care leave, gender identity or expression, genetic information, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran status, race, religion, *** (including pregnancy), ****** orientation, or any other characteristic protected by applicable laws, regulations and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request via the Accommodation request form.

Benefits/perks listed below may vary depending on the nature of your employment with Microsoft and the country where you work.

Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter