Senior Software Engineer
at Microsoft
Posted 4 hours ago
No clicks
- Compensation
- Not specified
- City
- Not specified
- Country
- Not specified
Currency: Not specified
Join a Microsoft AI Infrastructure team focusing on large language model optimization to design and implement high-performance, massively scalable infrastructure for deploying frontier LLM models. You will work on GPU kernel, compression, scheduling and parallelization optimizations, and maintain internal LLM inference infrastructure. The role emphasizes low-level performance optimization, cross-disciplinary engineering challenges, and shaping the future of AI at Microsoft.
We are seeking brilliant and passionate engineers to work with us on the most interesting and challenging problems of AI Infrastructure development. We are a team focusing on large language model optimization, and we are at the forefront of driving innovation in large-scale AI infrastructure. You will be instrumental in designing and implementing the high-performance, massively scalable infrastructure required to deploy frontier LLM models through innovative GPU kernel, compression, scheduling and parallelization optimizations, directly contributing to groundbreaking advancements in the field. This role is not just about maintaining systems; it's about architecting the future. If you are passionate about AI systems, low-level performance optimization, and solving hard cross-discipline engineering problems, we invite you to join us and help shape the future of AI at Microsoft. This is your opportunity to make a defining impact.
Responsibilities
- Keep up to date with and utilize the latest developments in LLM system optimization.
- Discover/solve impactful technical problems, advance state-of-the-art LLM technologies, and translate ideas into production.
- Optimize LLM inference workloads through innovative kernel, algorithm, scheduling, and parallelization technologies.
- Continuously maintain internal LLM inference infrastructure.
Qualifications
- A bachelor’s degree or higher in computer science, engineering, or a related field, PhD is preferred
- Strong programming skills in Python and C/C++
- 2+ years of experience in machine learning system development and optimization
Preferred Qualifications:
- 2+ years of experience in CUDA kernel development and optimization
- Experience in optimizing communication layer / kernels for deep learning systems
- Experience in machine learning model compression
- Experience on different hardware such as both NVIDIA and AMD GPUs is a plus - A growth mindset and a passion for learning new things
This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.
