Patrick LeGresley

Patrick LeGresley is a researcher in the NLP group within Applied Deep Learning Research at NVIDIA, where he focuses on system aspects of training large language models. He received his PhD in Aeronautics and Astronautics from Stanford University. In the past, he also worked at the Baidu Silicon Valley AI Lab, working on systems research and deep learning for speech recognition.
Avatar photo

Posts by Patrick LeGresley

Conversational AI

Scaling Language Model Training to a Trillion Parameters Using Megatron

Natural Language Processing (NLP) has seen rapid progress in recent years as computation at scale has become more available and datasets have become larger. At... 17 MIN READ
Conversational AI

State-of-the-Art Language Modeling Using Megatron on the NVIDIA A100 GPU

Recent work has demonstrated that larger language models dramatically advance the state of the art in natural language processing (NLP) applications such as... 9 MIN READ