apr 9, 2022 - Chinchilla large language model released

Description:

DeepMind has developed a new language model called Chinchilla, which has 70 billion parameters. Chinchilla significantly outperforms Gopher (280B parameters) and GPT-3 (175B parameters) on a large range of downstream evaluation tasks. This is despite the fact that Chinchilla is trained on a dataset that is half the size of the dataset used to train Gopher and GPT-3. The researchers believe that this is due to the use of a new training method called PROJUNN, which allows for more efficient updates to unitary matrices. PROJUNN is a promising new method for training large language models, and it could lead to the development of even more powerful language models in the future.

Added to timeline:

Date:

Images:

YouTube: