9 avr. 2022 - Chinchilla large language model released
Description:
DeepMind has developed a new language model called Chinchilla, which has 70 billion parameters. Chinchilla significantly outperforms Gopher (280B parameters) and GPT-3 (175B parameters) on a large range of downstream evaluation tasks. This is despite the fact that Chinchilla is trained on a dataset that is half the size of the dataset used to train Gopher and GPT-3. The researchers believe that this is due to the use of a new training method called PROJUNN, which allows for more efficient updates to unitary matrices. PROJUNN is a promising new method for training large language models, and it could lead to the development of even more powerful language models in the future.Ajouté au bande de temps:
Date:
9 avr. 2022
Maintenaint
~ Il y a 3 ans et 6 mois