T5 (Text-To-Text Transfer Transformer) is a family of language models from Google Research, introduced in the 2019 paper "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer" by Colin Raffel and eight co-authors. A transformer is a type of neural network for sequences such as text. T5 treats every language task as text-to-text: for example, the input "translate German to English: Das ist gut." gives the output "That is good."
The models were pre-trained on a large text corpus, the Colossal Clean Crawled Corpus (C4), and are then fine-tuned for one task; this is transfer learning. The authors released five checkpoints, from T5-Small (60 million parameters) to T5-11B (11 billion parameters). The t5 library loads and mixes datasets, fine-tunes and evaluates the models, on TPUs with Mesh TensorFlow or on a GPU with PyTorch through Hugging Face Transformers.
The code is open source under the Apache-2.0 license. Since July 2022 the authors recommend T5X, a newer implementation in JAX and Flax.