GPT-2 is a large language model from OpenAI, described in the 2019 paper "Language Models are Unsupervised Multitask Learners". It is a transformer, a type of neural network for sequences such as text, with 1.5 billion parameters in its largest version. It was trained on 40 GB of Internet text from 8 million web pages with one objective: predict the next word from all the previous words.
Given a prompt, GPT-2 writes a continuation that follows the style and topic of that text. Without task-specific training it also does basic reading comprehension, summarization, translation and question answering. Fine-tuning, which means further training on a smaller dataset, steers it toward one kind of text; OpenAI's example was product reviews conditioned on star rating and category. Because of concerns about misuse, OpenAI released it in stages: a small model in February 2019, larger ones later, and the full 1.5B model with code and weights on 5 November 2019.
The official GitHub repository holds the code and models under a modified MIT license. It is archived: the code is provided as-is, with no updates expected.