CodeFormer is a face-restoration model: it takes a blurry, noisy or otherwise damaged face in an image and produces a clean, detailed face. Shangchen Zhou, Kelvin C.K. Chan, Chongyi Li and Chen Change Loy of S-Lab, Nanyang Technological University, made it and published it at NeurIPS 2022. The task is called blind face restoration because the kind of damage in the input is not known.
The model first learns a codebook, a fixed set of learned parts of high-quality face images, and a decoder that rebuilds faces from these parts. A Transformer, a type of neural network that models a whole sequence at once, then reads the low-quality face and predicts which codebook entries make up the clean face. A weight w trades quality against fidelity: a smaller w gives a cleaner face, a larger w stays closer to the input. Scripts also process whole images and videos, colorize faces and fill in masked parts of faces.
The PyTorch code and pretrained models are on GitHub under the S-Lab License 1.0, which permits non-commercial use; commercial use requires contacting the authors.