Transformer models or Transformers (launched by Google in 2017) are a type of neural network primarily used for natural language processing tasks.
The model can weigh the importance of different parts of the input (by applying a set of mathematical techniques called self-attention) and allows for massive parallel processing.
Unlike other models, transformers can handle longer documents and maintain longer conversations without context being lost. Most importantly, performance does not plateau as model complexity becomes exponentially larger. This is a significant improvement over earlier models that had a tendency to unlearn when trained to undertake new tasks.