BTC
ETH
SOL
XRP
BNB
TON
DOT
LTC
POL
HBAR
NEAR
ARB
UNI
The neural network design behind today’s language models. It reads all the words of a text at once and works out which ones matter to each other.
AI Drafted with AI help · Reviewed by Keshav Aggarwal · Updated
In simple words
When you read “The cat sat on the mat because it was tired”, you know “it” means the cat. A transformer is built to make those links between words.
It looks at every word and asks: which other words should I pay attention to?
The technical side
The transformer was introduced by Google researchers in the 2017 paper “Attention Is All You Need”. Its key part, self-attention, lets each token weigh every other token in the context, and its maths runs efficiently in parallel on GPUs.
GPT stands for generative pre-trained transformer, and nearly every major LLM, as well as many image, audio and video models, is built on transformers.
Transformer kya hai?
Transformer woh design hai jis pe ChatGPT jaise models bane hain. Ye har shabd ko baaki shabdon se jod ke matlab samajhta hai.
Related terms
- LLM (Large Language Model)AIAn AI model trained on huge amounts of text so it can read and write language, like the models behind ChatGPT, Claude and Gemini.
- TokenAIA small piece of text, often part of a word, that an AI model reads and writes. AI APIs charge by the number of tokens.
- GPU (Graphics Processing Unit)AIA computer chip that does thousands of small calculations at once. First made for video games, it now trains and runs most AI models.