N NexChron

Transformer

Definition A neural network architecture that uses self-attention mechanisms to process sequences of data in parallel, enabling highly efficient training at scale. It is the foundation of virtually all modern large language models.

In Depth

Introduced in the 2017 paper "Attention Is All You Need," the transformer replaced recurrent architectures with multi-head attention layers that weigh the relevance of each token to every other token. This parallelism allows training on massive datasets using GPU clusters. Transformers now power models across text, image, audio, and video domains.

Browse more terms

AI Agent AI Alignment Agentic AI Artificial General Intelligence Artificial Superintelligence Attention Mechanism AutoML Benchmark Bias (AI) Chain-of-Thought Computer Vision Deepfake Diffusion Model Edge AI Embedding Federated Learning Few-Shot Learning Fine-Tuning Function Calling GPT