In Depth
Emergent abilities are capabilities that are absent in smaller models but appear in larger ones, seemingly arising from scale rather than being explicitly programmed. Examples include in-context learning, chain-of-thought reasoning, arithmetic ability, and the capacity to follow novel instructions. These abilities appear to 'emerge' suddenly as models cross certain size thresholds rather than improving gradually.
The phenomenon was documented in a 2022 paper by Google researchers who catalogued over 100 emergent abilities across different model families. However, subsequent research has debated whether emergence is truly sudden or an artifact of how performance is measured. Some researchers argue that with continuous metrics rather than threshold-based ones, improvements appear gradual rather than sudden.
Regardless of whether emergence is truly discontinuous, the practical reality is that larger models can do qualitatively different things than smaller ones. This has significant implications: it makes scaling outcomes partially unpredictable, it means smaller models may fundamentally lack certain capabilities regardless of training approach, and it creates both excitement (unexpected capabilities) and concern (unexpected risks) as models continue to scale. Understanding emergent abilities is central to debates about AI safety and development strategy.