In Depth
PaLM (Pathways Language Model) was introduced by Google in April 2022 as one of the largest dense language models at the time, with 540 billion parameters. It was trained using Google's Pathways system, which efficiently distributes training across thousands of TPU chips. PaLM 2 followed in 2023 with improved multilingual, reasoning, and coding capabilities.
PaLM demonstrated several emergent abilities that smaller models lacked, including the ability to explain jokes, solve multi-step math problems, and generate code from natural language descriptions. These capabilities appeared to emerge suddenly as the model scaled, contributing to research on scaling laws and emergent abilities in large models.
PaLM 2 powered Google's Bard chatbot before the transition to Gemini. It was also available through Google Cloud's Vertex AI platform for enterprise applications. While PaLM has been largely superseded by the Gemini model family, its contributions to understanding scaling behavior and training efficiency remain influential in the field.