ChatGPT works by predicting the next word (technically, token) in a sequence, using a massive neural network trained on a large portion of the internet's text. Despite the simplicity of that core mechanism, the scale and sophistication of the system produces remarkably human-like conversation.
The architecture is based on the transformer model, introduced by Google in 2017. Transformers use a mechanism called "attention" that lets the model consider the relationships between all words in a passage simultaneously. When processing "The cat sat on the mat because it was tired," attention helps the model understand that "it" refers to "the cat," not "the mat."
Training happens in phases:
Phase 1 — Pre-training: The model reads trillions of words from books, websites, Wikipedia, code repositories, and other text sources. It learns to predict the next word given the preceding context. This phase takes months on thousands of specialized GPUs and costs tens of millions of dollars. Through this process, the model learns grammar, facts, reasoning patterns, and even some common sense.
Phase 2 — Supervised fine-tuning: Human trainers write example conversations showing ideal assistant behavior. The model learns to respond helpfully, follow instructions, and maintain a conversational tone rather than just completing text.
Phase 3 — RLHF (Reinforcement Learning from Human Feedback): Human raters compare multiple model responses and rank them from best to worst. A reward model is trained on these preferences, and then reinforcement learning optimizes ChatGPT to generate responses humans prefer. This is what makes it feel helpful rather than robotic.
When you send a message, your text gets broken into tokens (roughly word fragments). The model processes all tokens through its neural network layers, using attention to understand context and relationships. It then generates a response one token at a time, with each new token influenced by everything that came before it.
Important limitations to understand: ChatGPT doesn't truly "know" things — it has learned statistical patterns about how language works. This means it can confidently state incorrect information (hallucinations), it can't access real-time data unless connected to tools, and its knowledge has a training cutoff date. It doesn't have memory between separate conversations unless that feature is specifically enabled.
The model has billions of parameters (GPT-4 is estimated at over 1 trillion) stored as numerical weights that encode everything it has learned. Your conversation doesn't change these weights — they're fixed after training.