Embeddings are numerical representations of data — text, images, audio, or any other content — in the form of dense vectors (lists of numbers). They capture the meaning and relationships of data in a way that computers can process, compare, and search efficiently. Embeddings are one of the most practically useful concepts in modern AI.

The core idea: An embedding converts something meaningful (a word, sentence, document, or image) into a list of numbers — typically 256 to 3,072 numbers. Items with similar meanings end up with similar numbers. "King" and "queen" have embeddings that are closer to each other than "king" and "bicycle."

How embeddings capture meaning:

In embedding space, mathematical relationships correspond to semantic relationships:

  • Vector("king") - Vector("man") + Vector("woman") ≈ Vector("queen")
  • Vector("Paris") - Vector("France") + Vector("Japan") ≈ Vector("Tokyo")

These relationships emerge automatically from training — nobody programs them explicitly. The model learns that these words appear in similar contexts and places them accordingly in high-dimensional space.

Practical applications:

Semantic search: Instead of keyword matching, embeddings let you search by meaning. Searching "how to fix a broken pipe" would also find documents about "plumbing repair" and "pipe leak solutions" because their embeddings are similar. This is dramatically better than keyword search for real-world queries.

Recommendation systems: Embed user preferences and available items in the same space. Recommend items whose embeddings are closest to the user's preference embedding. Netflix, Spotify, and Amazon all use embedding-based recommendations.

RAG (Retrieval-Augmented Generation): Embed your documents, store them in a vector database, and when a user asks a question, find the most relevant documents by embedding similarity, then feed them to a language model. This is how most enterprise AI chatbots work.

Clustering and classification: Group similar items automatically. Embed customer reviews and cluster them to discover common themes. Embed support tickets and classify them by topic.

Anomaly detection: Normal transactions form clusters in embedding space. Fraudulent transactions have embeddings that fall outside these clusters.

Using embeddings in practice:

The OpenAI embedding API costs roughly $0.0001 per 1,000 tokens — extremely cheap. You can embed millions of documents for a few dollars. Open source alternatives like Sentence-Transformers are free. The typical workflow is: generate embeddings once, store them in a vector database (Pinecone, Weaviate, Chroma, pgvector), and query by similarity at runtime.

Key considerations: Embedding quality depends on the model used. Different embedding models work better for different use cases — some excel at short queries, others at long documents. Embedding dimensions affect storage costs and search speed. Most applications work well with 768-1,536 dimensional embeddings.