Embeddings + Vector DBs

Q: How does the model discover the 1500 dimensions?

The neural network discovers them by predicting masked words across billions of examples, compressing its learned intuition into numbers.

Q: Why is "King - man + woman = queen" significant?

It proves embeddings capture real semantic structure, allowing arithmetic operations on meaning itself.

Q: Why use Approximate Nearest Neighbors (ANN)?

ANN (like HNSW) provides very fast results, trading occasional misses for massive speed gains necessary for scale.

Q: When should I use Cosine Similarity versus Euclidean Distance?

Cosine similarity is the default for text, measuring the angle between vectors; Euclidean distance is common for images, measuring straight line distance.

Outcome: Compare dims 768/1536/3072 on retrieval quality Curated video (LearnThatStack): Embeddings & Vector Databases Explained — https://www.youtube.com/watch?v=rw1YfQQttfo (verified live via yt-dlp 2026-09-24). Pointer: llms-genai-for-practitioners/11 (top_k, FAISS, Pinecone); shell: courses/video-scripts/genai-rag-agents/02.md.

10 minutesVideo LessonPDF notes

🎯 Free Guest Mode: You are learning for free. Sign in to save your completion progress and quiz answers.

Ready to continue?

Mark this lesson as complete when you're ready to proceed.

Key moments

Meaning vs Exact Match — Traditional databases struggle with human meaning, requiring a mathematical solution for concepts like cloud-like shoes.
Defining Embeddings — An embedding converts data into numerical coordinates where distance between points captures similarity in meaning.
High-Dimensional Space — Meaning is captured across 1500 dimensions, representing invisible traits discovered by the model.
Embedding Generation — Embeddings are extracted from a neural network trained to predict masked words in billions of sentences.
Vector Database Necessity — Vector databases are needed because brute force comparison of millions of vectors is computationally impossible.
Distance Metrics — Similarity is measured using distance metrics, primarily cosine similarity for text and Euclidean distance for images.
HNSW Indexing — HNSW builds layered graphs with long-range connections (highways) and local connections (streets) to narrow the search space quickly.
Trade-offs and Filters — The search is Approximate Nearest Neighbors (ANN) for speed, and strong vector DBs support combined similarity and metadata filtering.
Key Applications — Core applications include semantic search, Retrieval Augmented Generation (RAG), recommendations, and multi-modal search.
Complete Mental Model — Embeddings turn meaning into geometry, and vector databases make that geometry searchable at scale for AI applications.

PDF notes

Bottom line: You will learn how embeddings convert human meaning into mathematical coordinates (vectors). You will be able to explain how vector databases store and index these vectors using algorithms like HNSW. This foundation allows you to build modern AI applications like semantic search and Retrieval Augmented Generation (RAG).

This lesson on Embeddings + Vector DBs is hands-on and example-driven. You will learn how embeddings convert human meaning into mathematical coordinates (vectors). You will be able to explain how vector databases store and index these vectors using algorithms like HNSW. This foundation allows you to build modern AI applications like semantic search and Retrieval Augmented Generation (RAG).

What You'll Be Able To Do

Convert a conceptual description into the mathematical definition of an embedding.
Differentiate between cosine similarity and Euclidean distance metrics.
Explain how the HNSW algorithm accelerates vector similarity search.
Identify the core components required for a Retrieval Augmented Generation (RAG) system.
Contrast the capabilities of traditional databases with vector databases.

Topics Covered in Embeddings + Vector DBs

Meaning vs Exact Match (0:00 - 0:45) — Traditional databases struggle with human meaning, requiring a mathematical solution for concepts like cloud-like shoes.
Defining Embeddings (0:45 - 1:30) — An embedding converts data into numerical coordinates where distance between points captures similarity in meaning.
High-Dimensional Space (1:30 - 2:45) — Meaning is captured across 1500 dimensions, representing invisible traits discovered by the model.
Embedding Generation (2:45 - 3:30) — Embeddings are extracted from a neural network trained to predict masked words in billions of sentences.
Vector Database Necessity (3:30 - 4:30) — Vector databases are needed because brute force comparison of millions of vectors is computationally impossible.
Distance Metrics (4:30 - 5:15) — Similarity is measured using distance metrics, primarily cosine similarity for text and Euclidean distance for images.
HNSW Indexing (5:15 - 6:30) — HNSW builds layered graphs with long-range connections (highways) and local connections (streets) to narrow the search space quickly.
Trade-offs and Filters (6:30 - 7:15) — The search is Approximate Nearest Neighbors (ANN) for speed, and strong vector DBs support combined similarity and metadata filtering.
Key Applications (7:15 - 8:30) — Core applications include semantic search, Retrieval Augmented Generation (RAG), recommendations, and multi-modal search.
Complete Mental Model (8:30 - 9:00) — Embeddings turn meaning into geometry, and vector databases make that geometry searchable at scale for AI applications.

SQL Cheat Sheet

Embedding — Convert data (text, image) into a list of numbers (vector)
```
SELECT embed('shoes that look like clouds');
```

Vector Database — Stores vectors and indexes them for fast similarity retrieval

SELECT * FROM vectors ORDER BY vector_column <-> query_vector LIMIT 10;

Traditional Database — Stores structured data for exact matches based on columns
```
SELECT * FROM products WHERE color = 'red' AND price < 50;
```
Cosine Similarity — Measures the angle between two vectors to determine similarity
```
SELECT cosine_distance(v1, v2) AS score;
```
HNSW (Indexing) — Hierarchical structure that dramatically narrows the vector search space
```
CREATE INDEX idx_hnsw ON vectors USING HNSW (vector_column);
```

Semantic Search — Finds results based on the intent or meaning of the query

SELECT document_text FROM documents WHERE embedding <-> 'reduced cloud costs' LIMIT 5;

Comparison Table

Feature	Traditional Database	Vector Database
Primary Goal	Exact match retrieval	Similarity and meaning retrieval
Data Type	Structured rows/columns	High-dimensional numerical vectors
Indexing Method	B-trees, hash tables	HNSW (Approximate Nearest Neighbors)

Common Pitfalls

Mistake: Comparing vectors generated by different embedding models. Avoid: Always use the exact same model for query and stored data.
Mistake: Using brute force search for millions of vectors. Avoid: Implement intelligent indexing like HNSW to narrow the search space.
Mistake: Relying only on similarity without filtering. Avoid: Combine similarity search with metadata filters (e.g., category, price).

FAQs

How does the model discover the 1500 dimensions? The neural network discovers them by predicting masked words across billions of examples, compressing its learned intuition into numbers.
Why is "King - man + woman = queen" significant? It proves embeddings capture real semantic structure, allowing arithmetic operations on meaning itself.
Why use Approximate Nearest Neighbors (ANN)? ANN (like HNSW) provides very fast results, trading occasional misses for massive speed gains necessary for scale.
When should I use Cosine Similarity versus Euclidean Distance? Cosine similarity is the default for text, measuring the angle between vectors; Euclidean distance is common for images, measuring straight line distance.

Common pitfalls

Mistake: Comparing vectors generated by different embedding models. Avoid: Always use the exact same model for query and stored data.
Mistake: Using brute force search for millions of vectors. Avoid: Implement intelligent indexing like HNSW to narrow the search space.
Mistake: Relying only on similarity without filtering. Avoid: Combine similarity search with metadata filters (e.g., category, price).

You type shoes that look like clouds into a search bar. Traditional databases would panic. There's no column called cloud-likeness, but somehow you get exactly what you wanted. That's embeddings and vector databases working together. And if you're building anything with AI right now, search, recommendations, chatbots, rag applications, these two concepts are the foundation. In this video, we'll look at both of these and hopefully get a better understanding. Databases are incredible at exact matches. Show me all red shoes under $50. Done. The query maps perfectly to columns. But there's a problem. Comfortable, standing all day, concert, where's the column for that? Traditional databases store data, but humans communicate in meaning, and meaning doesn't fit in rows and columns. We needed a way to capture meaning itself, mathematically. A way to make computers understand that puppy and dog are related even though they share no letters. Enter embeddings. An embedding is this: take any piece of data, a word, a sentence, an image, and convert it into a list of numbers. That's it. But these aren't random numbers. They're coordinates, a position in space. Imagine a simple 2D space. We plot king somewhere. Now let's add more words. King and queen end up close together. Apple and banana cluster together. King and banana? Far apart. Embeddings capture meaning as distance. Similar concepts become nearby points. Now, how do we go from two dimensions to the 1500 that real models use? Imagine describing movies with two traits, how funny it is and how scary. Comedies here, horror here, horror comedies like Shaun of the Dead in between. But two traits can't capture everything. Is it romantic, animated, a sequel, set in space? Each trait is a dimension. 1500 dimensions means the model captures 1500 different aspects of meaning. Things like, is this formal or casual, positive or negative, technical or conversational, a question or a statement, abstract or concrete? We don't program these. The model discovers them from patterns in billions of examples. And while we can't visualize 1500 dimensions, the math works identically. Similar things remain nearby points. 1500 invisible traits. But do they actually capture meaning? King - man + woman = queen. Arithmetic on meaning. When researchers at Google demonstrated this consistently in 2013, it proved embeddings capture real semantic structure, not just surface patterns. Remember our shoe search? Shoes that look like clouds becomes a vector that lands near cushioned sneakers, pillowy foam runners, soft sole trainers, because the embedding captures the feeling you're describing. Where do these meaningful numbers come from? A neural network trained on a deceptively simple task. The model sees billions of sentences with a word hidden. Its job? Predict what's missing. To predict well, it must understand what cats do, what sitting means, what surfaces exist. That understanding, learned from patterns across billions of examples, gets compressed into the network's internal state. We extract that internal state. That's your embedding, the model's learned intuition frozen into numbers. Today, you rarely train these yourself. You call an API. A few lines to convert language into mathematical coordinates. Now you have millions of these. How do you search them without comparing every single one? This is exactly what vector databases were built for. The challenge is simple to state, brutal to solve. You have a query embedding, a million stored embeddings, find the most similar ones. The brute force approach, calculate the distance between your query and every single vector. For a million 1536 dimension vectors, that's over 1.5 billion calculations per query. Vector databases solve this with intelligent indexing. They don't compare against everything. They dramatically narrow the search space. Instead of searching an entire library book by book, you go to the right section, then the right shelf, then scan just the nearby books. Section, shelf, scan. That's the principle. Now let's see how vector databases actually pull it off. Vector databases combine three core ideas. First, distance metrics. How do we define similar? The most common is cosine similarity, measuring the angle between vectors. Same direction, score of one. Perpendicular, zero. Opposite, negative one. There's also Euclidean distance, straight line distance between the arrow tips. Cosine similarity is the default for text. Euclidean common for images. Most databases support both. Second, indexing. This is where the performance magic happens. One popular algorithm is HNSW, Hierarchical Navigable Small World. The name actually explains how it works. Small world comes from six degrees of separation. In social networks, any two strangers connect through surprisingly few mutual friends. HNSW builds the same structure for vectors. The insight, it builds the graph in layers, and that's what makes it fast. Why does this help? Imagine navigating with only local connections. You'd take hundreds of tiny hops just to cross the space. The top layer fixes that. Long range connections let you skip most of the distance in a few jumps, then you drop down for precision. Think GPS navigation. Top layer is highways, long distance jumps to the right region. Bottom layer is local streets, small precise moves. Parks right next to its nearest neighbors. Instead of a million distance calculations, a few hundred. Milliseconds instead of seconds. One trade-off to note. This is approximate nearest neighbors. HNSW might occasionally miss the absolute closest vector, but it finds very good neighbors very fast. For search, recommendations and rag, that trade-off is almost always worth it. Third, filtered search. Real applications don't just need most similar. They need most similar where category equals footwear and price under 100. Strong vector databases combine similarity search with metadata filters in a single efficient query. One common mistake, never mix embeddings from different models. A vector from OpenAI and a vector from Google exist in completely different mathematical spaces. Comparing them is meaningless. It's like mixing Celsius and Fahrenheit. Same number, completely different meaning. Always embed your queries with the exact same model you used for your stored data. Let's see what you'll actually build with this. Semantic search. User types reduced cloud costs. Your system returns documents about AWS spend optimization and infrastructure budgeting, even though those exact words weren't in the query. The embedding understood the intent, not just the keywords. Retrieval augmented generation, or rag, the backbone of AI assistance that actually know your data. Embed your documentation, store the vectors. When users ask questions, retrieve the relevant chunks, pass them to an LLM as context. Instant expert on your content. Recommendations. Embed products by their descriptions and attributes. Embed user behavior as preference vectors. Find where they overlap. Multi-modal search. Models like CLIP embed images and text into the same vector space. Search your photo library with natural language. Beach sunset with the dog finds the right memories. No manual tagging required. Let's lock in the complete mental model. Raw data comes in, text, images, audio, whatever you're working with, an embedding model converts it to vectors, numerical coordinates that capture meaning in high dimensional space. A vector database stores those vectors in intelligent index structures designed for fast similarity retrieval. Queries get embedded with the same model. The database finds similar vectors in milliseconds, not minutes. Your application uses those results for search, rag, recommendations, whatever you're building. Embeddings turn meaning into geometry. Vector databases make that geometry searchable at scale. Together, a core building block for AI powered applications. That search for cloud like shoes, now you know exactly what's happening under the hood. The query becomes a vector. The vector finds its neighbors. The neighbors are your results. Meaning captured as math, searchable at scale. And this is how it works. If this helps, please like and share. Thanks for watching and see you in the next one.

Frequently asked questions

How does the model discover the 1500 dimensions?