Blogs

Day 3: Memory Systems & Vector Foundations

2025-08-1620 min read
Day 3: Memory Systems & Vector Foundations

Welcome to Day 3 of our AI workshop series! Today we dive deep into memory systems and vector foundations. Through hands-on demonstrations and interactive exercises, you'll understand how AI systems remember, search, and retrieve information. From conversation context to vector databases, we'll build RAG systems step by step using real examples and practical analogies.

🕐 Workshop Schedule (10 AM - 2 PM)

10:00-11:00: Memory Foundation

11:00-12:00: Manual RAG Demo

12:00-1:00: Why Vector Hype?

1:00-2:00: Vector Search in Action

Hour 1: Memory Foundation (10:00-11:00)

The Conversation Analogy (30 minutes)

Core Teaching Example

“Right now, our conversation is going in my short-term memory”

“I remember you asked about Day 2 workshop planning earlier”

“But when you ask about vector databases, I search my knowledge base like searching a book”

“That's long-term memory through retrieval”

Interactive Demo

We'll demonstrate memory limitations in real-time:

  1. Start fresh ChatGPT conversation
  2. Ask it to remember 5 customer names throughout chat
  3. Keep chatting until it forgets first names (context limit)
  4. Show contrast with RAG system that never forgets

Bridge to AI Systems

  • Short-term = conversation context (limited, expensive)
  • Long-term = knowledge retrieval (unlimited, searchable)

Hour 2: Manual RAG Demonstration (11:00-12:00)

Breaking Down R-A-G Live (45 minutes)

“I'll be the AI agent - let's build RAG step by step”

Step 1: The Query

Student asks: “What's the latest AI safety regulations?”

Step 2: Retrieval (R) - Live Google Search

  • “I need external knowledge. Let me search...”
  • Live demo: Google search “AI safety regulations 2024”
  • Show results on screen
  • “I'm the Retrieval component”

Step 3: Augmentation (A) - Building Context

  • Show original prompt vs augmented prompt with search results
  • “See how I augmented my knowledge with fresh information?”

Step 4: Generation (G) - Create Response

  • Put augmented prompt into ChatGPT
  • Show the enhanced response
  • “AI now gives current, accurate information”

Student Exercise (15 minutes)

Students become “human RAG systems”:

  • Each gets different query (iPhone 16, mortgage rates, etc.)
  • They manually perform R-A-G process

Hour 3: Why All the Vector Hype? (12:00-1:00)

“So why is everyone obsessed with vector search?” (10 minutes)

Set the stage:

  • “We just saw 6 different retrieval methods”
  • “But 90% of RAG tutorials only show vectors. Why?”
  • “Let's understand what makes vectors special”

The Magic of Understanding Meaning (15 minutes)

Real examples to show the power:

SQL search: “refund policy” won't find “return procedure”

Keyword search: “cheap” won't find “affordable” or “budget-friendly”

Vector search: understands these are all related concepts

Key insight: “Vectors capture meaning, not just words”

How Sentences Become Vectors (20 minutes)

The Transformation Journey:

Step 1: Sentence → Tokens

“The refund policy is generous”

Becomes: [“The”, “refund”, “policy”, “is”, “generous”]

Show tokenization in action

Step 2: Tokens → Numbers (Embeddings)

Each token gets converted to a vector (list of numbers)

“refund” might become [0.2, -0.1, 0.8, 0.3, ...]

Analogy: “Like GPS coordinates, but for words”

Step 3: Understanding Embeddings as Locations
  • “Every word has a location in high-dimensional space”
  • Similar words live near each other
  • Visual: Show 2D representation of word clusters

The Famous King-Queen Example (15 minutes)

Classic demonstration:

  • “king” has coordinates in vector space
  • “man” has coordinates in vector space
  • “woman” has coordinates in vector space
  • “queen” has coordinates in vector space
The math magic:

king - man + woman ≈ queen

“Remove the ‘maleness’ from king, add ‘femaleness’, get queen”

Mind-blowing moment: “The AI discovered gender relationships just from text!”

More examples:

  • Paris - France + Italy ≈ Rome
  • walking - walk + ran ≈ running
  • “The AI learns relationships between concepts”

Hour 4: Vector Search in Action (1:00-2:00)

How LLMs Work: The Simple Truth (20 minutes)

“Let me show you how every LLM actually works:”

The Universal LLM Process:

Sentence → Tokens → Embeddings → BLACK BOX → Embeddings → Tokens → Sentence

Breaking it down:

  1. Input: “What's your refund policy?”
  2. Tokenize: [“What”, “'s”, “your”, “refund”, “policy”, “?”]
  3. Convert to embeddings: [numbers, numbers, numbers...]
  4. BLACK BOX: Complex neural network processing
  5. Output embeddings: [different numbers...]
  6. Convert back: [“Our”, “refund”, “policy”, “allows”, “30”, “days...”]

What's in the Black Box?

  • Pattern Recognition: “I've seen similar questions before”
  • Attention Mechanisms: “What words are important here?”
  • Context Understanding: “Based on what we've discussed...”
  • Knowledge Retrieval: “I remember from training that...”
  • Reasoning Circuits: Logical connections between concepts
  • Language Generation: “Here's how to say this clearly”

Key Message: “You don't need to understand the black box to use it - like driving a car!”

The Vector Search Connection (20 minutes)

“Here's the beautiful part - we can use these same embeddings for search!”

The Realization:

  • LLMs convert text to embeddings anyway
  • Those embeddings capture meaning
  • We can use them for similarity search
  • “It's the same numbers, just used differently!”

Vector Search Process:

  1. Query: “refund policy” → embeddings
  2. Documents: All converted to embeddings beforehand
  3. Search: Find closest embeddings (distance calculation)
  4. Same embeddings, different purpose!

Why This is Powerful:

  • No need for separate search technology
  • Same understanding of meaning as the LLM
  • Perfect compatibility between search and generation

Embedding Search = Distance Calculation (20 minutes)

Simple explanation:

  • “Embedding search is just measuring distances between points”
  • Query vector: “What's your return policy?”
  • Document vectors: All company documents
  • Find documents with shortest distance to query
  • Visual: Show dots on a map, finding closest ones

🎯 Interactive Vector Search Animation

Let's see exactly how this works! Follow along as we transform “The quick brown fox” into vectors and search for “a fast dark-colored fox”. Use the controls below to step through the process.

Vector Search Animation

Understanding how semantic search works under the hood

Progress1 / 6

Step 1: Sentence to Tokens

A sentence is broken down into individual words or 'tokens'

Original Sentence:
“The quick brown fox jumps over the lazy dog”
↓ Breaking into tokens ↓
The
quick
brown
fox
jumps
over
the
lazy
dog

Why distance = similarity:

  • Close in space = similar in meaning
  • Far apart = different concepts
  • Demo: Show how “refund” and “return” are close, but “refund” and “pizza” are far

Hands-on Activities

Hands-on Vector Exploration (20 minutes)

  • Use embedding playground/tool to show word relationships
  • Students input their own examples
  • See how different words cluster together
  • “Aha moment”: Watch them discover semantic relationships

Vector Database Implementation

Why Start with Vectors? (20 minutes)

  • Most flexible for natural language
  • Handles ambiguity well
  • Good starting point for document-heavy use cases
  • But remember - just one tool in retrieval toolkit

Hands-on Vector Setup (40 minutes)

  • Use pre-built functions approach
  • Students create knowledge base with their documents
  • Test different search queries
  • Compare results with manual Google search approach

Key Takeaways

🎯 What You'll Walk Away With

  • Memory Systems Understanding: Clear distinction between short-term conversation context and long-term knowledge retrieval
  • RAG Demystified: Hands-on experience building RAG systems manually before automation
  • Vector Intuition: Deep understanding of why vectors capture meaning and enable semantic search
  • Practical Skills: Ability to implement vector-based retrieval systems using real tools
  • Mathematical Insights: Understanding how embeddings work without getting lost in complexity

🚀 Ready for Implementation?

By the end of this workshop, you'll have a solid foundation in memory systems and vector search. You'll understand not just how to use these technologies, but why they work and when to apply them.

The combination of practical demonstrations, interactive exercises, and hands-on coding gives you the confidence to build your own RAG systems from scratch.