Notes

Vector Memory Store

An agent's context window is small and the things it ought to remember are not. The usual way out is to keep memories outside the model and fetch only the relevant ones back. A vector memory store is the database that makes that possible: text is turned into embeddings — numeric vectors that place similar meanings near each other — and stored so the agent can later retrieve by similarity rather than by exact keyword.

How the agent uses it

  • Write — a conversation turn, tool result or extracted fact is embedded and stored with metadata: who, when, which session, which task.
  • Read — the current step is embedded too, and the nearest stored vectors come back as candidate memories.
  • Inject — the top results are placed into the prompt, so the model sees a handful of relevant memories instead of an entire history.

Why it matters, and where it bites
This is what lets an agent recall a preference stated three sessions ago, or avoid repeating a tool call that already failed. Chroma, Pinecone, Qdrant, FAISS and pgvector all serve this role, usually behind a framework's memory interface. The failure modes are specific: similarity is not relevance, so a semantically close but outdated memory is retrieved with full confidence; nothing expires unless something deletes it, so contradictory facts pile up; and pulling back too many chunks crowds the very context window the store was meant to protect. A working memory layer needs write discipline — what is worth storing, scoped how, superseded by what — as much as it needs the index.

A vector memory store is a database that holds an agent's memories as embeddings — numeric representations of meaning — so they can be retrieved by semantic similarity to the current step rather than by exact match. It is the standard way to carry knowledge beyond a bounded context window, letting an agent recall facts from earlier sessions. Retrieval quality is the limit: similarity is not relevance, and stale memories resurface unless something expires them.

Imagine a filing cabinet you search by describing what you need, rather than by remembering the exact label on the folder. Ask for "that thing about the delayed order" and it hands you the right file. A vector memory store works like that.

An AI can only hold a small amount of information in mind at once. So the useful things are kept outside it — notes from past conversations, results it found earlier — and pulled back in when they match whatever it is doing now. That is how an assistant remembers a preference you mentioned last month. The catch is that it finds things that sound related, and sounding related is not the same as being right or still true.