MKLab · tech talk
Agentic RAG
The story of Norm, a chat bot who learns to look things up.
Space to begin
I · Born knowing nothing
Meet Norm.
Norm is a newborn chat bot. On his own he cannot say a word, so we give him a brain to talk to: a large language model. Every question travels to the LLM, every answer travels back.
II · Memory
Norm forgets.
The LLM has no memory between calls. Every question arrives alone, as if the conversation never happened. So we add a layer in between: an API with session memory that stores the conversation and replays the whole session to the LLM, every single time.
III · The wall
Norm only knows the public internet.
Your company's knowledge was never in the training data. And you cannot send petabytes along with every question. Something has to pick the right few pages, at the right moment.
Interlude · Embeddings
Meaning becomes geometry.
Ahead of time, every document is split into chunks, and an embedding model turns each chunk into a vector: a direction in meaning space. Not the exact words, but what the chunk is about. Similar meaning, similar direction.
Interlude · Vector search
The same trick, backwards.
The question becomes a vector too, through the same model. Finding relevant knowledge is now geometry: find the vectors pointing the same way in a forest of meaning.
Interlude · Augmentation
A better prompt.
The best matching chunks are stitched into the prompt as context. The LLM now answers with knowledge it was never trained on. That is retrieval augmented generation.
IV · Agency
Norm learns to look things up.
The vector database becomes a tool. The model decides when to search, what to search for, and whether one search was enough. That decision loop is what makes it agentic.
Interlude · Traditional vs agentic
The difference is the decision.
Traditional RAG is a pipeline: every question triggers one retrieval, the results are stitched in, the model answers, even when the search missed or was never needed. Agentic RAG puts the model in charge: it decides whether to search, rephrases the query, judges the results, and searches again until it has enough. Fewer wasted lookups, better answers on hard questions.
V · The whole picture
Agentic RAG, end to end.
Question in. Memory recalled, a decision made, knowledge retrieved, and the answer streamed back token by token. Every piece is something we added. Nothing here is magic.
VI · Where everything lives
Every piece runs somewhere.
A question we often get: can I run RAG on my own computer? Yes. You need a harness to orchestrate it all, an embedding model and an LLM. Your documents, the vector store and the harness usually live with you. The models, with the laptops we have today, usually live with a vendor in the cloud, reached over an API.
VII · Completely local
The whole thing on one laptop.
Nothing in the architecture requires the cloud. You need a runtime for a small local LLM, a local embedding model, a vector store, a file will do, your documents, and a harness to orchestrate it all. The trade is honest: smaller models and slower answers, but nothing ever leaves the machine.
MKLab · EST. MMXXV
That was the whole story.
This is the end of the presentation. Thank you for watching. More tech talks are on the way.
