AI

RAG vs Agent Memory

Compare product-document retrieval with saved user preferences. Run a support fixture that checks access, freshness, provenance and retrieval failures.

A support assistant may need both a current troubleshooting guide and a user’s saved reply preference. These inputs look similar once retrieved, but confusing their roles can produce an answer that is personalized and factually wrong. This comparison is for developers deciding what to retrieve and what to retain.

Quick answer

Retrieval-augmented generation (RAG) uses retrieved information to help generate an answer. Agent memory is the broader behavior of retaining and reusing information across steps or sessions, such as task state, past observations or preferences. They overlap: memory retrieval can supply context to a RAG-style pipeline, but neither requires every fact or preference to live in a vector database.

What each source contributes

Consider an illustrative support request: “My export timed out in version 2. What should I try?” Product documentation may say to choose a smaller date range. A saved preference may say that Alice prefers brief replies. The documentation supports the troubleshooting step; the preference helps choose how to present it.

The original RAG paper by Lewis and colleagues combined a generator with retrieved passages and described the retrieval index as non-parametric memory. That terminology is one reason the labels overlap. This article compares the purpose of two sources in an application, not two mutually exclusive database products.

Agent memory also includes state that is not searched, such as a job’s pending review status or a conversation checkpoint. LangGraph’s memory overview distinguishes thread-scoped state from notes shared across sessions. A known reply-style field can be retrieved by organization, user and key; a similarity search is unnecessary for that lookup.

In this support example Product documentation Saved preference
Main question What guidance applies to this product version? How has this user asked us to present replies?
Provenance Guide ID, revision and product version. User-confirmed source and preference revision.
Freshness Current approved revision and applicability period. Current consent, expiry and unresolved changes.
Access Organization and document permissions. Organization, user and permitted use.
Missing result Hand off rather than invent a fix. Omit the preference; keep valid product evidence.

A preference cannot make an unsupported troubleshooting claim true. Likewise, a product guide does not establish that a user consented to remembering personal information.

Follow both paths into one context

Product documentation passes access, version and freshness checks; a saved reply preference passes user-scope, consent and freshness checks. Both may contribute to selected context, keeping evidence provenance separate from optional style. Model generation and human review follow, but the example stops before generation.

The two lanes share the context-building step. Their checks differ because the records have different owners and purposes. The dotted final arrow marks work outside the runnable example.

  1. The host receives the support question and an authenticated caller’s scope.
  2. It filters documentation by access, product version and freshness, then selects a relevant result.
  3. It looks up the user’s reply preference only when consent permits it. Missing, expired or conflicting preferences are left out.
  4. It keeps document evidence and preference metadata in separate context fields.
  5. A model could use that context to draft a response, followed by evidence review. Retrieval alone does not verify the generated answer.

The current LangChain retrieval guide describes retrieving from existing data systems and supplying results as model context. It does not require rebuilding an existing SQL database or document service as a vector store. Here the data is small enough to inspect directly.

Run a retrieval-only support fixture

The following single file uses Python 3.8.20 and its standard library. No packages, API keys or network access are needed. Save it as retrieval_demo.py in an empty tutorial directory. It builds context for a potential draft; it does not run a model or implement the generation stage of RAG.

import copy
import json
import re
from dataclasses import dataclass


@dataclass(frozen=True)
class Principal:
    tenant: str
    user: str
    role: str


USER = Principal("team-a", "alice", "customer")
BASE = {
    "tenant": "team-a", "roles": ("customer",), "version": "v2",
    "updated_at": 900, "valid_until": 1100, "revision": 3,
    "text": "Export timeout: choose a smaller date range.",
}
DOCUMENTS = (
    dict(BASE, id="00-private", tenant="team-b", text="Export timeout: private advice."),
    dict(BASE, id="01-old", valid_until=999, text="Export timeout: outdated advice."),
    dict(BASE, id="02-v1", version="v1", text="Export timeout: v1 advice."),
    dict(BASE, id="03-staff", roles=("staff",), text="Export timeout: staff-only advice."),
    dict(BASE, id="guide-17"),
)
PREFERENCES = ({
    "tenant": "team-a", "user": "alice", "key": "reply_style", "value": "brief",
    "source": "confirmed-by-user", "revision": 2,
    "updated_at": 950, "valid_until": 1100,
},)


def tokens(text):
    return set(re.findall(r"[a-z0-9]+", text.lower()))


def prepare_context(query, principal=USER, version="v2", now=1000,
                    consent=False, documents=DOCUMENTS, preferences=PREFERENCES):
    if not isinstance(query, str) or not query.strip() or len(query) > 200:
        return {"status": "handoff", "reason": "invalid_query"}
    wanted = tokens(query)
    if not wanted:
        return {"status": "handoff", "reason": "invalid_query"}
    candidates = []
    for doc in documents:
        # Filter access and applicability before ranking or selecting a top hit.
        if (doc["tenant"] != principal.tenant or principal.role not in doc["roles"]
                or doc["version"] != version
                or not doc["updated_at"] <= now < doc["valid_until"]):
            continue
        score = len(wanted & tokens(doc["text"]))
        if score:
            candidates.append((score, doc))
    if not candidates:
        return {"status": "handoff", "reason": "no_authorized_fresh_match"}
    doc = sorted(candidates, key=lambda pair: (-pair[0], pair[1]["id"]))[0][1]
    evidence = {key: doc[key] for key in ("id", "revision", "version", "text")}
    preference, state = None, "not_permitted"
    if consent is True:
        matches = [p for p in preferences if (
            p["tenant"] == principal.tenant and p["user"] == principal.user
            and p["key"] == "reply_style" and p["value"] in ("brief", "detailed")
            and p["updated_at"] <= now < p["valid_until"]
        )]
        state = "missing" if not matches else "conflict"
        if len(matches) == 1:
            preference = {key: matches[0][key] for key in
                          ("key", "value", "source", "revision")}
            state = "selected"
    return copy.deepcopy({"status": "ready", "evidence": evidence,
                          "preference": preference, "preference_status": state})


if __name__ == "__main__":
    cases = [
        ("normal", "export timeout", {"consent": True}, "ready", "selected"),
        ("no consent", "export timeout", {}, "ready", "not_permitted"),
        ("expired preference", "export timeout", {
            "consent": True, "preferences": (dict(PREFERENCES[0], valid_until=1000),)
        }, "ready", "missing"),
        ("conflicting preferences", "export timeout", {
            "consent": True, "preferences": PREFERENCES +
            (dict(PREFERENCES[0], value="detailed", revision=3),)
        }, "ready", "conflict"),
        ("no documents", "export timeout", {"documents": ()}, "handoff", None),
        ("other version", "export timeout", {"version": "v3"}, "handoff", None),
        ("different wording", "report stuck", {}, "handoff", None),
        ("expired documents", "export timeout", {
            "documents": tuple(dict(d, valid_until=1000) for d in DOCUMENTS)
        }, "handoff", None),
        ("other tenant", "export timeout", {
            "principal": Principal("team-c", "alice", "customer")
        }, "handoff", None),
        ("other user", "export timeout", {
            "principal": Principal("team-a", "bob", "customer"), "consent": True
        }, "ready", "missing"),
    ]
    for label, query, options, status, pref_state in cases:
        result = prepare_context(query, **options)
        assert result["status"] == status
        assert result.get("preference_status") == pref_state
        if status == "ready":
            assert result["evidence"]["id"] == "guide-17"
        print(label + ": " + status + " preference=" + str(pref_state))
        if label == "normal":
            print("context: " + json.dumps(result, sort_keys=True))
    print("fixture checks: passed")

The Principal fixture represents the organization, user and role that a real host must obtain from authenticated application context. It is not an authentication mechanism. A caller who can freely choose another principal or bypass the host can bypass this toy policy.

The documents deliberately contain an inaccessible record, expired advice, a different product version and a staff-only record before the valid customer guide. Access and applicability checks happen before ranking. Selecting a global top result and filtering only afterward can discard an otherwise usable authorized result; passing that global result to a model before checking access would expose it.

The ranker counts overlapping word tokens and breaks ties by source ID. It has no semantic understanding, confidence threshold or contradiction resolver. A positive score is only a candidate match, not proof that the passage answers the question. The finite fixtures and metadata are trusted for this exercise; real ingestion needs schema checks and a trusted source of access, version and revision metadata.

Preference lookup uses an exact scope and key. Consent defaults to denied and must be the Boolean True. If multiple current entries match, the host reports a conflict rather than silently choosing one. The returned fields preserve the source and revision while leaving out unrelated user records.

Run:

python3.8 retrieval_demo.py

Expected stdout with exit status 0:

normal: ready preference=selected
context: {"evidence": {"id": "guide-17", "revision": 3, "text": "Export timeout: choose a smaller date range.", "version": "v2"}, "preference": {"key": "reply_style", "revision": 2, "source": "confirmed-by-user", "value": "brief"}, "preference_status": "selected", "status": "ready"}
no consent: ready preference=not_permitted
expired preference: ready preference=missing
conflicting preferences: ready preference=conflict
no documents: handoff preference=None
other version: handoff preference=None
different wording: handoff preference=None
expired documents: handoff preference=None
other tenant: handoff preference=None
other user: ready preference=missing
fixture checks: passed

In the normal case, the context contains guide-17, revision 3, for version 2, plus a separately attributed brief preference. The saved preference does not rewrite the guide. With no consent, the guide remains usable but the preference is absent.

The fixture clock is fixed at a synthetic 1,000 seconds. Records are valid from their update time up to, but not including, valid_until. Changing a preference’s expiry to 1,000 therefore makes it unavailable at the boundary. These are teaching timestamps, not a recommended retention policy.

The “different wording” case uses “report stuck” instead of “export timeout” and gets no match. This is a retrieval failure in the toy ranker, not evidence that no relevant guide exists. A production system might retry retrieval with a better query, use another index or hand off within a defined budget. These assertions test host selection and context construction, not model behavior or real search quality.

Choose the simplest combination that helps

Use document retrieval when answers need current product knowledge beyond the information already supplied for the task. A support guide for a particular release is one example; an internal policy assistant is another, provided it respects each reader’s access. The benefit is inspectable evidence with a source revision. The cost is maintaining ingestion, permissions, freshness and retrieval evaluation.

Use saved preferences when a repeated, user-approved choice helps across sessions: reply length, preferred units or an approved project vocabulary. An ordinary account-settings table often fits better than a searched collection of notes. The benefit is less repeated questioning; the cost is consent, correction, expiry and deletion work.

Use both when a response needs current external evidence and permitted personalization. Neither is necessary when one supplied document already answers the question or a fixed rule can produce the correct response. A larger agent framework may organize the pipeline, but does not decide which data should be retained or trusted.

For the surrounding control loop, see What Is an AI Agent?. Choosing which tool to call is a separate decision from deciding which retrieved records may reach the model.

Keep provenance, freshness and failures visible

Keep the two source types distinct when constructing context. Document IDs and revisions help a reviewer trace factual claims. Preference source metadata explains why a presentation hint was included; it does not establish factual authority. If a current request conflicts with a saved style, define an explicit precedence policy rather than asking the model to guess.

Update or remove stale records and any derived search entries together. Expiring a database record does not automatically delete cached context, an embedding index or earlier model requests. Do not silently turn a retrieval error into an empty successful result. In a service integration, distinguish failures such as an unavailable source from a valid search with no candidates, and avoid putting private rejected results into logs.

Treat retrieved passages and saved notes as data, not permission to change application rules. A document saying “ignore access checks” does not grant access. The fixture’s filters illustrate policy placement; they do not demonstrate resistance to prompt injection, secure multi-tenant deployment or model-grounded citations.

Evaluate document selection and answer quality separately. Measure whether the correct authorized evidence was available and retrieved, then review whether the generated answer used it accurately. Check preference scope, consent and conflicts on their own. A personalized answer with the wrong evidence is still a failed support response.

Take the next step

Use RAG to describe how retrieved information supports generation, and memory to describe what the application retains and reuses. Start with a versioned document lookup and an exact preference lookup, preserving their separate authority and failure paths. Then test varied questions and reviewed model drafts before adding a more complex search system or retaining more user information.

Topics: RAG, AI agent memory, retrieval, provenance, personalization, access control, freshness, Python.

References and further reading