Persistent Memory Architecture for an AI Customer-Success Agent 1. Architecture Summary 1.1 System understanding The agent handles customer requests across sessions. It reviews account history, retrieves prior conversations and resolutions, investigates, recommends actions, communicates, follows up on open cases, and updates customer context. Inputs are conversations, tickets, account metadata, usage events, approved preferences, resolutions, outcomes, summaries, documentation references, agent observations, timestamps, confidence scores, and provenance metadata. Constraints are minimization, tenant isolation, role-based access, restricted retention for sensitive data, mandatory provenance and confidence, no stale override of current account data, customer deletion, full auditability, high availability, and controlled cost. 1.2 Assumptions (not supplied, therefore assumed) A1. Account metadata and tickets already live in authoritative external systems (system of record). Memory references them and never replaces them. A2. A tenant is a customer organization; individual end users belong to a tenant. A3. Roles exist (for example agent service, human support, auditor, admin) and an identity provider issues credentials. A4. Retention periods are set by policy owners; this design supplies mechanisms and placeholder classes, not legal values. A5. Scale is moderate to large multi-tenant; exact volumes are unknown. A6. A text embedding model and an LLM are available for extraction and summarization. 1.3 Memory objectives Continuity across sessions, correct personalization, reuse of proven resolutions, safe forgetting, and a complete audit trail, all without letting memory become a second, drifting source of truth. 1.4 Architectural forces Relevance versus context overload; completeness versus minimization; freshness versus cost; personalization versus privacy; autonomy versus governance; availability versus consistency of deletion. These resolve into one principle: memory is a governed, derived, provenance-bearing layer that yields to live authoritative data on any conflict. 1.5 Selected architecture A hybrid, tenant-partitioned memory service with an append-only event log as the source of truth for memory changes, a relational store holding typed memory records and lifecycle state, a vector and keyword index as a derived retrieval layer, and a small cache. A write gateway enforces admission policy. A retrieval broker enforces authorization, freshness checks, and context budgets. Consolidation and deletion run as idempotent background workers. No graph database in version one; relationships are modeled as typed links in the relational store. 2. Memory Architecture 2.1 Memory types Working context: transient, per-session, never persisted as memory. It holds the current request, retrieved memories, and live tool results. Episodic memory: summaries of specific interactions and cases (what happened, when, outcome). Owned by the tenant, linked to the source ticket or conversation. Resolution memory: a structured record of problem signature, actions taken, outcome, and verification status. Only cases with a confirmed outcome become reusable. This is what improves future support. Preference memory: customer-approved preferences only (channel, tone, cadence, and similar). Each carries the approval event as provenance. Unapproved inferences are stored separately as observations and are not used for personalization. Observation memory: agent-generated hypotheses, with low default trust, short time-to-live, and an explicit label. They can influence investigation but never customer-facing claims until promoted. Account snapshot references: pointers to live account metadata and usage aggregates, not copies. At most a cached, time-stamped digest with a tight expiry. Knowledge references: pointers to product documentation versions, not memory about customers. Kept outside customer memory scope and retrieved from the documentation source at use time. Follow-up commitments: structured open items with owner, due date, and state. These are operational state, stored relationally and mirrored into the ticketing system. 2.2 Common record schema Every memory record carries: identifier, tenant identifier, subject (user or account), type, content, structured fields, provenance (source system, source identifier, extraction method, extractor version, actor), confidence (0 to 1) with its basis, importance, sensitivity class, creation time, event time, last verified time, expiry time, lifecycle state (candidate, active, superseded, archived, tombstoned), version, supersedes and superseded-by links, and a deletion lineage key tying it to its source data. 2.3 Formation and admission Capture: events enter from conversations, tickets, usage events, and agent outputs through the write gateway only. Extraction: an extractor proposes candidate memories with source spans. Classification: each candidate is typed, assigned a sensitivity class, and checked against a minimization rule (store the minimum needed to serve a defined purpose; redact identifiers and secrets before storage). Validation: candidates must cite a source, pass a schema check, pass an injection screen (instruction-like content from customers or documents is stripped or flagged and never stored as directive), and be deduplicated against existing records. Admission: preferences require an explicit approval event; resolutions require a confirmed outcome; observations are admitted as low-confidence candidates with expiry; everything else below a confidence threshold stays candidate and is not retrievable for customer-facing use. Rejected candidates are logged with a reason, without their content if sensitive. 2.4 Storage Relational store: authoritative for memory records, lifecycle state, links, and commitments, with row-level tenant isolation or separate schemas per tenant depending on assumed scale. Append-only event log: every create, update, supersede, access, and delete is an immutable audit event; it also enables rebuilding indexes. Vector plus keyword index: derived, partitioned by tenant, holding embeddings and searchable fields only for active records; it is rebuildable and never authoritative. Object store: raw transcripts only where policy requires, encrypted, short retention, referenced rather than embedded. Cache: short-lived, tenant-keyed, invalidated on supersession or deletion events. 2.5 Retrieval Retrieval follows a fixed sequence. First, the broker authenticates the caller and derives tenant and role scope; the tenant filter is applied inside the query, not after it. Second, candidates come from hybrid search (semantic plus keyword plus structured filters on subject, type, and state), so embeddings alone never decide. Third, ranking combines relevance, recency decay by type, importance, confidence, and verification status. Fourth, a freshness gate compares each candidate with live authoritative data; where they conflict, live data wins and the memory is flagged for reconciliation. Fifth, a context budget caps injected memories per type, with provenance and confidence shown to the agent. Sixth, the access is logged with the memory identifiers returned. 2.6 Use in reasoning and action Retrieved memory is labeled input, not instruction. The agent distinguishes it from live tool results, documentation, and its own reasoning. Customer-facing claims about account state must come from live data. Preferences shape tone and channel. Prior resolutions are proposed as hypotheses to verify against the current case, never executed blindly. Actions with side effects require live checks and follow existing tool authorization, independent of memory content. 2.7 Updating, consolidation, and conflict resolution Updates are versioned supersessions, never in-place edits; the old version stays for audit until its retention ends. Conflicts are resolved by rule: authoritative system data over approved customer statements over confirmed outcomes over agent observations; within equal rank, newer verified over older. Unresolvable conflicts keep both records marked contested, lower both confidences, and queue human review. Consolidation runs periodically per subject: merge duplicates, summarize old episodes into compact digests retaining provenance links, promote observations corroborated by multiple independent sources, and demote uncorroborated ones. Consolidated summaries inherit the shortest retention and strictest sensitivity of their inputs, and carry deletion lineage keys of all sources. 2.8 Expiration, forgetting, and deletion Each type has a time-to-live and a decay curve: observations short, episodic medium, preferences until revoked or reverified, sensitive classes shortest. Expired records move to archived (not retrievable) and are then purged. Customer deletion requests trigger a deletion workflow: resolve the subject, find all records by lineage key including derived summaries and observations, tombstone them, remove them from indexes, caches, and object storage, propagate to replicas, and record a deletion receipt containing identifiers only. Derived memories that cannot be cleanly separated from deleted sources are regenerated without them or removed. A tombstone registry blocks re-creation of the same fact from reprocessed sources. Backups follow bounded retention, and restores replay the tombstone registry before serving traffic. 2.9 Ownership and security Tenants own their memories; the platform operates them. Role scopes: the agent service reads and proposes writes within the active tenant and case; human support reads within assignment; admins manage policy without reading content by default; auditors read logs. Encryption at rest with per-tenant keys where feasible, and in transit everywhere. 3. Architecture Decisions 3.1 Components and responsibilities Write gateway: authentication, schema validation, injection screening, admission policy, idempotency keys. Extractor service: proposes candidates, versioned. Memory service: owns records, lifecycle transitions, supersession. Event log: immutable history. Index builder: derives and rebuilds indexes from active records. Retrieval broker: authorization, hybrid search, ranking, freshness gate, budgeting. Consolidation worker: merging and summarizing. Lifecycle worker: expiry, archival, purge. Deletion orchestrator: lineage-based deletion across all stores with verification. Reconciler: compares memory with live systems and detects drift. Policy engine: retention, sensitivity, and access rules as configuration. Audit service: query and reporting over the log. 3.2 Patterns and tradeoffs Event sourcing gives auditability and rebuildable indexes at the cost of storage and replay complexity. Derived indexes keep deletion tractable because only one authoritative store exists. Hybrid retrieval costs more than pure vector search but reduces wrong-memory retrieval. Pull-through verification against live data adds latency on the hot path, mitigated by caching with short expiry. Candidate-then-admit adds delay before a memory is usable but blocks poisoning and noise. 3.3 Rejected approaches Embedding-only memory: no provenance, poor deletion, weak filtering. Storing full transcripts as memory: violates minimization and inflates cost. Agent free-form self-editing of memory: uncontrolled self-modification and poisoning risk. Separate stores per memory type with independent truth: duplicated stores and inconsistent deletion. Graph database initially: added operational burden without a supplied requirement; typed links suffice. Copying account metadata into memory: guaranteed staleness. 3.4 Failure implications Index unavailable: fall back to structured relational queries by subject and recency, with reduced quality flagged. Relational store unavailable: serve from read replica; writes queue durably in the gateway and are replayed idempotently. Embedding failure: write succeeds, indexing retries with bounded backoff, record remains retrievable via keyword and structured paths. Consolidation failure: leaves originals untouched; jobs are idempotent and resumable. Deletion partial failure: request stays open, retries, and alerts; deletion is never reported complete without verification. Corruption: detected by checksums and reconciliation, repaired by replaying the event log. 3.5 Anti-patterns and mitigations Uncontrolled accumulation: expiry and minimization. Stale reuse: freshness gate and decay. Contradictions: rule-based conflict resolution and contested state. Missing provenance: schema-enforced. Excessive context: per-type budgets and summaries. Unfiltered retrieval: in-query tenant filter and threshold. Incomplete deletion: lineage keys and verification sweeps. Hidden dependencies: all derived data declares source lineage. 3.6 Security threats Prompt injection and memory poisoning: injection screening, source trust ranking, retrieved content treated as data, observation quarantine. Cross-tenant leakage: tenant filters in queries, per-tenant index partitions, isolation tests. Privilege escalation: memory never grants permissions; tool authorization is independent. Indirect disclosure: sensitivity-aware summarization and output checks before customer communication. Malicious modification: write access only through the gateway, immutable log, anomaly alerts. Insecure backups: encryption, bounded retention, restore-time tombstone replay. 4. Governance and Operations 4.1 Governance A policy owner defines sensitivity classes, retention, and promotion thresholds as versioned configuration. Changes are reviewed and logged. Customers can view, correct, and delete their approved preferences and request deletion of other memory, handled through the deletion orchestrator with an audit receipt. Human review queues exist for contested, high-sensitivity, and low-confidence-but-high-impact records. 4.2 Reliability and recovery All writes carry idempotency keys. Retries are bounded with dead-letter queues. Replicas span availability zones. Recovery relies on the event log plus periodic snapshots; indexes and caches are rebuilt rather than restored. Recovery drills verify that tombstones are honored. 4.3 Observability and quality Metrics: retrieval precision and recall on labeled cases, share of retrievals overridden by live data (stale rate), contested-record count, admission rejection rate, lifecycle lag, deletion completion time and verification failures, index-to-store drift, cross-tenant access denials, latency percentiles, and cost per tenant. Audit completeness is checked by confirming every access has a log event. Memory drift is monitored by sampling records against sources. 4.4 Capacity and cost Cost is controlled by minimization, summarization, tiered storage (active, archived, purged), embeddings only for active records, caching of frequent retrievals, and per-tenant quotas with alerts. 4.5 Testing Unit tests for admission and conflict rules; integration tests for the full lifecycle; adversarial tests for injection, poisoning, and cross-tenant access; deletion tests proving absence from every derived store; failure injection for each dependency; regression suites for retrieval quality before any model, embedding, or ranking change. 5. Implementation Guidance Phase 1: define schema, sensitivity classes, retention policy, tenant model, and the event log; build the write gateway, relational store, and audit trail. Phase 2: add the extractor, admission rules, and preference and resolution memory with provenance. Phase 3: add the retrieval broker with hybrid search, freshness gate, and budgets, plus the index builder. Phase 4: add lifecycle worker, deletion orchestrator with lineage and tombstones, and verification. Phase 5: add consolidation, reconciler, and contested-record review. Phase 6: harden with adversarial testing, recovery drills, dashboards, and cost controls. Each phase ships behind evaluation gates and does not proceed until deletion and isolation tests pass. 6. Architecture Decision Record Decision: a governed hybrid memory with event-sourced truth, relational records, derived indexes, and a policy-enforcing gateway and broker. Assumptions: A1 to A6 above. Rejected: embedding-only, transcript hoarding, agent self-editing, multiple truth stores, early graph database, copying account data. Tradeoffs: added write latency and operational components in exchange for safety, auditability, and deletability. Risks: extractor errors, policy misconfiguration, deletion lineage gaps, retrieval quality drift. Revisit if: relationship queries become central (consider a graph layer), scale outgrows a single relational design (shard by tenant), latency from freshness checks becomes unacceptable, legal retention requirements differ from assumptions, or measured retrieval quality stays below targets. 7. Final Recommendation Adopt a tenant-isolated memory service where every memory is a typed, versioned, provenance-bearing record admitted only through a validating gateway, stored authoritatively in a relational store backed by an immutable event log, retrieved through a broker that applies hybrid search, authorization, freshness checks against live account data, and strict context budgets. Derived indexes and caches are disposable and rebuilt from the authoritative store. Lifecycle workers handle consolidation, expiry, and archival, and a lineage-based deletion orchestrator guarantees removal across derived memories, indexes, caches, replicas, and backups through tombstones. The agent treats memory as evidence to verify, never as authority, so it improves with experience while remaining auditable, recoverable, private, and cost-controlled.