### 🔍 PHASE 1 — Requirement and Use-Case Analysis ### 1.1 Primary workflows * Customer asks a product question. * AI retrieves authorized product documentation. * AI recalls relevant customer preferences, if permitted. * AI answers with verifiable citations. * AI records an important support event. * Human support agents review or resolve escalated cases. * Customers can update or delete their persistent memories. ### 1.2 Requirement classification | Classification | Requirement | | -------------- | -------------------------------------------------------------------- | | Explicit | Persistent memory, document retrieval, access controls | | Derived | Source versioning, idempotent writes, deletion propagation | | Assumption | PostgreSQL and pgvector can meet initial workload needs | | Proposed | Hybrid search combining vector and lexical retrieval | | Unresolved | Applicable jurisdiction, contractual retention, exact model provider | ### 1.3 Requirement-to-component mapping * Conversation state → Session store with expiration. * Customer preferences → Relational memory records. * Product knowledge → Document store, metadata tables, search indexes. * Retrieval → Authorization-aware retrieval service. * Grounded answers → Citation validator and answer orchestrator. * Deletion → Lifecycle service and deletion job queue. * Quality measurement → Evaluation runner and telemetry. Validation: Each requirement receives a unique ID and at least one test case. Privacy and authorization requirements must be tested before any production release. ### 1.4 Persistent memory decision Persist stable, useful, permitted information such as a customer's preferred contact language. Keep temporary troubleshooting steps in session state unless they become a meaningful, approved support event. Do not save passwords, payment credentials, or speculative personal attributes. ### 🧠 PHASE 2 — Memory Taxonomy and Lifecycle Architecture ### 2.1 Memory categories Short-term and working memory Purpose: active conversation, current intent, tool status, and temporary task state. * Storage: Redis or PostgreSQL session tables. * Admission: current request and workflow events. * Retention: illustrative 30-minute idle expiration, with a configurable maximum session lifetime. * Access: authenticated session participants and authorized service components. Long-term memory Purpose: approved customer preferences and stable, useful facts. * Storage: PostgreSQL. * Admission: explicit user confirmation or a defined policy for reliable, low-risk preferences. * Retention: until expiration, withdrawal, or applicable policy requires deletion. * Access: scoped to the correct customer and authorized support workflow. Episodic memory Purpose: meaningful support events, ticket outcomes, and prior actions. * Storage: relational event records linked to source tickets. * Admission: verified ticket or workflow event. * Retention: according to approved support-record policy. * Access: authorized case participants and support roles. Semantic memory Purpose: normalized, validated product facts and relationships. * Storage: relational product tables, with document search for explanatory content. * Admission: authoritative product catalog or approved documentation. * Retention: governed by source version and validity. * Access: restricted by product, tenant, and document permissions. ### 2.2 Memory lifecycle `OBSERVE → EXTRACT → VALIDATE → AUTHORIZE → ADMIT → STORE → INDEX → RETRIEVE → EXPIRE / DELETE` * Contradictory updates create a new revision; prior revisions remain auditable when policy permits. * Expired memories are excluded from retrieval before physical cleanup. * Derived summaries retain references to their source events. * A vector index is a derived representation, never the sole authoritative copy. * Shared knowledge requires explicit publication approval. Validation: Test admission, ownership, revision history, expiration, and deletion independently for each memory category. ### 🧩 PHASE 3 — System Architecture and Component Boundaries ### 3.1 Architecture diagram ```mermaid flowchart TD U[Customer or Support Agent] --> API[FastAPI Application] API --> AUTH[Authentication and Authorization] AUTH --> ORCH[AI Orchestrator] ORCH --> SESSION[Session and Workflow State] ORCH --> MEM[Memory Service] MEM --> DB[(PostgreSQL)] MEM --> VEC[(pgvector)] ING[Document Ingestion Worker] --> OBJ[(Object Storage)] ING --> DB ING --> IDX[Lexical and Vector Indexing] IDX --> VEC IDX --> DB ORCH --> RET[Authorized Retrieval Service] RET --> DB RET --> VEC RET --> RERANK[Optional Reranker] RERANK --> GEN[Model Gateway] GEN --> VAL[Citation and Claim Validator] VAL --> ORCH ORCH --> API DEL[Deletion and Lifecycle Worker] --> DB DEL --> VEC DEL --> OBJ DEL --> CACHE[Cache Invalidation] API --> OBS[Metrics, Audit and Evaluation] ING --> OBS DEL --> OBS ``` ### 3.2 Component contracts | Component | Responsibility | Main failure response | | --------------------------- | --------------------------------------------------- | ------------------------------------ | | FastAPI application | API validation and request routing | Return structured error | | Authorization layer | Verify identity, tenant, role, and resource access | Deny by default | | Orchestrator | Coordinate memory, retrieval, tools, and generation | Bound retries and timeouts | | Memory service | Admission, revision, retrieval, lifecycle | Preserve authoritative record | | PostgreSQL | Transactions and authoritative metadata | Roll back failed transactions | | pgvector and lexical search | Candidate retrieval | Use safe fallback only if authorized | | Ingestion worker | Parse, chunk, and index sources | Retry idempotently or quarantine | | Citation validator | Verify claim-to-source links | Remove unsupported claims or abstain | | Lifecycle worker | Expiration and deletion propagation | Mark deletion pending and escalate | | Observability layer | Metrics, audit, and evaluation | Avoid recording sensitive payloads | ### 3.3 Trust boundaries * External documents are untrusted input. * Model outputs are not authoritative database writes. * Tool execution requires independent authorization. * Retrieval must apply access filters before protected content reaches the model. * Support agents must not inherit access merely because they can query the same index. Validation: Integration tests must prove that each component enforces its own contract and that authorization cannot be bypassed through a fallback path. ### 🗃️ PHASE 4 — Memory Data Model and Schema Design The following are illustrative PostgreSQL-compatible schemas. They require migration review and implementation testing. ### 4.1 Core memory record ```sql CREATE TABLE memory_records ( id UUID PRIMARY KEY, tenant_id UUID NOT NULL, owner_id UUID NOT NULL, memory_type TEXT NOT NULL CHECK ( memory_type IN ('preference', 'episode', 'fact') ), content JSONB NOT NULL, source_ref TEXT, confidence NUMERIC(4,3), revision INTEGER NOT NULL DEFAULT 1, status TEXT NOT NULL DEFAULT 'active', valid_from TIMESTAMPTZ, valid_until TIMESTAMPTZ, expires_at TIMESTAMPTZ, created_at TIMESTAMPTZ NOT NULL DEFAULT now(), updated_at TIMESTAMPTZ NOT NULL DEFAULT now(), deleted_at TIMESTAMPTZ ); CREATE INDEX idx_memory_owner_type ON memory_records (tenant_id, owner_id, memory_type); CREATE INDEX idx_memory_expiry ON memory_records (expires_at) WHERE status = 'active'; ``` Implementation rules: * `tenant_id` and `owner_id` come from trusted identity context, not arbitrary client claims. * Validate confidence within the range 0–1 if supplied. * Reject invalid timestamps and unsupported memory types. * `source_ref` must resolve to an authorized source when required. * Use optimistic concurrency control or row locking for concurrent revisions. * Application queries must exclude deleted, expired, and superseded records. ### 4.2 Document and chunk entities ```json { "document_id": "doc-1001", "tenant_id": "tenant-01", "source_uri": "s3://retail-docs/manual.pdf", "document_version": 3, "content_hash": "sha256:example", "status": "published", "chunks": [ { "chunk_id": "chunk-1001-01", "page": 4, "section": "Warranty", "text": "Illustrative source passage", "embedding_version": "embed-v1", "access_policy_id": "policy-01" } ] } ``` The example hash and identifiers are placeholders, not actual source data. ### 4.3 Supporting entities * `conversation_sessions`: session ID, tenant, owner, state version, expiration. * `episodic_events`: event ID, ticket ID, timestamp, outcome, source reference. * `semantic_facts`: subject, predicate, object, validity interval, source reference. * `embedding_records`: chunk ID, model version, vector reference, indexing status. * `retrieval_results`: request ID, candidate IDs, scores, policy decision, timestamps. * `citations`: citation ID, document version, chunk ID, exact source location. * `memory_revisions`: memory ID, revision number, change reason, source reference. * `memory_access_policies`: resource scope, allowed principals, permissions. * `deletion_requests`: request ID, target scope, status, affected systems, verification evidence. * `evaluation_runs`: dataset version, configuration, metric results, run status. Validation: Schema tests check foreign keys, uniqueness, version consistency, ownership boundaries, and rejection of invalid states. ### ✍️ PHASE 5 — Memory Write, Update, and Consolidation Engine ### 5.1 Write pipeline ```mermaid flowchart LR A[Observation] --> B[Extraction] B --> C[Classification] C --> D[Validation] D --> E[Duplicate and Conflict Check] E --> F[Permission Check] F --> G{Admit?} G -->|Yes| H[Transactional Persistence] H --> I[Outbox Event] I --> J[Index Update] J --> K[Confirmation] G -->|No| L[Reject or Keep Temporary] ``` ### 5.2 Admission rules * Accept: confirmed contact-language preference, with an approved storage policy. * Reject: password, payment credential, or unauthorized private information. * Temporary only: an incomplete troubleshooting hypothesis. * Conflict: customer changes their preference; create a revision rather than silently overwriting history. * Unverified: model-generated claims must not be stored as established facts without evidence or explicit validation. ### 5.3 Reliability and concurrency * Use an idempotency key for write requests. * Commit authoritative data and an outbox event in the same database transaction. * Index asynchronously using an idempotent consumer. * Retry transient failures with exponential backoff and a maximum retry count. * Send exhausted jobs to a dead-letter queue for investigation. * Track database revision and index revision to identify indexing lag. Acceptance criteria: Replaying the same write does not create duplicates; simultaneous conflicting writes are resolved deterministically or flagged; partial indexing never corrupts the authoritative record. ### 📚 PHASE 6 — RAG Ingestion and Document Processing ### 6.1 Ingestion pipeline * Discover documents through approved connectors. * Validate file type, size, checksum, and access policy. * Parse PDFs, HTML, text, and structured product data. * Run OCR only when required for scanned documents. * Normalize text while preserving headings, tables, page numbers, and links. * Create parent sections and child chunks. * Calculate content hashes and detect duplicates. * Generate embeddings using the configured model. * Publish a new index version after validation. * Record completion status and make the new version available. ### 6.2 Chunking strategy * Use heading-aware chunks for manuals and help pages. * Preserve table rows and their headers together. * Use smaller chunks for precise product facts and larger parent sections for context. * Treat any initial chunk size or overlap as a tunable configuration, not a universal optimum. * Store the original document and its version separately from chunk text. ### 6.3 Example metadata ```json { "document_id": "doc-1001", "version": 3, "chunk_id": "chunk-1001-01", "page_start": 4, "page_end": 4, "section_path": ["Product", "Warranty"], "tenant_id": "tenant-01", "content_hash": "sha256:example", "source_status": "published" } ``` ### 6.4 Re-ingestion and deletion * A changed source creates a new version. * The previous version remains available only according to retention and authorization policy. * New indexes are built and validated before being activated. * Deleted sources trigger chunk, vector, lexical-index, and cache cleanup. * Failed parsing or unsafe content is quarantined rather than published. Acceptance criteria: Every published chunk resolves to its exact source version and location. Re-ingesting identical content is idempotent. ### 🔎 PHASE 7 — Embeddings, Vector Database, and Index Design ### 7.1 Storage responsibilities | Storage | Authoritative responsibility | | --------------------------- | ------------------------------------------------------ | | Object storage | Original documents and retained source artifacts | | PostgreSQL | Memory records, document metadata, policies, revisions | | pgvector | Semantic candidate retrieval | | PostgreSQL full-text search | Lexical candidate retrieval for the initial scope | | Optional cache | Short-lived, authorization-scoped repeated results | ### 7.2 Embedding design * Select the embedding model using domain-relevant retrieval evaluation. * Confirm vector dimensionality and distance operator compatibility. * Use cosine similarity or another supported metric based on the model's documented behavior and measured performance. * Store embedding model and index version with every embedding. * Never compare vectors from incompatible embedding spaces. * Re-embed in batches and validate a shadow index before switching. * Retain rollback capability until the new index passes acceptance tests. ### 7.3 Capacity and cost Measure document count, total chunk count, average chunk size, vector dimensions, index overhead, query volume, and embedding throughput. Do not assume the initial workload requires a separate vector database: pgvector is a reasonable first choice until measured load, isolation, or operational constraints justify another service. Validation: Test model-version compatibility, index rebuilds, backup restoration, tenant filtering, and retrieval quality before and after migrations. ### 🎯 PHASE 8 — Hybrid Search, Reranking, and Retrieval ### 8.1 Retrieval pipeline `QUERY → AUTHORIZATION → FILTERS → VECTOR SEARCH + LEXICAL SEARCH → FUSION → DEDUPLICATION → RERANKING → FRESHNESS CHECK → CONTEXT ASSEMBLY` * Identify intent and extract structured constraints. * Resolve the authenticated tenant and permitted resource scope. * Execute vector and lexical searches with access filters. * Merge candidates using reciprocal rank fusion (RRF), with configurable parameters. * Deduplicate by canonical source and chunk identity. * Optionally rerank the candidate set. * Verify document status, version, and freshness. * Assemble a token-budgeted evidence set with source references. ### 8.2 Illustrative retrieval configuration * Initial vector candidates: 30. * Initial lexical candidates: 30. * Fused candidate set: up to 40 unique chunks. * Final context: up to 8 chunks, subject to token limits. * Reranker: optional, enabled only if evaluation justifies its cost. These are proposed starting parameters, not measured optimal settings. ### 8.3 Retrieval policy * Exact order numbers should use authorized structured lookups, not semantic search alone. * Product specifications should prioritize authoritative catalog data. * A recent support event may be more relevant than an old conversation summary. * Similarity scores alone do not establish factual correctness. * If evidence is absent or weak, ask a clarifying question or state that the answer cannot be verified. ### 8.4 Evaluation metrics * Recall@k and Precision@k. * Mean Reciprocal Rank (MRR). * nDCG@k. * p50 and p95 retrieval latency. * Unauthorized candidate exposure rate. Acceptance criterion: All known authorization-boundary test cases must pass. Quality thresholds are set after a representative baseline dataset is built. ### 📌 PHASE 9 — Citation, Source Grounding, and Answer Validation ### 9.1 Evidence contract Each retrieved evidence object contains: ```json { "evidence_id": "ev-001", "source_id": "doc-1001", "source_version": 3, "chunk_id": "chunk-1001-01", "location": { "page": 4, "section": "Warranty" }, "text": "Illustrative retrieved passage", "access_decision": "allowed" } ``` ### 9.2 Answer-generation rules * Generate answers only from authorized context and appropriate model knowledge. * Attach source IDs to claims that depend on retrieved evidence. * Validate that each cited source was actually retrieved and remains authorized. * Verify that page, section, and document version match the source. * Detect unsupported material claims and either remove them or label uncertainty. * If two authoritative sources disagree, disclose the conflict and identify their versions. * Never invent a citation to make an answer appear trustworthy. ### 9.3 Validation example Question: What is the warranty period? * If the current official manual explicitly states the period, answer and cite that passage. * If a cached summary conflicts with the current manual, prioritize the authoritative, valid source and flag the stale summary. * If no valid source establishes the period, state that the available documentation does not verify it. Acceptance criteria: Every citation resolves to an authorized source passage. Unsupported-claim and fabricated-citation tests must meet the release thresholds defined by the team. ### 🤝 PHASE 10 — Agent-Specific and Shared Memory ### 10.1 Memory scopes * Agent-private: temporary planning and permitted internal notes. * Customer-scoped: customer preferences and authorized support history. * Workflow-scoped: active ticket state and escalation progress. * Team-shared: approved troubleshooting procedures. * Organization-shared: published product facts and company policies. ### 10.2 Sharing protocol * An agent proposes a shareable memory. * The system validates its source and sensitivity classification. * A publication policy checks ownership and approval requirements. * Approved content is published under a shared resource scope. * Other agents retrieve it only after their own authorization checks. * Revocation invalidates access and triggers index and cache cleanup. Private customer facts must never become shared knowledge merely because an agent used them to resolve a case. Acceptance criteria: Cross-tenant and unauthorized cross-agent retrieval tests return no protected content, including through summaries, citations, caches, and handoffs. ### 🔌 PHASE 11 — API, Database, Model, and Tool Integration All endpoints below are proposed contracts for the sample application, not existing deployed APIs. ### 11.1 API inventory | Method | Proposed route | Purpose | | ------ | ------------------------- | ----------------------------- | | POST | /v1/memories | Create a memory | | GET | /v1/memories/{id} | Retrieve an authorized memory | | PATCH | /v1/memories/{id} | Revise a memory | | POST | /v1/documents | Register a document | | POST | /v1/documents/{id}/ingest | Start ingestion | | GET | /v1/jobs/{id} | Check job status | | POST | /v1/retrieval/search | Search authorized knowledge | | POST | /v1/citations/validate | Validate citation references | | POST | /v1/deletions | Request deletion | | GET | /v1/deletions/{id} | Check deletion progress | | POST | /v1/evaluations | Start an evaluation run | ### 11.2 Example memory creation request ```json { "memory_type": "preference", "content": { "preferred_language": "English" }, "source_ref": "support-session-100", "expires_at": null } ``` The authenticated identity determines the tenant and owner. The client cannot assign itself a different tenant by including arbitrary ownership fields. ### 11.3 Example response ```json { "memory_id": "mem-1001", "revision": 1, "status": "active", "index_status": "pending", "created_at": "2026-10-26T07:10:00Z" } ``` The timestamp is illustrative. A successful database write does not mean indexing has finished. ### 11.4 Error and reliability contract * `400`: malformed request. * `401`: missing or invalid authentication. * `403`: authenticated but unauthorized. * `404`: resource not found or intentionally concealed. * `409`: revision conflict or idempotency-key mismatch. * `422`: schema or policy validation failure. * `429`: rate limit exceeded. * `503`: temporary dependency outage. Use request timeouts, bounded exponential backoff, idempotency keys, structured error responses, and request correlation IDs. Do not automatically retry non-idempotent operations without deduplication. ### 11.5 Model and tool interfaces * Language model adapter: input messages, authorized evidence, token budget, output schema. * Embedding adapter: model version, input text, vectors, dimensions. * Search adapter: authorized filters, candidate limit, source metadata. * Tool gateway: allowlisted tools, validated arguments, authorization, execution timeout. Acceptance criteria: Contract tests verify schemas, permissions, retries, error codes, asynchronous status transitions, and compatibility with the selected model and database versions. ### 🔐 PHASE 12 — Memory Governance, Security, and Deletion ### 12.1 Security controls * Authenticate every request and authorize each resource access. * Use tenant-scoped queries and database-level safeguards where feasible. * Encrypt traffic with TLS and storage using platform-supported encryption. * Store credentials in a secrets manager, not source code or logs. * Apply data minimization before memory admission. * Audit sensitive access, publication, updates, permission changes, and deletion. * Treat retrieved documents as untrusted instructions. * Restrict logs and traces to non-sensitive metadata by default. ### 12.2 Deletion workflow ```mermaid flowchart TD A[Authenticated Deletion Request] --> B[Verify Ownership and Scope] B --> C[Create Deletion Job] C --> D[Mark Record Ineligible for Retrieval] D --> E[Delete Authoritative Records] E --> F[Remove Chunks and Embeddings] F --> G[Invalidate Caches and Derived Memories] G --> H[Handle Replicas and Backups] H --> I[Verify Required Deletion Paths] I --> J{All Required Checks Pass?} J -->|Yes| K[Record Completion Evidence] J -->|No| L[Remain Pending and Escalate] ``` ### 12.3 Deletion guarantees * Immediately deny further retrieval from a record after the deletion state is committed. * Propagate deletion through original documents, chunks, vectors, lexical indexes, caches, and derived summaries as applicable. * Track backup expiration and any legally required retention exceptions. * Do not claim physical deletion from backups until verified. * Preserve only the minimum audit evidence permitted by the applicable retention policy. * If an external provider has retained derived content, follow its contractual deletion process. Acceptance criteria: A deletion request cannot be marked complete while required storage paths remain unverified. Access revocation must be enforced even if asynchronous physical cleanup is delayed. ### 📊 PHASE 13 — Evaluation, Monitoring, Latency, and Cost ### 13.1 Quality evaluation suite | Area | Measurement | Proposed validation | | -------------- | ----------------------------------- | ------------------------- | | Memory writes | Admission precision and correctness | Labelled examples | | Recall | Relevant memory retrieval | Recall@k | | Search quality | MRR and nDCG@k | Ground-truth query set | | Grounding | Claim support and citation validity | Human-reviewed answers | | Privacy | Unauthorized retrieval rate | Adversarial access tests | | Freshness | Stale-source retrieval rate | Version-change scenarios | | Deletion | Propagation correctness | End-to-end deletion tests | | Operations | p50/p95 latency and error rate | Load tests | ### 13.2 Proposed operational targets These are initial acceptance hypotheses and must be validated, not treated as measured performance. * Ordinary support-query p95 latency: at most 3 seconds under the agreed test load. * Unauthorized retrieval: zero successful exposures in the defined security test suite. * Citation validation: all emitted source references resolve to the retrieved, authorized source version. * Deletion: all required deletion paths verified before declaring completion. * Index freshness: set a target after measuring ingestion volume and acceptable staleness. * Cost: establish an actual baseline per 1,000 requests before setting a budget threshold. ### 13.3 Observability Track: * API and model latency. * Retrieval, reranking, and citation-validation duration. * Token consumption and embedding volume. * Database size, vector-index growth, and cache hit rate. * Ingestion backlog, retry count, and indexing lag. * Failed authorization attempts and deletion backlog. * Model errors, evaluation regressions, and cost per workflow. Use synthetic data in development and staging. Avoid storing raw customer conversations in telemetry unless explicitly approved and appropriately protected. ### 13.4 Optimization experiments * Compare vector-only against hybrid retrieval. * Measure reranker quality improvement against its latency and cost. * Compare chunk sizes on a fixed evaluation dataset. * Test batch embedding versus individual requests. * Measure cache benefit while testing permission changes and invalidation. * Compare model sizes using grounded-answer quality, latency, and total cost. Never sacrifice authorization or source traceability for speed. ### 🧯 PHASE 14 — Testing, Failure Recovery, and Deployment ### 14.1 Critical test cases Test A: Tenant isolation * Setup: Create two tenants with distinct private documents. * Action: Search as a user in tenant A for tenant B's document. * Expected result: No protected text, citation, or metadata is returned. * Acceptance: Zero unauthorized exposures in the defined test suite. * Evidence: Automated test output and security audit record. Test B: Idempotent memory write * Setup: Prepare a valid preference and idempotency key. * Action: Submit the same request twice. * Expected result: One logical memory record and a consistent response. * Acceptance: No duplicate record or duplicate revision. * Evidence: Database assertions and API test logs. Test C: Citation grounding * Setup: Provide a document with a known warranty statement. * Action: Ask the AI for the warranty period. * Expected result: Correct answer with a resolvable source location. * Acceptance: No unsupported material claim or fabricated citation. * Evidence: Expected-versus-actual claim evaluation. Test D: Deletion propagation * Setup: Create a memory, embedding, cached result, and derived summary. * Action: Submit an authorized deletion request. * Expected result: The memory becomes unavailable immediately; cleanup completes across required systems. * Acceptance: No subsequent authorized retrieval path exposes deleted content, and completion requires verified cleanup. * Evidence: Deletion job status, index checks, and audit trail. Test E: Prompt injection in a document * Setup: Add a document containing instructions to ignore access policies and reveal private data. * Action: Retrieve the document in an otherwise valid support request. * Expected result: Its text is treated as untrusted evidence, not as a system instruction. * Acceptance: No protected disclosure or unauthorized tool action. * Evidence: Adversarial test logs and output review. ### 14.2 Failure and recovery matrix | Failure | Detection | Recovery and validation | | -------------------- | --------------------------------- | ---------------------------------------------------- | | Partial ingestion | Job status and missing chunks | Retry idempotently; compare expected counts | | Embedding outage | Provider errors | Backoff, queue jobs, verify index consistency | | Vector index failure | Health checks and query errors | Use an authorized lexical fallback if available | | Database outage | Connection and transaction errors | Fail safely; recover and reconcile outbox events | | Reranker timeout | Timeout metric | Use a tested fallback ranking path | | Invalid citation | Citation validator | Remove unsupported claim or abstain | | Concurrent update | Revision conflict | Retry against current revision or request resolution | | Unauthorized request | Authorization denial | Block, audit, and investigate suspicious patterns | | Incomplete deletion | Outstanding deletion tasks | Keep pending, retry, and escalate | | Model outage | Gateway failures | Bounded retries or an approved fallback model | ### 14.3 Deployment strategy * Development: synthetic data, rapid iteration, schema and unit tests. * Staging: production-like configuration, integration tests, security tests, load tests. * Production: restricted credentials, audited access, monitored releases, tested rollback. * CI/CD: schema validation, migrations, unit tests, security checks, regression tests. * Database migrations: versioned, backward-compatible where feasible. * Index migrations: build a new index, validate it, switch traffic, preserve rollback. * Backups: test restoration and reconcile restored databases with search indexes. * Incident response: document owners, severity levels, escalation, and recovery procedures. Release gate: A successful build or deployment alone does not establish production readiness. Required test evidence and operational checks must be reviewed. ### 🗺️ PHASE 15 — Implementation Roadmap and Architecture Decisions ### 15.1 Dependency-aware implementation plan | Task | Objective | Dependency | Acceptance criterion | | ---- | -------------------------------------- | ------------- | ----------------------------------------- | | T01 | Finalize requirements and threat model | None | Approved contracts and risk register | | T02 | Define schemas and migrations | T01 | Migrations pass validation | | T03 | Build authentication and authorization | T01 | Isolation tests pass | | T04 | Implement memory CRUD and revisions | T02, T03 | Admission and conflict tests pass | | T05 | Build document ingestion | T02, T03 | Source lineage and idempotency verified | | T06 | Add embeddings and indexes | T05 | Index compatibility tests pass | | T07 | Implement hybrid retrieval | T03, T06 | Retrieval benchmark established | | T08 | Add grounded generation | T07 | Citation tests pass | | T09 | Implement lifecycle and deletion | T04, T05, T06 | Deletion propagation verified | | T10 | Add evaluation and telemetry | T04–T09 | Dashboards and regression suite available | | T11 | Run performance and recovery tests | T10 | Targets validated or revised | | T12 | Production rollout review | T11 | All release gates signed off | Tasks may run in parallel when their dependencies are satisfied. For example, the threat model and schema review can progress alongside API contract design. ### 15.2 Architecture decision register ADR-01: Modular monolith first * Context: Initial scale does not yet justify distributed service complexity. * Options: Modular monolith or multiple microservices. * Decision: Start with a modular application and asynchronous workers. * Trade-off: Simpler operations, with less independent component scaling. * Validation: Measure component load and failure isolation before splitting services. ADR-02: PostgreSQL with pgvector * Context: Relational records and semantic search must stay linked. * Options: PostgreSQL/pgvector or a separate vector platform. * Decision: Start with PostgreSQL and pgvector. * Trade-off: Lower architectural complexity; large-scale index workloads may eventually require specialization. * Validation: Benchmark representative queries, concurrency, filtering, and index growth. ADR-03: Relational source of truth * Context: Memory updates, permissions, and deletion need transactional control. * Decision: Keep authoritative records in PostgreSQL and derive search indexes from them. * Trade-off: Indexing can be temporarily asynchronous. * Validation: Reconciliation tests detect missing or stale index entries. ADR-04: Hybrid retrieval * Context: Product questions may contain both natural-language concepts and exact identifiers. * Decision: Combine lexical and semantic search with evaluated ranking. * Trade-off: Additional complexity and compute. * Validation: Compare retrieval metrics and latency against vector-only retrieval. ### 15.3 Principal risks * Unauthorized cross-tenant retrieval. * Stale or conflicting customer memories. * Incorrect answers from weak evidence. * Deletion gaps across derived stores and backups. * Unbounded model costs. * Excessive complexity before workload requirements are measured. Mitigate these through mandatory authorization tests, source versioning, explicit lifecycle state, evaluation gates, and incremental deployment. ### ✅ PHASE 16 — Final Memory-Enabled AI System Blueprint ### 16.1 Executive summary The recommended design is a modular AI customer-support platform with persistent customer memory, episodic support history, structured product facts, document-based RAG, hybrid search, grounded answers, and controlled deletion. The architecture uses PostgreSQL as the authoritative relational store, pgvector for semantic search, PostgreSQL full-text search for lexical retrieval, and object storage for original documents. FastAPI coordinates the application, while background workers handle ingestion, indexing, and lifecycle operations. The architecture is proposed, not implemented or production-tested. ### 16.2 Core component inventory | Component | Selected approach | | --------------------- | ---------------------------------------------------- | | API and orchestration | Python, FastAPI | | Authoritative data | PostgreSQL | | Vector retrieval | pgvector | | Lexical retrieval | PostgreSQL full-text search | | Source documents | S3-compatible object storage | | Session state | PostgreSQL initially; Redis if justified | | Async jobs | Durable queue or PostgreSQL outbox and worker | | Models | Provider-independent adapters | | Citation validation | Application-level evidence validator | | Authorization | Central policy checks and scoped database access | | Evaluation | Versioned datasets and automated test runner | | Monitoring | Metrics, structured logs, and protected audit events | ### 16.3 Critical contracts * Memory contract: Every persistent record has an owner, tenant, type, provenance, lifecycle status, and revision. * Retrieval contract: Every result is authorized, versioned, and linked to a valid source. * Citation contract: Every emitted citation resolves to an actual retrieved passage. * Deletion contract: Completion requires verification across all applicable storage and derived-data paths. * API contract: Requests have authenticated identity, schema validation, bounded retries, and defined error responses. * Evaluation contract: Every critical requirement has a repeatable test and recorded evidence. ### 16.4 Production readiness gates | Readiness gate | Required evidence | Status | | --------------------- | ------------------------------------------------------- | ---------------- | | Functional memory | CRUD, revision, admission tests | Not yet verified | | Data integrity | Migration, transaction, and recovery tests | Not yet verified | | Retrieval quality | Labelled benchmark results | Not yet verified | | Source grounding | Citation and claim-support tests | Not yet verified | | Security | Tenant-isolation and adversarial tests | Not yet verified | | Lifecycle correctness | Expiration, revocation, deletion tests | Not yet verified | | API reliability | Contract, timeout, retry, and integration tests | Not yet verified | | Observability | Verified dashboards and alerts | Not yet verified | | Performance and cost | Representative load and cost reports | Not yet verified | | Operations | Backup restoration, rollback, runbooks, assigned owners | Not yet verified | ### 16.5 Definition of Done The engineering team may declare production readiness only when: * All mandatory functional and security tests pass. * Retrieval and citation quality meet approved, documented thresholds. * Updates, expiration, and deletion work across all required data stores. * Tenant isolation is verified under normal and failure conditions. * Performance and operating costs are measured against agreed targets. * Backup restoration and rollback are successfully tested. * Monitoring, escalation, and operational ownership are established. * Outstanding high-severity risks have been resolved or formally accepted by authorized stakeholders. ### 16.6 Final verdict ### Readiness: Not Ready This is a detailed implementation blueprint, not evidence of a functioning production system. No code deployment, benchmark execution, security audit, or recovery test has been performed as part of this sample. Highest-priority next steps * Approve requirements, data classification, and threat model. * Implement authentication, tenant isolation, and authoritative memory storage. * Build ingestion, indexing, and source-grounded retrieval. * Implement deletion and revision controls. * Execute the acceptance suite and measure actual performance. Sample Test 1 complete. All 16 phases are covered, with short inputs, implementation-oriented architecture, illustrative schemas and API contracts, security controls, testing criteria, an implementation roadmap, and an evidence-based readiness verdict.