1️⃣ Memory Architecture Overview ✅ What should be stored Stable user preferences (e.g., “I prefer evening workouts”) Long-term goals (e.g., “I want to learn English in 3 months”) Repeated patterns (e.g., morning routine habits) Saved notes / to-do tasks User-specific facts needed for personalization ❌ What should NOT be stored Sensitive personal data (political views, medical info) Temporary chat fluff (“ok”, “nice”, “haha”) Highly emotional or reactive statements Anything the user says “don’t save” 🧩 Memory Boundaries Only store information with future usefulness. Keep memories factual, concise, and user-approved. Expire outdated facts automatically. 🔄 High-Level Memory Flow Message → Extract → Classify → Vectorize → Store (Vector/JSON) → Retrieve (Top-k) → Merge → Respond --- 2️⃣ Memory Categories (Recommended) Category Purpose User Profile Basic non-sensitive info for personalization. Preferences Choices about food, content, schedule, routines. Past Actions Tasks completed, reminders done, historical signals. Frequent Context Repetitive information that helps future responses. Goals & Tasks Long-term objectives and to-do lists. Notes / Summaries User saved notes, journaling, summaries. System Knowledge Internal rules, responses, meta instructions. --- 3️⃣ Data Extraction Rules ✔ Save when: The user expresses a preference (“I like short workouts”). The user sets a goal or task (“Remind me to walk daily”). They correct the assistant with stable info (“My name is…”) They repeat the same info multiple times → becomes a preference. ✘ Ignore when: Emotional venting Uncertain statements (“maybe… I guess…”) Off-topic casual chat Temporary facts (“I’m eating now”) 📉 Noise Reduction Compress long sentences to 1–2 lines. Remove emojis, fillers, greetings. 🔔 Trigger Updates When: “Remember this”, “note this”, “save this” Pattern repetition (appears 3+ times) New stable preference stated --- 4️⃣ Embedding + Vector Rules Embedding Model: BGE / OpenAI text-embedding-3-large / Instructor-xl Chunk Size: 200–400 characters Metadata: category, timestamp, source message, confidence score Similarity Threshold: 0.75–0.82 Top-k Retrieval: k = 3–5 memories --- 5️⃣ Retrieval Logic Retrieve on: New question requiring personal context Planning tasks Goal updates Repeated patterns Pull 3–5 top memories only. Filter by: Category relevance Time (recent > older) Confidence score Build a context window: Short-term + retrieved long-term memories Deduplicate Compress into bullet points --- 6️⃣ Memory Update System Append when new facts appear. Overwrite when the user corrects info. Merge if two memories are almost identical. Auto-expire old tasks after completion. Versioning: Keep last 3 versions of any goal. --- 7️⃣ Long-Term Memory Workflow [Extract] → [Clean] → [Classify] → [Embed] → [Store] → [Retrieve] → [Summarize] → [Respond] --- 8️⃣ Short-Term vs Long-Term Memory Roles ✨ Short-Term Last 10–20 messages Temporary context for ongoing tasks Conversation continuity 🧠 Long-Term Preferences Goals Long-standing tasks Stable identity facts 🔗 Interaction Short-term context determines when long-term memory is needed. Long-term memory enriches responses when helpful. --- 9️⃣ Safety & Boundaries Never store: Health, political, or religious info Passwords or financial data Use consent triggers (“save this”) Auto-expire seasonal or outdated notes Compress memories to avoid overfitting Avoid using sensitive embeddings --- 🔟 30-Day Implementation Plan Week 1 — Architecture Planning Define memory buckets Set rules & boundaries Choose vector DB (Pinecone/Chroma) Week 2 — Extraction Logic Build keyword triggers Add classification layer Create summarization templates Week 3 — Vector DB Setup Implement embeddings + metadata Build retrieval pipeline Add fallback JSON store Week 4 — Testing & Refinement Test with real conversations Tune thresholds Fix merging, expiry logic Add monitoring --- 11️⃣ Action Checklist [ ] Define categories [ ] Implement extraction triggers [ ] Create summarization prompts [ ] Connect to vector DB [ ] Implement Top-k retrieval [ ] Add overwrite & expiry rules [ ] Start testing real dialogues ---