chat-memory

name	chat-memory
description	Use when user needs long-term memory for chatbots. Triggers on: chat memory, conversation history, long-term memory, chatbot memory, memory retrieval, persistent memory, remember conversations.

Chat Memory

Implement long-term memory for chatbots — remember and retrieve relevant conversation history across sessions.

When to Activate

Activate this skill when:

• User needs persistent conversation memory for a chatbot
• User mentions "remember", "long-term memory", "conversation history"
• User wants a chatbot that recalls past interactions
• User needs to personalize responses based on history

Do NOT activate when:

• User only needs in-session context → use standard context window
• User needs document search → use rag-toolkit:rag
• User needs user recommendations → use rec-system

Interactive Flow

Step 1: Understand Memory Scope

"What should the chatbot remember?"

A) User preferences (favorite topics, communication style)

• Long retention, low decay
• e.g., "User prefers technical explanations"

B) Conversation context (discussed topics, mentioned names)

• Medium retention
• e.g., "User mentioned they're working on project X"

C) Factual information (user-provided facts)

• Variable retention
• e.g., "User's dog is named Max"

D) All of the above

Which types matter most?

Step 2: Retention Strategy

"How long should memories last?"

Type	Retention	Decay
Preferences	Permanent	None
Recent context	Days-weeks	5% per day
Old context	Compressed	Summarized

Step 3: Confirm Configuration

"Based on your requirements:

• Memory types: All (preferences, context, facts)
• Retrieval: Semantic similarity + time decay
• Compression: Auto-summarize after 30 days

Proceed? (yes / adjust [what])"

Core Concepts

Mental Model: Human Memory

Think of chat memory like human memory:

• Working memory: Current conversation (context window)
• Long-term memory: Past conversations (vector database)
• Recall: Retrieve relevant memories when needed

┌─────────────────────────────────────────────────────────┐
│                   Chat Memory System                     │
│                                                          │
│  User Message: "How's my Python project going?"          │
│                         │                                │
│       ┌─────────────────┼─────────────────┐             │
│       │                 │                 │             │
│       ▼                 ▼                 ▼             │
│  ┌─────────┐    ┌─────────────┐   ┌─────────────┐      │
│  │ Recent  │    │   Memory    │   │   Memory    │      │
│  │ Context │    │  Retrieval  │   │   Retrieval │      │
│  │(session)│    │ (semantic)  │   │   (keyword) │      │
│  └────┬────┘    └──────┬──────┘   └──────┬──────┘      │
│       │                │                 │              │
│       ▼                ▼                 ▼              │
│  ┌──────────────────────────────────────────────────┐  │
│  │              Combined Context                     │  │
│  │                                                   │  │
│  │  Recent: "We discussed Python yesterday"         │  │
│  │  Memory: "User started Python project 2 weeks    │  │
│  │          ago, learning Flask for web app"        │  │
│  │  Memory: "User mentioned deadline is next month" │  │
│  └──────────────────────┬───────────────────────────┘  │
│                         │                               │
│                         ▼                               │
│  ┌──────────────────────────────────────────────────┐  │
│  │                   LLM Response                    │  │
│  │  "Based on our previous conversations, I know    │  │
│  │   you're building a Flask web app. Since your    │  │
│  │   deadline is next month, let me help you..."    │  │
│  └──────────────────────────────────────────────────┘  │
│                                                          │
│  [Store: "User asked about Python project progress"]    │
└─────────────────────────────────────────────────────────┘

Memory vs RAG

Aspect	Chat Memory	RAG
Source	Past conversations	Documents
Updates	Every conversation	Batch indexing
Personal	Per-user	Shared
Decay	Time-based	Usually none

Implementation

from pymilvus import MilvusClient, DataType
from openai import OpenAI
import time
import uuid

class ChatMemory:
    def __init__(self, uri: str = "./milvus.db"):
        self.client = MilvusClient(uri=uri)
        self.openai = OpenAI()
        self.collection_name = "chat_memory"
        self._init_collection()

    def _embed(self, text: str) -> list:
        """Generate embedding"""
        response = self.openai.embeddings.create(
            model="text-embedding-3-small",
            input=[text]
        )
        return response.data[0].embedding

    def _init_collection(self):
        if self.client.has_collection(self.collection_name):
            return

        schema = self.client.create_schema()
        schema.add_field("id", DataType.VARCHAR, is_primary=True, max_length=64)
        schema.add_field("user_id", DataType.VARCHAR, max_length=64)
        schema.add_field("role", DataType.VARCHAR, max_length=16)  # user/assistant
        schema.add_field(, DataType.VARCHAR, max_length=)
        schema.add_field(, DataType.INT64)
        schema.add_field(, DataType.VARCHAR, max_length=)
        schema.add_field(, DataType.FLOAT)  
        schema.add_field(, DataType.FLOAT_VECTOR, dim=)

        index_params = .client.prepare_index_params()
        index_params.add_index(field_name=, index_type=, metric_type=)
        index_params.add_index(field_name=, index_type=)
        index_params.add_index(field_name=, index_type=)

        .client.create_collection(
            collection_name=.collection_name,
            schema=schema,
            index_params=index_params
        )

     ():
        
        embedding = ._embed(content)

        .client.insert(
            collection_name=.collection_name,
            data=[{
                : (uuid.uuid4()),
                : user_id,
                : role,
                : content,
                : (time.time()),
                : session_id,
                : importance,
                : embedding
            }]
        )

     () -> :
        
        embedding = ._embed(query)

        results = .client.search(
            collection_name=.collection_name,
            data=[embedding],
            =,
            limit=limit * ,  
            output_fields=[, , , ]
        )

        memories = []
         hit  results[]:
            memory = {
                : hit[][],
                : hit[][],
                : hit[][],
                : hit[][],
                : hit[]
            }

            
             apply_decay:
                days_ago = (time.time() - memory[]) / 
                decay =  ** days_ago  
                memory[] = memory[] * decay * ( + memory[] * )
            :
                memory[] = memory[]

            memories.append(memory)

        
        memories.sort(key= x: x[], reverse=)
         memories[:limit]

     () -> :
        
        filter_expr = 
         session_id:
            filter_expr += 

        results = .client.query(
            collection_name=.collection_name,
            =filter_expr,
            output_fields=[, , ],
            limit=limit
        )

        results.sort(key= x: x[])
         results

     () -> :
        
        
        messages = [{
            : ,
            : 
        }]

        
        memories = .retrieve_relevant(user_id, message, limit=)
         memories:
            memory_text = .join([
                
                 m  memories
            ])
            messages.append({
                : ,
                : 
            })

        
        recent = .get_recent(user_id, session_id, limit=)
         msg  recent:
            messages.append({: msg[], : msg[]})

        
        messages.append({: , : message})

        
        response = .openai.chat.completions.create(
            model=,
            messages=messages,
            temperature=
        )
        assistant_message = response.choices[].message.content

        
        .store_message(user_id, , message, session_id)
        .store_message(user_id, , assistant_message, session_id)

         assistant_message

     ():
        
        
        results = .client.query(
            collection_name=.collection_name,
            =,
            output_fields=[, , ],
            limit=
        )

         r  results:
             content_snippet  r[]:
                .client.upsert(
                    collection_name=.collection_name,
                    data=[{: r[], : }]
                )
                 
         


memory = ChatMemory()

user_id = 
session_id =  + ((time.time()))


response = memory.chat(user_id, , session_id)
()

response = memory.chat(user_id, , session_id)
()


new_session =  + ((time.time()))
response = memory.chat(user_id, , new_session)

()

Memory Strategies

1. Sliding Window + Retrieval

# Combine recent context (window) with relevant memories (retrieval)
recent_messages = get_recent(limit=5)  # Last 5 messages
relevant_memories = retrieve_relevant(query, limit=5)  # Top 5 relevant

context = recent_messages + relevant_memories

2. Memory Compression

def compress_old_memories(self, user_id: str, days_threshold: int = 30):
    """Summarize and compress old memories"""
    cutoff = int(time.time()) - (days_threshold * 86400)

    old_memories = self.client.query(
        collection_name=self.collection_name,
        filter=f'user_id == "{user_id}" and timestamp < {cutoff}',
        output_fields=["content"],
        limit=100
    )

    if len(old_memories) > 10:
        # Summarize with LLM
        content = "\n".join([m["content"] for m in old_memories])
        summary = self.llm.summarize(content)

        # Store summary, delete originals
        self.store_message(user_id, "summary", summary, importance=0.8)
        self.delete_old_memories(user_id, cutoff)

3. Importance Detection

def detect_importance(self, content: str) -> float:
    """Automatically detect if content is important"""
    # Keywords indicating important info
    important_keywords = ["always", "never", "prefer", "hate", "love",
                          "my name", "birthday", "deadline", "important"]

    content_lower = content.lower()
    matches = sum(1 for kw in important_keywords if kw in content_lower)

    return min(0.5 + (matches * 0.1), 1.0)

Common Pitfalls

❌ Pitfall 1: Retrieving Irrelevant Memories

Problem: Bot mentions unrelated past conversations

Fix: Increase similarity threshold

memories = [m for m in memories if m["similarity"] > 0.7]

❌ Pitfall 2: Memory Overload

Problem: Too many memories in context, confuses LLM

Fix: Limit memories, summarize if needed

memories = retrieve_relevant(query, limit=3)  # Only top 3

❌ Pitfall 3: Privacy Leaks

Problem: Memory from one user leaks to another

Fix: Always filter by user_id

filter=f'user_id == "{user_id}"'  # ALWAYS include

❌ Pitfall 4: Stale Context

Problem: Bot keeps mentioning outdated information

Fix: Apply time decay

decay = 0.95 ** days_ago
final_score = similarity * decay

When to Level Up

Need	Upgrade To
Search documents	rag-toolkit:rag
Multi-user shared knowledge	Combine with RAG
Real-time streaming	Add message queue
Complex memory graphs	Consider Neo4j

References

• RAG for documents: rag-toolkit:rag
• Embedding models: core:embedding
• Vertical guides: verticals/