ChatGPT Memory advanced: personal vs team, GDPR, evaluation metrics

First 3 Memory articles covered 'what / how to architect / how to use'. This article is 'Memory's last production piece': personal vs team differences, compliance boundaries, evaluation metrics, version control and deletion.

1. Personal vs team Memory

Personal Memory and team Memory are essentially different:

Three sharing modes:

1. Strict separation - personal Memory only user_id dimension; team Memory team_id + user_id composite key. Default config.
2. Shared pool - team Memory visible to all, but each memory has creator user_id (traceable). For doc collaboration / shared notes.
3. Progressive sharing - default personal, memory marked shareable=true enters team pool. Compromise.

Recommendation: mode 1 (default strict separation) + mode 3 (important memory shared after marking).

class MemoryType(Enum):
    PERSONAL = "personal"
    TEAM_SHARED = "team_shared"

class Memory:
    id: int
    user_id: str                    # always set
    team_id: str | None             # set only for TEAM_SHARED
    shareable: bool = False
    memory_text: str
    embedding: vector(3072)
    confidence: float
    version: int = 1
    deleted: bool = False
    created_at: timestamp

# Query: personal memory
memories = db.query("""
    SELECT * FROM memories
    WHERE user_id = :user_id AND team_id IS NULL AND deleted = false
    ORDER BY version DESC LIMIT 10
""", user_id="alice")

# Query: team memory (including personal memories shared to this team)
memories = db.query("""
    SELECT * FROM memories
    WHERE (user_id = :user_id OR team_id = :team_id)
      AND deleted = false
    ORDER BY version DESC LIMIT 10
""", user_id="alice", team_id="team_42")

2. Privacy and compliance

GDPR Article 17 (Right to be forgotten)

Requirement: user can request deletion of all their personal data, organization must:

• delete immediately (withinwithin 30 days)
• including all replicas + cache + backup
• provide deletion proof

Memory system implementation:

def gdpr_delete(user_id: str) -> dict:
    """GDPR Article 17 deletion - cascade across all stores"""
    start = time.time()

    # 1. Delete from vector store
    embedding_ids = db.query("SELECT id FROM memories WHERE user_id = ?", user_id)
    vector_store.delete(ids=[m.id for m in embedding_ids])

    # 2. Delete from primary DB
    count = db.execute("""
        DELETE FROM memories
        WHERE user_id = ?
          AND (deleted = false OR deleted = true)
    """, user_id).rowcount

    # 3. Delete from caches
    cache.delete(f"user:{user_id}:memory_top_k")
    cache.delete(f"user:{user_id}:memory_summary")

    # 4. Mark audit log (keep for compliance per EU AI Act)
    audit_log.record(
        event="gdpr_delete",
        user_id=user_id,
        deleted_count=count,
        timestamp=datetime.utcnow(),
    )

    # 5. Notify user
    send_email(user_id, subject="Your memory data has been deleted", body=...)

    return {
        "status": "deleted",
        "user_id": user_id,
        "deleted_count": count,
        "duration_ms": (time.time() - start) * 1000,
    }

CCPA (California Consumer Privacy Act)

Similar to GDPR, but has opt-out concept (user can choose not not to sell data). Memory system needs to support:

• user export all Memory data (JSON format)
• user mark Memory as 'do not sell'

EU AI Act

Memory / training data audit requirements for AI systems:

• high-risk categories (medical / financial / educational) must audit training data source
• user has right to know 'why did AI answer this way' - may involve Memory recall chain
• AI Act Article 13: training data must be traceable

3. Memory evaluation metrics

Production Memory must monitor 3 core metrics:

def evaluate_memory_precision(top_k_memories, ground_truth_useful):
    """LLM-as-judge evaluates each recalled memory"""
    useful = 0
    for mem in top_k_memories:
        judge = judge_memory_useful(mem, ground_truth_useful)
        if judge["useful"]:
            useful += 1
    return useful / len(top_k_memories)

def evaluate_memory_recall(top_k_memories, all_useful_memories):
    """How many of all useful memories were recalled"""
    top_k_ids = {m.id for m in top_k_memories}
    useful_ids = {m.id for m in all_useful_memories}
    return len(top_k_ids & useful_ids) / len(useful_ids)

def evaluate_memory_drift(user_id, k=100):
    """Drift = memory lost recall quality over 30 days"""
    # Sample 100 memories that user_id wrote 30+ days ago
    old_memories = db.query("""
        SELECT * FROM memories
        WHERE user_id = ? AND created_at < now() - interval '30 days'
        LIMIT ?
    """, user_id, k)

    drift_count = 0
    for mem in old_memories:
        # Re-query with current embedding model and check if still top-K for same query
        # (you need original query — keep it in memory metadata)
        still_recalled = recall_test(mem.text, mem.metadata.original_query)
        if not still_recalled:
            drift_count += 1

    return drift_count / len(old_memories)

Run monthly, output dashboard:

Month | precision | recall | drift
2026-08 | 0.86 | 0.74 | 7%
2026-09 | 0.84 | 0.72 | 9%   <- drift approaching threshold
2026-10 | 0.81 | 0.70 | 12% <- drift alert! need to check index

4. Memory version control

Every memory write gets a version stamp, easy to rollback:

CREATE TABLE memories (
  id BIGSERIAL PRIMARY KEY,
  user_id TEXT NOT NULL,
  memory_text TEXT NOT NULL,
  embedding vector(3072),
  memory_type TEXT,  -- 'personal' / 'team_shared'
  confidence FLOAT,
  version INTEGER DEFAULT 1,
  deleted BOOLEAN DEFAULT false,
  created_at TIMESTAMPTZ DEFAULT now(),
  updated_at TIMESTAMPTZ DEFAULT now()
);

-- Index (by user_id + version sort)
CREATE INDEX ON memories (user_id, version DESC);
CREATE INDEX ON memories (user_id, deleted, version DESC);

On write:

def write_memory(user_id, memory_text, memory_type="personal"):
    """Soft-delete old version + insert new version"""
    with db.transaction():
        # 1. Mark existing memory as superseded
        db.execute("""
            UPDATE memories
            SET deleted = true, updated_at = now()
            WHERE user_id = ? AND memory_text_hash = ? AND deleted = false
        """, user_id, hash(memory_text))

        # 2. Insert new version
        new_version = db.query("""
            SELECT COALESCE(MAX(version), 0) + 1
            FROM memories WHERE user_id = ?
        """, user_id)[0]

        embedding = embed(memory_text)
        db.execute("""
            INSERT INTO memories (user_id, memory_text, embedding, memory_type, version)
            VALUES (?, ?, ?, ?, ?)
        """, user_id, memory_text, embedding, memory_type, new_version)

Rollback:

def rollback_memory(memory_id):
    """Mark current as deleted + restore previous version"""
    with db.transaction():
        mem = db.query("SELECT * FROM memories WHERE id = ?", memory_id)
        # Find all versions with same content hash, keep the highest non-deleted version
        previous_version = db.query("""
            SELECT * FROM memories
            WHERE user_id = ? AND memory_text_hash = ?
              AND version < ? AND deleted = false
            ORDER BY version DESC LIMIT 1
        """, mem.user_id, mem.text_hash, mem.version)

        if previous_version:
            db.execute("UPDATE memories SET deleted = false WHERE id = ?", previous_version.id)

5. Memory deletion mechanisms

User-initiated withdrawal (UI Settings → Memory → Delete All)

def user_delete_memory(user_id):
    """User-initiated deletion via ChatGPT UI"""
    # Delete all memories for user
    db.execute("DELETE FROM memories WHERE user_id = ?", user_id)
    vector_store.delete(where={"user_id": user_id})
    cache.delete(f"user:{user_id}:memory_*")
    return {"status": "deleted", "user_id": user_id}

GDPR request (API / email)

def gdpr_delete_request(user_id, request_id):
    """GDPR Article 17 - process formal deletion request"""
    # Audit log
    audit_log.record(
        event="gdpr_request_received",
        user_id=user_id,
        request_id=request_id,
    )

    # 30-day deadline (typically complete in 1 day)
    result = gdpr_delete(user_id)

    audit_log.record(
        event="gdpr_request_completed",
        user_id=user_id,
        request_id=request_id,
        result=result,
    )

    return result

Auto expiration (Retention)

RETENTION_RULES = {
    "medical": 7 * 365,    # 7 years
    "financial": 5 * 365,  # 5 years
    "general": 2 * 365,    # 2 years
    "eu_user": 0,          # immediate (GDPR)
}

def retention_sweep():
    """Daily cron to mark expired memories as deleted"""
    for category, days in RETENTION_RULES.items():
        if days == 0:
            continue  # eu_user handled via gdpr_delete

        cutoff = datetime.utcnow() - timedelta(days=days)
        expired = db.query("""
            UPDATE memories
            SET deleted = true, scheduled_at = now()
            WHERE category = ? AND created_at < ? AND deleted = false
            RETURNING id
        """, category, cutoff)

        audit_log.record(
            event="retention_expired",
            category=category,
            expired_count=len(expired),
        )

6. Memory recall LLM summary vs full

Memory summary with LLM trade-off:

Recommendation: start with LLM summary (save token); when user reports 'AI got it wrong' switch hybrid; high-sensitivity scenarios (medical / legal) directly use full original.

FAQ

1. How do team Memory and personal Memory coexist?

Three modes: (1) strict separation - personal Memory only user_id dimension, team Memory team_id + user_id composite key; (2) shared pool - team Memory visible to all, but each memory has creator user_id; (3) progressive sharing - default personal, memory marked shareable=true enters team pool. Recommendation (1) + (3): default strict separation, important information shared after marking.

2. How to handle GDPR right to be forgotten?

Three steps: (1) receive user deletion request (API or UI) → trigger immediately; (2) backend bulk delete that user_id's all memory (including personal + team references + associated embedding); (3) delete audit log (except compliance retention required) + notify user. Technical: DELETE FROM memories WHERE user_id = ? + DELETE FROM memories WHERE json_contains(shared_with, ?) AND user_id != ?. Note: all associated cache / replicas must clear.

3. How to measure Memory precision?

Three steps: (1) prepare 100+ test queries (with happy + boundary); (2) run query → recall top-K memory; (3) human or LLM-as-judge evaluates each recall whether truly useful (yes → 1, no → 0). precision = sum(useful) / K. LLM-as-judge precision usually correlates > 0.85 with human. Target > 0.8.

4. How to monitor Memory drift?

Metric: same user_id writes a memory at T0, T+30 days recall same query, check if recalled memory still in top-K. drift = 1 - (T+30 still in top-K ratio). Target < 10%/month. High drift means: (1) vector store index issue; (2) memory writes too many causing new memory to preempt; (3) time decay logic not working. Recommendation: monthly sample 100 existing memory check recall quality.

5. Must Memory use LLM summary?

Not required. Memory summary with LLM aims to compress token + extract structured fields, but: (1) summary adds latency + 5-10% cost; (2) summary may lose details; (3) summary LLM occasionally hallucinates. Production optional modes: (1) full original message; (2) full LLM summary; (3) hybrid; (4) user switchable. Recommendation: start with LLM summary (save token), switch full when data sensitive.

Next steps

• Want Memory architecture? Read ChatGPT Memory architecture: long context, vector retrieval, and cross-session persistence.
• Want Memory complete manual? Read ChatGPT Memory: The Complete User Manual.
• Want Memory high-value use cases? Read ChatGPT Memory: 12 High-Value Use Cases (Tech Stack to Family).