Introduction
Large Language Models (LLMs) have transformed natural language processing, yet they remain fundamentally limited in their ability to manage external knowledge. While LLMs excel at generating fluent text, they suffer from critical issues:
- Hallucination: generating plausible but incorrect information when knowledge is absent
- Context limitations: context windows constrain the amount of retrievable knowledge
- Retrieval inefficiency: standard RAG systems load excessive context, increasing latency and cost
Retrieval-Augmented Generation (RAG) addresses some of these issues by grounding LLM outputs in retrieved documents [1]. However, current RAG architectures have significant limitations:
- Flat retrieval - all documents searched equally, regardless of relevance
- No verification - no mechanism to detect when the LLM fabricates information
- Context bloat - retrieving top-k chunks floods the context window
We propose Memory Palace, a hierarchical memory system for LLMs inspired by the ancient method of loci [2]. Rather than flat vector search, Memory Palace organizes knowledge into domain-specific indices with multi-hop retrieval—routing queries through hierarchical structure to minimize context while maximizing precision.
Contributions
We present a novel LLM memory architecture with four key innovations:
Hierarchical Domain Index: A three-level index structure that reduces retrieval context by 97% compared to flat RAG, enabling efficient scaling to large knowledge bases.
Verification Tokens: Embedded tokens in memories that allow deterministic detection of LLM hallucination with F1=0.92—without requiring additional model inference.
SMASHIN SCOPE Encoding: A systematic method for encoding knowledge into structured, retrievable memories with multi-channel redundancy for robust retrieval.
Red Queen Protocol: Named after Lewis Carroll’s Through the Looking-Glass (“It takes all the running you can do, to keep in the same place”), an adversarial pre-learning framework with configurable rounds that proactively strengthens weak memories, reducing retrieval requirements by up to 37%.
Research Questions
We address the following questions for LLM memory systems:
- RQ1: Does hierarchical retrieval improve accuracy compared to flat RAG?
- RQ2: Can verification tokens effectively detect LLM hallucination?
- RQ3: What context reduction is achievable while maintaining retrieval quality?
- RQ4: How does Memory Palace scale with corpus size compared to standard approaches?
- RQ5: How does adversarial pre-learning (Red Queen) affect retrieval efficiency?