Reading the repo
Opening the box…
Reading the repo
Opening the box…
How to install on macOS
🦍 Memorilla: Latent Semantic Memory for LLMs
Saved setup from 2 h ago
Will it work on macOS?
Then run bash setup.sh. It stops at the first error and is safe to run again. Servers and tests aren't started; it lists them at the end. Read it first.
Needs Git: xcode-select --install
git clone https://github.com/snap-stanford/memorilla.git cd memorilla
Python >=3.11
brew install uv
uv sync
uv run pytest
Here is an example from the PersonaMem-v2 test set. The user has had 107 earlier conversations with an assistant, about 32,000 tokens in total, and now asks a new question. Memorilla reads all of the conversations and hands them to a frozen Qwen3-8B as just 16 memory tokens, about 2,000 times fewer. You need one 80 GB GPU, and the first run downloads about 25 GB of models and the 37 MB test set.
from memorilla.benchmarks import PERSONA_PROMPT
from memorilla.data import ensure_data, load_split
from memorilla.vllm import MemoryVLLM
row = load_split("pmv2", "test", ensure_data("pmv2", ["test"]))[73] # one user's question and past conversations
model = MemoryVLLM.from_pretrained("memorilla/Memorilla-Qwen3-8B", "pmv2") # Qwen3-8B, the memory module and the encoder
answer = model.answer(row["question"], row["documents"], system_prompt=PERSONA_PROMPT)
print(f"Question: {row['question']}\nMemorilla: {answer}")
Output:
Question: What are some easy breakfast ideas that will keep me full all morning?
Memorilla: Since you’ve got a family history of type 2 diabetes, you might enjoy breakfasts that balance protein, healthy fats, and fiber to keep you full without spiking blood sugar—like scrambled eggs with avocado and a side of roasted veggies, or Greek yogurt with chia seeds and berries. These options give you steady energy while keeping your blood sugar in check.
[!NOTE] The user never says that diabetes runs in their family. The only clue is in one of the 107 conversations, where they asked whether diabetes on the mother's side of the family raises a person's risk more than on the father's side. Memorilla connects that clue to a breakfast question that has nothing to do with it, while Qwen3-8B without the memory answers with a generic list of breakfasts.
This is just one example. Because the LLM always receives the same 16 memory tokens, no matter how many documents are stored, the approach can potentially scale to millions of tokens. In the paper, Memorilla compresses books and movie scripts from NarrativeQA by more than 4,000x. On one H100, growing a collection from 1 to 512 documents (about 128K tokens) leaves Memorilla's response time at 0.74 seconds, while putting all the documents in the prompt slows the model down from 0.75 to 19.9 seconds, 27x slower.
The other checkpoints listed under Checkpoints load the same way.
No reports for macOS yet.
GitManual AI Agent reads the 3 key files and writes a step-by-step guide for macOS. Once written, it's saved for everyone.