Memory Skill — long-term recall

What it does#

At the end of a session Bea writes a diary entry about it. On every turn, the entries that resemble what is happening right now are retrieved and injected into her prompt. That is the whole loop.

Everything Bea remembers lives in one transactional SQLite file, data/bea.db. Embeddings are computed locally by fastembed and stored as float32 blobs in the memories table.

src/core/skills/memory/
├── memory.py      MemorySkill — the capability
└── generator.py   DiaryGenerator — one LLM call, JSON out

src/core/memory/           (the storage layer, shared with everything else)
├── db.py          SQLite in WAL + a lock; loads sqlite-vec when available
├── rag.py         embed, recall, recall_split, forget
├── embedder.py    fastembed / ONNX on CPU, lazy
└── schema.sql

Recall — two blocks, never one#

context_for(batch) runs on every batch and returns what she remembers. It comes back split by who said it:

[LONG TERM MEMORY]
- [3 days ago] marco said he's building a redstone farm

[THINGS YOU SAID BEFORE — your own past lines, not established facts.
 You made some of them up; don't treat them as true just because you said them.]
- [yesterday] you told chat you own four ferraris

Bea invents on purpose. Without that split her own inventions come back as retrieved facts, she builds on them as if they were true, and the fiction compounds into incoherence. memories.source is 'person' or 'bea', and rag.recall_split() returns two lists.

There is no recall tool. Memory is injected on every turn, so asking for it would only cost a second embedding and a slow round-trip for something already in front of her.

How a result is ranked#

rag.py embeds the query, computes cosine similarity against the candidates, and mixes in recency so a recent memory beats a marginally more similar old one. When sqlite-vec is installed it is used as a coarse pre-filter (over-fetching VEC_OVERFETCH × k candidates, since it orders by L2 and we re-rank by cosine); without it the pure-Python path runs over the same rows and returns the same thing.

min_similarity (default 0.35) is the floor below which a result is not a recollection, just noise.

The diary#

A page is written from the stream of one session, and there is one page per session. Three paths write it; each checks whether the page exists first, so none of them writes one twice:

DreamSkill.run_dream()                      the main path
    └─ memory_skill.write_pages(sittings)     every session the dream read, awaited
Brain.create_new_session()
    └─ memory_skill.process_previous_session(session_id)
shutdown()
    └─ memory_skill.save_all_pending()        the session still open: a safety net

each of them:
    └─ DiaryGenerator.generate_diary(transcript)
            ├─ background model, JSON mode
            └─ {"diary_content", "tags", "user_id"}
    └─ rag.remember(text, scope="diary", ...)

A boot opens a new session without rotating, so a session whose shutdown save did not complete has no page until the next dream reads it.

The generator runs on the background model pool, never on the mind's — a diary is not worth the good model, and it must not compete with the part of her that talks to people.

On shutdown, save_all_pending() is awaited by shutdown() in src/cli.py, so the current session is written on a clean stop, a crash and a Ctrl+C. It guards on existence, so a double call is harmless.

Every shutdown step runs through run_to_completion(), which shields it from the cancellation Ctrl+C aims at main and bounds it with its own deadline (SAVE_GRACE, 30s — the save talks to an LLM). A step that is interrupted or runs out of time is logged and skipped; the steps after it still run, because the last of them is what stops the skills and kills their subprocesses.

POST /memory/save triggers the same pass for the live session from the dashboard.

The embedding model#

The default is multilingual on purpose: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2. With an English-only model, non-English sentences collapse into the same region of the space and retrieval becomes close to random.

Vectors from two different models are not comparable, so changing embedding_model re-embeds everything: rag.ensure_model() checks the recorded model at startup and rebuilds the store if it moved.

The model (~220 MB) is downloaded from Hugging Face on first use into embedding_cache_dir, not at startup — so on a fresh install it is the first thing she remembers that pays for it. uv run bea --setup offers to fetch it at the end, which is the same download with a progress bar in front of it.

No account and no key: the repo is public. When the download is refused anyway — a shared IP hitting Hugging Face's anonymous rate limit — the log says so once and names HF_TOKEN, which .env is read for like any other secret.

If the model cannot be loaded at all, the brain logs the error and keeps going: the roster, the person cards and the hot facts still work — only recall is lost.

Configuration#

"memory": {
  "enabled": true,
  "db_path": "data/bea.db",
  "embedding_model": "sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2",
  "embedding_cache_dir": "data/embeddings_cache",
  "min_similarity": 0.35
}

Old configs carrying "local", "default" or an empty string as embedding_model are migrated at load time (embedder.resolve_model), because fastembed rejects those names.

Importing a Chroma-era store#

tools/migrate_to_sqlite.py lifts a Chroma diary and the JSON roster, people, recent-facts and self-lore files into bea.db.

make migrate

runs it with --dry-run first; re-run without the flag to apply. It needs the migrate extra (chromadb), which is not installed by default.

Related#

• Social memory — who people are, on top of the same database
• Dream — the nightly pass that turns raw sessions into durable memory
• Architecture → Memory