Skip to content

We're open source. If actrone-memory has been useful to you, a star on GitHub means a lot to us.

Memory retrieval pipeline

How the Context Manager assembles the context window before each LLM call: parallel fetch from L1 and L2, token budget allocation, hybrid relevance ranking, and recency decay.

The 4-phase pipeline

Note

This pipeline is identical in both open-source libraries (actrone-memory for Python and for TypeScript). In code, retrieve_context / retrieveContext returns the assembled window (recent_turns / recentTurns, episodic_memories / episodicMemories, plus the tokens used and retrieval latency), so you can inspect exactly what was included without a hosted account.
  1. Your system prompt stays yours: the library assembles memory within the shares of the budget below and leaves the rest for your system prompt and the current turn. It never includes or prunes your prompt.
  2. Parallel fetch: L1 (last N turns, up to max_session_turns) and L2 (semantic search, admitting memories at or above the embedder's calibrated relevance threshold) are queried in parallel. Both calls run concurrently, whether the backend is the zero-dependency in-memory store or Redis + Qdrant in production, so the fetch takes as long as the slower of the two. On the Python library's bundled benchmark (in-memory store, keyword embedder), a whole retrieval takes 0.27 ms at the median and 1.43 ms at p95, from one run on a development machine; with Redis and Qdrant, add your network round trips. Run python -m actrone_memory.benchmark to measure your own.
  3. Token-budget allocation: the budget you pass is split by fixed fractions: session turns (L1) get 35% and episodic memories (L2) get 25%, and the remaining 40% is left for your system prompt and the current turn. Items are included greedily in priority order within each share; an unused share is not handed to the other.
  4. Relevance ranking: by default (hybrid_retrieval / hybridRetrieval: true), L2 candidates are ranked by Reciprocal Rank Fusion across three independent rankings: dense (vector) similarity, lexical (BM25) match, and recency. Each ranking contributes 1 / (k + rank) to a memory's fused score, so a memory doesn't need to win on embeddings alone: a strong keyword or very recent match can still surface it. If you disable hybrid retrieval, ranking falls back to a simple weighted blend: 0.7 × cosine_similarity + 0.3 × recency, where recency falls linearly from 1 for a memory written now to 0 for one 30 days old or older.

Reranking (opt-in)

For an extra precision pass on top of hybrid retrieval, enable cross-encoder reranking: the pipeline over-fetches the top candidates from the RRF stage, then rescores them with a cross-encoder model that sees the query and each candidate together (slower, but more accurate than comparing embeddings independently) before truncating to the token budget.

.env
# Python: needs the [onnx] extra
ACTRONE_RERANK_ENABLED=true
ACTRONE_RERANK_TOP_K=20   # candidates sent to the reranker before truncation

In TypeScript, pass a Reranker as the reranker option of MemoryManager.create(); rerankTopK in config sets the window.

Tip

Reranking is off by default in both languages. It costs extra latency for a precision gain that mostly matters once an agent has thousands of stored memories. Enable it once you see irrelevant L2 hits slipping through hybrid retrieval alone.

Configuring the pipeline

Every value in the pipeline above is configurable, via environment variables or a config object in Python, via the config option on MemoryManager.create() in TypeScript. See Configuration for the full reference; the fields that affect retrieval specifically:

python
from actrone_memory.config import MemoryConfig

config = MemoryConfig(
    max_session_turns=50,        # L1 cap
    # relevance_threshold: leave unset to use the embedder's calibrated value
    max_episodic_memories=500,   # L2 candidates considered
    hybrid_retrieval=True,       # RRF across dense + BM25 + recency
    rerank_enabled=False,        # opt-in cross-encoder rerank
    relevance_weight=0.7,        # only used when hybrid_retrieval=False
    recency_weight=0.3,          #   (must sum to 1.0 with relevance_weight)
)

Relevance threshold tuning

Similarity scores are not comparable across embedding models, so one number cannot suit them all: the keyword-only hashing embedder scores relevant text around 0.24, while bge-small scores unrelated text around 0.48. Each built-in embedder therefore carries the threshold it was calibrated for, measured on a labelled set of 48 relevant and 528 unrelated query and memory pairs, and that is the default. An embedder that declares none, such as OpenAI's text-embedding-3-small or your own, uses 0.72 in Python and 0.7 in TypeScript; measure it on your own data before relying on that. Lower the threshold to retrieve more (potentially less relevant) context; raise it to keep the window focused.

Tip

Test threshold values directly against your own store, call retrieve_context / retrieveContext with a candidate query and inspect the returned scores before wiring the value into your agent.
EmbedderCalibrated thresholdMeasured recall and precision
bge-small-en-v1.5 (the [onnx] extra, or fastembed in TypeScript)0.6388% recall, 84% precision
all-MiniLM-L6-v2 (Python [local] extra)0.4088% recall, 91% precision
Hashing (no extra installed)0.3042% recall, 36% precision: keyword matching only
Was this page helpful?