RembrREMBR
Back to Blog
1 March 2026· 12 min read

Building a Production MCP Memory Service: What We Learned

engineering
postgresql
mcp
vector-search

Rembr is a memory-as-a-service platform exposed over the Model Context Protocol. This is the engineering post: the decisions that weren't obvious, the things that broke, and what we'd do differently.

Why Postgres, Not a Dedicated Vector DB

The obvious choice for vector search is a dedicated vector database — Pinecone, Weaviate, Qdrant. We chose Postgres with pgvector. Here's why.

Agent memory isn't just vectors. It's structured data: tenant IDs, categories, timestamps, visibility flags, relationship types, PII markers. A dedicated vector DB forces you to maintain two systems — one for vector similarity, one for everything else — and joins across them at the application layer. That's slow and fragile.

Postgres lets you do this in a single query:

SELECT m.id, m.content, m.category,
       1 - (m.embedding <=> $1::vector) AS similarity
FROM memories m
WHERE m.tenant_id = $2
  AND m.category = ANY($3)
  AND m.created_at >= $4
ORDER BY m.embedding <=> $1::vector
LIMIT 20;

Vector similarity, tenant isolation, category filter, and date range — all in one query, all inside one transaction, with ACID guarantees. No cross-system join. No eventual consistency edge cases.

The trade-off is that pgvector's ANN performance tops out below the scale of a dedicated vector DB. For agent memory workloads (tens of thousands to low millions of vectors per tenant), it's more than sufficient. If you're indexing a 100M-document corpus, use a dedicated system. If you're storing AI agent memories, Postgres is the right call.

HNSW vs IVFFlat: The Indexing Decision

pgvector supports two index types: IVFFlat and HNSW.

IVFFlat clusters vectors into buckets and searches only the closest buckets. It's fast to build and has a small memory footprint — but recall degrades sharply as you add more vectors if you don't rebuild or tune the number of lists.

HNSW (Hierarchical Navigable Small World) builds a multi-layer graph structure. Build time is slower and memory usage is higher (roughly 8 bytes × m × vectors), but query performance is more consistent as the corpus grows, and it doesn't require periodic rebuilds.

We went with HNSW. The parameters that matter:

CREATE INDEX idx_memories_embedding ON memories
  USING hnsw (embedding vector_cosine_ops)
  WITH (m = 16, ef_construction = 64);
  • m = 16 — edges per node. Higher m improves recall at the cost of index size. 16 is a good default for most workloads.
  • ef_construction = 64 — search width during index build. Higher values improve index quality but slow build time. 64 is balanced.

Adaptive ef_search: The Tuning Problem Nobody Talks About

Here's the problem: ef_search controls how many candidates the HNSW graph explores during a query. Higher values = better recall, lower values = faster queries. The default is 64.

But the right value depends on corpus size. A tenant with 500 memories doesn't need the same ef_search as a tenant with 500,000. Setting a single global value means you're either over-searching small tenants (slow for no benefit) or under-searching large ones (poor recall).

Our solution: adaptive ef_search, set per-query using SET LOCAL:

-- Tier logic (inside the query transaction)
SET LOCAL hnsw.ef_search = CASE
  WHEN memory_count < 10000   THEN 40
  WHEN memory_count < 100000  THEN 64
  WHEN memory_count < 500000  THEN 100
  ELSE 128
END;

SELECT ... FROM memories ORDER BY embedding <=> $1 LIMIT 20;

SET LOCAL is transaction-scoped — it resets automatically at commit/rollback. No risk of a high-value setting leaking to other queries. We implement this in a VectorSearchService that reads the tenant's memory count, selects the appropriate tier, and wraps the search in a transaction.

Multi-Tenancy: Row-Level Security, Not Application Filters

Every query that touches memory data could theoretically return another tenant's data if a WHERE tenant_id = $x clause is accidentally omitted. Application-layer filtering is a single point of failure.

We enforce isolation at the database layer with Postgres row-level security:

ALTER TABLE memories ENABLE ROW LEVEL SECURITY;

CREATE POLICY tenant_isolation ON memories
  USING (tenant_id = current_setting('app.current_tenant')::uuid);

The application sets app.current_tenant at the start of each connection. If a query somehow misses the tenant filter, RLS catches it. The security property holds even under application bugs.

One subtlety: RLS has a performance overhead. The policy predicate is evaluated for every row the query touches before filtering. With a compound index on(tenant_id, category, created_at), the planner pushes the tenant filter into the index scan and RLS overhead becomes negligible.

The MCP Interface

MCP (Model Context Protocol) defines a standard for AI models to call external tools. A Rembr MCP server exposes memory operations as tools — the model can call them the same way it would call a web search or a code interpreter.

The core tools:

// Store a memory
memory({ operation: "create", content: "...", category: "facts" })

// Retrieve relevant memories
search({ operation: "query", query: "user's preferred stack", limit: 10 })

// Find semantically similar memories
search({ operation: "similar", memoryId: "uuid", limit: 5 })

// Detect contradictions
contradictions({ operation: "detect" })

The model decides when to call these tools based on its understanding of the task. You don't need to hardcode memory reads into your agent loop — the model learns to retrieve context when it needs it and store information when it's likely to be useful later.

Embedding Model Consistency: The Footgun

Here's a mistake that's easy to make and expensive to fix: switching embedding models after you have data.

Vector similarity only works if all vectors were generated by the same model. Mix embeddings from nomic-embed-text andtext-embedding-3-small in the same index and your similarity scores are meaningless — you're comparing vectors in different geometric spaces.

We enforce model consistency at the migration level: the embedding model name is stored in a schema metadata table. Any attempt to write an embedding generated by a different model is rejected. Migrations that change the model trigger a full re-embedding job.

What We'd Do Differently

  • 1.
    Start with hybrid search from day one. We added BM25 full-text search later. Retrofitting it onto an existing schema is messier than designing for it upfront. Always include a tsvector column in your initial migrations.
  • 2.
    Track embedding model per-row, not just globally. During a migration, you'll have a mix of old and new embeddings. A global model setting doesn't handle the transition period well. A per-row model column does.
  • 3.
    Instrument vector search latency from the start. ef_search tuning is a data-driven exercise — you need latency percentiles and recall metrics to tune it. Add slow-query logging (>200ms) in v1 so you have a baseline when you need to optimise.

The Stack

  • Database: PostgreSQL 15 + pgvector, pgBouncer for connection pooling
  • Embeddings: Ollama (nomic-embed-text) for local; fallback to OpenAI text-embedding-3-small
  • Server: Node.js, TypeScript, Fastify
  • Protocol: MCP 2024-11-05 (Streamable HTTP transport)
  • Infra: Kubernetes, ArgoCD, Prometheus + Grafana, Falco
  • UI: Next.js 15, React, D3.js, react-force-graph-2d

Read next: ← Why Agent Memory Matters: The Context Problem in Multi-Agent Systems