Enterprise RAG: permissions, freshness and index migrations
Design retrieval that respects changing permissions and document versions. A good answer delivered to the wrong person is a system failure.
- •Enforce tenant permissions before context construction.
- •Plan deletion propagation and a reversible embedding migration.
- •Estimate raw vector storage without hiding index overhead.
Sources & further reading
Original RAG paper: retrieval and generation components ↗vLLM prefix caching: distinguish KV reuse from answer caching ↗In-Depth Architecture & Mechanics
Interview brief
Design an assistant for 500 organizations sharing a search platform. A document can be edited or revoked while a conversation is in progress. The example numbers are interview assumptions, not measured service limits.
Keep authority outside the model
Resolve the user and organization at the gateway. Enforce authorization in retrieval and again before passing evidence to the generator. Use the source of truth for permissions, with a defined cache invalidation policy. The model cannot grant access, and a hidden citation is not a substitute for preventing unauthorized context from entering the prompt.
Build a versioned ingestion path
Represent document_id, source_version, tenant_id, ACL reference, content hash, parser version, embedding model and ingestion timestamp. Make ingestion idempotent on document and version. Publish an index generation only when its records and metadata are ready. Use tombstones for deletions and make query serving honor revocations before asynchronous compaction finishes.
Estimate the storage before picking a database
Assume 10 million chunks and 1,536-dimensional float32 embeddings. Raw vector storage is 10,000,000 × 1,536 × 4 = 61.44 GB, or about 57.22 GiB. This excludes graph/index overhead, text, metadata, replicas and backups. With three copies, vectors alone occupy 184.32 GB. Estimate the candidate-set effect of restrictive tenant filters and benchmark filtered recall and latency.
Change embeddings without mixing spaces
Create a new index generation, backfill deterministically, dual-write subsequent changes and compare shadow queries using the matching query encoder. Atomically switch the serving alias after quality, authorization and freshness checks. Preserve rollback data and propagate deletions to both generations. Vectors from unrelated embedding models are not comparable merely because their dimensions match.
Cache and failure drill
Key answer caches by tenant, authorization scope or policy version, query normalization and index generation. Revalidate access before returning cached evidence. If a permission service fails, choose an explicit fail-closed policy for private content. Measure deletion propagation, stale-answer rate and isolation tests as well as relevance.
Source changes → durable log → parse + version → embed → staged index Permission changes → authoritative policy → retrieval filter + cache invalidation Staged index → shadow evaluation → atomic serving alias