Multi-Tenant AI Needs Isolation You Can Prove
Telling the model to only use one customer's data is an instruction, not a guarantee. Isolation has to live where the data lives.
In short
5 things that decide this
- 01In a multi-tenant AI product, one customer's assistant answering from another customer's records is not a bug to patch. It means isolation was never built into the architecture.
- 02A system prompt that says 'only use this customer's data' is an instruction to the model, not a control the system enforces. Models follow instructions unevenly under load and under attack.
- 03Isolation belongs in the data layer: every retrieval query scoped to a tenant id before the model ever sees a result, not after.
- 04Go4Gr8's coaching platform and TankAware's site-management platform both run this way — tenant scoping and role tiers built into the query path, not the prompt.
- 05The test for whether isolation is real: try to retrieve another tenant's data through the assistant and confirm the query returns nothing, not that the model declines to repeat it.
A prompt is not the access control
Here's a common shortcut in early multi-tenant AI builds. You put every tenant's data in one shared index, then tell the model in the system prompt to only answer from the current customer's records. It works in the demo. Every test question comes from the tenant the demo is showing.
It stops working the moment retrieval pulls in a passage from a different tenant. Now the model has to refuse it, every time, under phrasing you never tested. A prompt is a request you're making, not a rule you're enforcing. Your database query with no tenant filter returns whatever matches, and the model reads whatever the query returns.
In a single-tenant product, a data leak is a bug. Ship a multi-tenant product, and it's a breach of contract with every customer whose data got exposed. That difference changes where your fix has to sit.
Scope the query before the model sees a result
The fix isn't a smarter prompt. Tag every record with its tenant id at the point it enters your system, then filter on that id before retrieval runs, not after. Do this, and your model never receives a candidate result from outside the tenant, so there's nothing for it to leak.
This is the same discipline as row-level security in a normal application. Your app already scopes a SQL query to the logged-in account. Give your retrieval pipeline the same scoping, applied at the query, not left to the model to police in the output.
Permission tiers add a second axis on top of tenant scoping. A tenant isn't one flat account. Take Go4Gr8's coaching platform: an organization admin, a team lead and an individual user each need a different slice of the data. Their AI agents have to respect that split too. Building the organization and role model was the real engineering work behind the platform's onboarding. Its chat interface just sits on top.
TankAware runs a sharper version of the same pattern. Each client company operates in its own environment under a dedicated subdomain, with five access tiers from Super Admin down to Contractor. A contractor at one petroleum site can't pull inspection data from a site they aren't assigned to. Build yours this way, and that boundary sits in the role and tenant model, not in whether the AI declines to mention it.
- 01Tag every record with its tenant id at ingestion, not at query time.
- 02Filter retrieval before the model sees results, so there is nothing to leak.
- 03Add role tiers on top of tenant scoping where users inside one account need different access.
Prove it by trying to break it
A prompt-level fix looks the same as a data-layer fix in a normal test. Ask your assistant about another tenant's data, and it says it can't help. Checking what actually happened underneath the refusal is the only way to tell them apart.
With a real data-layer boundary, your query never returned another tenant's rows. Nothing reached the model to refuse. With a prompt-only boundary, the query returned the data and the model chose not to repeat it that time. That second version has no floor: under a different phrasing, a longer context, or a model update, the same request can succeed.
Test for the difference directly. Log what your retrieval step actually returned, and compare it to what the model said back. Say a cross-tenant record shows up in the retrieved set even once, alongside a polite refusal. Your isolation lives in the wrong layer, and the next request isn't guaranteed to refuse.
- IngestionEvery record tagged with its tenant id as it enters the system.
- RetrievalQuery filtered by tenant id before the model sees a candidate result.
- ModelSees only what already passed the filter. Nothing left to leak.
- PromptAn instruction on top, never the boundary itself.
Move the boundary left of the model, and there's nothing for a prompt to police.
Some of the systems we have shipped
Questions this raises
01How do you isolate tenant data in a multi-tenant LLM application?+
Tag every record with its tenant id when it enters the system, and filter retrieval by that id before the model sees any result. Your model should never receive a candidate answer from outside the requesting tenant. A system prompt telling the model to ignore other tenants' data isn't a substitute. It depends on the model following the instruction every time, under every phrasing.
02Is a system prompt enough to keep customer data separate in a SaaS AI product?+
No. A system prompt is an instruction the model can follow inconsistently, especially under adversarial phrasing or after a model update changes its behavior. Real isolation happens in your retrieval query, scoped by tenant id, so there's no cross-tenant data left for the model to see or repeat.
03What is per-customer data separation in an AI SaaS product?+
It means each customer's data is tagged and filtered at the database or vector-index level. One account's queries can never retrieve another account's records. Role tiers add a further split inside one tenant, so different users at the same customer see only what their role allows.
Related
- AI, automation and custom software for property managers →Multi-tenant systems built for property, with this isolation pattern in mind.
- SaaS MVP development →Where tenant and role modeling gets built in from the first release.
- Retrieval-augmented generation →The retrieval mechanism this isolation has to wrap around.
- How to add RAG to an existing SaaS product →A retrofit path, including the filtering trap in approximate vector search.
