Haven
Personal data security that never ships your messages to a vendor
havenprotect.app- Model
- Gemma 4 31B, FP8
- Serving
- vLLM in a rootless container
- Hardware
- One 48 GB NVIDIA card
- Measured latency
- 4.0 s average
- Cold starts
- None — model stays resident
The scanner has to read the most private thing a person owns
Haven is a desktop application that finds sensitive data a person has forgotten they exposed. A tax return attached to a chat thread four years ago. A password typed in plain text to a colleague. A photograph of a passport sent to a landlord. The premise only works if the software can look at message content, because that is where the exposure is.
That creates an uncomfortable requirement. To tell a user that their own messages are safer with Haven installed, the product has to be able to say exactly where every message went — and “we send it to a large AI provider and they have promised not to keep it” is not an answer that survives the question.
Three tiers, and only the third one leaves the device
Haven’s answer is not to avoid inference. It is to make the off-device portion small, bounded, visible, and served from hardware the company controls end to end.
Deterministic pattern matching. Card numbers, key formats, national ID shapes — anything a well-written rule can settle without a model. Runs on every message.
A small bundled classifier, shipped as signed ONNX weights and updated over its own release channel. Also runs on every message. Between them, Tiers 1 and 2 resolve the overwhelming majority of traffic without a single byte leaving the device.
The tiebreaker. When the local classifier cannot reach a confident verdict — roughly 0.8% of messages in measured real inboxes — a bounded excerpt of at most 2 KB goes out in a single round trip, gets a verdict, and is discarded. Every one of those calls is visible to the user in the application’s own network activity log.
A retention policy is a promise. A private GPU is a property of the system.
Every major inference provider will tell you they do not train on API traffic and do not retain it beyond an abuse-monitoring window. Those statements are almost certainly true. They are also unverifiable from the outside, revocable by a terms-of-service update, and subject to a subpoena served on someone other than you.
Running an open-weights model on a dedicated card changes the category of the claim. There is no third party in the request path, so there is no third-party retention to promise about. The guarantee stops being a policy Haven asks users to trust and becomes a fact about where the bytes went.
This is also why the carve-out is stated plainly rather than buried. Haven does not claim it never sees content — it claims that a bounded excerpt of a small minority of messages is evaluated on infrastructure it operates, and it shows the user each time that happens.
The data being classified is private message content, so a third-party inference API is disqualified on principle rather than on price. Owning the box is what makes the no-retention promise enforceable instead of contractual.
What the deployment actually is
Gemma 4 31B in FP8, served by vLLM inside a rootless container, on a single 48 GB card. A reverse proxy in front handles admission control and TLS termination, and its request logging is content-stripped by configuration rather than by convention. The model is loaded once and stays loaded.
4.0 s
Average request latency across 654 lifetime requests
~1.4 K
Prompt tokens per Tier 3 call, by design — excerpts are bounded
~0.8%
Of messages that reach Tier 3 at all
The 48 GB budget turned out to have room in it. The server was originally configured to claim 92% of the card, most of that a pre-allocated key-value cache far larger than the real prompt sizes needed. Re-tuning that budget downward freed enough memory to bring a second, smaller model onto the same card without measurably moving the first one’s latency — which is a fair description of what a dedicated card buys you. On shared or serverless capacity that tuning decision is not yours to make.
At this volume, an API would be cheaper. That is not the point.
Haven is in alpha. Tier 3 currently sees on the order of 53 requests a day. At roughly 1.4 K prompt tokens each, a commodity inference API would bill that at a rounding error, and it would be dishonest to present this deployment as a cost optimisation. It is not one yet.
What the dedicated card buys today is the architectural guarantee, and it buys it at a price that does not move. The interesting property is what happens next: Tier 3 volume scales with installed seats, so a workload that is a rounding error at alpha becomes a real monthly line item at ten thousand users and a serious one at a hundred thousand. The card is already paid for at all three volumes. That is the trade — you pay early for capacity you have not filled, in exchange for a bill that stops being a function of growth.
For a workload with a genuine floor, that crossover is the whole analysis. For a workload with privacy constraints, it is a bonus on top of a decision that was already made.
- Haven is a pre-release product. These are alpha-stage numbers from a small user base, not a capacity study.
- Fifty-three requests a day does not exercise a 48 GB card. The latency figures say the deployment is healthy; they say nothing about its ceiling.
- The privacy argument here is about infrastructure topology. It is not a security audit of the application, and it does not substitute for one.
Have a workload shaped like this one?
A resident open-weights model on a card you don't share is a different cost and privacy story than a per-token API. Tell us what you're running and we'll size it.