--- title: "Private GPU Use Cases — What People Actually Run | Bit Refinery" url: "https://bitrefinery.com/private-gpu-use-cases" description: "Real workloads running on Bit Refinery private GPU infrastructure: the models, the hardware, the measured latency, and why each one left the per-token API behind." lastmod: "2026-08-24" source: "auto-generated from SSG HTML" --- USE CASES # What people actually run on a private GPU Not benchmarks. Production workloads, with the model named, the card named, and the latency measured. Most writing about self-hosted inference argues from a spreadsheet. These are applications running on hardware in our racks right now. Each page says what the workload is, why it is not sitting on a per-token API, what it costs to serve, and — where the work is genuinely unfinished — what has not been proven yet. The common thread ## Four reasons a workload leaves the API No workload moves to dedicated hardware for one reason alone, but in practice it is some combination of these four. Every case study below is an instance of at least two. ### The data is not allowed to leave Some inputs disqualify a third-party API before the pricing conversation starts. Private message content, patient records, unreleased designs, anything under a contractual no-disclosure clause. A promise that a vendor will not retain your prompts is a contract. A model on a card you rent alone is a fact. ### The load is steady, not bursty Per-token pricing is a good deal for spiky, low-volume work and a bad one for a job that runs the same shape of request all day. Once a workload has a floor, a dedicated card turns a variable bill into a fixed one — and the arithmetic usually crosses over sooner than people expect. ### The model needs to already be warm Serverless GPU platforms load weights on demand, so the first request after a quiet period pays for the load. When a model stays resident on hardware nobody else is scheduling onto, there is no cold start to design around and no queue you do not control. ### The weights need to stay put An open-weights model on your own hardware does not get deprecated, silently re-tuned, or repriced. For anything with a published evaluation or a regulated audit trail, reproducing last quarter's output matters more than access to the newest frontier model. Case studies ## The workloads LIVE IN PRODUCTIONConsumer security software ### Haven Personal data security that never ships your messages to a vendor Haven scans a person’s own cloud accounts and devices for sensitive data they have forgotten about — a tax return in a chat thread, a password sent in plain text. Almost all of that classification runs on the user’s own machine. The small fraction the local model cannot decide runs on a GPU in Bit Refinery’s rack. Model Gemma 4 31B, FP8 Serving vLLM in a rootless container Hardware One 48 GB NVIDIA card [Read the Haven case study](https://bitrefinery.com/private-gpu-use-cases/haven) DEMO LIVE · EVALUATION IN PROGRESSElectric utilities · vegetation management ### LimbWatch Vegetation hazard triage from a phone photo, on one GPU Photograph a tree near a power line and get back a structured work ticket: severity score, hazard type, conductor type, species, tree health, and the pole tag number read off the tag. An open-weights vision model does the reading, served from a single card. The demo is live; the fine-tune and the published accuracy table are not done yet. Model Gemma 4 12B (Apache 2.0) Serving vLLM with guided decoding Hardware One RTX 6000 Ada 48 GB [Read the LimbWatch case study](https://bitrefinery.com/private-gpu-use-cases/limbwatch) More case studies land here as workloads move into production and their operators agree to be named. The shape of it ## One card, one resident model, a flat bill Both workloads on this page have the same architecture, and it is an unglamorous one. A single 48 GB card. One open-weights model loaded once and left there. An inference server in front of it, TLS at the edge, and no scheduler deciding to evict anything. That is the entire trick. 48 GB One card is enough for a 12B vision model or a 31B text model in FP8 0 Cold starts. Weights are resident, so the first request of the day costs the same as the thousandth Flat Monthly cost that does not move when usage doubles, and no egress metering on responses ## Where does your workload sit? If your inference load has a floor, or your inputs are the kind you would rather not hand to a vendor, the arithmetic is worth doing. We will do it with you, honestly, including the cases where an API is still the right answer.