LimbWatch
Vegetation hazard triage from a phone photo, on one GPU
Try the live demo- Model
- Gemma 4 12B (Apache 2.0)
- Serving
- vLLM with guided decoding
- Hardware
- One RTX 6000 Ada 48 GB
- Measured latency
- 15.4 s p95, single request
- Schema validity
- 100% over 18 runs
The demo is live and you can put your own photograph through it. The accuracy work is not finished. There is no published evaluation table yet, because the labelled photo set and the fine-tune that the table would compare against are still in progress. Everything below is either a measured performance number or an honest statement about what has not been measured.
A photograph in, a work ticket out
Utility vegetation management is a triage problem at enormous scale. Somebody has to look at a tree growing toward a conductor and decide whether it is a routine trim, a scheduled job, or a truck dispatched today. That judgement is currently made by trained inspectors walking circuits, and the bottleneck is how many circuits a person can walk.
LimbWatch takes two to four phone photographs and returns a structured ticket: a severity score on a published five-level rubric, hazard type, conductor type, a species guess, tree health, the pole tag number read off the tag, access notes, and a recommended action. The output is schema-constrained, so it drops into a work queue rather than into a paragraph somebody has to re-read.
The severity rubric is versioned and published alongside the tool, generated from the model’s own system prompt so the documentation and the behaviour cannot drift apart. It is one person’s rubric, deliberately written down so it can be argued with.
Five levels, published so they can be argued with
A severity score is only worth something if the reader can see what it means. This is the full scale the model is scored against, reproduced from the versioned rubric rather than summarised. A vegetation manager who disagrees with where a level boundary sits is giving useful information, and they can only do that if the boundaries are visible.
- 1No action needed
Clear of all conductors by more than 10 ft. Healthy, no lean, no deadwood over the span.
- 2Monitor
Growth trending toward the line, 3 to 10 ft clearance. Attention within two to three growing seasons.
- 3Schedule routine trim
Under 3 ft and closing, or a healthy limb over the span with obvious growth direction. No contact.
- 4Priority trim
Contact likely within one growing season. Deadwood over the span, a lean toward the conductor, or a declining tree in range.
- 5Immediate
In contact with a conductor, resting on the line, or a failing tree within striking distance of primary conductors.
Modifiers push a score up a level (primary distribution rather than a service drop; a species with a known failure habit near lines; visible ice or storm damage; a span crossing a road or a structure) or down one (an insulated service drop to a single house; an ornamental that cannot reach the conductor at maturity).
What one card does with a 12B vision model
These are performance numbers against the base model, before any fine-tuning. They describe how the deployment behaves, not how accurate it is — those are different claims and only one of them has been tested.
| Analysis latency, single request | ~15 s (p95 15.4 s over 18 runs) |
|---|---|
| Four concurrent requests | All complete, 16.6 s wall clock |
| Schema validity | 100% over 18 runs |
| Output tokens per ticket | 409–481 against a 2048 ceiling |
| VRAM | 42.9 GB of 49 GB, resident, no cold starts |
The design target was under 20 seconds at p95, and the deployment clears it. The more interesting line is the concurrency one: four simultaneous requests finished 1.1 seconds later than a single request did. At this scale, on a card nothing else is scheduled onto, concurrency is effectively free — the model is already resident and the batch simply fills capacity that was otherwise idle.
That is the property serverless GPU platforms cannot sell you. With weights loaded on demand, the first photograph after a quiet afternoon pays the load cost, and a field inspector standing under a tree experiences that as the application being broken.
There is no accuracy table yet, and inventing one would defeat the purpose
The point of the project is to show that a fine-tuned open model on one card measurably beats the base model, and to publish both numbers. Producing that table requires several hundred labelled photographs, a QLoRA run, and an evaluation harness scoring both models against the same held-out set. The photographs are being collected. None of the rest has happened.
A plausible-looking accuracy figure could be written into this page in about a minute, and it would poison the single asset the project exists to produce. So the space stays empty until the harness fills it.
Where it gets things wrong
Documented from building it. Anyone evaluating a vision model for field work should read a list like this before they read a benchmark, because these are the errors that survive a good average score.
Communications wires read as power
The lowest wires on a pole are usually telephone, cable, or fibre — not power. A model that scores clearance against a fibre drop is confidently wrong in a way that looks perfectly reasonable on screen. The prompt calls this out explicitly. How well that holds is an open question until the evaluation runs.
Depth from a single photograph
Distinguishing "under 3 ft" from "3 to 10 ft" is a depth judgement made from a 2D image with no scale reference in frame. This is expected to be the weakest field in the output, and saying so is more useful than a confidence score that hides it.
Pole tag OCR on weathered tags
Legible tags read well. Weathered, angled, or distant tags do not — and the model has a documented habit of producing a plausible number rather than admitting it cannot read one. The prompt pushes hard toward returning null. A dedicated OCR pass on a cropped region may beat fine-tuning here.
Species identification at distance
Bark and leaf detail carry species identification, and neither survives a wide shot. Low confidence is the correct answer more often than the model currently gives it.
One card covers serving and the fine-tune that has not happened yet
A 48 GB card serves this model in bf16 with room to spare, and a quantised deployment fits in 24 GB. The same class of card also runs the QLoRA fine-tune in a few hours per attempt — which matters, because fine-tuning is not one run, it is a dozen, and each one is an experiment you would rather not be metered on.
The model server accepts no connections from the internet. It binds an internal interface, and the public web tier reaches it over an encrypted tunnel. The web tier itself is a small VM that does almost nothing — an important detail for anyone sizing a similar build, since the temptation is to over-provision the part of the stack that is not the bottleneck.
No visitor photograph submitted to the public demo is retained. That is a property of where the thing runs, not a paragraph in a privacy policy.
A vision model reading every submitted photo is a steady, predictable load rather than a bursty one, which is exactly the shape that per-image API pricing punishes and a resident model on owned hardware handles for a flat monthly cost.
Have a workload shaped like this one?
A resident open-weights model on a card you don't share is a different cost and privacy story than a per-token API. Tell us what you're running and we'll size it.