Ask a parking operations manager how confident they are in the occupancy numbers coming off their sensor network, and most will hedge. Vendor sales sheets tend to advertise accuracy figures in the high nineties — 95 percent, 97 percent, sometimes higher — and those numbers get built into procurement decisions, budget justifications, and public dashboards without much independent verification. The sensor gets installed, the dashboard populates, and unless something goes visibly wrong, the accuracy claim mostly goes untested.

That’s what makes a report the City of Bellevue, Washington quietly published earlier this year worth a much closer look than it’s gotten outside a fairly narrow circle of curbside-technology specialists. Bellevue ran one of the more methodologically careful independent tests of camera-based curbside monitoring sensors done to date — five vendors, a shared test site, months of ground-truth video comparison, and a hard 95 percent accuracy target set in advance. The result: not one vendor met it. For anyone currently evaluating occupancy-detection technology, whether for on-street parking, loading zones, or garage guidance, the report is a useful corrective to how accuracy gets marketed versus how it actually performs in the field.

What Bellevue Actually Tested

Bellevue’s pilot grew out of its participation in Transportation for America’s Smart Cities Collaborative, which selected Bellevue as one of three cities — alongside Boston and Minneapolis — to run curbside management pilots focused on new monitoring technology. Bellevue’s piece of that effort became a formal technology evaluation: install multiple vendors’ camera-based sensors along a single, busy stretch of curb in its downtown core, and measure how accurately each system detected vehicle presence, absence, and dwell time against a human-verified ground truth.

The test site sat along 106th Avenue NE between NE 4th and NE 6th Streets — a corridor with on-street parking, commercial loading zones, an EV charging zone, passenger drop-off areas, and bus and shuttle stops, which is to say a realistic mix of the curb-use conflicts that make occupancy data valuable in the first place. Five vendors ultimately participated across three evaluation rounds run between November 2020 and May 2021: Automotus, the University of Washington’s Smart Transportation Application & Research (STAR) Lab, Cleverciti, Verizon, and Umojo. All five deployed camera-based systems, though the underlying approaches varied — some paired cameras with edge computing that processed video on-site, others streamed footage to cloud dashboards, and one added GPS-corner detection to sharpen vehicle boundaries.

Project staff reviewed the same video feeds manually to build a ground-truth dataset, then compared each vendor’s reported occupancy events against it using a confusion-matrix approach — the same basic statistical framework used to evaluate any binary detection system, whether it’s a spam filter or a diagnostic test. That produced standard performance measures: sensitivity, specificity, positive and negative predictive value, and overall accuracy, each checked against the city’s 95 percent target using formal statistical tests rather than eyeballing the numbers.

The Headline Result: No Vendor Cleared the Bar

Under the report’s most stringent matching criteria — requiring a vendor’s reported start and end times to fall within 30 seconds of the ground truth, at the correct stall — accuracy across the five systems ranged from roughly 5 to 79 percent. Loosen the matching criteria to a more forgiving zone-level standard, and the range shifted upward, to roughly 42 to 88 percent. Either way, every single vendor fell statistically short of the 95 percent goal, a gap the report’s authors confirmed with t-tests rather than leaving to interpretation.

That’s a wide spread, and it’s worth sitting with the shape of it rather than just the headline. Under the strict criteria, three of the five systems landed in the 60-to-80-percent accuracy range — good enough, the report noted, that they “may enable use of these systems for some applications,” but well short of the reliability a city would want for anything tied to enforcement or automated payment. The other two systems performed dramatically worse under strict matching, though their numbers improved substantially once the matching criteria relaxed — a pattern that points less at raw detection failure and more at a mismatch between how a given vendor’s system logs an event and how the ground-truth methodology defined one. That distinction matters enormously for anyone reading vendor accuracy claims: a system can look like it’s failing badly when the real issue is a timestamp or boundary-definition mismatch, not a blind camera.

Why Do Camera-Based Systems Struggle With Something This Basic?

Detecting whether a parking space is occupied sounds like it should be a solved problem by 2022 — cameras are cheap, computer vision has improved enormously, and plenty of demo videos show clean, confident detection boxes drawing around every vehicle in frame. The gap between demo conditions and a real downtown corridor over months of data collection is where accuracy erodes.

The Bellevue report and vendor-side technical material point to a consistent set of culprits. Obstruction is the biggest one: parked delivery vehicles, bike racks, overhanging trees, temporary construction barriers, and even other parked cars can block a camera’s clean line of sight to the space it’s supposed to monitor. Weather and lighting are close behind — rain, glare, and specifically sun reflection off wet pavement or vehicle glass were flagged by the project team as anecdotal factors degrading detection, even though the study wasn’t designed to isolate weather as a controlled variable. Camera angle and mounting height introduce their own error, since a sensor positioned to see one zone clearly may have a partially obstructed or foreshortened view of an adjacent one it’s also responsible for. And because most of these systems apply computer-vision models trained on some reference dataset, real-world edge cases — an unusual vehicle shape, a food truck, a trailer — can confuse detection in ways a lab benchmark never surfaces.

There’s also a data-engineering problem sitting underneath the detection problem. Several of Bellevue’s vendors could only deliver curbside data as offline CSV exports rather than a live API feed, and getting timestamps synchronized to a common clock across five independently operated systems turned out to be nontrivial. The report’s authors were explicit that in early rounds, they couldn’t always tell whether a low accuracy score reflected the camera actually missing a vehicle, or the matching algorithm failing to correctly pair a correctly detected event with its ground-truth counterpart because of a clock drift or an inconsistent data schema. That’s a mundane-sounding issue, but it’s exactly the kind of thing that doesn’t show up in a vendor’s own marketing accuracy figure, which is typically generated under controlled conditions the vendor itself designs.

Is This a Camera-Specific Problem, or a Sensor-Accuracy Problem in General?

It would be convenient to conclude that camera-based detection is simply the weak link and that in-ground or radar-based sensors solve the accuracy problem outright. The evidence for that is mixed at best. In-ground magnetometer sensors — pucks embedded in or bonded to the pavement that detect the magnetic disturbance a vehicle’s mass creates overhead — have a real structural advantage over cameras: they’re not affected by obstruction, weather, or lighting, because they aren’t looking at anything. Vendors in that category commonly advertise accuracy in the 95-to-98-percent range under standard conditions, and radar-based in-ground sensors, which some manufacturers position as more resistant to interference from adjacent traffic and metal debris than pure magnetometers, have performed well in head-to-head trade testing against older sensor generations.

But magnetometer accuracy isn’t universally that clean in practice, either. Vendors in this space have separately reported that independent field testing against certain electric vehicles — whose battery packs and drivetrain layouts differ from internal-combustion vehicles in ways that can alter the magnetic signature a sensor is calibrated to detect — has produced accuracy closer to 60 percent against an advertised 95-to-98-percent spec. That’s the mirror image of Bellevue’s camera finding: a sensing technology that performs well under the conditions it was validated against, and considerably worse against a real-world case its validation didn’t anticipate. The pattern across both technology families is the same lesson, not two different ones: advertised accuracy figures describe performance under specific test conditions, and any city procuring at scale should assume field performance will regress toward those conditions’ limits, not away from them.

Occupancy Accuracy and Duration Accuracy Are Not the Same Metric

One of the more practically useful findings buried in Bellevue’s report is that a vendor can perform reasonably well on one accuracy dimension and poorly on another, and a city that only asks about “accuracy” in the abstract won’t know which one it’s getting. The pilot measured two distinct things: whether a system correctly flagged that a space was occupied or vacant at all (the event-detection problem covered above), and, separately, how closely a system’s reported occupancy duration matched the ground truth once an event was correctly identified.

On duration, results were considerably stronger across the board. Under the most stringent matching criteria, every vendor’s average difference from ground-truth duration came in under a minute, and most systems correctly estimated duration within plus-or-minus 30 seconds for well over 90 percent of qualifying events. That’s a meaningfully different accuracy story than the sub-95-percent occupancy detection numbers, and it matters operationally: a system that struggles to reliably flag every occupancy event but estimates duration well once it does catch one is suited to different applications — trend analysis and average-dwell-time studies, say — than a system a city plans to use for real-time enforcement, where missing the event in the first place is the failure mode that counts.

What Should a City Do With This Right Now?

For a planner or parking operations lead currently in a vendor evaluation cycle, Bellevue’s findings don’t argue against occupancy-sensing technology — the pilot’s own conclusion was that several systems performed well enough to support “some applications,” not that the technology is unworkable. What the report argues for is a more skeptical, more structured procurement process.

Ask for accuracy figures broken down by matching criteria, not a single headline number. Bellevue’s results shifted by 20 to 40 percentage points depending on whether matching was scored at the strict stall-and-timestamp level or a looser zone level. A vendor quoting one number without specifying the matching methodology behind it isn’t giving a city enough information to compare against another vendor’s claim, let alone its own operational needs.

Separate the occupancy-detection accuracy question from the duration-accuracy question in any RFP. They measure different things and a system’s fitness for enforcement, wayfinding, dynamic pricing, or research use depends on which one matters more for that use case.

Run a ground-truth comparison before committing to full-scale deployment, even a short one. Bellevue’s own preliminary rounds caught data-formatting and clock-synchronization issues that would have quietly corrupted accuracy numbers if the city had gone straight to a production rollout without a verification step first.

Treat weather, obstruction, and edge-case vehicle types as procurement questions, not just engineering footnotes. Asking a vendor directly how their system performs in rain, glare, partial obstruction, or against vehicle types outside their core test set is a reasonable diligence step for either camera-based or in-ground sensor technology, given how consistently both categories show performance gaps tied to exactly those conditions.

None of this makes occupancy sensing a bad investment — the underlying case for digitized curb and parking data hasn’t weakened. It does mean that a 95-percent accuracy claim on a spec sheet is a starting point for questions, not a number a city should build a budget or an enforcement program around without independent verification.

Frequently Asked Questions

What did Bellevue’s curbside sensor pilot actually measure?

The pilot compared five camera-based curbside monitoring vendors against a human-verified ground truth, measuring both whether each system correctly detected when a parking or curb space was occupied versus vacant, and how accurately it estimated the duration of each occupancy event, at a test site in downtown Bellevue, Washington.

Did any vendor hit the city’s 95 percent accuracy target?

No. Under the report’s most stringent matching criteria, accuracy across the five vendors ranged from roughly 5 to 79 percent; under a looser zone-level standard it ranged from roughly 42 to 88 percent. Every system fell statistically short of the 95 percent goal the city had set in advance.

Are in-ground magnetometer sensors more accurate than cameras?

Under standard conditions, in-ground magnetometer and radar sensors are typically less affected by weather, obstruction, and lighting than cameras and commonly advertise accuracy in the 95-to-98-percent range. But field testing against atypical vehicle types has shown accuracy for some magnetometer systems dropping well below spec, so no single sensor category is immune to real-world performance gaps.

What should a city ask vendors before buying occupancy sensors?

Ask for accuracy figures broken down by matching methodology rather than a single headline number, request separate figures for occupancy-detection accuracy and duration accuracy, and ask directly how the system performs under obstruction, adverse weather, and against vehicle types outside the vendor’s core test set. A short independent ground-truth comparison before full deployment is worth the added time.

The City of Bellevue’s full Curbside Technology Performance Assessment Report, including its complete methodology and vendor-level results, is available from the City of Bellevue. For background on the Smart Cities Collaborative program that funded the pilot, see Transportation for America.