From pixels to a managed presence signal

Two ribbon cables, two independent faults

Camera Module 3 Wide bring-up on September 26 initially failed because the Pi-side ribbon contact orientation was wrong, following incorrect guidance. With power removed, the connection was corrected. The IMX708 Wide then enumerated and produced a 4608 × 2592 image with usable exposure, color and focus. The loose mounting left the image upside down; electrical success had not solved positioning.

The AI HAT+ 26 TOPS was mechanically installed using its supplied short PCIe ribbon. On September 27 the Pi booted normally, but the external PCIe link was down and Hailo was absent. Updating software did not correct a reversed physical connection. Powering down and correcting that separate ribbon allowed Hailo-8 enumeration and the device node to appear.

The camera’s Standard–Mini ribbon and the HAT’s PCIe ribbon serve different interfaces. Treating “the ribbon cable” as one generic item obscures the fault boundary. A camera regression check after the accelerator work confirmed that the camera still captured successfully.

The inference path and its units

The exercised live path used the IMX708 Wide camera, PiSP processing, a 640 × 640 BGR888 inference stream and Hailo YOLOv8 object detection. The main image used for bounding-box coordinates was 2304 × 1296. These are different coordinate spaces; geometry must use the dimensions belonging to the reported box.

A model benchmark processed 1,549 frames at 309.35 frames per second. That number describes the accelerator/model test. It is not the end-to-end camera rate and does not include every stage needed for a responsive companion.

A separate roughly ten-second 1280 × 720 H.264 recording contained 292 frames. Together these checks exercised useful parts of the stack, while still leaving latency and recognition accuracy as separate questions.

Throughput did not equal recognition quality

Person detection was useful enough to develop the presence gate. Broader object labels were unreliable: a vertical bottle was labeled as a surfboard, and a horizontal bottle was labeled as an airplane in 287 of 291 frames. Zero dropped frames would not make those labels correct.

The AI still-capture path also remained unqualified. Switching into a 1920 × 1080 still configuration removed the low-resolution stream and led to a HailoRT readiness failure. Continuous live/video behavior did not prove that a mode switch was safe. The record kept that failed path open rather than generalizing the successful video result.

Why presence must be evaluated once per frame

The first presence gate used a 0.50 confidence threshold, five confirmation frames and ten missing frames. Its bug was structural: it updated the person streak while walking individual object detections. An unrelated object could reset the streak even when the same frame contained a person.

The correction aggregated the entire frame first, retained the best qualifying person box and updated the gate once at the frame boundary. This separates object enumeration from temporal state. The following pseudocode illustrates that decision boundary; it is not a downloadable production implementation.

for each completed frame:
    person = best qualifying person across all detections
    if person exists:
        consecutive_present += 1
        consecutive_absent = 0
    else:
        consecutive_absent += 1
        consecutive_present = 0
    evaluate confirmation / loss transition once

Time, geometry and camera placement

Sixty missing frames is about two seconds only at 30 frames per second. A frame-count threshold is not a fixed wall-clock guarantee. A proposed 0.50/0.40 dual-threshold scheme was not adopted at this checkpoint.

Horizontal position work used the box center, x + width/2, normalized against the 2304-pixel main-image width. Normalizing against the 640-pixel inference width would give incorrect positions. Very close boxes were large and unstable; moving farther away, around six feet, improved the observations. Left/center/right calibration remained incomplete.

A camera tilted upward could exclude a reclining user. That is a field-of-view problem before it is a confidence-threshold problem. Physical placement needed adjustment before adding software tolerance.

Turning the experiment into a managed service

The perception process was first exercised manually, then placed under systemd with restart-on-failure and a five-second restart delay. Stop checks looked for orphaned processes and confirmed the camera could reopen. A restart produced a fresh confirmation event.

Boot enablement followed those checks. The September 27 boot observation recorded a first event around two seconds after service activation, no restarts during the short observed interval and a 42.8 °C temperature. These are bounded commissioning observations, not an endurance claim.

The health checks covered syntax, devices, camera availability, temperature and throttling. They did not establish a continuous fresh-frame heartbeat. CFG-002 preserved the checkpoint; file-copy backups were not a proven full-system restore.