Skip to content

0033 — Pluggable face detector (UltraFace | YuNet)

  • Status: IMPLEMENTED (2026-07-24, MR !99). faceDetector seam + YuNet shipped and defaulted; validated by a full re-cull of the Lyndsey-Denis shoot — 718 faces (+13% vs UltraFace's 636) at the shared 0.70 confidence floor (a modest, genuine gain, not over-detection), verdicts keep 77 / maybe 377 / reject 102 (vs 84/370/102 — the usual safe keep→maybe drift, reject unchanged). Keypoints are available for 0032 Stage 3.
  • Owner: —
  • Realises: the 0032 §8 investigation — adopt YuNet (MIT, box + 5 keypoints, strong on small/side faces) as a selectable face detector, defaulting to it, behind a small interface so UltraFace and YuNet can be A/B-compared over time. The keypoints also unblock the 0032 Stage-3 rotation path (roll from the eye landmarks) without a licence-encumbered model.
  • Depends on: 0004 (face detect/crop), 0013 (ONNX runtime), 0031 (model hosting — YuNet rides a future krites-models release; cache-resolved from its MIT source until then).

1. Why

The detector is currently hard-wired to UltraFace RFB-320 (boxes only, no keypoints, tight portrait boxes that the FaceMesh dislikes — 0032). The 0032 §8 spike measured YuNet as a clear, licence-clean upgrade: MIT weights (Shiqi Yu), box + score + 5 keypoints (both eyes, nose, mouth corners), better coverage incl. small/side faces, and a decode we validated to the exact pixel against OpenCV's reference. Making the detector pluggable (rather than a wholesale swap) lets us default to YuNet while keeping UltraFace one config flip away — the general principle that the more backends are pluggable, the more we can measure and refine over time (mirrors the FaceAnalyzer / Inpainter / strategy seams, 0001 §5).

2. Design

  • faceDetector interface in pkg/face/onnx: detect(ctx, img) ([]detection, error) + close(). detection gains an optional keypoints []point (nil for UltraFace; the 5 YuNet landmarks otherwise) — carried through NMS.
  • Two implementations: the existing UltraFace detector, and a new yunetDetector (pkg/face/onnx/yunet.go).
  • Config: face.detector = yunet (default) | ultraface, via a new onnx.Detector type + Config.Detector (defaulted in withDefaults). Selected in analyzer.openDetector.
  • Model resolution: faceprovider.resolveModels picks the detector model by face.detector (models.YuNet vs models.UltraFace); face.detector_model still overrides. models.YuNet is a plain checksum-pinned Model (MIT source URL, SHA 8f2383e4…), like UltraFace, until it rides a signed krites-models release (0031).

YuNet specifics (validated in the spike)

  • Input [1,3,640,640] float, BGR, 0–255, no mean/scale (OpenCV swapRB=false). The image is resized to 640², detections scaled back to image coords.
  • Anchor-free decode over strides 8/16/32 (grids 80²/40²/20², one prior per cell):
  • score = sqrt(cls · obj)
  • cx = (col + bboxΔ.x)·s, cy = (row + bboxΔ.y)·s, w = exp(bboxΔ.w)·s, h = exp(bboxΔ.h)·s
  • keypoint k: x = (col + kpsΔ.x_k)·s, y = (row + kpsΔ.y_k)·s
  • row-major cell index; confidence floor then the shared nms.

3. Constraints

  • cgo-free / WASM line unchanged — the detector lives in the native pkg/face/onnx adapter (already behind ORT/purego), not the deterministic engine.
  • Non-destructive. The detector choice changes detection boxes → analysis values, so switching is a re-cull (AnalysisVersion bump). Keypoints are not persisted yet (they feed future rotation, in-memory).
  • Not wired to MCP/studio settings in this spec — config-key + default only; a studio toggle can follow.

4. Tests

  • Unit (yunet_test.go, pure): the decode on synthetic per-stride tensors → known boxes + keypoints; the config default resolves to YuNet.
  • INT_TEST (yunet_integration_test.go, env-gated): the real model on a committed face fixture detects the face with a plausible box + 5 keypoints (eyes near the expected positions).
  • just ci + just wasm-check green.

5. Success

A face.detector: yunet default that runs end-to-end (detect → mesh) with detection parity/upside vs UltraFace on the shoot, one config flip back to ultraface, and the keypoints available for the 0032 Stage-3 rotation work.