LYBO OS — OS-native AI interop

LYBO OS is the umbrella for running one governed agent across every OS-native AI stack. The runtime treats each platform model as a normal routing tier: it *leverages* the OS engine where it exists (zero download, NPU-fast, private), *augments* what the OS API lacks, and *degrades gracefully* to the built-in engine and deterministic floor when the surface is missing or denied by policy.

Register a surface

// Host probe result crosses the FFI as JSON:
let surface: OsAiSurface = serde_json::from_str(r#"{
  "api": "android_ml_kit_gen_ai",
  "available": true,
  "os_model_version": "nano-4",
  "features": ["streaming", "structured_output", "prefix_caching"]
}"#)?;
runtime.register_os_ai_surface(surface);
// Unavailable surfaces are ignored — report every probe unconditionally.

Three tiers, one nervous system

Sync is store-and-forward: structured deltas queue at each tier and trickle up when bandwidth allows (differential sync, redaction at every egress boundary). Offline is a first-class state, not an error — Tiers 1–2 keep answering with local knowledge when the WAN drops.

Multimodal: pixels and audio as governed citizens

Photos and voice get the same structural guarantees as text (runtime media/ module + the `@lyboai/media` npm package for JS hosts). Media crosses the bridge by reference (content-hash id + path, never base64); OS surfaces unlock vision/audio tasks via probed features (Gemini Nano 4 image input; Windows NPU OCR/speech/description; Apple OCR system tools).

• Scrub-on-ingest: JPEG EXIF (GPS, device serials) is stripped *before* hashing — an unscrubbed original never exists in the store; retention is policy-enforced.
• Grounded vision: extractions return claims with evidence + per-field confidence; numeric/text claims are cross-checked against an independent OCR pass — a transposed invoice amount escalates even at 0.97 confidence.
• The floor is a human: low confidence, unconfirmed critical claims or empty extractions → needs_human_review — for pixels there is no deterministic tier, so the system never guesses.
• Cross-modal RAG: the Node tier's media index answers "find photos like this corrosion" via caption ⊕ embedding hybrid search — with an honest empty result below threshold.
• Streaming voice: audio sessions with rolling partials, barge-in, and a bounded buffer that force-finalises instead of OOMing a field device.
• Mesh envelopes: raw media crossing a tier needs on-device consent (the hub can't mint it) + recorded redactions; raw-to-cloud is off by default and LocalOnly mode always wins.

Guardrails & observability (structural, not prompt-deep)

• Schema-first outputs — every structured task declares a JSON schema; results are validated and repaired in-core, even when the OS API has no native structured output (e.g. Phi Silica).
• Confidence gating — low-confidence generations fall through the cascade to the deterministic floor instead of guessing. The agent never invents an answer to keep the conversation going.
• Grounded knowledge — answers draw on signed pack knowledge and governed RAG, not open-ended recall.
• Human-in-the-loop consent — side-effecting tools (payments, messages, records) park for approval; the hub can never bypass on-device consent.
• Glass-box traces — every route decision (including skipped tiers and why), tool call and repair is logged with a verifiable trace chain (recent_traces / verify_trace_chain).
• Eval harness — packs pass an eval gate before signing; the router cascade itself is covered by the runtime's 100+ test suite.

Try it now: the [live LYBO OS demo](/lyboos/demo.html) routes one skill across all four runtimes, or open the native sample projects in examples/lyboos-* on GitHub.