cto-legends · module specification · studio · v0.1.1
legends-ambient-intelligence: Passive Audio Capture, SHA-256 USB Recorder Ingestion, Offline ASR & Vault Distillation CLI
Passive audio pipeline CLI for SHA-256 deduplicated USB recorder ingest, offline Whisper/NeMo Parakeet ASR, rollover recovery, and vault distillation.
GitHub Repository (https://github.com/avalonreset/legends-ambient-intelligence) · Handoff: cto-legends handoff legends-ambient-intelligence
legends-ambient-intelligence (executable CLI `legends-ambient`, Python module `legends_ambient`) is the passive audio ingestion, offline transcription, and knowledge-distillation pipeline for the CTO Legends ecosystem. While `legends-hyperyap` handles real-time push-to-talk dictation at the keyboard, `legends-ambient-intelligence` turns voice memos, local microphone directories, and dedicated portable USB hardware voice recorders (such as the `KT5000` dictaphone) into content-addressed archives and source-cited Obsidian vault notes for `legends-empire`. It discovers audio files across local directories or mounted removable volumes (`RECORDED` subfolder), computes SHA-256 digests against all historical `**/manifest.json` records in `~/.legends-ambient/archive`, and copies only new recordings into dated `YYYY/MM/DD/<slug>/raw/` packages without mutating or deleting source media.
After verifying post-copy SHA-256 integrity and writing both `manifest.json` and `SHA256SUMS.txt` (rolling back the entire ingest directory on any checksum mismatch), the CLI can cleanly dismount removable USB media via OS-native volume ejection (`Win32_Volume.Dismount` on Windows, `diskutil eject` on macOS, `umount` on Linux). Its pluggable offline ASR engine (`transcribe`) runs local `faster-whisper` (with VAD filtering and word timestamps), the `whisper` CLI, or GPU-accelerated NVIDIA NeMo Parakeet (`nvidia/parakeet-tdt-0.6b-v3`), generating five immutable sidecar artifacts per recording (`.txt`, `.json`, `.srt`, `.vtt`, `.words.json`) plus a Markdown `index.md`. To solve hardware dictaphone file-split rollovers, `recover` analyzes timestamp gaps (`<= 2.0s`) and mid-thought continuation conjunctions (`and`, `but`, `because`, `then`, `also`, `which`, `that`, `or`, `nor`, `yet`) to stitch predecessor and latest audio tails, while `distill` extracts action items, decisions, and themes into frontmatter-tagged Obsidian Markdown notes.
Deterministic Execution Boundary
Why Raw Frontier LLM Prompts Cannot Replace Deterministic Execution
A raw Frontier LLM cannot mount or dismount a `KT5000` USB voice recorder volume via `Win32_Volume`, compute SHA-256 checksums across gigabytes of historical WAV/MP3 files to prevent duplicate ingestion, run offline `faster-whisper` or CUDA NeMo Parakeet models locally, or inspect filesystem timestamps to detect a 15-minute hardware recorder rollover split. Uploading raw multi-hour ambient recordings to a cloud chat window also violates offline privacy boundaries, hits file-size limits, and discards word-level `.words.json`, `.srt`, and `.vtt` timing sidecars.
- Content-Addressable SHA-256 Deduplication & Atomic Rollback: `Archiver.archive()` scans every existing `**/manifest.json` in the archive root, filters incoming recordings by uppercase SHA-256 hash, re-hashes each copied file in `YYYY/MM/DD/<slug>/raw/`, and deletes the staging directory if any byte fails verification.
- Non-Destructive USB Recorder Discovery & Safe Volume Eject: `DeviceSource` locates mounted removable recorders by filesystem label (default `KT5000`), scans `RECORDED/` non-destructively (`source_mutated: false`), and executes OS-level volume dismount (`Win32_Volume.Dismount`, `diskutil eject`, or `umount`) when `--eject` is passed.
- Five-Artifact Offline ASR Sidecar Generation: `legends-ambient transcribe` runs local `faster-whisper` (`word_timestamps=True`, `vad_filter=True`), `whisper` CLI, or NeMo `ParakeetTranscriber` offline, deterministically emitting `.txt`, `.json`, `.srt`, `.vtt`, `.words.json`, and a linked `transcripts/index.md` table.
- Split-File Rollover & Mid-Thought Continuation Recovery: Portable recorders split long monologues at fixed 15-minute intervals. `analyze_prompt_recovery` checks whether the gap between predecessor `last_modified` and latest `recording_start` is `<= 2.0s` or if the latest transcript begins with a continuation regex (`^(and|but|because|then|...)\b`), planning a stitched `predecessor-tail` + `latest-tail` recovery clip.
CLI Subcommands & Capability Surface
legends-ambient doctor [--device-label KT5000] [--json] | status — Offline Environment & Hardware Readiness Diagnostics: Runs six offline readiness checks: Python 3.11+ runtime verification, archive root write permissions, `ffmpeg` and `ffprobe` availability on `PATH`, `faster-whisper` or `whisper` CLI installation, NVIDIA NeMo + CUDA GPU availability for Parakeet ASR, and removable USB volume mount detection. `status` reports the active archive root, count of known SHA-256 hashes, and mounted device path.legends-ambient ingest (--source <dir> | --device) [--device-label KT5000] [--slug <slug>] [--what-if] [--eject] — SHA-256 Deduplicated Local & USB Recorder Ingestion: Discovers audio files from a local folder or a mounted USB voice recorder (`KT5000`), compares file SHA-256 digests against all historical `manifest.json` records, and previews (`--what-if`) or copies only unarchived recordings into `<archive-root>/YYYY/MM/DD/<slug>/raw/`. Emits `manifest.json`, `SHA256SUMS.txt`, and a JSON `IngestReceipt` (`source_mutated: false`), optionally dismounting the USB volume cleanly with `--eject`.legends-ambient transcribe <target> [--backend whisper|parakeet|mock] [--model <name>] — Pluggable Offline Whisper & NeMo Parakeet Transcription: Transcribes an ingested session directory or individual audio file completely offline using `faster-whisper` (with VAD filtering and word-level probabilities), the local `whisper` CLI, or NVIDIA NeMo `ParakeetTranscriber` (`nvidia/parakeet-tdt-0.6b-v3` on CUDA). Writes `.txt`, `.json`, `.srt`, `.vtt`, `.words.json`, and `index.md` into the `transcripts/` subdirectory.legends-ambient recover (--source <dir> | --device) [--transcript <file>] [--latest-tail 15] [--predecessor-tail 5] — Rollover Split & Prompt Continuation Recovery: Detects when a spoken prompt or monologue was cut across two files by a hardware recorder's rollover limit. Evaluates both timestamp proximity (`boundary_gap_seconds <= 2.0`) and opening continuation cues (`and`, `but`, `because`, `then`, `also`, `which`, `that`, `or`, `nor`, `yet`) to output a JSON `PromptRecoveryPlan` (`include-predecessor-tail` vs. `latest-only`) with ordered `RecoveryClip` segments.legends-ambient distill <transcript.json|.txt> [--vault <dir>] [--slug <name>] — Provenance-Linked Obsidian & Empire Vault Distillation: Parses transcript text or JSON metadata to extract action items (matching `need to`, `should`, `must`, `remember to`, `todo`), decision takeaways (`decided`, `going with`, `approved`), and top non-stopword themes. Formats an Obsidian-compatible Markdown note with `type: ambient-distill` YAML frontmatter, task checkboxes (`- [ ]`), and full audio/ASR provenance (`source_audio`, `duration_seconds`, `language`, `model`).
Raw Frontier LLM vs. legends-ambient-intelligence Execution Matrix
- Ingestion & Media Safety: Raw LLM (Cannot read block volumes, track historical file hashes across sessions, or safely unmount removable USB hardware.) vs. Module (Non-destructive copy (`source_mutated: false`), global SHA-256 deduplication across `**/manifest.json`, post-copy hash verification, and OS volume dismount.) — `legends-ambient ingest --what-if` previews `new_archived` vs. `already_archived`; `SHA256SUMS.txt` and `manifest.json` record verified digests.
- Offline ASR & Timing Sidecars: Raw LLM (Requires uploading private voice memos to cloud APIs and returns unstructured prose without word-level timing or subtitle files.) vs. Module (Runs local `faster-whisper` or CUDA NeMo Parakeet offline and writes 5 deterministic artifacts (`.txt`, `.json`, `.srt`, `.vtt`, `.words.json`) plus `index.md`.) — `legends-ambient doctor` verifies `whisper_asr` / `parakeet_asr` and `audio_tools`; `transcripts/index.md` links every generated sidecar.
- Recorder File-Split Continuity: Raw LLM (Treats each audio file in isolation, losing the first half of a thought when a USB dictaphone rolls over at a 15-minute boundary.) vs. Module (`recover` computes `boundary_gap_seconds` (`<= 2.0s` tolerance) and scans opening tokens against `CONTINUATION_PATTERN` to stitch predecessor tails.) — `legends-ambient recover` emits a structured JSON `PromptRecoveryPlan` with `rollover_detected`, `starts_mid_thought`, `reason`, and `clip_plan`.
- Knowledge Vault Provenance: Raw LLM (Summarizes text ad hoc without preserving source audio filenames, measured durations, ASR model identifiers, or immutable raw transcripts.) vs. Module (Keeps raw ASR artifacts immutable in `transcripts/` and exports separate `type: ambient-distill` Markdown notes with YAML provenance to `--vault`.) — `legends-ambient distill` validates transcript JSON schema and writes `<slug>.md` with `source_audio`, `duration_seconds`, and `ASR Model` metadata.
Explicit Safety Boundaries & Exclusions
Operator FAQ
- Does `legends-ambient ingest` delete or modify recordings on my USB voice recorder?
- No. Ingestion is strictly non-destructive (`source_mutated: false`). The archiver computes SHA-256 checksums for discovered files, compares them against every `manifest.json` in your archive root, copies only new files into a dated `YYYY/MM/DD/<slug>/raw/` folder, verifies the copied file hashes, and—when `--eject` is passed—cleanly dismounts the USB volume so you can unplug it safely.
- How does `legends-ambient recover` detect and repair split thoughts across recording files?
- Hardware voice recorders frequently split continuous sessions into fixed 15-minute files. `legends-ambient recover` sorts recordings chronologically and checks two deterministic signals: whether the timestamp gap between the predecessor file's close (`last_modified`) and the latest file's start is `<= 2.0` seconds (`rollover_detected`), or whether the latest transcript begins with a mid-thought continuation conjunction (`and`, `but`, `because`, `then`, `also`, `which`, `that`, `or`, `nor`, `yet`). If either triggers, it outputs an `include-predecessor-tail` clip plan combining the predecessor's final 5 minutes with the latest recording's 15-minute tail.
- What ASR backends and output formats does `legends-ambient transcribe` support?
- It supports `--backend whisper` (using `faster-whisper` with VAD filtering and word-level timestamps, or falling back to the local `whisper` CLI) and `--backend parakeet` (using NVIDIA NeMo `nvidia/parakeet-tdt-0.6b-v3` on CUDA-capable GPUs), plus `mock` for offline pipeline testing. Every transcribed file produces five sidecar files in `transcripts/`—`.txt`, `.json`, `.srt`, `.vtt`, and `.words.json`—alongside a Markdown `index.md` summary table.
All 19 CTO Legends Packages (1 Router + 18 Modules)
cto-legends (v0.2.5) — Single-skill Python CLI router and isolated environment manager that resolves 18 SHA-256-pinned modules on demand without MCP startup bloat.legends-empire (v0.3.2) — SHA-256 Markdown knowledge foundation with recoverable NTFS and POSIX transactions, .raw/ evidence quarantine, and read-only vault stewardship.legends-hetzner (v0.1.3) — Hetzner Cloud CLI toolkit with Cost-Optimized CX/CAX stock radar, Fractional CTO sizing, SHA-256 server plans, 10-check preflight, and VPS sniping.legends-coolify (v0.2.1) — Read-only Coolify PaaS CLI for self-hosted app inventory, secret-free env key inspection, backup schedule audits, deployment history, and runbooks.legends-cloudflare (v0.1.1) — Zero-dependency Cloudflare DNS CLI enforcing SHA-256 preview-first single-record upserts, exact-ID deletes, setting preservation, and API readback.legends-astro (v0.2.1) — Astro v7 migration CLI that extracts HTML and WordPress source contracts, scaffolds Content Layer collections, and verifies dist/ build parity.legends-gohighlevel (v0.3.0) — Python 3.11+ GoHighLevel CLI for API v2 CRM, Workflows v3, Agent Studio AI, and 5-layer agency RBAC/1.05x rebilling with SHA-256 preview and readback.legends-geogrid (v0.2.2) — Circular-lattice Google Maps rank tracker, GBP name-stuffing audit workbench, and 18-page vector PDF strategy studio for coding agents.legends-dataforseo (v0.1.6) — Zero-dependency Python API and CLI for DataForSEO v3 SERP, Maps, GBP, and keyword research with offline previews and evidence banking.legends-firecrawl (v0.3.2) — Agent-first CLI wrapper (lfc and lax) for official Firecrawl web extraction and Alexandria queries with full local JSON evidence banking.legends-github (v0.2.1) — Deterministic Python CLI for GitHub repository audits, DataForSEO keyword and SERP demand, 20-topic synthesis, non-destructive READMEs, and OG checks.legends-grant (v0.3.0) — Standard-library Python CLI for 56-jurisdiction US grant discovery, SQLite evidence banking, qualification checks, and proposal verification.legends-jev (v0.2.1) — Zero-dependency Python CLI for TypeSafe Jev 1.14 optical recognition (OCR), storefront signage checks, pixel bounding boxes, and typed UI decisions.legends-obs (v0.1.2) — Guarded Windows OBS Studio CLI over WebSocket v5 for hardware encoder planning, pre-mutation rollback snapshots, and FFprobe-verified recording canaries.legends-captions (v0.1.1) — Deterministic video caption QA and ASS rendering CLI with contextual ASR repair, NeMo Forced Aligner word timing, spoken acronym locks, and proof frames.legends-yt-dlp (v0.1.2) — Paced yt-dlp orchestration CLI with SHA-256 binary verification, channel item ledgers, opt-in Mullvad Lockdown guards, and Parakeet/NFA word-level search.legends-stable-audio-3 (v0.1.3) — Hardware-aware Stable Audio 3 CLI for VRAM-calibrated local Medium batches, guarded Large REST jobs, cue-trimmed streaming mixes, and float WAV mastering.legends-hyperyap (v0.1.1) — Offline desktop dictation runtime running INT8 ONNX NVIDIA Parakeet TDT 0.6B v2 locally with CUDA-first inference, active-window paste, and terminal hooks.legends-ambient-intelligence (v0.1.1) — Passive audio pipeline CLI for SHA-256 deduplicated USB recorder ingest, offline Whisper/NeMo Parakeet ASR, rollover recovery, and vault distillation.