cto-legends · module specification · studio · v0.1.1
legends-hyperyap: Offline NVIDIA Parakeet TDT 0.6B v2 Voice Dictation & Active-Window Paste Engine
Offline desktop dictation runtime running INT8 ONNX NVIDIA Parakeet TDT 0.6B v2 locally with CUDA-first inference, active-window paste, and terminal hooks.
GitHub Repository (https://github.com/avalonreset/legends-hyperyap) · Handoff: cto-legends handoff legends-hyperyap
legends-hyperyap (alias `hyperyap`) is a privacy-first, native desktop speech-to-text application (built on Tauri, Rust, and ONNX Runtime, adapted from MURmure under AGPL-3.0) that turns spoken English into instant keystrokes in whatever window currently holds focus. Rather than routing microphone audio to cloud transcription APIs or confining speech input to a single browser tab, it executes NVIDIA's English-optimized `parakeet-tdt-0.6b-v2` model locally via the `smcleod/parakeet-tdt-0.6b-v2-int8` ONNX conversion (`encoder-model.int8.onnx`, `decoder_joint-model.int8.onnx`, `nemo128.onnx`, and `vocab.txt`, ~665 MB). Once the initial model bundle is downloaded, core transcription runs 100% offline with zero telemetry, preferring CUDA 12 + cuDNN 9 hardware acceleration on Windows x64 and automatically falling back to CPU when a compatible NVIDIA GPU is unavailable.
Architecturally, `legends-hyperyap` is registered in `cto-legends` as a `guided` native desktop module (`recipe.mode: "guided"`) because OS-level microphone hooks, Accessibility permissions, and global input injection live outside isolated Python virtual environments. The runtime captures audio via toggle-to-talk or push-to-talk global shortcuts, applies user-defined custom dictionary substitutions and formatting rules, writes the transcript transiently to the system clipboard, synthesizes a paste event into the active application, and restores the prior clipboard state—keeping transcript history in volatile memory by default unless disk persistence (capped at the last 5 transcripts) is explicitly enabled. On Windows workstations, `install.ps1` and the companion hotkey helper (`hotkeys/` Rust crate and `hyperyap-hotkeys.ahk`) add `F13`, `CapsLock`, and mouse side-button (`XButton1`/`XButton2`) bindings plus process-aware terminal paste/undo handling across 12 terminal emulators.
Deterministic Execution Boundary
Why Raw Frontier LLM Prompts Cannot Replace Deterministic Execution
A raw Frontier LLM in a chat window has no OS-level global hotkey hook, cannot capture low-latency PCM audio from a hardware microphone while another IDE or terminal is focused, cannot run an INT8 ONNX Token-and-Duration Transducer (`parakeet-tdt-0.6b-v2`) offline on local CUDA/CPU hardware, and cannot synthesize native clipboard paste/restore sequences into arbitrary desktop windows. Cloud-based voice chat modes also transmit raw voice recordings to remote servers and trap the resulting text inside the browser conversation rather than inserting it at the OS cursor.
- On-Device INT8 ONNX Parakeet TDT Inference: Executes `encoder-model.int8.onnx`, `decoder_joint-model.int8.onnx`, and `nemo128.onnx` locally on the workstation with automatic CUDA-to-CPU fallback, guaranteeing zero cloud audio transmission and offline reliability after initial model setup.
- Atomic Clipboard Swap & Active-Window Injection: Captures speech while any editor, terminal, or browser is focused, applies deterministic custom dictionary and casing rules, injects the text into the active cursor via transient clipboard swap, and restores the user's previous clipboard contents.
- Process-Aware Terminal Helper & Screenshot Path Paste: On Windows, the companion hotkey helper detects 12 terminal processes (`wezterm-gui.exe`, `windowsterminal.exe`, `pwsh.exe`, `alacritty.exe`, `warp.exe`, etc.) to provide smart terminal copy/paste, bounded paste undo, and automatic clipboard screenshot-to-PNG file path conversion.
- Keyring-Isolated Optional LLM Post-Processing & Localhost API: Keeps LLM Connect disabled by default; when enabled for local Ollama or HTTPS endpoints, stores API keys exclusively in the OS secure keyring (Windows Credential Manager, Apple Keychain, Linux Secret Service), blocks unencrypted HTTP key transmission, and strips secrets from `.hyperyap` exports.
CLI Subcommands & Capability Surface
smcleod/parakeet-tdt-0.6b-v2-int8 (CUDA 12 / CPU fallback) — Offline NVIDIA Parakeet TDT 0.6B v2 INT8 Transcription: Runs the English-focused NVIDIA Parakeet TDT 0.6B v2 speech recognition model locally via INT8 quantized ONNX weights (~665 MB). On Windows x64, the engine automatically probes for a compatible NVIDIA GPU with CUDA 12 and cuDNN 9 runtime and falls back cleanly to CPU execution when unavailable.Record shortcut | Ctrl+Shift+Space | Ctrl+Alt+Space | Ctrl+Shift+X | Ctrl+Shift+1..4 | Escape — Global Shortcut Matrix & Toggle/Push-to-Talk Modes: Provides configurable OS-wide shortcuts for starting/stopping recording (in toggle-to-talk or push-to-talk mode), pasting the last transcript (`Ctrl+Shift+Space`), triggering LLM-assisted recording (`Ctrl+Alt+Space`), invoking Command mode (`Ctrl+Shift+X`), switching across four LLM mode slots (`Ctrl+Shift+1` through `Ctrl+Shift+4`), and cancelling (`Escape`).powershell -ExecutionPolicy Bypass -File install.ps1 [-KeepConfig] — Windows Workstation Installer & Terminal-Aware Hotkey Helper: Automates full Windows workstation provisioning: removes legacy MURmure installs, downloads the 6 Parakeet INT8 ONNX model files with curl retry logic, deploys `%APPDATA%/com.avalonreset.hyperyap/settings.json`, and configures the AutoHotkey v2 / Rust hotkey helper mapping `F13`, `CapsLock`, and Mouse Back to record and Mouse Forward to Enter across 12 supported terminals.Settings -> Dictionary & Formatting Rules | Import/Export (.hyperyap) — Custom Dictionary, Formatting Rules & Portable `.hyperyap` Profiles: Applies deterministic post-recognition vocabulary replacements and formatting rules for technical identifiers, proper nouns, CLI commands, and casing before pasting. Configurations can be exported and imported as `.hyperyap` bundles while keeping API keys strictly inside the native OS keyring.POST http://127.0.0.1:4800/api/transcribe (-F "audio=@file.wav;type=audio/wav") — Localhost HTTP Transcription API for Tool Integration: Exposes an opt-in local HTTP server bound strictly to `localhost` / `127.0.0.1` (default port `4800`, no CORS) so external scripts and local tools can submit `.wav` files (up to 100 MB, automatically resampled to 16 kHz) for sequential Parakeet transcription with custom dictionary rules applied.Settings -> LLM Connect (Ollama / HTTPS) | Voice Mode (Wake Word) — Optional LLM Connect & Hands-Free Wake-Word Voice Mode: Supports optional transcript post-processing through a local Ollama instance or user-configured OpenAI-compatible HTTPS endpoint (blocking API keys over plaintext HTTP). Also includes an opt-in local wake-word Voice Mode that monitors speech segments on-device and immediately discards raw audio after wake-word evaluation.
Raw Frontier LLM vs. legends-hyperyap Execution Matrix
- Audio Privacy & Network Boundary: Raw LLM (Streams live microphone audio to remote cloud ASR/LLM servers and retains transcripts in remote conversation logs.) vs. Module (Transcribes 100% locally via ONNX Parakeet TDT; zero telemetry; transcripts stay in RAM by default and are cleared on exit.) — Disconnect network after initial ~665 MB model download and verify record-and-paste loop in a local editor (`docs/FIRST-RUN.md`).
- Application Focus & Cursor Injection: Raw LLM (Requires switching windows to a browser tab, speaking, and manually copying/pasting output back into the target IDE or terminal.) vs. Module (Global OS shortcut records in background while any window is focused, auto-pastes via transient clipboard swap, and restores prior clipboard.) — `Ctrl+Shift+Space` replays last transcript; clipboard contents before dictation remain intact after paste when `Copy to clipboard` is disabled.
- Terminal & Developer Ergonomics: Raw LLM (No awareness of terminal emulator keybindings, accidental command execution, or clipboard screenshot workflows.) vs. Module (Windows hotkey helper recognizes 12 terminal processes (`wezterm-gui.exe`, `windowsterminal.exe`, `pwsh.exe`, etc.), enabling bounded paste undo and PNG screenshot path paste.) — Verify `hyperyap-hotkeys.ahk` / `hotkeys` helper active against supported terminal process names and tray menu pause toggle.
- Credential & Upgrade Governance: Raw LLM (Stores endpoint tokens in plaintext config files or shell scripts that leak into dotfiles and repository commits.) vs. Module (Stores LLM API keys in OS keyring (Credential Manager / Keychain / Secret Service), strips keys from `.hyperyap` exports, and refuses plaintext HTTP with keys.) — Inspect `%APPDATA%/com.avalonreset.hyperyap/settings.json` and exported `.hyperyap` files to confirm zero secret values are written.
Explicit Safety Boundaries & Exclusions
Operator FAQ
- Why does `legends-hyperyap` use NVIDIA Parakeet TDT 0.6B v2 INT8 instead of a multilingual model?
- `legends-hyperyap` is purpose-built for fast, private, everyday English dictation into code editors, terminals, and chat apps. The English-focused `nvidia/parakeet-tdt-0.6b-v2` model converted to INT8 ONNX (`smcleod/parakeet-tdt-0.6b-v2-int8`, ~665 MB) fits cleanly into a local desktop runtime, runs completely offline after first download, and executes with low latency on either CUDA 12 or CPU fallback.
- How does `legends-hyperyap` differ from `legends-ambient-intelligence`?
- `legends-hyperyap` handles active, foreground voice-to-text typing: you press a shortcut, speak, and it pastes directly into your focused application cursor without saving recordings to disk by default. `legends-ambient-intelligence` handles the passive, background audio pipeline: SHA-256 deduplicated ingestion from USB voice recorders (`KT5000`) or folders, batch offline Whisper/NeMo transcription, 15-minute split-file prompt continuation recovery, and Markdown distillation into an Obsidian/Empire vault.
- Does enabling LLM Connect or exporting `.hyperyap` settings expose my API keys or transcripts?
- No. By default, LLM Connect is disabled and no transcripts are written to disk. If you configure LLM Connect, API keys are stored exclusively in your operating system's native keyring (Windows Credential Manager, Apple Keychain, or Linux Secret Service), never written to `settings.json` or exported `.hyperyap` files, and blocked from being transmitted over unencrypted HTTP.
All 19 CTO Legends Packages (1 Router + 18 Modules)
cto-legends (v0.2.5) — Single-skill Python CLI router and isolated environment manager that resolves 18 SHA-256-pinned modules on demand without MCP startup bloat.legends-empire (v0.3.2) — SHA-256 Markdown knowledge foundation with recoverable NTFS and POSIX transactions, .raw/ evidence quarantine, and read-only vault stewardship.legends-hetzner (v0.1.3) — Hetzner Cloud CLI toolkit with Cost-Optimized CX/CAX stock radar, Fractional CTO sizing, SHA-256 server plans, 10-check preflight, and VPS sniping.legends-coolify (v0.2.1) — Read-only Coolify PaaS CLI for self-hosted app inventory, secret-free env key inspection, backup schedule audits, deployment history, and runbooks.legends-cloudflare (v0.1.1) — Zero-dependency Cloudflare DNS CLI enforcing SHA-256 preview-first single-record upserts, exact-ID deletes, setting preservation, and API readback.legends-astro (v0.2.1) — Astro v7 migration CLI that extracts HTML and WordPress source contracts, scaffolds Content Layer collections, and verifies dist/ build parity.legends-gohighlevel (v0.3.0) — Python 3.11+ GoHighLevel CLI for API v2 CRM, Workflows v3, Agent Studio AI, and 5-layer agency RBAC/1.05x rebilling with SHA-256 preview and readback.legends-geogrid (v0.2.2) — Circular-lattice Google Maps rank tracker, GBP name-stuffing audit workbench, and 18-page vector PDF strategy studio for coding agents.legends-dataforseo (v0.1.6) — Zero-dependency Python API and CLI for DataForSEO v3 SERP, Maps, GBP, and keyword research with offline previews and evidence banking.legends-firecrawl (v0.3.2) — Agent-first CLI wrapper (lfc and lax) for official Firecrawl web extraction and Alexandria queries with full local JSON evidence banking.legends-github (v0.2.1) — Deterministic Python CLI for GitHub repository audits, DataForSEO keyword and SERP demand, 20-topic synthesis, non-destructive READMEs, and OG checks.legends-grant (v0.3.0) — Standard-library Python CLI for 56-jurisdiction US grant discovery, SQLite evidence banking, qualification checks, and proposal verification.legends-jev (v0.2.1) — Zero-dependency Python CLI for TypeSafe Jev 1.14 optical recognition (OCR), storefront signage checks, pixel bounding boxes, and typed UI decisions.legends-obs (v0.1.2) — Guarded Windows OBS Studio CLI over WebSocket v5 for hardware encoder planning, pre-mutation rollback snapshots, and FFprobe-verified recording canaries.legends-captions (v0.1.1) — Deterministic video caption QA and ASS rendering CLI with contextual ASR repair, NeMo Forced Aligner word timing, spoken acronym locks, and proof frames.legends-yt-dlp (v0.1.2) — Paced yt-dlp orchestration CLI with SHA-256 binary verification, channel item ledgers, opt-in Mullvad Lockdown guards, and Parakeet/NFA word-level search.legends-stable-audio-3 (v0.1.3) — Hardware-aware Stable Audio 3 CLI for VRAM-calibrated local Medium batches, guarded Large REST jobs, cue-trimmed streaming mixes, and float WAV mastering.legends-hyperyap (v0.1.1) — Offline desktop dictation runtime running INT8 ONNX NVIDIA Parakeet TDT 0.6B v2 locally with CUDA-first inference, active-window paste, and terminal hooks.legends-ambient-intelligence (v0.1.1) — Passive audio pipeline CLI for SHA-256 deduplicated USB recorder ingest, offline Whisper/NeMo Parakeet ASR, rollover recovery, and vault distillation.