cto-legends · module specification · studio · v0.1.2
legends-yt-dlp: Paced Media Capture, Item Ledgers, Opt-In Mullvad Guards & Parakeet/NFA Clip Intelligence
Paced yt-dlp orchestration CLI with SHA-256 binary verification, channel item ledgers, opt-in Mullvad Lockdown guards, and Parakeet/NFA word-level search.
GitHub Repository (https://github.com/avalonreset/legends-yt-dlp) · Handoff: cto-legends handoff legends-yt-dlp
legends-yt-dlp is a Windows-first Python 3.11+ control plane and PowerShell CLI (scripts/legends-yt-dlp.ps1) that wraps the official standalone yt-dlp.exe binary into a repeatable plan, inventory, preflight, paced pull, verification, and post-capture intelligence workflow. One-off yt-dlp shell commands routinely fail in creator production due to unverified binaries, missing JavaScript extractors, accidental cookie/account leakage, duplicate pulls, unresumable channel rips, and unpaced bulk requests that trigger source blocks. legends-yt-dlp manages the upstream yt-dlp.exe lifecycle (verifying SHA-256 checksums against upstream SHA2-256SUMS and enforcing a 90-day binary freshness gate) and isolates every job inside a self-contained batch directory with manifest.json, urls.txt, yt-dlp.conf, and a resumable download archive.
Architecturally, every batch enforces anonymous operation by default (no browser cookies, no cookie files, no .netrc, and no inherited user config) alongside 15-to-45-second inter-item pacing and an explicit --bulk speed bump for batches over 50 URLs. For operators who want network leak protection during larger batches, an opt-in --with-vpn layer integrates the official Mullvad CLI—verifying connected state, Lockdown mode on, split tunneling off, and LAN sharing blocked before preflight passes, and automatically disabling Lockdown and disconnecting with --wait upon batch completion. Once local media is verified via verify (checking archive IDs, info JSON sidecars, byte counts, and ffprobe readability), the optional intelligence subsystem runs external CrispASR (Parakeet TDT 0.6B v3) transcription and NVIDIA NeMo Forced Aligner (NFA) CTM refinement to power exact contiguous-token search, FFmpeg clip planning, and Obsidian/Empire transcript vault exports.
Deterministic Execution Boundary
Why Raw Frontier LLM Prompts Cannot Replace Deterministic Execution
When asked to download or clip videos, an unguided LLM typically strings together ad-hoc yt-dlp flags—often passing --cookies-from-browser (exposing the user's primary Google account to automated rate limits), hammering endpoints without sleep intervals, or leaving a VPN kill-switch enabled after the script exits. Deterministic CLI orchestration is required to enforce binary integrity, anonymous extraction, fail-closed VPN state transitions, and strict word-ledger alignment.
- Upstream SHA-256 Binary Verification & 90-Day Freshness Gate: legends-yt-dlp never bundles a static yt-dlp.exe in git. The yt-dlp install/update/check commands fetch the official release from yt-dlp/yt-dlp, verify its SHA-256 digest against upstream SHA2-256SUMS, and fail readiness if the local binary is older than 90 days.
- Enforced Anonymous Config & Pacing Speed Bumps: Preflight deterministically blocks runs unless yt-dlp.conf enforces anonymous extraction (forbidding browser cookies, account logins, and user-level configs), injects 15–45s request sleeps, requires --yes for live pulls, and requires --bulk above 50 URLs.
- Fail-Closed Mullvad Lockdown Lifecycle (--with-vpn): In opt-in --with-vpn mode, preflight verifies Mullvad is connected with Lockdown on, split-tunnel off, and LAN blocked. When run --yes reaches a terminal state, it deterministically turns Lockdown off first, disconnects with --wait, and verifies disconnection so the workstation is never stranded.
- Strict Sequence-Matched NFA Word Ledger Refinement: In the post-capture intelligence layer, intelligence align nfa writes a *.words.pre-nfa.jsonl backup and refuses to import NVIDIA NeMo Forced Aligner CTM timings unless the CTM word count and normalized token sequence exactly match the existing transcript ledger.
CLI Subcommands & Capability Surface
legends-yt-dlp onboard | doctor [--production] | yt-dlp install/update/check — Managed yt-dlp Binary Lifecycle & Readiness Doctor: Downloads and SHA-256-verifies the official Windows yt-dlp.exe binary, enforces a 90-day freshness threshold, and checks ffmpeg and JavaScript runtime availability for modern YouTube extraction. Passing --production or --with-vpn extends readiness checks to validate Mullvad Lockdown, split-tunnel, LAN, and auto-connect posture.legends-yt-dlp inventory <url> --name <batch> | ledger <manifest.json> --refresh — Channel & Playlist Inventory + Reconciled Item Ledgers: Expands channels or playlists into manifest.json, urls.txt, yt-dlp.conf, and items.jsonl without downloading media, supporting optional --rights and --rights-file provenance attachments and folder policies (auto, batch, by-uploader, flat). Running ledger --refresh reconciles item statuses (planned, inventoried, archived, downloaded, blocked) against archive files, info JSON sidecars, and disk media.legends-yt-dlp preflight <manifest> | run <manifest> [--dry-run | --yes] | verify <manifest> — Preflight-Gated, Paced Batch Execution & Verification: Blocks live downloads until preflight confirms manifest integrity and anonymous cookie/auth policy, prints a legal-use notice, and enforces 15–45s pacing (requiring --bulk over 50 URLs). After execution, verify validates download archive entries, ledger-scoped info JSON sidecars, media byte counts, and ffprobe stream readability.legends-yt-dlp setup production | run <manifest> --yes --with-vpn | mullvad disconnect-test — Opt-In Mullvad VPN Guard, Safe Recovery & Clean Shutdown: Provides an optional leak-prevention wrapper around the official Mullvad CLI when --with-vpn is requested, recovering from transient tunnel drops while strictly stopping on source-side captchas, logins, or rate-limit blocks without rotating relays. Automatically disables Lockdown and disconnects Mullvad after terminal runs unless --keep-vpn is passed.legends-yt-dlp intelligence doctor --require-gpu | intelligence transcribe <manifest> --all — Local CrispASR Parakeet Transcription & GPU Truth Gate: Extracts mono 16 kHz WAV audio via FFmpeg from verified local batch media and invokes an external CrispASR binary (Parakeet TDT 0.6B v3) to populate normalized timestamped word ledgers (words/<video_id>.words.jsonl). Enforces intelligence doctor --require-gpu as a hard truth gate that passes only when CrispASR reports a compiled CUDA, Vulkan, or Metal backend.legends-yt-dlp intelligence align nfa | intelligence search | clips plan/render | vault build — NFA Word-Boundary Alignment, Exact Clip Planning & Empire Vault Export: Refines word boundaries via external NVIDIA NeMo Forced Aligner CTM output, performs exact contiguous token searches across local word ledgers, generates reviewable FFmpeg clip plans (clip-plan.json) before rendering with --yes, and exports Obsidian-compatible transcript pages (intelligence/vault/<video_id>.md) mapped for Legends Empire vaults.
Raw Frontier LLM vs. legends-yt-dlp Execution Matrix
- Authentication & Account Safety: Raw LLM (Passes --cookies-from-browser chrome by default, exposing the operator's personal account to automated rate limits or bans.) vs. Module (Enforces strict anonymous yt-dlp.conf generation with zero cookies, zero account auth, and ignored user-level configs.) — preflight inspects batch config and fails closed if any cookie, login, or .netrc directive is present.
- Rate Limits, Blocks & VPN Behavior: Raw LLM (Loops retries aggressively or scripts IP/relay rotation to evade platform blocks and throttles.) vs. Module (Enforces 15–45s pacing, requires --bulk over 50 URLs, and halts immediately on captcha, login, or block signals.) — Run reports record exact stop conditions; --with-vpn only recovers local tunnel drops and never rotates relays to bypass blocks.
- Post-Run Network Lockdown State: Raw LLM (Enables Mullvad Lockdown mode in a script and exits, leaving the workstation without internet when the VPN disconnects.) vs. Module (Terminal run --yes --with-vpn and mullvad shutdown disable Lockdown first, disconnect with --wait, and verify state.) — Post-run Mullvad status readback confirms disconnected state with Lockdown off; mullvad disconnect refuses if Lockdown is on.
- Transcript Search & Clip Cutting: Raw LLM (Searches fuzzy auto-captions and cuts FFmpeg clips at rough segment timestamps that chop mid-word.) vs. Module (Builds normalized word ledgers via CrispASR/Parakeet, refines timings via NFA CTM, and writes reviewable clip-plan.json.) — Strict sequence verification before NFA CTM import (with *.words.pre-nfa.jsonl backup) and explicit --yes gate on clips render.
Explicit Safety Boundaries & Exclusions
Operator FAQ
- Is Mullvad VPN required to use legends-yt-dlp?
- No. Ordinary authorized downloads and batch workflows run without Mullvad, a VPN subscription, or a .env account file. Mullvad integration (--with-vpn) is strictly an opt-in leak-prevention guardrail for operators who want fail-closed Lockdown protection during larger batch pulls.
- Why does legends-yt-dlp refuse to use browser cookies or logged-in accounts by default?
- Attaching personal browser cookies (--cookies-from-browser) to automated yt-dlp runs exposes your primary account to platform rate-limiting and enforcement. legends-yt-dlp enforces anonymous extraction with conservative 15–45s pacing by default, and immediately pauses for human review if a source requests authentication or presents a captcha.
- How does post-capture transcription work without bloating the Python package with PyTorch?
- The core legends-yt-dlp package has zero third-party Python dependencies. For post-capture analysis, intelligence transcribe invokes an external C++ CrispASR binary (running Parakeet TDT 0.6B v3) on verified local media, and intelligence align nfa optionally invokes an external Python environment for NVIDIA NeMo Forced Aligner word-boundary refinement.
All 19 CTO Legends Packages (1 Router + 18 Modules)
cto-legends (v0.2.5) — Single-skill Python CLI router and isolated environment manager that resolves 18 SHA-256-pinned modules on demand without MCP startup bloat.legends-empire (v0.3.2) — SHA-256 Markdown knowledge foundation with recoverable NTFS and POSIX transactions, .raw/ evidence quarantine, and read-only vault stewardship.legends-hetzner (v0.1.3) — Hetzner Cloud CLI toolkit with Cost-Optimized CX/CAX stock radar, Fractional CTO sizing, SHA-256 server plans, 10-check preflight, and VPS sniping.legends-coolify (v0.2.1) — Read-only Coolify PaaS CLI for self-hosted app inventory, secret-free env key inspection, backup schedule audits, deployment history, and runbooks.legends-cloudflare (v0.1.1) — Zero-dependency Cloudflare DNS CLI enforcing SHA-256 preview-first single-record upserts, exact-ID deletes, setting preservation, and API readback.legends-astro (v0.2.1) — Astro v7 migration CLI that extracts HTML and WordPress source contracts, scaffolds Content Layer collections, and verifies dist/ build parity.legends-gohighlevel (v0.3.0) — Python 3.11+ GoHighLevel CLI for API v2 CRM, Workflows v3, Agent Studio AI, and 5-layer agency RBAC/1.05x rebilling with SHA-256 preview and readback.legends-geogrid (v0.2.2) — Circular-lattice Google Maps rank tracker, GBP name-stuffing audit workbench, and 18-page vector PDF strategy studio for coding agents.legends-dataforseo (v0.1.6) — Zero-dependency Python API and CLI for DataForSEO v3 SERP, Maps, GBP, and keyword research with offline previews and evidence banking.legends-firecrawl (v0.3.2) — Agent-first CLI wrapper (lfc and lax) for official Firecrawl web extraction and Alexandria queries with full local JSON evidence banking.legends-github (v0.2.1) — Deterministic Python CLI for GitHub repository audits, DataForSEO keyword and SERP demand, 20-topic synthesis, non-destructive READMEs, and OG checks.legends-grant (v0.3.0) — Standard-library Python CLI for 56-jurisdiction US grant discovery, SQLite evidence banking, qualification checks, and proposal verification.legends-jev (v0.2.1) — Zero-dependency Python CLI for TypeSafe Jev 1.14 optical recognition (OCR), storefront signage checks, pixel bounding boxes, and typed UI decisions.legends-obs (v0.1.2) — Guarded Windows OBS Studio CLI over WebSocket v5 for hardware encoder planning, pre-mutation rollback snapshots, and FFprobe-verified recording canaries.legends-captions (v0.1.1) — Deterministic video caption QA and ASS rendering CLI with contextual ASR repair, NeMo Forced Aligner word timing, spoken acronym locks, and proof frames.legends-yt-dlp (v0.1.2) — Paced yt-dlp orchestration CLI with SHA-256 binary verification, channel item ledgers, opt-in Mullvad Lockdown guards, and Parakeet/NFA word-level search.legends-stable-audio-3 (v0.1.3) — Hardware-aware Stable Audio 3 CLI for VRAM-calibrated local Medium batches, guarded Large REST jobs, cue-trimmed streaming mixes, and float WAV mastering.legends-hyperyap (v0.1.1) — Offline desktop dictation runtime running INT8 ONNX NVIDIA Parakeet TDT 0.6B v2 locally with CUDA-first inference, active-window paste, and terminal hooks.legends-ambient-intelligence (v0.1.1) — Passive audio pipeline CLI for SHA-256 deduplicated USB recorder ingest, offline Whisper/NeMo Parakeet ASR, rollover recovery, and vault distillation.