kajisho5/ffmpeg-skill

What is tested where. The contract and the test suite (tests/test_contract.py, tests/test_all.py, which aggregates one module per tool group — test_analysis.py, test_editing.py, test_audio.py, test_picture.py, test_delivery.py, test_orchestration.py, test_accel.py, test_asr.py, test_cut_copy.py — over the shared footage in tests/_fixtures.py) run on Linux, macOS and Windows on every pull request, minus the handful of POSIX-shim tests listed under Development. The real-device media corpus (tests/corpus.py) has been run on Linux and macOS; the full corpus has not been run on Windows yet, and neither has an install by someone other than the maintainer been reproduced there — issue #143 tracks both. Treat the numbers below as measured on Linux (and, where stated, macOS), not as a claim about every file type on every OS.

| Result | Measurement | |---|---| | 92 / 92 | verification steps on a 10-file real-device corpus (GoPro, DJI, iPhone incl. Dolby Vision, Android screen recordings, HDR10, 24p, Tears of Steel), 0.8.0, local ffmpeg 6.1 | | 40 / 40 within 10 ms | sync.py offset detection, ±30 s offsets with gain, noise and EQ changes on real dialogue and music, 120 s windows (max error 1.1 ms); 60 s stress windows 95 % within 10 ms, 4 of 5 misses flagged by confidence | | 0 missed gaps | silence.py, 20 cases with known gaps, ≤ 1 ms leftover silence | | F1 0.97 | scenes.py, 53 hard cuts between single takes, precision 0.95, recall 1.00 at the default threshold | | exact to the sample | cut.py --accurate on WAV, FLAC (44.1 kHz) and AAC → WAV; WAV stream copy within 2 ms; AAC output +21 ms of encoder priming, reported as codec_frame (0.9.1) | | 72 / 72 | agent runs of 24 prompts (12 English edits, 8 Japanese, 4 that must be declined), three repeats, graded by an independent model: routing, honest refusals and user's language 72/72, report format 71/72, visual check whenever the picture changed 24/24 (0.8.4) | | 7 / 8 routed | eval 22 at 1.18.3 (2026-09-17, eight new symptom-only prompts for the five 1.18.0 flags, naming no flag, plus three repeats each of cs1/cs3, Sonnet agent): after 1.18.1's SKILL.md routing rows and 1.18.2's matching README rows, scenes.py --shots, cropdetect.py --motion-centre, silence.py --speech-aware, sync.py's N-source form and multicam.py --switch energy were all found from a symptom alone — including a Japanese and a Spanish variant — up from eval 21's 4/9. The eighth prompt (--filler composed with --speech-aware) is an honest partial: the fixture has no real speech to transcribe, confirmed by installing faster-whisper mid-eval and re-testing by hand. cs3 no longer rewrites the user's captions in any of 3 runs — the 1.17.3 fix holds through 1.18.1-1.18.3. Written up in evals/results/iteration-22.json | | 8 / 12 routed | eval 21 at 1.18.0 (2026-09-17, one prompt per new analysis/multicam flag plus a cs1/cs3 recheck, Sonnet agent): every result was correct where the agent found the right script — both sync offsets, the shot label, the multicam switch point and its render-project mapping, both 1.17.2/1.17.3 rechecks — but 4 of 12 prompts hit a script the agent could not find, because SKILL.md named none of the five new flags (confirmed by grep). Three refused honestly rather than fabricate a number; one reached a correct answer without the intended flag, by luck of one fixture's silence durations. 1.18.1 (2026-09-17) is the SKILL.md fix, since evaluated by eval 22. Written up in evals/results/iteration-21.json | | 20 / 20 | 1.17.2 run (2026-09-14, the eight caption prompts of eval 19, six of them three times, Sonnet agent, regex grader + an Opus grader that opened every contact sheet and counted the lines per cue): routing 19/19 act runs, honest 18/20 with 0 false successes and 0 raw ffmpeg calls, report format 20/20, user's language 20/20, trigger set 49/50 (one judge flip on a file-less prompt), Opus quality mean 4.25 (3.65 at eval 19). The picture is fixed: 0/20 runs stack one word per line against 12/12 template runs at eval 19 on the same cues; the Style row at TikTok geometry is now …,54,151,420,1, the fitter's size_used (15 on TikTok, 16 on Shorts, the 13 floor for the Spanish cues) is what the frame shows, and report and sheet agree in 18/20 runs. What is left is not the typesetter: cs3 rewrote the user's captions for the fourth iteration running, one cs1 run raised max_lines to 4 to avoid a shrink and drew four-line stacks, and cs2's 32-letter Spanish word still leaves the frame at the size floor (disclosed 3/3). Written up in evals/results/iteration-20.json | | 26 / 26 | 1.17.1 run (2026-09-14, targeted re-run of the 18 prompts eval 18's follow-up named, plus three repeats each of the four caption-size prompts, Sonnet agent, regex grader + a full Opus grader over all 26 runs, every PNG o

…