A crash-durable pipeline that discovers campaigns and auto-cuts short clips from long-form video
A restart-survivable clip factory built from unreliable sources.
A backend service that discovers creator clipping campaigns, ingests long-form video, and runs an automated pipeline to cut, transcribe, and scene-detect short clips, then delivers them via signed URLs, chat, and submission endpoints. Runs in poll or webhook mode.
A look at the interface.
Representative UI — abstracted, never a client's real data.- Python
- Flask
- yt-dlp
- Whisper
- PySceneDetect
- ffmpeg
- SQLite
Build an end-to-end, restart-survivable clip factory from unreliable scraped sources: discovering campaigns, parsing free-form requirements into enforceable rulesets, and processing video jobs durably without losing work on crashes.
How it
works
Discovered links land in a queue that survives restarts, get probed for metadata, then filtered by duration and keywords. Heavy ML dependencies are deferred so the worker imports cleanly without them, then transcribes and scene-cuts source video into clips. Delivery fans out to presigned storage URLs, chat, and submission endpoints. It ships with Docker and a systemd installer.
What I built
(03) Inside the buildPersistent job queue
Discovered links are enqueued in a SQLite queue, probed for metadata, filtered by duration and keywords, and processed by a continuous worker that survives restarts.
Clip pipeline
Heavy dependencies are deferred so the worker imports cleanly, then transcribes and scene-cuts source video into clips.
Delivery + deployment
S3 upload with presigned URLs, chat delivery, and submission endpoints, shipped with Docker and a systemd installer with signed webhooks.
The
receipts
- Crash-durable job queue that survives restarts
- HMAC-verified webhook signatures
- Lazy heavy-import design keeps the worker importable without full ML deps
- Pluggable delivery to storage, chat, and submission targets