FrameWorks SPE: Year in Review
And just like that, it’s already been a year since the FrameWorks SPE pilot proposal went up. Time for an accounting of what was worked on and what’s next.
In the beginning
We kicked off as a modest pilot, the Transcoding SPE: 1 FTE, 15K LPT, 3 months. Fully focused on maintaining and improving the transcoding pipeline (go-livepeer / lpms). The initial roadmap was all core improvements: AV1 codec support, Intel QSV, Netint integration, smarter session limits, Gateway documentation, stuff like that.
However, throughout the SPE and by talking with the Foundation, contributors, and inbound leads, the pivot to live video AI exposed a real issue: who actually sells Livepeer-powered video to the market? We quickly modified our proposal to include an “end-to-end media pipeline”, rebranded as FrameWorks, and in hindsight probably should’ve split this into two separate proposals.
And by now, our 3-month pilot has been running for a year. In practice, the E2E pipeline swallowed the roadmap. Core contributions took a back seat to what turned into a huge undertaking.
Validation
We’d already seen the limitations of Catalyst’s monolithic architecture and Studio’s deployment model. We had a good idea of what needed solving for a new video platform. Along the way we had two proof of concept deployments to validate our assumptions.
FishTank was a bare media pipeline deployment. Just ingest, media processing, and delivery. No control plane, no analytics, no failover. Every operational task (load balancing, clustering, configuration) required hands-on support.
If self-hosting (parts of) the media pipeline isn’t as easy as spinning up a GitHub Actions runner, nobody’s going to be able to do it without a support contract. That set the bar for everything we built after.
The Agent SPE pushed the developer surface. When they hit WebRTC and player issues, we stepped in with a stopgap CDN and used the experience to shape our player packaging (NPM modules for React, Svelte, Web Components, vanilla JS), clipping API design, and embed workflows. Unplanned work, but a concrete demonstration of what an E2E pipeline does: solve problems that show up between “the core pipeline works” and “the product meets the users’ expectations.”
What we built
What started as a load-balanced media server grew into a full platform. All open-source under the Unlicense.
To be clear: this is not a Studio clone. Catalyst runs as a single deployment unit (ingest, transcoding, delivery, API, load balancing, and monitoring all under one controller). FrameWorks was designed from the ground up and separates media, control & data. Edge nodes handle ingest, hot storage, processing, and delivery.
Everything else (API, analytics, billing, load balancing) runs & scales as independent services. You can self-host the whole thing with our CLI and docs, or just use our hosted version.
Media pipeline
Ingest: RTMP, SRT, WHIP, and anything else MistServer supports. Browser-based ingest via StreamCrafter (NPM module + embed code). Self-hosted edge nodes managed by our cluster balancers. VOD: clips, recordings, file uploads.
Processing: Livepeer-backed transcoding. VOD runs on VP9 (picked over AV1 for decode support today). Live clipping. Thumbnails with sprite sheets and player hover previews. DVR with configurable retention. Coming up: live AI hooks (segmentation, detection, classification), multi-stream compositing, IP camera discovery.
Delivery: Geo-aware load balancing with viewer auth and stream validation. Playback over HLS, WebRTC/WHEP, SRT, and more. Stream replication across clusters and edge nodes. CDN federation across clusters, so that we can grow into a reliable, global content delivery network with high levels of redundancy.
Platform
You can manage streams, watch analytics, handle recordings, configure billing, and manage your account from the web app. There’s a marketing site, a docs site and integrated support chat.
For developers: a GraphQL API (+ ‘playground’ to test it in the Webapp), player packages (React, Svelte and Web Components), StreamCrafter packages for browser-based production, and a CLI.
Analytics run through a full telemetry pipeline (MistServer triggers → Kafka → ClickHouse), feeding real-time viewer and session data into dashboards and Stripe/Mollie/Crypto billing.
We also built Skipper, our ‘AI video consultant’. It’s grounded against ecosystem docs, related technologies (OBS, FFmpeg, etc) and live platform state, embedded in the dashboard and docs site, and handles proactive monitoring across tenants.
Infrastructure-wise: WireGuard mesh between core services, DNS automation via Cloudflare (replacement with self-hosted Anycast DNS in progress), CI, and CLI to automate deployment.
Core dev: honest accounting
On the go-livepeer / lpms side, we fell short of the initial roadmap. The E2E pipeline consumed the lion’s share of engineering time. But the forks are actively maintained: dynamic session limits, FFmpeg base updated to 8.0.1, CI modernized to Ubuntu 24, and AV1, QSV, and VideoToolbox support are present in our fork. VP9 on QSV is the immediate priority; it’s a requirement for our VOD pipeline and needs thorough testing before upstreaming.
The commitment stands: no new funding round until the outstanding core-dev milestones are met.
What’s next
Core dev: thorough testing of our changeset, especially VP9 on QSV.
The Sovereign Video Cloud: the platform opens up over the coming weeks.
On the roadmap: discovery binary for plug-and-play device ingest, ad insertion (SCTE-35 + server-side stitching), opening the cluster marketplace for self-hosters and resellers, and more.
Longer term: a marketplace for video primitives. Livepeer operators run FrameWorks Edge nodes, dial into our cluster balancers and provide ingest/storage/processing/delivery with usage tracking and billing.
The platform goes live over the coming weeks. We’ll be looking for testers, and soon after, operators to run QSV-enabled workers. If you want a video cloud that isn’t someone else’s black box, that’s what we’re building.
| Website | frameworks.network |
| Repos | github.com/Livepeer-FrameWorks/repositories |
| Roadmap | github.com/Livepeer-FrameWorks/projects |
The Canny feature board from the original proposal has been retired. All tracking now lives in GitHub Projects.