Livepeer Workflow Kit

Retroactive Grant Applications
1

What did you build?

One sentence. What is the output?

Livepeer Workflow Kit — the first framework that lets AI agents compose and run Livepeer media workflows — plus two new open-source Livepeer runners (diarized/streaming audio transcription and Florence-2 vision) that are deployed as live capabilities on the Livepeer Modules Gateway.

Why does it matter?

One to two sentences. What problem does it solve, and for whom on the network?

It is the first outside application to integrate Mike Zoop’s DAO-funded Livepeer Modules Gateway, and it breaks Livepeer out of being primarily for media creators into a media-intelligence backend for anyone driving an agent through everyday or professional tasks. This framework enables users/agents to use Livepeer to address tasks like:

  • “Extract the slides from this video,”
  • “Capture and transcribe this Google Meet,”
  • “Cut these home videos into scenes.”

Because Roboflow is the workflow core, every new Livepeer module becomes a reusable block, so the framework — and the network’s reach into agent ecosystems like OpenClaw and Hermes — scales without rewrites; the two new runners also give orchestrators reusable diarized-transcription and vision capabilities that are essential primitives for agentic tasks.

The Roadmap item for this project expounds on the problem this project is trying to solve. It was the outcome of the Roadmap Session the foundation organized with a number of other project builders.

Link to the work

Direct link to the merged PR, deployed tool, published docs, or equivalent. No link = no review.

Evidence of impact

How is this being used? Who benefits? Quantify where possible (e.g. number of users, PRs merged, integrations adopted).

  • First outside application integrated with the Livepeer Modules Gateway: first Roboflow → Livepeer gateway calls succeeded, with paid usage and billing observed through the gateway.

  • 2 new network runners deployed and published live by orchestrators (openai:audio-transcriptions diarized/streaming model; florence-2 vision), reusable by any caller via OpenAI- and Roboflow-compatible APIs.

  • Agent-native: given the framework with no example, multiple different agents independently figured out how to capture and transcribe a meeting from a one-line “take this stream and get this data” prompt.

  • Beneficiaries: agent/end users gain Livepeer access for use in everyday task based workflows; the gateway operator gains a first live outside integration with paid usage; orchestrators gain two new deployable runner capabilities.

Community proof points

Link to the Discord thread or other signal. Reminder: you should also get 2-3 Orchestrators commenting their support for this work.

Amount requested

USD-equivalent (max $5,000). Include a brief breakdown if the amount is above $2,000.

$5,000 USD-equivalent in LPT. Development work breakdown:

  • Livepeer Workflow Kit (85%)
    • Core framework
    • Livepeer-aware Roboflow blocks
    • CloudSPE integration
    • Ingest paths
    • Session runner
    • Agent skill
  • Audio diarized transcription runner (10%) — openai:audio-transcriptions capability
    • NeMo diarized ASR
    • OpenAI-compatible bounded transcription
    • True streaming (WebSocket) transcription
    • Speaker diarization with segment/word timestamps
  • Florence-2 vision runner (5%) — florence-2 vision capability
    • Florence-2 screen, slide, image, and visual-text understanding
    • OpenAI-compatible vision chat route
    • Direct vision analysis route
    • Roboflow LMM-compatible routes
4 Likes
Pre-Proposal: Network Engineering SPE
2

This is a great concept. While the building blocks themselves are valuable, I think the real breakthrough here is the move toward a truly dynamic, user-defined workflow system.

By providing a framework that allows AI agents to compose and execute these tasks on the fly, you’re shifting the network’s value proposition from static media processing to an agile, agentic intelligence backend. This capability to dynamically chain modules—turning “media ingestion” into “task-based intelligence”—is exactly what will unlock more complex, autonomous use cases that go well beyond simple media transformation. Looking forward to seeing how these workflows evolve as the ecosystem begins to leverage this flexibility.

1 Like
3

Thanks Mike. I will say, your Livepeer Modules Gateway stack makes it easy for someone like myself to simply focus on building use cases, and abstracting away the network complexities. This platform can role into self hosting more of the stack or adopting other gateway tech, but what you built gives an easy path for users and made getting to PoC fast. Huge kudos to you and your team!

4

Ok but how are you gonna make orchestrator manage software that needs to be run/loaded on the gpu? How do you see this workflow exist next to other workflows which already utilize some gpus ie daydream? The model where orchestrators need to hot load the models which are in demand did not work, babysitting orchestrators is not possible that way, we tried it with many different ai models tried to be run on the netowrk by several parties. Nothing besides daydream left and it was nightmare to coordinate programatically on the network - gateway- orchestrator level. Having 10 different docker images loaded for 10 different projects/models eventually canibalize itself and none of the project can be run at the proper quality level. And orchestrators wont keep it loaded if it is not making money. Currently orchestrator keep DAYDream loaded due to incentives, nobody will unload daydream and load whatever different without incentives. Software stack is only getting more fragmented and less managable to maintain network-wise. Ehh livepeer is doomed and the longer I think about it the more I bias towards livepeer as rent gpu server and pay for time market. So everybody can use our gpus as they like without the need of coordinating by orchs cause user who rents the servers takes the responsibility for this. That is the only way to abstract software stack from the network stack which should only act as a payment layer.

2 Likes
5

Yeah, those are real network challenges. I’m not suggesting this is a solution for those deeper challenges.

With this framework, any offering Livepeer has could be put into a block that agents can use to compose workflows with.

My thoughts on this is most AI services are about providing model access… not raw GPU. So there is a market for having persistent models that services can access, which cannot be accomplished in the same manner with being a reseller of raw GPU compute. The question comes back to what models are folks running, which goes back to your question. A lot of Livepeer’s focus that I’ve seen has been around generative media, which is being well done through products like Daydream. I think there is a lot of growth happening in more agentic productivity sector thanks to Openclaw, Hermes, and frontier labs focusing on desktop applications and enterprise markets, and I think there could be a place for Livepeer in that growth.

What I’m trying to do is pull together capabilities that fit in the agentic productivity sector and put them into a platform that makes them one-shot accessible to these kind of productivity agents. The runners I’ve been focused on are not heavy models, and so far there have been orchestrators willing to run them.

My hope is to keep building and keep expanding this platform to make more of Livepeer capabilities accessible to agents, especially productivity agents, in a “one shot” style. This was a PoC, so still a lot more that could be done, but I think this captures how Livepeer could be used in a productivity agent setting. Give Livepeer a play in the productivity space.

2 Likes
6

This is very cool @shane, great work. Given that this is posted as a retroactive grant application, my question is around impact. For context, here are some selected quotes from the Treasury proposal:

The primary bar is impact, not completion. Each application must name the problem, who had it, and who is already using the solution. Describe what was built, link to the work, and state the amount requested. Projected usage does not count.

The concrete impact signals that the Review team will look at are all about usage and adoption:

If I’m not mistaken, this is a POC that you successfully used to transcribe a WC call. But is this being used in the wild at the moment? Have you observed agents (aside from your own) finding this workflow and using it to access the Livepeer network?

2 Likes
7

@rickstaa setup a Roadmap Session which is where we went into depth on this. You can see the output of it here. I’ve updated application so it is more explicit.

While I don’t think any amount of work that caps at $5k will have “usage and adoption” in the traditional commercial sense (which focuses on # of users and market reach), I agree impact is important to define, so here are a few touch points:

I know of another builder that is now accessing Livepeer through Mike’s gateway stack and is using the Audio diarized transcription runner I built. The runner is live on the network. Just yesterday I was working with @MikeZupper to get them setup.

Also, as I mentioned in the proposal, this is the first application to build on a Livepeer Modules gateway and that was regularly discussed in the watercooler calls. By being the first, Mike and I worked closely together to validate the UX and change things that needed to be changed, ultimately improving his product.

So we are talking about very modest impact… I’m not trying to over hype or exaggerate. Does it measure up to what would be expected from a $5k grant for 3 weeks of work and collaboration? I believe so :slight_smile: I think the “spirit of the law” is to weed out grifters and give an entry point for good faith builders that are output oriented, which is why I believe this grant applies to this project.

Hopefully that helps. I do believe there is market potential for Livepeer in the productivity workflow space and this project is the right core. I hope to keep building and go from this PoC to something that is notable in the very competitive agentic market space.

3 Likes
8

Thanks for laying this out clearly. I appreciate the good-faith work here and the fact that you’re not overhyping it.

The collaboration with Mike, the series of validation calls (watercoolers and roadmap session), and another builder already using the stack are meaningful impact signals. (However as a SPE member, I’m aware I may be biased)

One thing that would help strengthen the case further for the community is if that builder could share publicly about how they’re using your stack, and what their experience has been.

Thanks again for pushing this forward.

1 Like
9

@shane Thank you for the official application post as this gives visibility as well as history for community members keen to follow the progress. Next step is the review from the team for confirmation.

1 Like
10

Retro Grant AI Pre-Screen — Livepeer Workflow Kit

Applicant: shane
Application: Livepeer Workflow Kit — Retroactive Grant Applications
Amount requested: $5,000 USD-equivalent in LPT
Screened against: Network Engineering SPE — Retro Grants Funding Process (rubric + execution standards)

Verdict

:white_check_mark: READY FOR REVIEW — Recommend APPROVE

The application clears every gate (scope, shipped & reviewable, community proof point) and triggers no Decline or Return condition. Under the rubric’s impact matrix it maps to high importance × narrow confirmed reach → Approve. It is good-faith, substantive, in-scope work with a validated first integration.

Rubric scorecard

Criterion Result Notes
Scope (3 eligible areas) :white_check_mark: Pass Fits Developers / 5-Minute API (agentic harness tooling, onboarding scaffolding) and Orchestrators / Tooling & Infrastructure (two reusable runners)
Shipped & reviewable :white_check_mark: Pass (with caveat) Open-source Apache-2.0 repos; runners live in the Open Clearinghouse catalog; direct links valid. :warning: Applicant repeatedly frames it as a “PoC,” which sits in tension with the “complete, not proof-of-concept” execution standard
Community proof point (required) :white_check_mark: Met Three watercooler demos, the Foundation roadmap session, and Mike Zupper’s public positive signal
Impact — importance of need :yellow_circle: Moderate–high Tied to a prioritized roadmap item (agentic workflows); first outside integration of the DAO-funded Livepeer Modules Gateway
Impact — breadth already solved :yellow_circle: Narrow One builder being onboarded + the gateway operator.
Quality & completeness :white_check_mark: Ready PoC Runners are live and complete; the Workflow Kit framework is PoC-stage but can be leveraged by other builders.
Proportionality :yellow_circle: Justified High Full $5k (the maximum) is justified given narrow confirmed breadth and community engagement
Red-flag check (AI-slop / speculative) :white_check_mark: Clean Substantive, good-faith, real external collaboration — not built without evidence of need

Matrix placement: high importance × narrow reach → Approve.

Sources: Livepeer Workflow Kit thread · Retro Grants Funding Process — Network Engineering SPE · Skill:Review Retroactive Grant

3 Likes
11

@shane
For the record, please add your wallet address to the grant application.

12

Verdict: Approved, with the understanding that funding for the complete turnkey solution is contingent on future grants.

The project already provides meaningful agentic primitives: an agent-facing workflow construction skill, composable Livepeer/Roboflow blocks, dynamic LOC capability and route discovery, workflow JSON authoring, a generic run-session runtime, and persisted artifacts such as status.json, summaries, transcripts, visual results, and event logs.

Integration of the Livepeer Modules Gateway / payment clearing house is already agent-consumable at the API level. It exposes capability, route, work-unit, and pricing metadata through authenticated endpoints, including price_per_work_unit_wei, work_unit, interaction mode, provider, and worker route. I was able to independently verify real funded execution through its hosted service for catalog/route discovery, Florence-2 vision paid inference, and Nemo transcription request. This demonstrates live orchestrator-backed usage, not just local framework code.

The remaining gap in truly agentic workflows is not core capability anymore since fresh agent can use the Livepeer pipelines if it has repo access, the skill, runtime support, a configured paid key, and the right media ingress path. It is the turnkey flow where any agent receives a skill URL, API key, and media input and completes arbitrary media work without setup.

Future of fully agent-native pipelines should come in the form of next milestone that should be a versioned skill/registry URL, unified machine-readable workflow manifest, hosted or one-command runtime for common tasks, in-agent credential granting, unified media ingress, async workflow job/status/artifact APIs, and workflow-level cost estimation built on top of payment clearing house existing pricing APIs.

Overall, this is a credible framework-level contribution with verified Livepeer network integration and useful new runner capabilities. The next step is turning the existing primitives into a cohesive, self-describing agent runtime that a blank agent can consume directly.