The Agent SPE is requesting 15,000 LPT in funding for inference credits to support AI-driven services within the Livepeer ecosystem. These credits will be utilized by both the Agent SPE and Cloud SPE Gateways to ensure continued accessibility for developers.
Introduction
With the integration of AI Agents into the Livepeer network, it is essential to provide free inference for new app developers leveraging the infrastructure. Initially, we received 1,000 LPT in inference credits; however, this allocation is projected to deplete before the three-month period originally estimated.
To maintain uninterrupted access for developers and facilitate ongoing development, we are seeking additional funding to extend our networkâs availability. This funding will also aid in our efforts in launching the Agent Live Stream feature (VTuber plugin) for Eliza.
Fund Management & Security
The requested 15,000 LPT will be held securely in a 2/4 multisig Safe wallet managed by the Agent SPE. When needed, funds will be converted to ETH to replenish Gateway balances, ensuring continuous service availability.
Record Keeping & Transparency
The Agent SPE maintains comprehensive transaction records for all expenditures and fund allocations. These records are available for review upon request, ensuring transparency and accountability in fund utilization.
Conclusion
By allocating these inference credits exclusively for AI model inference, we can:
Encourage Orchestrators to maintain inference pipelines.
Incentivize the deployment of popular AI models requested by end users.
Strengthen the Livepeer AI infrastructure, fostering broader adoption and innovation.
We appreciate the communityâs support in enabling AI-powered applications on the Livepeer network.
First and foremost, thank you, @Titan-Node, for putting forward this pre-proposal. I genuinely support the idea of funding gateways to drive broader adoption. To help ensure clarity and strengthen the proposal, I have a couple questions/comments:
V-Tuber Plugin and Inference Credits
Could you clarify why the V-Tuber Plugin development is included under this inference credit proposal?
If the two are closely related, an updated breakdown showing how much of the requested funding will go towards the plugin vs. the inference credits would be very helpful.
Time-frame, Fund Management, and Reporting
How long is this proposal intended to last (i.e., over what period do you expect these 15,000 LPT to be spent)?
If part of the funding remains unused beyond that time-frame, would you consider returning it to the treasury?
Given that the initial 1,000 LPT was depleted faster than expected, would the Agent SPE be open to providing periodic (e.g., quarterly) funding updates to help the community track spending and understand any unexpected usage spikes?
Could you clarify why the V-Tuber Plugin development is included under this inference credit proposal?
The development of the VTuber plugin is not within the scope of this proposal. Funds will only be used for credits for VTuber inference. I think ChatGPT did not make that clear, but by providing inference credits for VTuber (which could be expensive) would be aidding in the development of the plugin, per se.
If the two are closely related, an updated breakdown showing how much of the requested funding will go towards the plugin vs. the inference credits would be very helpful.
see above
How long is this proposal intended to last (i.e., over what period do you expect these 15,000 LPT to be spent)?
There is no set time limit, we would just fund the gateways until the LPT is depleted.
If part of the funding remains unused beyond that time-frame, would you consider returning it to the treasury?
I think we would rather just keep it until public gateways have spent it, regardless of the timeframe. If we canât spend it within a reasonable time frame then we likely have a bigger adoption issue at hand.
Given that the initial 1,000 LPT was depleted faster than expected, would the Agent SPE be open to providing periodic (e.g., quarterly) funding updates to help the community track spending and understand any unexpected usage spikes?
The 1,000 LPT has not been depleted yet, itâs just a projection that it wonât last the three month period. And yes, we have SPE record keeping that tracks all SPE spending, including inference spend, and can be shared upon request. We can also offer quarterly Gateway spending report based on on-chain activity if the community would like to see that.
Are there any business models behind this? What are the plans for making it sustainable? Relying solely on free inference credits without a clear business model doesnât seem very promising. Implementing a top-up gateway for users accessing gateways could be a good idea. How does the cost of inference compare to other agent endpoints?
What is your pricing strategy? Have you considered creating an alpha testing group with free inference on their nodes? There is currently little work in the AI space, and many orchestrators have idle GPUs. Would it not be wise to lower prices to gain tractionâat least until G/O becomes reliable enough to be a viable alternative to centralized endpoints?
I worry this could follow the same path as other startups that initially gained usage due to free inference but ultimately failed to gain traction.
Asking for inference credits could be an option, but only once you have a reliable and proven working proof of concept. From what Iâve observedâthough I could be wrongâthere are still significant issues at the gateway > orchestrator level, and much work remains to be done.
Subsidizing work is not the difficult part; the real challenge is making it sustainable. At this stage, Iâm not convinced that some agent users and developers would choose Livepeer over centralized endpoints. I would love to see some data/statictis/comparision and business model for the future. For the moment I do not even know how cheap/expensive/reliable livepeer is compared to other endpoints.
Thank you, Titan, and I support any reasonable project that will help the adoption and development of the Livepeer network.
Of course, itâs difficult to know if the adoption will succeed, but itâs indeed also important to support the orchestrators so that they maintain sufficient infrastructure to accommodate any potential AI demand.
Moreover, for this purpose, it might be good to inject AI prompt requests in regions other than North America, in order to also support orchestrators in other regions.
Distributing AI prompt requests globally not only bolsters the networkâs resilience but also encourages participation from orchestrators worldwide.
This global engagement can lead to a more decentralized and robust infrastructure, ensuring that regions like Europe are equally prepared to handle potential AI demands.
By supporting orchestrators across various regions, we cultivate a stronger, more adaptable Livepeer network capable of meeting diverse and growing needs.
As mentioned in the introduction, this proposal originates from the shortness of LPT credits targeted for inference in this proposal. It would be very interesting to know how fast these credits are being spent. You already mentioned that this data is available upon request, so we agree having access to this information would be crucial for future funding in Livepeer Treasury.
Instead of making this information available only upon request, it would be great to standardise this reporting, along with the progress made by the SPE, in a balanced cadence. As part of GovWorksâ Contribution Efficiency Framework we think this type of information fits perfectly to highlight the advancements SPEs make and improve tracking of key technical information.
Regarding Fund Security. I guess the Agent SPE members you mention are the ones stated in the Agent SPE Proposal. For security, could you explicitly confirm them and include it in the final Pre-Proposal text?
It seems that the budget requested is in excess to what the Agent SPE will be needing, are there other projects that can request it until depletion? What is a reasonable timeframe you see these funds lasting for?
A lot of great discussion and feedback on this one. Couple bullets from meâŚ
Definitely supportive of infra credits from the treasury that help with the testing and establishment of fit and growth for the agent-based category youâre building around. They also play a key role in routing fees to the Oâs who are spending their time iterating on the infrastructure to support these use cases. So during a bootstrapping phase, this makes sense. I also think @Karolak raises some good points, that itâs helpful to understand the potential FUTURE models that youâre considering to actually make it sustainable beyond just free inference forever. That way people know theyâre releasing funding in service of some end goal.
The point was also mentioned in discussion about âestablishing the true cost, reliability, and scale of the Livepeer network as it comes to AIâ, as those are still unknown. And then theyâll need to be improved over time. I was chatting with the Cloud SPE about a similar type of proposal whereby the use credits to establish this through a global testing and reporting framework. Stand by for more on that front potentially, but I agree that this shouldnât be the goal of the agent SPE or this credits proposal.
I like the proposal! As previously mentioned, Iâm a fan of offering free credits for the Livepeer network.
@Karolak has some fair points, but looking at it from a pragmatic point of view: All it does is redistributing general protocol inflation to those who actually provide meaningful GPU power to the network.
Asking for inference credits could be an option, but only once you have a reliable and proven working proof of concept. From what Iâve observedâthough I could be wrongâthere are still significant issues at the gateway > orchestrator level, and much work remains to be done.
IMO, this is an argument FOR free credits. We can learn important things from more real world usage/stress tests regarding what is currently missing or not yet fully working.
Itâs probably easier to get feedback and actual usage if you can advertise it as âbeta, but freeâ than âbeta, but you still have to payâ.
I do support the idea of a fully open reporting like @Jose_StableLab mentions. It might give good info for future treasury proposals as well as pricing insights.
We are at the beginning of an immense technological advancement with Livepeer contributing to the global system of AI, Agents and inference.
With any ground breaking projects there is always a vast testing phase with the goal of providing an excellent, high performance product and end user experience. We are in this testing phase. We need to ensure the Livepeer network can handle the work we endeavour to attract and also build a product with maximum functionality, permanence and value.
On the topic of sustainability:
Create a sustainable and high performance inference service.
Test, benchmark and use with plugins that we are currently building, to ensure this system and gateways are robust.
We have an excellent engineer @define supported by @Titan-Node who is building the V-Tuber plugin which requires significant inference to initialise. This, we feel has huge potential for sustainability. Plus all the agent building frameworks we are working with - which use inference for agent interaction, image and video creation.
Market to the end user.
Would you be able to seek out the reasons these startups did not gain traction, so we can avoid their pitfalls please? That would be super helpful!
Standardised reporting formats is a good idea to assist any future SPEs to utilise templates etc. We currently are on a monthly progress reporting schedule, our first monthly report will be available by February 19, 2025.
Yes we can absolutely proceed with open reporting, this is one of the reasons we wanted to build the system right from the start - all hours spent, are recorded. All treasury funds are tracked and accounted for.
I feel we are all working for a common goal - to make Livepeer succeed and gain global traction. We have never been closer to this goal - lets be part of this AI movement and respond swiftly.
Thereâs likely a cheaper and more reliable centralized solution, which is why itâs crucial to analyze market pricing. If we can achieve a comparable level of quality, weâll need to offer a lower price to attract customers.
To be honest, T2I and I2V didnât gain traction because their quality wasnât great. However, LLMs are a different story. Theyâre among the most usable models right now, but theyâre also relatively easy to hostâmeaning pricing competitiveness is key. This should be a fundamental starting point for discussion.
At this point, I donât even know how much inference costs or how we compare to others in the market. Do Eliza agents already have offerings in place? We can certainly spend LPT on inference, but the key question is: what price for what quality will customers be willing to pay?
I believe the proposal should include concrete pricing detailsâfor example, specifying what percentage of centralized suppliersâ prices weâll be paying for each model (e.g., 30% of AWS/Google prices). I canât justify paying more than existing suppliers just to test whether G-O works.
This could initially be tested with volunteering Orchs unless larger GPUs are required, in which case renting would be justified. Since the Treasury will be covering inference costs, we need a clear breakdown of how much weâll be paying for each model.
I donât find this pricing competitive at allâthey are nearly identical. Why would developers choose Livepeer in its current state over the competition? Do these prices account for the payment system issues, where the Gateway is overpaying? IMHO, there should be at least a 50% reduction.
What about the business model? Do you believe you can win the market with the same pricing and current quality? Right now, it looks like an LPT redistribution to four selected orchestrators at prices higher than those in the free market. The selection algorithm has flaws, and there are numerous other issues as well.
Throwing credits at this level seems like a complete waste. Sure, a bridge can be built with public money at higher costs than a private corporation would incurâbut a bridge is still a bridge, so who cares? review upon request does not look good in terms of transparency.
Itâs disappointing that none of the feedback was considered in the final proposal published on Explorer, especially when the discussion wasnât finished.This just gives me DAO proposal vibes and Iâm against this.
Keep in mind the Livepeer network only charges for output tokens and all input tokens are free.
So itâs likely much less than 50% compared to competitors.
We am not focusing on competing with other providers. That is an uphill battle. Rather we should be focusing on new pipelines like the VTuber pipeline and make it exclusive to the Livepeer network which is all image-to-image inference and costs a lot.
Yes and this needs to be fixed. But imagine telling devs to launch their own gateway to pay for inference on the network, it would be a terrible experience right now and weâd lose every single one of them.
Battle testing the network, fixing bugs and paying Orchs to improve their operations is a waste? Iâd rather fix these issues before we allow any dev or end user to experience how horrible it is to run an Gateway right now.
Which other SPEs has public books for people to review?
Yes, well⌠we are pretty busy trying to actually build a product that users might want. I understand the criticism for the DAO, which was giving away funds explicitly to devs, but these are just fees paid back to Orchetrators for running their machines at huge discounts compared to other service providers.
I would also like to comment that the amount of credits we are requesting is equivalent to giving away 15 hours of LPT inflation (every round mints 23,413 LPT) to help fix and promote the network.
I certainly support the idea of providing inference credits using the treasury, but do have some concerns:
I get that we want to support node operators that take the effort & CapEx to add AI jobs, but there should be more pushback on pricing. This is evidenced by the fact that the initial 1000 LPT wonât even last 3 months of mostly testingâŚ
Personally when I look at the price sheets (and considering that we only charge for output tokens) I agree with @karolak that the prices are too high by at least 50%.
The proposal went live without changes, while there is an ongoing discussion here with good feedback being shared, which hooks into the next point:
We can also offer quarterly Gateway spending report based on on-chain activity if the community would like to see that.
Yes we can absolutely proceed with open reporting
That wouldâve been great to include in the final proposal. Since costs are already being tracked dilligently, it should be trivial to generate some charts on allocations and burn rate and share a brief quarterly report.
Since you already have data from the initial 1000 LPT I would have appreciated more data on that as well, as opposed to only mentioning âthis allocation is projected to deplete before the three-month period originally estimatedâ. Itâs important to understand in what rate these funds are being depleted and on what.
the amount of credits we are requesting is equivalent to giving away 15 hours of LPT inflation
âWe print too much money, so we might as well spend it.â. Framing it as â15 hours of inflationâ takes away from the value that it represents. Fact is we would be converting over $100K worth of LPT into ETH over time. Which is fine given the goal of this proposal, but funding requests should always be evaluated on their merit, not by comparison to inflation rates.
Keep in mind that these parameters are the MAX price the gateway is willing to pay. The default selection algo is 70% weighted towards price so Orchs can easily lower their price and build a proper competitive market to get more work.
Also note this is for the Agent SPE gateway only, not the Cloud SPE Gateway which is the default gateway for the Eliza integration.
We are being transparent with our max pricing to help guide Orchestrators, maybe that was a mistake and we should keep it private since all weâve gotten is criticism lol.
We discussed this proposal at the Monday water cooler chat, hence why it has been proposed this week.
We are requesting inference credits - funds to facilitate the testing and fixing of bugs on Livepeer in order to provide a robust inference system to our potential customers. Surely we need to have a fully functioning network?
We are not asking for funds to spend unwisely, we are asking for inference credits for the greater good of Livepeer to create a robust inference system, which also pays our loyal Orchestrators in the process.
For transparency, I will get the Gateway spending report published this week for your review. I will also prepare our first monthly report his week so we can show what the Agent SPE has achieved so far.
Thanks for your inputs and we really appreciate the ongoing support of the community.
