Agent SPE: Inference Credits Pre-Proposal

Pre-Proposal: Inference Credits for Agent SPE

Abstract

The Agent SPE is requesting 15,000 LPT in funding for inference credits to support AI-driven services within the Livepeer ecosystem. These credits will be utilized by both the Agent SPE and Cloud SPE Gateways to ensure continued accessibility for developers.

Introduction

With the integration of AI Agents into the Livepeer network, it is essential to provide free inference for new app developers leveraging the infrastructure. Initially, we received 1,000 LPT in inference credits; however, this allocation is projected to deplete before the three-month period originally estimated.

To maintain uninterrupted access for developers and facilitate ongoing development, we are seeking additional funding to extend our network’s availability. This funding will also aid in our efforts in launching the Agent Live Stream feature (VTuber plugin) for Eliza.

Fund Management & Security

The requested 15,000 LPT will be held securely in a 2/4 multisig Safe wallet managed by the Agent SPE. When needed, funds will be converted to ETH to replenish Gateway balances, ensuring continuous service availability.

Record Keeping & Transparency

The Agent SPE maintains comprehensive transaction records for all expenditures and fund allocations. These records are available for review upon request, ensuring transparency and accountability in fund utilization.

Conclusion

By allocating these inference credits exclusively for AI model inference, we can:

  • Encourage Orchestrators to maintain inference pipelines.
  • Incentivize the deployment of popular AI models requested by end users.
  • Strengthen the Livepeer AI infrastructure, fostering broader adoption and innovation.

We appreciate the community’s support in enabling AI-powered applications on the Livepeer network.

Titan
Agent SPE

Edited by ChatGPT

7 Likes

First and foremost, thank you, @Titan-Node, for putting forward this pre-proposal. I genuinely support the idea of funding gateways to drive broader adoption. To help ensure clarity and strengthen the proposal, I have a couple questions/comments:

  1. V-Tuber Plugin and Inference Credits

    • Could you clarify why the V-Tuber Plugin development is included under this inference credit proposal?
    • If the two are closely related, an updated breakdown showing how much of the requested funding will go towards the plugin vs. the inference credits would be very helpful.
  2. Time-frame, Fund Management, and Reporting

    • How long is this proposal intended to last (i.e., over what period do you expect these 15,000 LPT to be spent)?
    • If part of the funding remains unused beyond that time-frame, would you consider returning it to the treasury?
    • Given that the initial 1,000 LPT was depleted faster than expected, would the Agent SPE be open to providing periodic (e.g., quarterly) funding updates to help the community track spending and understand any unexpected usage spikes?
7 Likes

Could you clarify why the V-Tuber Plugin development is included under this inference credit proposal?

The development of the VTuber plugin is not within the scope of this proposal. Funds will only be used for credits for VTuber inference. I think ChatGPT did not make that clear, but by providing inference credits for VTuber (which could be expensive) would be aidding in the development of the plugin, per se.

If the two are closely related, an updated breakdown showing how much of the requested funding will go towards the plugin vs. the inference credits would be very helpful.

see above

How long is this proposal intended to last (i.e., over what period do you expect these 15,000 LPT to be spent)?

There is no set time limit, we would just fund the gateways until the LPT is depleted.

If part of the funding remains unused beyond that time-frame, would you consider returning it to the treasury?

I think we would rather just keep it until public gateways have spent it, regardless of the timeframe. If we can’t spend it within a reasonable time frame then we likely have a bigger adoption issue at hand.

Given that the initial 1,000 LPT was depleted faster than expected, would the Agent SPE be open to providing periodic (e.g., quarterly) funding updates to help the community track spending and understand any unexpected usage spikes?

The 1,000 LPT has not been depleted yet, it’s just a projection that it won’t last the three month period. And yes, we have SPE record keeping that tracks all SPE spending, including inference spend, and can be shared upon request. We can also offer quarterly Gateway spending report based on on-chain activity if the community would like to see that.

2 Likes

Are there any business models behind this? What are the plans for making it sustainable? Relying solely on free inference credits without a clear business model doesn’t seem very promising. Implementing a top-up gateway for users accessing gateways could be a good idea. How does the cost of inference compare to other agent endpoints?

What is your pricing strategy? Have you considered creating an alpha testing group with free inference on their nodes? There is currently little work in the AI space, and many orchestrators have idle GPUs. Would it not be wise to lower prices to gain traction—at least until G/O becomes reliable enough to be a viable alternative to centralized endpoints?

I worry this could follow the same path as other startups that initially gained usage due to free inference but ultimately failed to gain traction.

Asking for inference credits could be an option, but only once you have a reliable and proven working proof of concept. From what I’ve observed—though I could be wrong—there are still significant issues at the gateway > orchestrator level, and much work remains to be done.

Subsidizing work is not the difficult part; the real challenge is making it sustainable. At this stage, I’m not convinced that some agent users and developers would choose Livepeer over centralized endpoints. I would love to see some data/statictis/comparision and business model for the future. For the moment I do not even know how cheap/expensive/reliable livepeer is compared to other endpoints.

8 Likes

Thank you, Titan, and I support any reasonable project that will help the adoption and development of the Livepeer network.

Of course, it’s difficult to know if the adoption will succeed, but it’s indeed also important to support the orchestrators so that they maintain sufficient infrastructure to accommodate any potential AI demand.
Moreover, for this purpose, it might be good to inject AI prompt requests in regions other than North America, in order to also support orchestrators in other regions.

Distributing AI prompt requests globally not only bolsters the network’s resilience but also encourages participation from orchestrators worldwide.
This global engagement can lead to a more decentralized and robust infrastructure, ensuring that regions like Europe are equally prepared to handle potential AI demands.
By supporting orchestrators across various regions, we cultivate a stronger, more adaptable Livepeer network capable of meeting diverse and growing needs.

4 Likes

Hi! Thanks for putting up this proposal.

As mentioned in the introduction, this proposal originates from the shortness of LPT credits targeted for inference in this proposal. It would be very interesting to know how fast these credits are being spent. You already mentioned that this data is available upon request, so we agree having access to this information would be crucial for future funding in Livepeer Treasury.

Instead of making this information available only upon request, it would be great to standardise this reporting, along with the progress made by the SPE, in a balanced cadence. As part of GovWorks’ Contribution Efficiency Framework we think this type of information fits perfectly to highlight the advancements SPEs make and improve tracking of key technical information.

Regarding Fund Security. I guess the Agent SPE members you mention are the ones stated in the Agent SPE Proposal. For security, could you explicitly confirm them and include it in the final Pre-Proposal text? :slight_smile:

It seems that the budget requested is in excess to what the Agent SPE will be needing, are there other projects that can request it until depletion? What is a reasonable timeframe you see these funds lasting for?

1 Like

A lot of great discussion and feedback on this one. Couple bullets from me…

  • Definitely supportive of infra credits from the treasury that help with the testing and establishment of fit and growth for the agent-based category you’re building around. They also play a key role in routing fees to the O’s who are spending their time iterating on the infrastructure to support these use cases. So during a bootstrapping phase, this makes sense. I also think @Karolak raises some good points, that it’s helpful to understand the potential FUTURE models that you’re considering to actually make it sustainable beyond just free inference forever. That way people know they’re releasing funding in service of some end goal.

  • The point was also mentioned in discussion about “establishing the true cost, reliability, and scale of the Livepeer network as it comes to AI”, as those are still unknown. And then they’ll need to be improved over time. I was chatting with the Cloud SPE about a similar type of proposal whereby the use credits to establish this through a global testing and reporting framework. Stand by for more on that front potentially, but I agree that this shouldn’t be the goal of the agent SPE or this credits proposal.

5 Likes

I like the proposal! As previously mentioned, I’m a fan of offering free credits for the Livepeer network.

@Karolak has some fair points, but looking at it from a pragmatic point of view: All it does is redistributing general protocol inflation to those who actually provide meaningful GPU power to the network.

Asking for inference credits could be an option, but only once you have a reliable and proven working proof of concept. From what I’ve observed—though I could be wrong—there are still significant issues at the gateway > orchestrator level, and much work remains to be done.

IMO, this is an argument FOR free credits. We can learn important things from more real world usage/stress tests regarding what is currently missing or not yet fully working.
It’s probably easier to get feedback and actual usage if you can advertise it as “beta, but free” than “beta, but you still have to pay”.

I do support the idea of a fully open reporting like @Jose_StableLab mentions. It might give good info for future treasury proposals as well as pricing insights.

4 Likes

Thanks everyone for supporting the Agent SPE - we are making great progress!

Thank you too for your thoughts and comments on this Pre-Proposal for Inference Credits.

Hi @Karolak

We are at the beginning of an immense technological advancement with Livepeer contributing to the global system of AI, Agents and inference.

With any ground breaking projects there is always a vast testing phase with the goal of providing an excellent, high performance product and end user experience. We are in this testing phase. We need to ensure the Livepeer network can handle the work we endeavour to attract and also build a product with maximum functionality, permanence and value.

On the topic of sustainability:

  1. Create a sustainable and high performance inference service.
  2. Test, benchmark and use with plugins that we are currently building, to ensure this system and gateways are robust.
  3. We have an excellent engineer @define supported by @Titan-Node who is building the V-Tuber plugin which requires significant inference to initialise. This, we feel has huge potential for sustainability. Plus all the agent building frameworks we are working with - which use inference for agent interaction, image and video creation.
  4. Market to the end user.

Would you be able to seek out the reasons these startups did not gain traction, so we can avoid their pitfalls please? That would be super helpful!

There have been excellent advancements at the gateway level this week. Thanks to @Titan-Node @Wiser @MikeZupper @brad-ad-astra-video. Still, work does needs to continue, you are right.

Hi @Wiser

Exactly, thank you.

Hi @Jose_StableLab

Standardised reporting formats is a good idea to assist any future SPEs to utilise templates etc. We currently are on a monthly progress reporting schedule, our first monthly report will be available by February 19, 2025.

Yes we can absolutely proceed with open reporting, this is one of the reasons we wanted to build the system right from the start - all hours spent, are recorded. All treasury funds are tracked and accounted for.

I feel we are all working for a common goal - to make Livepeer succeed and gain global traction. We have never been closer to this goal - lets be part of this AI movement and respond swiftly.

4 Likes

There’s likely a cheaper and more reliable centralized solution, which is why it’s crucial to analyze market pricing. If we can achieve a comparable level of quality, we’ll need to offer a lower price to attract customers.

To be honest, T2I and I2V didn’t gain traction because their quality wasn’t great. However, LLMs are a different story. They’re among the most usable models right now, but they’re also relatively easy to host—meaning pricing competitiveness is key. This should be a fundamental starting point for discussion.

At this point, I don’t even know how much inference costs or how we compare to others in the market. Do Eliza agents already have offerings in place? We can certainly spend LPT on inference, but the key question is: what price for what quality will customers be willing to pay?

I believe the proposal should include concrete pricing details—for example, specifying what percentage of centralized suppliers’ prices we’ll be paying for each model (e.g., 30% of AWS/Google prices). I can’t justify paying more than existing suppliers just to test whether G-O works.

This could initially be tested with volunteering Orchs unless larger GPUs are required, in which case renting would be justified. Since the Treasury will be covering inference costs, we need a clear breakdown of how much we’ll be paying for each model.

1 Like

I have already provided the pricing offered by the Agent SPE Gateway and the calculations we derived from Eliza community.

In addition we have the pricing of our Gateway config file public for Orchs to see what we are willing the pay to keep an open and transparent market.

https://prices.livepeer-eliza.com/

3 Likes

I don’t find this pricing competitive at all—they are nearly identical. Why would developers choose Livepeer in its current state over the competition? Do these prices account for the payment system issues, where the Gateway is overpaying? IMHO, there should be at least a 50% reduction.

What about the business model? Do you believe you can win the market with the same pricing and current quality? Right now, it looks like an LPT redistribution to four selected orchestrators at prices higher than those in the free market. The selection algorithm has flaws, and there are numerous other issues as well.

Throwing credits at this level seems like a complete waste. Sure, a bridge can be built with public money at higher costs than a private corporation would incur—but a bridge is still a bridge, so who cares?
review upon request does not look good in terms of transparency.
It’s disappointing that none of the feedback was considered in the final proposal published on Explorer, especially when the discussion wasn’t finished.This just gives me DAO proposal vibes and I’m against this.

3 Likes

Keep in mind the Livepeer network only charges for output tokens and all input tokens are free.
So it’s likely much less than 50% compared to competitors.

We am not focusing on competing with other providers. That is an uphill battle. Rather we should be focusing on new pipelines like the VTuber pipeline and make it exclusive to the Livepeer network which is all image-to-image inference and costs a lot.

Yes and this needs to be fixed. But imagine telling devs to launch their own gateway to pay for inference on the network, it would be a terrible experience right now and we’d lose every single one of them.

Battle testing the network, fixing bugs and paying Orchs to improve their operations is a waste? I’d rather fix these issues before we allow any dev or end user to experience how horrible it is to run an Gateway right now.

Which other SPEs has public books for people to review?

Yes, well… we are pretty busy trying to actually build a product that users might want. I understand the criticism for the DAO, which was giving away funds explicitly to devs, but these are just fees paid back to Orchetrators for running their machines at huge discounts compared to other service providers.


I would also like to comment that the amount of credits we are requesting is equivalent to giving away 15 hours of LPT inflation (every round mints 23,413 LPT) to help fix and promote the network.

3 Likes

I certainly support the idea of providing inference credits using the treasury, but do have some concerns:

  • I get that we want to support node operators that take the effort & CapEx to add AI jobs, but there should be more pushback on pricing. This is evidenced by the fact that the initial 1000 LPT won’t even last 3 months of mostly testing…
    Personally when I look at the price sheets (and considering that we only charge for output tokens) I agree with @karolak that the prices are too high by at least 50%.

  • The proposal went live without changes, while there is an ongoing discussion here with good feedback being shared, which hooks into the next point:

  • We can also offer quarterly Gateway spending report based on on-chain activity if the community would like to see that.

    Yes we can absolutely proceed with open reporting

    That would’ve been great to include in the final proposal. Since costs are already being tracked dilligently, it should be trivial to generate some charts on allocations and burn rate and share a brief quarterly report.
    Since you already have data from the initial 1000 LPT I would have appreciated more data on that as well, as opposed to only mentioning ‘this allocation is projected to deplete before the three-month period originally estimated’. It’s important to understand in what rate these funds are being depleted and on what.

  • the amount of credits we are requesting is equivalent to giving away 15 hours of LPT inflation

    ‘We print too much money, so we might as well spend it.’. Framing it as ‘15 hours of inflation’ takes away from the value that it represents. Fact is we would be converting over $100K worth of LPT into ETH over time. Which is fine given the goal of this proposal, but funding requests should always be evaluated on their merit, not by comparison to inflation rates.

1 Like

Keep in mind that these parameters are the MAX price the gateway is willing to pay. The default selection algo is 70% weighted towards price so Orchs can easily lower their price and build a proper competitive market to get more work.

Also note this is for the Agent SPE gateway only, not the Cloud SPE Gateway which is the default gateway for the Eliza integration.

We are being transparent with our max pricing to help guide Orchestrators, maybe that was a mistake and we should keep it private since all we’ve gotten is criticism lol.

2 Likes

Hi @stronk Thank you for:

We discussed this proposal at the Monday water cooler chat, hence why it has been proposed this week.

We are requesting inference credits - funds to facilitate the testing and fixing of bugs on Livepeer in order to provide a robust inference system to our potential customers. Surely we need to have a fully functioning network?

We are not asking for funds to spend unwisely, we are asking for inference credits for the greater good of Livepeer to create a robust inference system, which also pays our loyal Orchestrators in the process.

For transparency, I will get the Gateway spending report published this week for your review. I will also prepare our first monthly report his week so we can show what the Agent SPE has achieved so far.

Thanks for your inputs and we really appreciate the ongoing support of the community.

2 Likes

Hi Everyone,

The Agent SPE’s monthly report and Inference credit report is complete and can be found in this Forum post.

Thank you for your support in building Livepeer for the future inference provider for the world!

4 Likes