Potential Demand Increase and Network Impacts

Transcoders
1

Hi Orchestrators! Over the past few months, the Livepeer Studio team has been engaged in conversations with a 1st-tier livestreaming platform (top 10 in the world by volume). We’re happy to share this customer is setting up to scale up traffic on Livepeer Studio in late February. We think this opportunity represents a big step towards establishing Livepeer network’s credibility in the global transcoding market.

As you might imagine, this was a very competitive deal; platforms of this size have extraordinary negotiating power, receive huge discounts on rack rates, and even have the ability to operate their own infrastructure. We’ve spent the past 4 months working through the transcoding cost to support this customer’s commercial and technical due diligence process. This message is the start of a conversation about its potential impact on the Livepeer network.

< Network Impact TLDR >

Traffic - We expect this customer to 3-5x current transcoding minutes on the Livepeer network as they ramp up traffic over a few months. Orchestrator nodes should see higher utilization. We suspect the current capacity in the network can handle this increase, but would like to gut check with you on this.

Pricing - PPP will need to reduce dramatically (and quickly) to 60-70ppp for it to be economically viable to support this user at this scale. They have pushed very hard on price, and even have alternatives that are similar in cost effectiveness.

Net result - In the short term, if node operators can handle the traffic increase, they should make about the same amount of ETH fees as today. In the longer term, we think this increase in traffic can attract more scaled applications, and thereby increasing fees for all.

Timing - We hope to see traffic start to ramp up in mid March. Our current plan is to roll out new pricing a few days prior to that, with the exact date still TBD.

< Details >

To administer this customer (and similar future customers) at breakeven, we need to achieve a cost of $0.50 / 1k mins (~$0.03 / hr) of transcoding for a full ladder (1080p/720p/480p/360p). This works out to around 70 wei per pixel. While there is clear evidence that the network is capable of providing high-quality service at this price point, we consider it highly unlikely that market forces will drive price low enough within the next 2-3 weeks.

In the ideal world, price adjustments like this would happen over a longer period of time, potentially via dynamic and permissionless price discovery. But given the technical complexity, the sensitive timing, and the potential significance of the deal, we are reaching out to directly solicit your feedback.

We also plan to increase the stake weight with this roll out to help increase market competition.

11 Likes
2

Hey Eric,

Congratulations on the deal! Seems positive for the future of Livepeer.

Some concerns and Q’s as an O:

This seems like amazing news for everyone other than Orchestrators. What would be the benefit for O’s reducing their PPP’s for Livepeer Studio, since, as other Orchestraors have discussed, increased streams don’t seem to translate to increased income?

Because Livepeer Studio is the #1 driver of traffic for the network, this PPP requirement would be a forced change if an O wants to receive traffic at all, and at the cost of increased hardware usage while pay doesn’t necessarily increase. The majority of O’s that are lucky enough to have stake are already relying on LPT as payment and this seems to continue pushing the status quo in that direction.

From an Orchestrator standpoint, this doesn’t make a lot of sense to me, so any clarity that can be provided there would be appreciated.

2 Likes
3

True. So delegators will need to make sacrifices as well. I’d suggest Os to max your Eth fee cut to 100% to make this happen.
This kind of increased traffic from web2 is what we’ve all been asking for. Now it’s here.

2 Likes
4

I support the reduction of my PPP to 50-70 for Livepeer Inc’s new client. Given my (current) capacity to handle only 60 concurrent streams, my operational costs are considerably lower than those of larger entities. I empathize with @Authority_Null’s concern that this adjustment may not directly benefit orchestrators. However, sacrifices are sometimes necessary to stimulate network adoption and fortify long-term profitability and ecosystem resilience. While advocating for this change comes more naturally for me as a minor player, I acknowledge the potential challenges it may present for more established operators, which could result in losses :confused:.

Consequently, I propose updating the go-livepeer binary to empower orchestrators to differentiate between streams from clients with whom they have agreements and those from other clients. This capability would enable them to maintain higher prices for the latter. My suggestion closely aligns with the issue I raised on GitHub (Allow distinct Pricing for VOD and Live Streams per Broadcaster in `pricePerBroadcaster` · Issue #2945 · livepeer/go-livepeer · GitHub) a few weeks ago and should be relatively straightforward to implement. While we can already distinguish our pricing between broadcasters, adding a field for broadcasters to differentiate between clients would be incredibly beneficial :thinking:.

Unfortunately, due to the demands of my ongoing graduation process, I haven’t been able to address this myself. @ericxtang, would it be feasible to allocate some FTE resources from your team (or a grant) to prioritise the implementation of this feature before your client begins utilizing the network? Doing so would empower orchestrators to make informed decisions regarding this and potential future agreements.

2 Likes
5

For reference @Authority_Null replied to this on Discord.

Wouldn’t this effect smaller O’s more since bigger O’s can rely on their stake to receive work without a massive reduction in PPP, however, smaller O’s are forced to reduce their PPP to the absolute lowest and see little return?

I agree that sacrifices are needed but we’re pretty sacrifice heavy on the lowering PPP and upping load on hardware side of things as of late, although I do think the new client sounds like a big positive for the network as a whole. I just see the continual decline of PPP a net negative for O’s with the current payout system.

My follow-up to that question is found on Discord.

In my previous comment, brevity was my aim :sweat_smile:. While I grasp that smaller orchestrators are impacted too, relying solely on inflation rewards seems more viable for us compared to our larger counterparts. My energy costs are manageable, VPS bandwidth charges aren’t a concern, and I actively utilize bots to source affordable hardware in the second-hand market. Ideally, I’d offset expenses with fees in ETH or USDT, but the current low demand presents a challenge. @oburs’ comment in the forum post is equally noteworthy. The implications stretch beyond orchestrators; there’s a potential indirect impact on delegators if the expected demand fails to materialize over time. Hence, I concur with your assessment that this deal could adversely affect the ecosystem, warranting a discussion.

Ultimately, I believe in a free market once the ecosystem attains better adoption. Nevertheless, I want to emphasize my willingness to forgo immediate profit for future ecosystem adoption. My proposal for the client-based PPP feature, as suggested in Discord, could help mitigate risks. If the deal enhances ecosystem visibility and attracts more participants, orchestrators could compete on the open market for this increased demand :thinking:. The outcome remains uncertain, but I foresee benefits for both profits and ecosystem resilience in the long run.

2 Likes
6

I’m all up for increased demand on network! Lowering PPP isn’t an issue that I can see. I’m all for more demand and exposure to the network
My biggest concern is can network really handle increased demand without adding a ton of additional hardware?
I’ll be totally honest and blunt here but I do believe everyone will need to scale up to handle demand. As I’ve mentioned in water cooler I still believe a lot of O’s are sharing 1 GPU between them, that’s there I can see an issue.
Would it be possible to simulate a load test with increased demand 3x to 5x as we speaking?

2 Likes
7

Hi @Authority_Null - thank you for expressing your concern.

I agree that fees won’t necessarily grow for each orchestrator based on this deal. However, there are 2 macro points to consider:

  1. Livepeer’s transcoding market competes with other centralized transcoding providers on a global scale. This price that we got from the customer is a strong signal that Livepeer’s transcoding cost needs to come down to be competitive at a large scale.
  2. We believe having a large customer like this will go a long way in establishing credibility for Livepeer in the video market, thus bringing more work to the network, and more fee opportunities for orchestrators.

So we hope this can be a long term benefit for the orchestrator community.

1 Like
8

Simulating a load test with 3-5x of the current load for the entire network is definitely theoretically possible, but can be a complex and expensive endeavor.

Given the timing, our current plan is to help the customer gradually ramp up traffic, while stay in close communication with the orchestrator community throughout that process. Based on conversations with select orchestrators, the turnaround time to add more transcoding capacity seems to be within a few days to 1 week.

3 Likes
9

Thanks for highlighting the Github proposal @rickstaa. Will respond in the Github thread to keep the discussion in context.

2 Likes
10

Upscaling can happen in days or a week, thats correct its feasable option.
In all recent changes with PPP some Os has downgraded their nodes, thats why I would love to see what capacity we have on network at that moment. Whatever orchestrators set on their configs will not be reflecting true capacity of O.

1 Like
11

I am very much on board with distinguishing between VOD and LIVE jobs (that feature is probably overdue), but have a quick clarifying question about the rest.

In this scenario, Livepeer Studio is the client with whom the O has an agreement - so setting price by broadcaster seems to fulfill the need as there isn’t a direct relationship between Livepeer Studio customers and orchestrators. Do you believe that such a relationship should exist? If yes, why?

My initial reaction is that this question is only relevant in the current environment with very few Bs. In a many-B environment, Bs would compete with one another to offer their clients lower prices by negotiating more effectively with Orchestrators

1 Like
12

In that scenario Bs could also compete with other Bs for compute and offer Os better price to secure more GPUS for themselves etc. Competition works both ways all depends on current supply and demand both work and gpus. Currently there are more GPUS than work so B have bigger leverage and Os need to take it or leave it. But that could hypothetically change in the future if there is more work than GPUs. AI work might be more profitable so Os could change over to AI work instead of transcoding. It is very dynamic environment and I belive in invisible free market’s hand taking care of the pricing.

Distinguishing VOD and LIVE jobs is important from operational point of view. Currently Orchs limit their work by session number. That has become unreliable way since vod jobs were implemented. Ie. gpu can take 20 live streams and 3 vod jobs or 10 vod jobs and 4 live streams. It would be ideal to be able to assign gpus to different job type. If demand surges I can imagine some gpus crashing if only session limit is used and domino effect might be real.

Personally I limit work with script basing on 10sec MA DEC/ENC live usage and increasing price dramatically once threshold is reached. That works great and unlocks some potential when compared to using session limit only. IE one can have 40 480p live sessions instead of using 20 session limit which might be too conservative in that case, but might be too little when some VOD work arrives.

Also counting decoded pixels is also important thing to implement into calculations. 4k INPUT uses much more resources than 1k input, 4k streams max out card really quick and it is not counted in the pricing.

I don’t understand why Os would need to distinguish between Studio Clients? That is not our scope of interest who they have deals with. Studio and any other B can setup their accepted pricing without notifying Os. Distinguishing between job types is important but not between clients. How would that look for other clients “without deal”? It is permissionless network and let it stay like this.

2 Likes
13

If Livepeer studio wanted to separate their price by customer (based on a large scale deal) they could always spin up a second B and a new address and tell the Os that this new B only accepts <70ppp. That would be a quick fix if they see it’s a good idea.

Otherwise the Titan Node Pool has roughly 900 session capacity right now with only 3-8 being utilized.

We are already running at 70ppp so a 3-5x increase would be very welcomed :slight_smile:

3 Likes
14

At this point, anything that will bring more work to the network is a good thing.

@ericxtang 70 wei / pixel may not sound bad now that ETH is reaching $3000 but any thoughts on how this price would pan out for periods of time when ETH drops below $1500 or lower?

1 Like
15

I agree with hthillman. The orchestrators’ customer is Livepeer Studio, and what Livepeer Studio negotiates with its own customers should not concern us. The only thing that should interest us is the price that Livepeer Studio pays to the orchestrators.

If Livepeer Studio wants, they can indeed create a second broadcaster to offer a different price to the orchestrators for some of their streams.

However, a possibility that I think would be interesting for the orchestrators in the future (when there are more different broadcasters) would be to be able to define the number of streams they accept from one broadcaster or another (Or rather, as a percentage of total capacity, since the number of streams can vary dynamically depending on the GPU capacity available at any given time).

This would allow them to allocate more or less of their resources according to the price offered by a broadcaster and not to saturate their capacity with a broadcaster that offers a very low price compared to another.

Finally, to determine the real encoding capacity of an orchestrator, it would be more accurate to use the number of pixels rather than the number of streams.

Indeed, how can we determine the number of streams that a GPU can support, given the difference between a 1080p stream to 480p at 24 fps and a 4k stream to 1080p/720p/480p at 60 fps?

Whereas the number of pixels that a GPU can process in encoding and decoding should be easy to determine.

1 Like
16

I believe part of this proposal was to peg the ppp to $ value of transcoding, such that it will fluctuate with the moving price of ETH. Hunter shared that when this is the case there will also be basic tools that O’s can use to update their pricing accordingly in an automatic fashion, though I could see O’s themselves iterating on that quicker and more effectively than a basic tool that ships with the node.

5 Likes
17

@Strykar Doug is correct - we’ll be adding the ability for Bs to set a static USD target price. This will most likely be a fast follow on the initial price change (as I expect Os will want a little time to see how the operational aspects of the price-chaining implementation work).

4 Likes
18

One thing I’d add here (Karolak alluded to this above) is that broadcasters will certainly be subject to supply and demand. A broadcaster must set a price that entices a sufficient set of orchestrators to support their traffic volume. This price may be different for different job types - for example, if fewer Os support AV1 or it requires newer hardware, then a broadcaster would likely have to pay more to receive adequate service.

6 Likes
19

This definitely seems like something that requires a zoom-out to see the benefits. I think most O’s are fine with this as LPT is more of the income driver than fees, but I do wonder how sustainable relying so heavily on inflationary rewards is if a balance isn’t found between B’s and O’s for fairly priced transcoding eventually.

4 Likes
20

Great to hear and congrats @ericxtang to you and all the amazing team at Livepeer Studio. When it will be announced?

3 Likes