Orchestrator Pool Modification pre-proposal

[kautukkundan]

Currently the orchestrator pool consists of 100 orchestrators based on their stake. In case a new orchestrator wants to join the pool, they must possess stake greater than the orchestrator with the lowest amount of stake in the pool. While doing so the old orchestrator gets evicted from the pool and the new orchestrator takes it’s place.

The issue with the current approach which I feel are:

1. Only 100 orchestrators can participate in video encoding at any time
2. Only the top few orchestrators get attention by the delegators as well as broadcasters

Going over each issue one by one

Only 100 orchestrators can participate in video encoding at any time

Given the most recent data from Livepeer explorer, it looks to me that this limit of 100 orchestrators isn’t much of an issue immediately.

This can be observed by the fact that the orchestrators with the lowest amount of stakes aren’t participating in the network anyway. Their reward calling ratio is much lower. In the lowest 20 orchestrators by stake, only a handful of them are calling reward. This is likely due to the fact that these orchestrators are not receiving enough jobs such that it is not profitable to call reward.

Moreover, a sizeable chunk of orchestrators have a performance score of 0 which directly means that they had not been performing any work. This is independent of the amount of stake that the orchestrator owns.

These points are indicators that there is a need to promote and incentivize users to actually come and perform transcoding and that each and every orchestrator in the pool should get a fair opportunity to compete.

In short, despite having a pool size of 100, many active orchestrators are not able to participate in the network. Once we get to a place where all 100 orchestrators are participating to their maximum potential only then it makes sense to think about making space for more than 100 orchestrators.

This brings us to the the second point

Only the top few orchestrators get attention by the broadcasters and delegators

There is huge gap (about 2.5 ETH) between the fees earned by the number 1 orchestrator in terms of fees and number 2 orchestrator in terms of fees. This difference is even larger between the top earners and the others.

This is a clear indication that almost all the work is being sent to these few whales and others are having a hard time competing. This has been a frequent pain point in the community chat.

This also creates a negative loop wherein the new delegators prefer staking towards the orchestrator already on top of the table thereby pushing the lower end even lower.

Thus the idea should be to provide a ‘fairer’ ground for all the orchestrators.

Solutions

I suggest approaching this in 2 phases

1. Increasing the network output by providing more job opportunities to orchestrators with lesser stake
2. Once this optimum is reached, more orchestrators will want to join the video mining pool. Then increase the capacity of active orchestrator pool

One approach I was thinking of goes like this

• We decrease the pool size
• Then decide who remains in the pool for that round based on lottery influenced by the total stake

The impact of these steps would be several folds

1. On decreasing the pool size, the reserve requirements for the broadcaster will decrease
2. In a smaller pool size, the probability of getting picked for transcoding will increase
3. Since stake requirement will influence the probability of getting selected per round, whales will still have a higher chance of getting picked but will also allow smaller orchestrators to compete and earn fees
4. This eliminates the minimum stake requirement for participation since any orchestrator will have a chance to be in the active set and will attract new orchestrators
5. This will also pave way for multi-delegation where in a delegator will want to stake towards multiple orchs in order to increase returns per round.
6. Thus the entirety of the stake will not be stagnant towards a single large orchestrator and also increase transaction frequency with the protocol

Issues

The biggest issue will be developing an algorithm to pick orchestrators for each new round.

• This transaction will not only be costlier than the current round initialization transaction as the entire orchestrator pool needs to be evaluated for selecting a sub-pool
• It will be more complex as the calculation of stake eligible for rewards per round will change.
• This will rely on pseudo-random number generation which is a complicated topic.
• Also, for multi-delegation support, there needs to be earnings accounting modifications which already complex today.

* I am still thinking about the selection algorithm and trying to find similar implementations used in other protocols. Will do a followup in this thread.

Summary

This post talks about modification to the orchestrator pool such that the participation of smaller orchestrators in the transcoding activity increases. This will not only benefit smaller orchestrators in terms of rewards and fees but also attract other new orchestrators (with low stake) to join the network.

Moreover, it also talks about modifying the pool such that an unlimited number of orchestrators can join and contribute to the video mining infrastructure!

Happy to get feedback and ideas!

[hthillman]

A few direct responses

There is huge gap (about 2.5 ETH) between the fees earned by the number 1 orchestrator in terms of fees and number 2 orchestrator in terms of fees. This difference is even larger between the top earners and the others.

This is a clear indication that almost all the work is being sent to these few whales and others are having a hard time competing. This has been a frequent pain point in the community chat.

I think the diagnosis is roughly correct, but I’d like to add some context that affects the approach.

First, the protocol actually tilts the selection a little towards smaller orchestrators; Vires’s share of fees earned is actually lower than his share of total active LPT stake. It’s fair to argue that the selection should be more redistributive, but I’m not convinced that this needs a protocol-level solution.

This also creates a negative loop wherein the new delegators prefer staking towards the orchestrator already on top of the table thereby pushing the lower end even lower.

Similarly, this is more a result of the display of orchestrators in the Explorer. In terms of ROI, it’s actually much better for a self-interested delegator to stake with a low-stake O than with a high-stake O. LPT ROI is entirely dependent on reward cut %, but ETH ROI depends on the delegator’s stake in relation to the total stake of the specific orchestrator . If I have 100 LPT and I stake it to an O with 1000 LPT, I’d get 10% of the fees that they share with delegators. But if I stake that 100 LPT to an O with 100 LPT, I’d get 50% of the fees that they share.

6. Thus the entirety of the stake will not be stagnant towards a single large orchestrator and also increase transaction frequency with the protocol

Since Confluence, we’ve seen an order-of-magnitude increase in redelegation activity - upcoming explorer updates are targeted to increase this activity as well. I agree that this is something to monitor, though!

2. In a smaller pool size, the probability of getting picked for transcoding will increase

I get what you’re suggesting here, but I’d argue it’s another example of something that can be solved independently

other questions/thoughts

• If we eventually increase the pool size, will we run into the same problem that we have today re: reserve requirements? It seems like we’ll have to increase it over time as demand increases…and then capital requirements go up again.
• I do think it’s likely that we’ll eventually have to institute some concept of an “active subset”, but I don’t think it’s fair to lock out certain orchestrators for a round.
• consistency of payments for orchestrators is a challenge as well

[vires-in-numeris]

I’ve already stated my point on Discord regarding this numerous times, so I’ll try to keep it short here

1. Stake concentration is not a security or any other concern in the Livepeer protocol.
2. There are already enough incentives present to stake towards a smaller O. I also wouldn’t say that stake is stagnant, we’ve seen quite some movement since the Arbitrum migration.

@kautukkundan

Thus the idea should be to provide a ‘fairer’ ground for all the orchestrators.

3. I don’t see how taking work a away from bigger O’s (that have been here for years and put a lot of work and time in) and giving it to newcomers is considered “fairer”.

@kautukkundan

Increasing the network output by providing more job opportunities to orchestrators with lesser stake

4. This “solution” simply does not work in a permissionless protocol. If I’d lose too many jobs to smaller O’s, I’d just join the club and also setup another (smaller) O. I can even use the same GPUs for that second smaller O. Maybe even a third and fourth if I can get a lot of work with only a small amount of stake… I think you can see where this is going

@kautukkundan

This will not only benefit smaller orchestrators in terms of rewards and fees but also attract other new orchestrators (with low stake) to join the network.

5. We have enough O’s (supply), what we need is more work (demand). More O’s with a constant amount of work just means that everyone gets less work.

[rafal]

Thanks @kautukkundan for the proposal! I completely agree on the point that we need to work on increasing the transcoding demand. Then, it would actually solve a lot of issues you described.

When it comes to decreasing the pool size and the lottery-based selection, let me add a few points (apart from what @hthillman and @vires-in-numeris already wrote).

1. For the reserve requirements, I think we can address this issue in some other way (maybe change the whole idea of reserve or introduce some O-O off-chain P2P communication).

2. I believe that decreasing the number of active Os is actually the opposite of what we want to achieve. The ideal Livepeer network state is when demand equals supply. Currently the demand is not that high, but when we finally manage to increase the demand, we need to be prepared to increase the supply. So, the solution with artificially limiting the supply in the protocol is not future-proof.

3. Another problem I see with randomly limiting the number of active Os is that it may actually be discouraging for new Os. Their LPT stake is low, so they have little change in the lottery-based pool selection.

4. Promoting low-on-LPT small Os as opposed to high-on-LPT big Os is questionable. One thing, as mentioned by @vires-in-numeris , is that you can always create multiple Os and “cheat the system”. Another thing is that we technically decrease the network security which is based on the LPT stake. Taking it into the corner case scenario, imagine that we select the new pool of 10 Os. We want to promote new small Os, so an O with a just 2 LPTs get selected for the whole round. Now, B has the 1/10 probability of using this new O which may actually not do its transcoding job correctly.

[yondon]

Thanks for sharing these thoughts @kautukkundan.

Response to Stated Issues

@kautukkundan

Only 100 orchestrators can participate in video encoding at any time

You’re right that this is not a problem today and that there is not an immediate need to expand past 100 orchestrators.

The actual problem that is more important to focus on is that as the # of active orchestrators increases the minimum reserve requirement for broadcasters increases as well [1]. And as a result, any decision making process for increasing in the # of active orchestrators would have to factor in the negative side effect of increasing the broadcaster minimum reserve requirement. So, it is worthwhile focusing on addressing this problem, not because there is an immediate need to increase the # of active orchestrators, but because we should make sure that when we do need to increase the # of active orchestrators we can do so without negatively impacting the broadcaster UX (additionally, addressing this problem would overall improve the broadcaster UX).

[1] The minimum reserve requirement right now is roughly equal to estimated tx cost of winning ticket redemption * # of active orchestrators. This requirement is the bare minimum required for an orchestrator to accept requests from a broadcaster and would result in a ~99% increase in prices for the broadcaster because at this reserve level the orchestrator would have a ~99% overhead for redeeming tickets which is covered by charging a premium on top of prices advertised to broadcasters. In order to reduce the premium to 1% (and a 1% overhead for orchestrators), the reserve requirement would be estimated tx cost of winning ticket redemption * # of active orchestrators * 100.

Response to Proposed Solutions

The problem with sampling a “sub pool” in-protocol based purely on stake and then only allowing Bs to select from this sub pool is that this sampling process does not consider each Os ability to actually perform. Additionally, the protocol (more specifically the contracts) is not aware of an O’s ability to actually perform largely due to the fact that it is very difficult and maybe even infeasible to prove anything on-chain about the latency of an O when processing requests.

If this design was used, then a failure scenario could be that a number of Os in the sub pool are unable to perform well or worse are offline. As a result, for as long as that sub pool is fixed (i.e. until we sample again to construct an updated sub pool), B will either have a more limited ability to failover in the event of a problem with a selected O (i.e. the O is at capacity) or worse the B will have no Os to select from if the entire sub pool cannot accept additional work.

The “sub pool” creation process already exists in a different form today except it all happens off-chain and is executed at the B level and the criteria for creating the pool is not based on stake. Instead, it looks roughly like this:

• B creates a pool based on all available active Os that includes the ones that respond within a timeout which ensures that a) the Os respond within a reasonable time frame for a simple request and b) the Os are online
• B runs a selection algorithm to select Os from the pool using stake as one of the criteria

While I am open to alternative ideas, generally I think selection (or pool creation for selection) and job distribution is better of being executed primarily off-chain by Bs because the protocol contracts are unable to acquire sufficient provably correct information to make those decisions for Bs in a way that ensures the probability of high QoS for Bs and can avoid bad failure scenarios.

Tying this back to # of active orchestrators <> minimum broadcaster reserve requirement, which I highlighted as the most important problem to focus on amongst the ones mentioned in the OP, I think we should either:

• Consider ways to align the reserve requirement with the # of Os that a B actually intends to use (as opposed to aligning the reserve requirement with the # of all active Os even if the B does not intend to use most of them)
  • Note: The proposal in the OP does sort of do this, but at the cost of defining the O sub pool in-protocol which has the problems mentioned earlier - better to define the O pool at the B level while also only requiring reserve funds proportional to expected usage of the members of the O pool
• Consider ways to avoid having the reserve requirement scale with the # of Os at all. One option that was suggested during DevConnect conversations was moving to model with a single slashable stake that does not need to scale with the # of Os (although as-is this has a few problems/considerations as well including it being more difficult to reason about security and it being easy for a B to accidentally get slashed for depleting its deposit)