MVP Performance Incentive Program Thread

Transcoders
1

Today is the start of the 1st round of the new experimental performance incentive program funded by a Livepeer community node grant!. The goals of this program are to kickstart experimentation with performance incentive programs to reward well performing orchestrators on the network and to generate empirical data that can be used by the community to design additional performance incentives for the network.

This program will last for the next 3 weeks until 1/4/20 and during this time well performing orchestrators will receive LPT performance rewards in addition to the LPT staking rewards that they are already earning. Orchestrator performance will be evaluated using data from the leaderboard.

This forum thread will be used to:

  • Establish rules for the program
  • Establish rules for a specific round of the program
  • Announce reward distribution results for a specific round of the program

Program Rules

  • The program will be divided into 1 week rounds
  • Orchestrators that return transcoded results that correctly correspond to input video sent for leaderboard transcoding tests will be eligible for consideration in rewards distribution
  • Rewards will be distributed by a program administrator account that holds the grant LPT
  • At the end of the round, a snapshot of the leaderboard 7 day view will be generated and reward distribution will be based on the data in this snapshot
  • The reward distribution rules will be fixed for the duration of around
  • The reward distribution rules can change in-between rounds
  • Livepeer Inc. operated orchestrators will be excluded from reward distribution

The administrator will distribute the rewards to the best of their ability based on the rules for a round and the leaderboard data, but given that this program is experimental and the leaderboard infrastructure is fairly new, the administrator does not provide any guarantees to participants that take actions based on the rules of the program.

Any changes to the rules for the next round will be announced at least 2 days before the start of the round. The reasons for allowing reward distribution rules to change in-between rounds are:

  • To adjust distribution based on results of previous rounds and community feedback
  • To allow for updates to leaderboard scoring that capture aspects of performance that are currently not incorporated into scoring (i.e. capacity/scale)

Round 1 (12/14/20-12/21/20) Rules

  • A total of 7000 LPT will be available as rewards
  • The 3 regions being considered are North America (Chicago), Europe (Frankfurt) and Asia (Singapore)
  • All orchestrators that have a total score ≥ 3 in any of the regions will split 4000 LPT equally
  • Each of the top 5 orchestrators in each region in terms of total score that also have a total score ≥ 3 will receive an additional 200 LPT
  • The leaderboard 7 day view snapshot will be taken on 12/21/20 at 3:00pm UTC
  • LPT rewards will be sent directly to the orchestrator’s on-chain registered address on 12/21/20 after the leaderboard snapshot is taken

A total score of 3 can be achieved with a ≥ 6.3 latency score which indicates real-time transcoding and a success rate of 50%

Round 1 Kickoff Report

Note: 0x3e2b450c0c499d8301146367680e067cd009db93 is excluded from the summary below per program rules because it is operated by Livepeer Inc.

At the start of round 1, the following orchestrators have a total score ≥ 3 in at least one region for the past week:

  • vires-in-numeris
  • majOrcha
  • Staked
  • Orcha
  • 0x942f0c28fb85ea0b50bfb76a3ecfa99861fa9b4b

Livepool.io and DeFiCrypto.dev are on the brink of joining this group as well.

Below is a snapshot of the leaderboard standings at the start of round 1 for the past week:

North America (Chicago)

Screen Shot 2020-12-14 at 9.13.33 AM
Screen Shot 2020-12-14 at 9.13.33 AM1619Ă—803 60.5 KB

Orchestrators currently eligible for the regional LPT rewards:

  • Orcha
  • majOrcha
  • Staked
  • 0x942f0c28fb85ea0b50bfb76a3ecfa99861fa9b4b

Europe (Frankfurt)

Screen Shot 2020-12-14 at 9.13.47 AM
Screen Shot 2020-12-14 at 9.13.47 AM1626Ă—808 59.5 KB

Orchestrators currently eligible for the regional LPT rewards:

  • vires-in-numeris
  • majOrcha
  • Orcha
  • Staked

Asia (Singapore)

Screen Shot 2020-12-14 at 9.13.53 AM
Screen Shot 2020-12-14 at 9.13.53 AM1638Ă—813 61 KB

Orchestrators currently eligible for the regional LPT rewards:

  • majOrcha
1 Like
2

Round 2 (12/21/20-12/28/20) Rules

The rules for round 2 will remain the same as the rules for round 1.

3

Round 1 Report

Leaderboard

Note: 0x3e2b450c0c499d8301146367680e067cd009db93 is excluded from the summary below per program rules because it is operated by Livepeer Inc.

At the end of round 1 (12/21/20 3:00pm UTC), the following orchestrators had a total score ≥ 3 in at least one region for the past week and each received 571.428571428 LPT for overall performance:

j1v and 0x10b21af759129f32c6064adfb85d3ea2a8c0209c are new additions to the group since the start of round 1.

Livepool.io and DeFiCrypto.dev are on the brink of joining the group as well.

Below is a snapshot of the leaderboard standings at the end of round 1:

North America (Chicago)

Screen Shot 2020-12-21 at 10.06.39 AM
Screen Shot 2020-12-21 at 10.06.39 AM1659Ă—931 68 KB

The top 5 orchestrators that also had a total score ≥ 3 that each received 200 LPT:

  • majOrcha
  • Orcha
  • Staked
  • 0x942f0c28fb85ea0b50bfb76a3ecfa99861fa9b4b
  • j1v

Europe (Frankfurt)

Screen Shot 2020-12-21 at 10.06.34 AM
Screen Shot 2020-12-21 at 10.06.34 AM1661Ă—938 66.2 KB

The top 5 orchestrators that also had a total score ≥ 3 that each received 200 LPT:

  • j1v
  • majOrcha
  • vires-in-numeris
  • Orcha
  • Staked

Asia (Singapore)

Screen Shot 2020-12-21 at 10.06.27 AM
Screen Shot 2020-12-21 at 10.06.27 AM1666Ă—942 66.7 KB

The top 5 orchestrators that also had a total score ≥ 3 that each received 200 LPT:

  • majOrcha
  • j1v

The full leaderboard 7 day view snapshot can be found here.

Reward Distribution Transactions

Onwards to round 2!

4 Likes
4

Livepeer stimulus cheques! Very very nice!

5

Round 3 (12/28/20-1/4/21) Rules

The rules for round 3 will remain the same as the rules for rounds 1 and 2.

7

Round 2 Report

Leaderboard

Note: 0x3e2b450c0c499d8301146367680e067cd009db93 is excluded from the summary below per program rules because it is operated by Livepeer Inc.

At the end of round 2 (12/28/20 3:00pm UTC), the following orchestrators had a total score ≥ 3 in at least one region for the past week and each received 500 LPT for overall performance:

DeFiCrypto.dev is a new addition to the group since the start of round 2.

Below is a snapshot of the leaderboard standings at the end of round 2:

North America (Chicago)

Screen Shot 2020-12-28 at 10.26.28 AM
Screen Shot 2020-12-28 at 10.26.28 AM2844Ă—1654 304 KB

The top 5 orchestrators that also had a total score ≥ 3 that each received 200 LPT:

  • Orcha
  • majOrcha
  • Staked
  • 0x942f0c28fb85ea0b50bfb76a3ecfa99861fa9b4b
  • j1v

Europe (Frankfurt)

Screen Shot 2020-12-28 at 10.26.19 AM
Screen Shot 2020-12-28 at 10.26.19 AM2850Ă—1648 305 KB

The top 5 orchestrators that also had a total score ≥ 3 that each received 200 LPT:

  • j1v
  • Staked
  • Orcha
  • majOrcha
  • vires-in-numeris

Asia (Singapore)

Screen Shot 2020-12-28 at 10.26.08 AM
Screen Shot 2020-12-28 at 10.26.08 AM2820Ă—1628 298 KB

The top 5 orchestrators that also had a total score ≥ 3 that each received 200 LPT:

  • j1v
  • Orcha
  • majOrcha
  • Staked

The full leaderboard 7 day view snapshot can be found here.

Reward Distribution Transactions

Onwards to round 3!

1 Like
8

Round 3 Report

Leaderboard

Note: 0x3e2b450c0c499d8301146367680e067cd009db93 is excluded from the summary below per program rules because it is operated by Livepeer Inc.

At the end of round 3 (1/4/21 3:00pm UTC), the following orchestrators had a total score ≥ 3 in at least one region for the past week and each received 444.444444444444444444 LPT for overall performance:

Livepeer Grants (Community Node) is a new addition to the group since the start of round 3.

Below is a snapshot of the leaderboard standings at the end of round 3:

North America (Chicago)

Screen Shot 2021-01-04 at 10.10.15 AM
Screen Shot 2021-01-04 at 10.10.15 AM2832Ă—1650 309 KB

The top 5 orchestrators that also had a total score ≥ 3 that each received 200 LPT:

  • majOrcha
  • Orcha
  • Staked
  • 0x942f0c28fb85ea0b50bfb76a3ecfa99861fa9b4b
  • Livepeer Grants (Community Node)

Europe (Frankfurt)

Screen Shot 2021-01-04 at 10.09.56 AM
Screen Shot 2021-01-04 at 10.09.56 AM2824Ă—1642 305 KB

The top 5 orchestrators that also had a total score ≥ 3 that each received 200 LPT:

  • j1v
  • Livepeer Grants (Community Node)
  • Orcha
  • majOrcha
  • 0x10b21af759129f32c6064adfb85d3ea2a8c0209c

Asia (Singapore)

Screen Shot 2021-01-04 at 10.09.42 AM
Screen Shot 2021-01-04 at 10.09.42 AM2844Ă—1642 303 KB

The top 5 orchestrators that also had a total score ≥ 3 that each received 200 LPT:

  • Livepeer Grants (Community Node)
  • j1v
  • Orcha
  • majOrcha
  • 0x10b21af759129f32c6064adfb85d3ea2a8c0209c

The full leaderboard 7 day view snapshot can be found here.

Reward Distribution Transactions

And that wraps up the last round of this experimental performance incentive program! This program was an initial iteration so stay tuned for more developments around incentive programs.

1 Like
9

Isn’t the Livepeer Grants (Community Node) also operated by Livepeer Inc. and thus should be excluded per program rules?

10

Good question.

I believe Livepeer Grants (Community Node) is operated by the grants team (@ericxtang @adamsoffer @nryan). While some members of the grant team are employed by Livepeer Inc., given that there are other cases of Livepeer Inc. team members operating nodes as individuals where the nodes are not a part of Livepeer Inc. operations it seemed reasonable to not exclude Livepeer Grants (Community Node) from reward distribution.

That being said, I can now certainly see a flaw in the original rules since there was not a formal definition of what a “Livepeer Inc. operated orchestrator” entailed. Without that formal definition, questions around whether a node operated by an individual/group with some affiliation with Livepeer Inc. (i.e. their place of employment) should be considered a “Livepeer Inc. operated orchestrator” are understandable. I think this is a good lesson for future programs. My takeaway is that there should be more formal definitions of node eligibility to avoid any ambiguity.

11

Yeah, The Community Node is independent from Livepeer Inc in it’s operations, but I do think we can exclude the Grants Node from any incentive programs reward distributions in the future.

12

@yondon

there was not a formal definition of what a “Livepeer Inc. operated orchestrator” entailed

I am wondering what the motivation behind the original flawedly-defined rule was. Perhaps it was about not allowing individuals whose full-time job is to work on Livepeer to be at an unfair advantage over those who are doing this independently / permissionlessly.

Or perhaps there was something about the fact that Livepeer Inc. defines the rules and runs the tests, so effectively anyone “on the inside” is at a substantial advantage over independent operators, as they can set the rules to increase their chances of winning.

That said, we don’t know for sure whether any of the other nodes are being run by Livepeer Inc. employees, so in many ways the ruling is kinda unenforcable anyway.

I believe Livepeer Grants (Community Node) is operated by the grants team (@ericxtang @adamsoffer @nryan)

It would be good to have some clarity on what it means for a node to be operated by 3 people?

For example, who holds the keys? Who operates and administers the infrastructure?

I can understand that having a committee to run a grants program (with proposals, grant decision making, etc) makes some sense, but I wonder whether it’s actually just 1 person holding the keys, making the withdrawals and payments, updating the software, administrating the server.

@nryan

The Community Node is independent from Livepeer Inc in it’s operations

How do you reach that conclusion? AFAIK, 2 out of 3 people on “the grants team” are current employees of Livepeer Inc. and my speculation is that the hardware being performance-tested is being operated by the Inc. employees.

My proposal would be that the payment made to the grants program should be returned, and added to the pool for future programs.

13

Agree that all the above insights about more clearly defined and purposeful rules at the start could help address this. Also just pointing out that the funds being returned and added to the pool for future programs, is in effect, the same as the grants node earning the reward. The LPT came from the grant node, and by earning the payout, is already back in the grant node for future grant programs. The remediation here would be the grant node sending the LPT back to itself.