Livepeer Testnet Implementation Proposal

Protocol
1

The problem

The original SPE proposal mentioned Testnet development as one of the main deliverables; however, it didn’t specify enough requirements to immediately start working on its implementation. Instead, we had to first evaluate the existing tooling and the actual business need, collect input from community members through interviews we conducted in February, run our observations through the Livepeer Security Committee, collect feedback from a watercooler in March, and receive additional use cases from the Livepeer Foundation in April. After systematising all input, we are ready to propose and develop a new testnet approach for the Livepeer protocol.

Research and what we found

Terminology and testnet types

Testing a protocol can be done through different means. While some approaches are clearly outdated, some other solutions can be more useful for certain use cases. Here, we provide a summary of the available testnet types:

  1. Local chain with fresh contracts (e.g., using geth)

    Outdated approach: No cheat codes, a need for a faucet, since all contracts are freshly deployed, all addresses differ from production + someone needs to maintain a script that deploys all contracts with up-to-date values. Even then, live usage data is missing (since fresh contracts haven’t been used).

  2. Local chain with fresh contracts and cheat codes (e.g., using anvil)

    Outdated approach: Same as type 1, but with a small improvement: a special faucet contract is no longer needed; test transactions can “pretend” to have any amount of LPT and/or to be someone else.

  3. Local fork with cheat codes (e.g., using anvil --fork-url)

    Up-to-date approach that requires no additional setup. Executing a command locally creates a local clone of the production network where all contracts have the same addresses as in production and the same historical data. Most useful for creating tests in the CI. Referred to as the local fork in this document.

  4. Remote chain with fresh contracts (e.g., Arbitrum Sepolia)

    Outdated approach: Same as type 1, with an additional disadvantage of maintaining a public rate-limited faucet, to avoid griefing attacks on the testnet. Usually referred to as a public testnet.

  5. Remote fork with cheat codes (RPC_URL="tenderly.co/...")

    Up-to-date approach that utilises an external service (such as Tenderly, BuildBear, etc) to create a persistent fork accessible over the network. Additional benefits: built-in faucet, block explorer, and transaction tracer (allows inspecting submitted transactions). Referred to as the protocol testnet in our proposal, but sometimes also referred to as devnet elsewhere**.**

  6. Other Livepeer services are running over the local fork or the protocol testnet. (RPC_URL="tenderly.co/..." docker compose)

    Since the main product that Livepeer provides is not a protocol itself, but the offchain compute service, to test the product, one needs to run additional services (e.g., indexer, explorer, transcoder, etc). Referred to as the product testnet in this document.

Existing tooling

Before proposing a new solution, we looked into which testnet-related tooling already exists and how it is being used by the community. TLDR: only type 1 and type 4 tooling were found, both of which are not maintained and outdated.

  • The old “contract addresses” documentation page listed two old public testnets (type 4), none of which are functional:
  • There are hardhat-based scripts inside the protocol repository (such as deploy/deploy_contracts.ts and deploy/deploy_poll.ts) to deploy some of the protocol contracts (prerequisite for type 1 approach).
    • Combined, those two scripts deploy and configure most contracts (Controller, Minter, LivepeerToken, TicketBroker, JobsManager, SortedDoublyLL, BondingVotes, RoundsManager, ServiceRegistry, MerkleSnapshot, Governor, Treasury, LivepeerGovernor, BondingManager, PollCreator).
    • Difference with production or fork-based tests:
      • Some contracts are missing, compared with the contracts in production: AIServiceRegistry, DelegatorPool, L2LPTDataCache, L2LPTGateway, L2Migrator.
      • The script does not seem to configure Proxies for the contracts that have them in production.
      • The script uses hard-coded constructor arguments and other parameters, meaning that newly deployed contracts can be configured differently from those in production.
  • There is a 4-year-old livepeer/geth-with-livepeer-protocol docker image with geth tool (type 1).
  • We couldn’t find an official guide for the fork-based testing (type 3 approach).
  • The private livepeer/governor-scripts repository has simulate command (that we implemented back in 2025) to create a fork-based testnet.

Insights from the interviews

Community members generally seem to have no established or maintained integration tests – tests that check that a tool or a service correctly interacts with the protocol – as it’s now assumed that the protocol is very stable (which is likely due to the protocol only receiving very few updates over the recent years).

  • The most common concern that was brought up when asked about testnets is that many tools have testing dependencies beyond the protocol itself:
    • Example 1. The Livepeer Explorer fetches most of its data from the Subgraph. So, in addition to simply pointing the Explorer app to a testnet RPC endpoint, the team testing it also needs to start a local Subgraph instance against the same endpoint.
    • Example 2. To test their platform, Embody would require an orchestrator to be active on the testnet. So, in order to test the platform end-to-end, in addition to the protocol, they also need to start an instance of an orchestrator or multiple ones.
  • Based on the above, the most common approach for testing products against the protocol is to manually check them against the mainnet network with a wallet that has a small amount of ETH for gas (and tickets). Notes:
    • Some protocol functionality (e.g., being in the earning pool, calling rewards, creating test-LIPs and treasury proposals) is, of course, not testable via this approach.
    • Since this approach uses actual mainnet ETH, it’s simply not suitable for the CI – it will be too expensive to run automated integration testing this way.

Insights from the provided use-cases

Livepeer Foundation provided (in a private document) concrete use cases in which a testing environment will be beneficial. The main takeaways for this proposal:

  • Certain testnet types are more beneficial for certain use cases. For example, most of the time, a local fork (type 3) is desired and sufficient for local testing of an external service. For example, to develop the Explorer or SubGraph, it’s sufficient to use a local fork and easily simulate test events using cheat codes.
  • In other cases, it is beneficial to have a persistent sharable state of the protocol (type 5). For example, if a protocol upgrade will introduce breaking changes, each team (Go-client, Explorer, SubGraph, etc) would benefit from having a single command to test compatibility of their service against the new version of the protocol, instead of either waiting for the production update or locally replicating the update.
  • Use cases that involve cross-team dependencies that require “product testnet” (type 6) are rare.

So overall, we’re fully aligned that there is no fit-for-all solution, and we need to make different approaches available for the different kinds of use cases. More specifically: make local fork-based testing (type 3) approachable and introduce a harness to create remote forks with protocol updates (type 5).

Our proposal

Based on the above, we are proposing the following scope:

  1. Sidestream will create a new public repository livepeer/protocol-updates for the protocol updates, which will replace the existing private repository lacking most of the described features. The new repository will include:

    • A script to create a protocol testnet (type 5) with any update.
    • Public repository structure (to improve transparency and allow everyone to use relevant testnet tooling).
    • Structure to account for both chains (Ethereum Mainnet or Arbitrum) and for different potential admins (Governor or Admin msig).
    • Tests
      • Base tests that run every time and validate the Controller’s addresses.
      • Specific tests that sanity check changes made by each update.
    • Archive with a few past upgrades as examples of this approach.
    • Readme with the documentation.

    Once publicly shared, it will be actively used by the protocol developers for all upcoming protocol updates, increasing the robustness of the future updates, and opening testnet tooling to everyone. Of course, security-related updates will still be developed in private until they are released onchain.

  2. Sidestream will create a new public repository, conceptually replacing the outdated geth-with-protocol. It will include:

    • Main readme that explains which testnet approach to choose for which use-case:

      • Local fork (type 3)

      • Protocol testnet (type 5)

      • Product testnet (type 6)

        Note: Initially it will contain only a single Docker service: the protocol itself. Sidestream wouldn’t be adding any additional Livepeer services ourselves, but expect that external teams can contribute services they maintain (Subgraph, Explorer, test setups for a transcoder and a ticket broadcaster).

    • A tiny Dockerfile that contains the latest anvil and creates a fork (conceptually replacing the existing geth-based Dockerfile).

    • Docker Compose config for a future product testnet (type 6).

    This approach, compared to a full production testnet, already solves most provided use cases while not overwhelming the security-focused SPE to maintain other services. It would allow developers to test client changes, and the Explorer team to test interfaces like staking/voting/etc. Keeping it in a separate repository from security-critical updates would allow the community to own and advance this setup over time.

Next steps

Sidestream will work on the two main deliverables outlined above. Once each is approved by the Foundation (or the Security Committee, where needed), we will open the repositories and publicly share them as a follow-up comment in this thread.

Request for feedback

We’re curious to hear how the proposed changes will affect the teams we already spoke to, but also, especially, those we haven’t reached so far. Please let us know how you plan to use this setup or if you think your use case wasn’t accounted for yet.

5 Likes
Proposal - Protocol R&D Special Purpose Entity
2

Thanks for providing this analysis, writeup, and recommendation. A couple quick thoughts…

  1. I definitely encourage others to reveal any key use cases they have which are NOT covered by this proposal. It’s always been easy to “ask” for a full testnet, but not actually clear why people need one…so it is helpful to know exactly what people are trying to do that would not be covered by this.

  2. Please keep in mind that the set of changes proposed in Livepeer 2.0 are pretty significant. As the final spec come out we’ll want to develop and test prototypes and migration mechanics, so it is worth taking into account that this approach is compatible with larger changes, as well as just incremental smaller ones. If it feels like there’s a better approach to support a larger protocol overhaul, it’s worth considering that now before wasting work on the incremental process. Note, I’m using relative terms like “large”, but I’m referring to many logic and state changes deployed within the current protocol most likely - not an entirely new set of protocol state.

2 Likes
3

Hey @drieddate_sidestream,

Great to see this shared with the community. Thanks for taking my feedback into account and working actively with the community on a setup that covers most needs without the complexity of maintaining a separate protocol testnet.

I’m looking forward to using this setup across the clients, Explorer, and subgraph. I’ve had to work around a lot of testing limitations in the past, so this should make things much faster. I also know the initial version has already helped improve protocol release triage and security.

To echo Doug: if community members see any needs this proposal doesn’t cover, please share your feedback, so sidestream can take it into account during their implementation.

1 Like
4

Thanks @dob for raising those concerns. To answer question 2, addressed at us: The proposed livepeer/protocol-updates repository structure will be better suited for larger protocol updates, as it will allow integration and sanity testing, not available in the current setup. Generally, the new structure (together with the adjusted process) should reduce the risk of protocol misconfiguration for an update of any size.

1 Like
6

Strong support for this. The fork harness is exactly right for protocol and contract work, and having it maintained in a real repo is overdue.

One gap I’d like to raise, which may simply be out of scope: an Anvil fork is local and ephemeral. It gives one person a deterministic sandbox, but it doesn’t give two people a shared one. There’s no way for someone running an orchestrator to test against someone else’s gateway — discovery, capability advertisement, PM ticket exchange, redemption — without pointing both at mainnet and spending real ETH.

That’s the situation a lot of us are actually in. I’ve spent the last few weeks testing a capability against mainnet with real deposits, because there is nowhere else to do it. It works, but “deploy it to mainnet and see” is a rough on-ramp for anyone building something new, and it’s a worse one for anyone who wants to test with somebody else.

So: is a shared, persistent deployment in scope here, or explicitly not? Either answer is useful — I’d just rather know which problem this is solving.

If it is in scope, one technical note. I’d suggest Arbitrum Sepolia over an L1 testnet. Winning tickets are held until their params expire and redemption fires on the next L1 block as seen from the L2, so the L1/L2 relationship is part of what needs testing — I lost an afternoon to tickets that looked stuck and were simply 23 L1 blocks early. An L1-only testnet wouldn’t reproduce that class of bug at all.

Happy to help run infrastructure for it if that’s useful.