Pre-Proposal: Network Engineering SPE II

Treasury
21

Thank you for the recent updates. @ECWireless, @shane, @MikeZupper, and @j0sh,

In addition to the above updates, you can find the budget ongoing here

the status of the milestones can be tracked via the following links:

22

@MikeZupper thanks for the Build Track update. I’d like to offer two things before the 16 Oct integration contracts are fixed, both from work on what a publisher actually hands to the network.

1. A shape for the capability schema. The architecture lists “input descriptions” under discovery, and the schema itself is still open pending the simple-infra review. I’ve been working from a frame where everything anyone publishes reduces to four kinds: text, audio, static image and moving image. A capability declares what it accepts and what it emits, in those four kinds. Captions accepts one audio and emits one text. An audiobook accepts one text and emits one audio. Transcoding with captions accepts one video and one audio and emits N video, N audio and N text. A job declares its tracks the same way, so matching a job to supply becomes mechanical rather than editorial.

The frame also carries extent: instant, finite or unbounded. Unbounded (live) is a structural difference, not a flag: the result is a feed published incrementally rather than one object. That matters if streaming moves out of stretch scope later, and it costs nothing to leave room for now. The write-up is here.

2. An Ethereum-key access adapter, as an extension. The design already leaves room for “supported identity/policy integrations” through an adapter, without forking core. I’d like the sovereignty-preserving self-custodial case to be one of them: the end user’s own Ethereum key as the credential, paying with their own Livepeer tickets through a signer they run, and results written to Ethereum Swarm under a postage stamp the user signs. The operator then processes and stores on the user’s behalf without ever holding their storage or their output.

This represents the shape of all my current work integrating the processing capabilities of Livepeer with the storage and distribution capabilities of Swarm, focused exclusively on Publisher end-user-centricity, without the need for any intermediary between the Publisher and Livepeer/Swarm. This runs end to end today. On 8 Oct a capability wrote its output to Swarm under the Publisher’s own stamp, and it came back out of a public gateway, named via ENS.

As well as empowering individual Publishers, this also paves the way towards empowering Publishing collectives, governed onchain by e.g. m-of-n individuals (e.g. Safe).

Neither asks the core to change. It’s a request that the schema and the access contract leave room for both, so that “self-sovereign” can reach the end user and not stop at the operator.

I can write either up against your integration contracts in whatever format helps.

1 Like
23

Delegator Track - Update #2

Period: 10/6 through 10/31

October focused on closing out old PRs, resolving delegation bugs, improving account history, and addressing security and maintenance issues. Milestone 1 is now 88% complete, with 23 of 26 tracked items closed. The remaining work focuses on Livepeer 2.0 UX design.

Delegation & Account History

Several long-standing delegation bugs were resolved, alongside improvements to how account activity is displayed.

Bug Fixes & Old PRs

Additional work focused on resolving existing bugs, improving accessibility, and clearing out older PRs.

Security & Maintenance

October also included dependency updates and improvements to the repository’s security scanning and CI configuration.

Livepeer 2.0 Research

Research into the Minimum Fee Stake (MFS) model was completed as part of Milestone 1.

The research explores how Livepeer 2.0 changes delegation incentives, particularly how fee share and stake share influence node operator rewards and create opportunities for delegators to move stake toward under-staked nodes.

This work provides a foundation for the remaining Milestone 1 UX design tasks.

Upcoming Work

November will focus on completing the remaining Milestone 1 research and design tasks, including the MFS delegation experience, reward-history views, and validator UX. Work will also continue on high and medium-priority bugs, repository maintenance, and preparations for implementing Livepeer 2.0 features.

24

Network Engineering SPE — Build Track Monthly Update: October 2026

October completed M2 and brought M3 to the halfway mark by task count. The shared builder engine is now an open-source package that has run paid jobs on public Livepeer orchestrators and matched each charge to the job that caused it. M2 closed on October 19, three days after its October 16 target, with nothing descoped. In M3, nine of 18 tasks are done and eight more are in open pull requests. The last is the funded seven-outcome run that M3 must deliver, which has not happened yet. Its dry run is due November 13 and the gate is November 20, but M3 can be completed as early as the end of this week, November 6. The December 31 delivery date is unchanged.

Author: Mike Zupper · Reporting period: October 1–31, 2026

Where the seven outcomes stand

September’s update named seven outcomes for builders. This table shows each one on October 31. M3 work merges to the releases/m3 branch; main still holds the M2 release.

Outcome Merged In open pull requests
1. Obtain access API keys with inspection, rotation and revocation; JWT tokens from an enterprise identity provider None
2. Discover capabilities and supply A discovery service with on-chain and static orchestrator directories; a live catalog None
3. Understand a price Advertised network rates in the catalog Source, age, and stale or missing markers on every rate (#77)
4. Invoke a capability REST and embedded use; immediate and streamed jobs; repeat-safe commands; paid failover None
5. Receive a result or actionable failure Inline results and failure kinds Distinct payment-failure reasons (#78); paged history and documented result access (#81)
6. Pay without a crypto wallet Clearinghouse Batteries allowances and the remote signer Funded configuration write-up (Builder Core Console #12)
7. Inspect usage and cost Cost per job and by class; reconciliation with Batteries Cost per app (#82). The events feed is held

Merged means built, tested and on the branch, not proven in the gate. At month end, CI on releases/m3 passed at 2edf903, with 1,534 tests at 98% coverage on SQLite and a separate PostgreSQL store suite. Across NE-Build-Track, 74 pull requests merged in October.

October’s work

  • Closed M2. At the October 16 checkpoint, four of M2’s seven tasks were accepted and paid jobs had succeeded on public orchestrators. The contract, payment and package tasks closed on October 19, when each charge matched its job and reconciliation balanced exactly on the tagged v0.1.0 builds. We showed the work at a demo on October 20 and posted the M2 update to the project board, so this is its first report in this thread.

  • Set the M3 plan. On October 21 we chose the Builder Core Console as the reference application, with streamed chat (Qwen) and image generation (FLUX schnell) as the representative jobs. The engine reports wholesale network cost only; retail pricing stays with each application. Streaming merged the same day.

  • Built the reference application (ISS-19, ISS-21, ISS-22). The Builder Core Console gained an Operator area for grants, funding, actors and keys, and a Builder area that runs the journey over REST. It was delivered on October 21 and accepted on October 27. An October 21 paid run of two streamed chats and three images cost $0.0092, matching the actor’s spent allowance.

  • Made cost reporting hold up under failure (ISS-16, ISS-17). A running paid job now reads pending, never unpaid. The engine stores each payment’s identifier before asking the signer to sign, so a charge stays attributed to its job even after a crash.

  • Added paid failover. In John’s change, the engine sends the paid runner request itself and can move a job to another orchestrator when a paid attempt is refused. An operator setting caps the paid attempts a job can fail over from, 2 by default, and the operator bears those payments. This answers the failover request made earlier in this thread; recourse for a job that fails partway through is not part of the accepted plan. A live paid failover between two orchestrators ran on November 1, just after this period, and spend matched exactly.

  • Made commands safe to repeat (ISS-11, ISS-14). Every command that can move money, create a credential or run paid work can now be repeated under the same operation reference without applying twice. Live checks on Arbitrum One on October 29 passed through the engine (37) and the Builder Core Console’s Operator page (14). This settles the retry rules carried from M2.

  • Separated discovery and finished standalone credentials (ISS-09, ISS-08). Discovery now runs as its own service with on-chain and static directories, and the signer only signs. Keys can be inspected and rotated, and key events are logged without secrets.

  • Started early on M4’s enterprise sign-in (ISS-33). The engine accepts JWT access tokens from an identity provider and has a read-only auditor credential. The Builder Core Console signs in through a provider, tested with Keycloak through a paid job.

  • Wrote guides for the remaining work. Implementation guides cover the remaining M3 tasks and every M4 and M5 task. None needs an upstream change.

In review at month end

Eight M3 tasks are in open pull requests: John’s #77, the stack from #78 to #83, and Builder Core Console #12. On October 30 we ran this work together on a local build against Arbitrum One. All 41 engine checks passed, three of them after fixes to the test runner rather than the engine, covering payment failures, access, secrets, outages, stream restarts and cost-sync replay. Network cost for the day was $0.011. Browser checks found two defects in the Builder Core Console, one fixed in its #12 and one waiting on #77. A rehearsal of the seven-outcome journey also completed, at about $0.00011 per chat and $0.0030 per image, with reconciliation balanced. None of this is the gate, which must run on merged code.

Working with upstream

  • Batteries and signer. Josh merged idempotency keys, narrower reads and attributed usage in Clearinghouse Batteries on October 6, and fixed the signer so it accepts orchestrators whose ticket recipient differs from their node address. Paid jobs on public orchestrators followed on October 14.
  • Python SDK. The M2 update listed SDK #77 as in review. Josh’s #81 replaced it and merged on October 21.
  • No forks. By October 28 the engine pinned livepeer/* for the SDK, node and Batteries instead of our forks, keeping one adapted copy of the SDK’s paid call with a test that fails when upstream changes it. No SDK release on PyPI includes the changes we need, so the engine pins the SDK by commit and can’t go to PyPI itself until one does.

Applications on the engine

  • ChatLPT embeds the engine and now shows the extra network cost of a failed-over reply.
  • Inference gateway, new this month, puts OpenAI- and Anthropic-compatible APIs on the engine. Its October 29 paid check reconciled 62 paid attempts exactly, on one orchestrator.

Open items and risks

  • Merge timing. The stack must merge in order, and the Builder Core Console then re-pins to the result. This is the main risk to the November 6 and 13 checkpoints.
  • Payment latency. A partner measured a 3.2–3.5 second wait before each paid inference reached the GPU pool. Their logs attribute 98% of it to chain RPC calls, and by our reading the node code makes 29 sequential, uncached calls per paid request. We now record where each attempt’s time goes, but nothing has reduced the wait yet. The fix may need changes in livepeer/node. The M3 gate doesn’t depend on it; real-time products will.
  • Usable supply. Discovery leaves out orchestrators with self-signed certificates, because the engine can’t call them. A late-October test excluded three and left two.
  • Still open from M2. We still lack a payment reference the caller owns, so a collision between payment identifiers reads as uncertain. The forged-manifest and rejected-ticket cases are tested but not yet shown live on the current stack.
  • Held by decision. The ISS-18 events feed stays held; John has written a delivery proposal for it. His design for media outputs and realtime sessions (ISS-27) is open for review.

Next: finish M3 (due November 20)

Our working checkpoints are Builder Core Console areas complete by November 6, a dry run by November 13 and the gate on November 20. The gate is one funded run on merged code, with evidence linking a builder key, live discovery and rate, execution, result, usage and final network-cost class. Spend across all runs is capped at $1. M4 starts November 23. The next monthly update will report the gate result.

The project board tracks every task. The accepted delivery specification links the architecture, required scope and milestone acceptance criteria through December 31.

1 Like
25

Validator Track — October Update

Period: October 1–31, 2026

October completed M2’s economic modeling and moved several verification ideas into working proofs of concept. The focus was understanding what validators can establish from available evidence, then building and testing ways to fill the gaps.

The M3 workbench carries this into a broader question: how can validation support both economic integrity and dependable service for customers? The working direction is scoring that addresses both, expanding the litepaper’s focus on dishonesty. Measures and reward effects remain under evaluation; poor service and dishonesty need distinct treatment.

Modeling — mapping the self-dealing profitability window

M2 is complete and reviewed. Its self-funding models established conditions under which self-funded work can increase an operator’s net return: fee share is below stake share, and the additional rewards retained exceed unrecovered payment and execution costs. Reaching the stake cap or facing higher costs can close that opportunity. Token value, customer fees and their distribution also move its boundaries.

An exploratory extension of M2 found 79 of 100 pools, representing 88.2% of stake, had some positive modeled opportunity. This is a conditional model result.

This window gives mechanism development a concrete target: improve data visibility and work verification, reduce the potential economic gain, and make evidence-supported dishonesty carry credible reward consequences. Verification alone is insufficient when self-funded jobs really happened. The goal is to reduce material, feasible gains while distinguishing abuse from legitimate internal work—not to penalize operators simply for being inside the window.

Data aggregation — building the evidence base for validation**

The M2 evidence study establishes the need for records beyond on-chain accounting. Fees and stake do not establish what a customer requested or received. Meaningful validation needs records connecting what was agreed to what happened:

  • Job identity: link requests, attempts, retries and payments.

  • Accepted terms: capability, model, price and service requirements.

  • Observed outcomes: delivery, performance and failures.

  • Evidence provenance and coverage: who observed what, missing records and operator counterevidence.

This is a likely baseline, rather than sufficient evidence for every score. Fingerprinting and TLS witnessing explore particular claims about whether the promised service was delivered; completion, latency and availability need their own observations and checks.

For these records to support reliable validation, evidence needs corroboration beyond the operator being assessed. A gateway could collect off-chain records as a trusted data provider or as part of validator responsibilities, complemented by selective proofs. Detailed records need bounded retention and disclosure rather than permanent on-chain storage. Independence, incentives, privacy and coverage—including failures—remain design questions.

Fingerprinting — testing which model performed the work

Pass: the bounded model-discrimination tests succeeded under controlled conditions. This supports continued development; it does not establish a deployment-ready verification system. PoC code, reports and results.

We built GPU experiments using compact summaries of models’ internal computations. Under controlled settings, these correctly distinguished the tested speech models in 36/36 held-out cases and closely related image models in 80/80 held-out generations.

A further individual-job prototype checked jobs against trusted reference runs and detected tested model substitutions. It also exposed important gaps: genuine evidence could be reused without fresh execution, and incorrect delivered images could pass without an independent delivery check. These results support further investigation, with replay protection, runtime variation and verification cost still open.

Judging — identifying model substitution from outputs

Block: the tested methods were not reliable enough to support validator enforcement of model identity. This finding concerns substitution detection, not every use of output-quality assessment. PoC code, reports and results.

We tested image quality scoring, AI judging, image-pattern analysis and comparisons across multiple outputs. The tested methods did not provide a reliable way to distinguish closely related models: the combined quality/visual method identified **22/36 held-out images** correctly. Other tests missed substitutions or flagged ordinary image compression as suspicious. This currently looks less promising as evidence for validator enforcement.

TLS witnessing — verifying calls to external providers

Possibility: the prototype demonstrated exchange verification, with practical cost, privacy and trust questions still open. PoC implementation and usage and limitations.

We built a TLSNotary-based prototype to test whether a witness could verify the destination, request and response of an external provider call. October tests covered real OpenRouter text and image responses, offline evidence checking, streaming and witness interruptions. Testing also uncovered and led to a fix for a request-verification flaw.

This is a tentative path worth pursuing for external-service verification. It establishes evidence of an exchange with a named endpoint; it does not prove the provider’s internal model. Credential privacy, witness independence, performance and failure handling remain research challenges.

Next Steps

These findings give M3.2 a starting set of evidence components to evaluate. Work continues on how gateway observations and selective proofs could support economic-integrity and QoS scores, who provides and reviews the data, and how rewards should respond. The combined mechanism is not yet built or selected. The planned Livepeer 2.0 testnet—our first implementation of the proposed 2.0 rules at the smart-contract and node levels—will let us test validation mechanisms and economic strategies together in a working 2.0 environment.