Livepeer’s probabilistic micropayment system has been a game-changer for decentralized transcoding, enabling a scalable and cost-effective way to pay orchestrators for their work.
However, as Livepeer expands into AI inference workloads—such as LLM requests or other computationally variable tasks—the current ticket-based system starts showing its limits. Generating a new ticket for every inference request can inflate costs for users, especially when the computational demand fluctuates wildly between tasks. To address this, I’d like to propose a smart contract-based solution that uses “persistent tickets” to better align costs with real resource usage, while keeping the system fair and decentralized. Here’s the idea—let’s discuss it!
The Problem
In the current setup, each transcoding segment or inference request triggers a new probabilistic ticket, which orchestrators redeem for payment. This works great for video transcoding, where tasks are relatively uniform in cost. But for AI inference:
- Cost Variability: A simple LLM request (e.g., translating a sentence) consumes far fewer resources than a complex one (e.g., generating a 1,000-word article), yet both generate a full ticket.
- Rising Costs: Every ticket comes with on-chain transaction overhead (even if minimized on Arbitrum), making frequent small requests expensive for users.
- Lack of Granularity: Users pay based on ticket generation rather than actual computational effort, leading to inefficiencies.
For AI use cases like Daydream or other real-time inference pipelines, this mismatch could discourage adoption unless we adapt the payment model.
The Proposal: Persistent Tickets via Smart Contract
What if we replaced per-request tickets with a persistent ticket system managed by a smart contract? Here’s how it could work:
- Initial Deposit and Persistent Ticket:
- A user deposits ETH or LPT into a smart contract when they start using the AI inference service.
- The contract issues a single “persistent ticket” tied to that deposit, representing a pool of credit the user can spend over multiple requests.
- Cost Estimation per Request:
- Before executing a request, the orchestrator estimates its cost (e.g., based on tokens generated, GPU time, or a standardized metric).
- This estimate is submitted to the smart contract, which checks if the ticket’s remaining credit covers the cost.
- Progressive Debit and Proof of Work:
- If approved, the contract debits the estimated cost from the ticket and signals the orchestrator to proceed.
- After completing the task, the orchestrator submits a proof of work (e.g., a hash or signature) to confirm execution. The contract adjusts the debit if the real cost differs slightly from the estimate.
- When the ticket’s credit is depleted, a new one is generated from the user’s deposit—or they’re prompted to top up.
- Preventing Abuse:
- User Protection: The user must approve the cost estimate upfront (via a signature), preventing orchestrators from overcharging.
- Orchestrator Accountability: Proof of work ensures tasks are completed, with reputation or slashing mechanisms for dishonesty.
- Cost Standardization: Both sides agree on a transparent cost metric (e.g., cost per token or per compute unit) baked into the protocol.
Benefits
- Cost Efficiency: Fewer tickets mean fewer on-chain transactions, slashing gas fees for users.
- Fair Pricing: Payments reflect actual resource usage, not an arbitrary ticket count.
- Scalability: This adapts to AI’s variable workloads, making Livepeer more attractive for inference tasks.
- Transparency: The smart contract serves as a public ledger, ensuring trust across the network.
Challenges and Mitigations
- Accurate Cost Estimation: Orchestrators need reliable ways to predict costs. We could standardize metrics (e.g., tokens processed) and use oracles or reputation to penalize bad estimates.
- Smart Contract Complexity: Adding this logic increases development effort and security risks. Thorough audits and phased rollouts could mitigate this.
- Latency: On-chain checks per request might slow things down. Off-chain batching with periodic on-chain settlement (like payment channels) could help.
Implementation in Livepeer
This could build on the existing TicketBroker contract:
- Extend it to support persistent tickets with a running balance.
- Define AI-specific cost metrics in the protocol (e.g., cost per token for LLMs).
- Integrate orchestrators to report costs and proofs of work to the contract.
A proof-of-concept could start with a testnet deployment, targeting a use case like Daydream or Vtubers to refine the system.
Why It Matters
As Livepeer grows into AI inference, aligning payments with real costs will be key to staying competitive. This proposal keeps the decentralized ethos intact while making the network more user-friendly and cost-effective for emerging workloads. It’s not about replacing the current system but evolving it to handle transcoding and AI seamlessly.
Discussion Questions
- What do you think of persistent tickets for AI workloads?
- Are there better ways to estimate and verify inference costs on-chain?
- Could this integrate with existing treasury or fee mechanisms?
- Any risks or edge cases I’m missing?
Looking forward to your thoughts—let’s brainstorm how to make Livepeer the go-to network for decentralized AI inference!