vllm-realtime: realtime speech transcription over Trickle + WebSocket

Decision: Approved

The Review Team approves the requested $500 in full, subject to PR #28 completing the normal merge process.

The contribution addresses the network engineering need established in Issue #2 by providing a self-contained realtime speech-transcription example using Trickle audio input, WebSocket transcript output, a GPU-free mock backend, and a real vLLM/Voxtral backend. It demonstrates streaming transcripts, live settings transport, lossless backpressure, detailed performance statistics, and both offchain and paid execution.

Independent review deployed the example through a registered Arbitrum mainnet orchestrator using go-livepeer v0.9.0 and publicly trusted TLS. Testing returned the correct transcript, delivered 14/14 segments without loss or retries, completed with a 0.437-second finalization tail, and recorded a paid PymtHouse session costing $0.005002.

The requested amount is modest and proportionate to the scope, technical complexity, network relevance, and useful developer-experience findings.

Before payment, PR #28 should be updated against the current main branch, its non-mergeable state resolved, and the normal CI, review, and merge process completed. It should also be updated to the current Livepeer gateway API; independent testing required a compatibility build because the submitted code targets an older mutable SDK branch.

The submitted client disables TLS certificate and hostname verification. Trusted certificate verification should be enabled by default, with any insecure mode restricted to an explicit local-development option.

The paid client currently makes only the reservation payment and discards the payment-session object. It should retain that object, send continuing payments during long-running streams, close it correctly, and include a paid-session regression test.

The live-language claim should either be demonstrated using a setting supported by the selected vLLM version or described as settings-transport functionality. The current implementation forwards the language field, but vLLM 0.24 ignores it.

The example currently assumes a locally hosted, unauthenticated remote signer. Its --signer option does not expose authentication headers or token refresh for hosted signers such as PymtHouse. This limitation does not justify reducing the award because the documented local-signer path exists and paid execution was independently verified.

Future Follow-Ups

  • Pin the Livepeer gateway commit, vLLM image, model revision, base image digest, and Python dependencies for reproducible builds.
  • Replace the global runner state with concurrency-safe, session-keyed state. Validate that /transcribe and /ws belong to the same Livepeer session, and prevent stale teardown callbacks from clearing a newer session.
  • Add bounded queues, strict audio and settings validation, meaningful backend health checks, and explicit error propagation so incomplete transcription cannot appear successful.
  • Add automated tests covering mock execution, 14/14 lossless delivery, concurrent-session isolation, TLS verification, settings updates, and continuing paid execution.
  • Add authenticated hosted-signer support with bearer headers and automatic token renewal.
1 Like