Aquareum: The Video Layer for Everything

[iameli]

Hey everyone. Adam and I have been working on a treasury proposal, and I’m so excited to finally share this with the community. This is an idea very close to my heart; something I’ve been working on in one way or another since I first quit my day job to start hacking on video in 2016. Please let me know what you think! Without further ado:

---

Aquareum: The Video Layer for Everything

Background

Social networking is in the middle of a decentralized revolution. The world is moving its social structures away from centralized platforms run by megacorporations to federated structures that better align with our values and needs as a society. Projects like Bluesky, Farcaster, Lens, and Nostr leverage blockchains and public key infrastructure to put control of data firmly in the hands of creators. They all obey the fundamental principle of decentralization: user actions are self-sovereign and immutable.

As these platforms continue to evolve, they will all want rich video functionality. (Some social networks, like TikTok, are themselves 90% rich video functionality.) Video, especially live video, is notoriously difficult and expensive to make work at scale. Legacy platforms such as Twitch, YouTube, and Periscope are massively unprofitable; they only work well due to immense expenditure of servers and bandwidth. Decentralized protocol designers are focused on their protocols, not technical details of video muxing, transcoding, and global low-latency distribution.

Livepeer and MistServer are great at video muxing, transcoding, and global low-latency video distribution. They have a wide range of rich video features, including Livestreaming, VoD, Clipping, Multistreaming, Transcoding, and AI video generation. All of that content can be efficiently muxed to out millions of users, using low-latency WebRTC. But Livepeer nodes don’t have a strong concept of a “user” — who exactly is allowed to stream on a Livepeer node? Livepeer isn’t a social network. It’s not our job to invent decentralized social.

There’s a natural opportunity here — by indexing decentralized social networks and treating those as the source of truth for the world’s freely-available video content, we immediately and permissionlessly enable rich video functionality for all of these platforms.

The end result:

• An iOS/Android/Web app that provides a Twitch/YouTube interface, instantly usable by millions of users.
• A single-binary node that anyone can run, anywhere.
• A set of user-sovereign primitives for expressing creator consent and provenance.
• A trustless protocol, such that any app can connect to any node and be confidently served content associated with the dSocial user.
• And millions of users that can start watching and creating video content immediately.

The Aquareum team is seeking funding from the Livepeer Treasury to deliver on this vision. Toward this end, we’re seeking 20,000 LPT. This funding will go toward compensating our team and renting server infrastructure capable of prototyping global video distribution.

The Team

Eli Mallon: Eli is a longtime core contributor to the Livepeer protocol and was the primary architect behind Livepeer Studio. He will be leading the backend, video development, and protocol design.

Adam Soffer: Adam was one of the earliest contributors to Livepeer, instrumental in building the Livepeer Subgraph, the Explorer, and the Livepeer Studio player and frontend. Adam also created the Web3 Index. He will be leading the design, frontend development, and indexing layers.

Eli and Adam will be leaving our positions at Livepeer Inc. to take on a vision that we see as core to delivering on the promise of decentralized video. We’re excited about participating in the decentralization of Livepeer by establishing another entity in the ecosystem.

The Plan

Aquareum Primitives

This is the schema. The top-level set of primitives are very familiar: Segment, Livestream, Clip, and Transcode. Most developers working with these primitives will only need to use a few to compose rich video experiences.They will all be tightly-coupled with React components, making frontend video development fun and easy.

The primitives will be signed by the content creators and include an expressive language for the allowed distribution of their content. Creators will, for example, be able to express licensing information or choose to make content “expire” after a certain amount of time. Aquareum nodes will respect these fields, giving creators control over the lifecycle of their content in the system.

These primitives will be natively anchored into the signing schemes utilized by decentralized social networks. They’ll work with Bluesky/Farcaster/etc natively. Behind the scenes, this set of primitives is expressible as a Atproto DAG-CBOR lexicon (for Bluesky) or an EIP-712 schema as utilized by Farcaster and Lens. Additionally, these primitives will be C2PA compliant — they’ll respect the industry standard for representing signed video with a coherent provenance chain. This means that C2PA tooling built into video editors and players will be able to confidently inform an end user of the veracity of the video. The EIP-712 schema also makes these primitives suitable for minting onto NFTs.

The end result is an API that provides you verifiable video content, provided by whichever node that is willing to serve it to you.

Aquareum Indexer

This app consumes the firehoses from the social networks, and is conceptually similar to a Graph Node subgraph. It then builds all of users’ video content into a big distributed database. it builds an internal state of video clips and livestreams, exposing a GraphQL interface to be consumed by the application. All data served by the indexer is signed and verifiable within its own cryptographic realm.

Our initial approach will be to deliver indexers for Farcaster and Bluesky, working closely with those communities.

Aquareum.app

An app that runs on iPhone, Android, and Web. A familiar frontend, like Twitch or YouTube. Connects to one or more Aquareum Nodes to display a library of video content. All content is accompanied with C2PA metadata — you can always be confident in what you’re looking at, even if you don’t trust the node you’re connected to. All of the video content is associated with, say, adamsoffer.eth or @iame.li.

Down the line there will also be creation tools — an embedded livestreaming interface and TikTok-style video editor. While new content will often be created and processed in tandem with an Aquareum node to offload the heavy lifting of video processing, the app will also be fully transcode-enabled, utilizing deterministic WASM-compiled encoding libraries.

Aquareum Node

This bundles the indexer, the app, and the full Livepeer + MistServer software stack into a single executable. It’s dead simple to boot it up, configure, and serve content to users over low-latency WebRTC.

It will also facilitate creation of new video content. Users may livestream from the app into Aquareum nodes, and both live and recorded content may be clipped, spliced, composited, and synthesized into new video content. The node leverages the Livepeer Network’s GPU processing to do the heavy lifting of video content creation, distribution, and synthesis. However, data is local to the user’s app/node until published onto an upstream social network, facilitating both statelessness and moderation. Aquareum nodes will utilize the moderation capabilities built into decentralized social networks to facilitate trust and safety.

The First Milestone

Aquareum is an ambitious vision, so it’s important to be clear about what we’re looking to accomplish in this phase.

• First and most boring, we’ll be establishing a business entity: Aquareum Inc. Aquareum will take a long time to make, and we’re in it for the long haul; this is a necessary step to ensure stability for our team.
• We will deliver the creation of the Aquareum node. This includes a build process for a single statically-linked binary. (This will deliver on a piece of the Livepeer Catalyst vision: a unified version of MistServer and go-livepeer).
• We will deliver a full-stack development environment for Aquareum that’s stupid easy to use. Unlike the old Livepeer Catalyst stack, which was opaque and difficult, this will be built from the ground up to encourage community contributions. There will be **one git repository with a single make command capable of building all components of the project, and we will be seeking community contributions from Day 1.
• The Aquareum node will run on Windows, macOS, or Linux on AMD64 or ARM64 processors. The Aquareum app will run on iOS, Android, and the Web. More platforms is better, but that’s a good start.
• Ship the design of the API primitives, with a focus on signed canonical representations, probably including DAG-CBOR, EIP-712, and C2PA schema.
• Build integrations with our first two social networks: Farcaster and Bluesky.
• 100% of what we build will be open-sourced under an MIT or similarly permissive license.

Livepeer Network Improvements

So why should the Livepeer Network participants care about this? Aside from the possibility of Aquareum’s success driving traffic to the Livepeer network processing, there are numerous specific protocol upgrades:

• We’ll be packaging up go-livepeer as a library that can be used within other code, dramatically expanding its use cases. This will make it possible to embed Livepeer Network transcoding in any application capable of linking a C library.
• We will be establishing a GitLab instance with hosted CI runners that will be available free of charge to other projects in the ecosystem. This server will also be used to facilitate project management.
• Aquareum will (finally!) give a coherent answer to how the Livepeer protocol handles scaled video distribution — video users run Aquareum nodes.
• We will seek to upgrade the Livepeer wire protocol to utilize C2PA signatures for transcoding. (This is how it would have been doing originally, if the C2PA had existed when the original protocol was developed.) If necessary, we will deliver the actual protocol improvements via an LIP. This will also dramatically increase the utility of Livepeer transcoding. Transcoded segments are presently mostly opaque. With these schema, transcoded Livepeer segments will instead be digital artifacts with a coherent provenance chain, identifying the user, the transcoder, the content, the licensing, and a million other things.

And one more thing: development will be livestreamed as much as possible — development, code review, project planning will be livestreamed on the Aquareum app and website. We’ll be scratching our own itch for a streaming platform while building everything out. We see this as one of the most honest ways to hold ourselves accountable; tune in and see exactly how the treasury funding is being spent! Our livestreaming premiere will be on Tuesday Jul 16 2024 10:00 PDT (12:00 EDT, 17:00 UTC); check the countdown at aquareum.tv.

[Karolak]

My wishes from a few days ago seem to be coming true. I had hoped for more “Livepeer studios” to focus on innovation rather than competition, given the ample space available in the ecosystem.

Regarding the proposal, it is quite technical, and incorporating visual aids such as schemas or graphs could help clarify the concept.

I also hope there are capable successors to maintain the “Livepeer in a Box” initiative. I am excited to see this progress!

[iameli]

My wishes from a few days ago seem to be coming true. I had hoped for more “Livepeer studios” to focus on innovation rather than competition, given the ample space available in the ecosystem.

Yeah - I saw that message and held off on replying, knowing that this was going live in a couple of days. I also think it will be healthy for the ecosystem to have a few more core entities developing on the software.

Regarding the proposal, it is quite technical, and incorporating visual aids such as schemas or graphs could help clarify the concept.

Great feedback, I’ll definitely be looking to clarify things as much as possible over the next few days. Here’s one thing that I’ll try and convey:

1. Streamer streams into Aquareum Node A
2. Viewer plays back from Aquareum Node B
3. Livestream is mirrored through those nodes trustlessly; the Aquareum app ensures the user is looking at the signed video content from the streamer.

But I’ll work on making that clear visually. I want to make a video explainer for this too.

[hthillman]

@iameli

20,000

Hi Eli - I’m very excited to see this proposal, especially the C2PA integration!

Past pre-proposals have included some dates / milestones (example from the AI SPE below). As a voter, this helps me understand the timeline over which the funds will be spend (and when additional funds might be required). It also gives voters a way to evaluate SPE success when considering follow-on funding.

Do you have a sense of what milestones might look like for Aquareum?

[iameli]

Thanks Hunter! Roughly, we’re looking for this grant to fund operations for an 8-12 month period to achieve the items listed in “The First Milestone” up above. I’m hoping to launch an (incredibly basic, single-stream) iOS/Android app in advance of the Jul 16 premiere so folks have a place to watch and receive push notifications. The stuff in the Livepeer Network Improvements section will be delivered incrementally over that timeframe.

I’ll work on a more detailed outline but that’s the general framework.

[iameli]

Edited proposal to include that 100% of what we make will be open-source and freely-licensed; so obvious to me that I didn’t even think to include it. (In particular, I don’t want us to pull an Arbitrum and force a Business Source License on y’all.)

[huangkuan]

I think the bet that “the world needs a video layer” is directionally exciting. It makes sense to have a native video layer to onboard millions of developers and consumers into the web3 social. But I have a few questions on the details of the plan:

1. With this proposal, are you trying to win the hearts of end consumers, developers or enterprises? I think it’s very hard to achieve all of them and it would require teams with different DNA to tackle these sectors.

2. Can you talk a little bit about the demand here? What pain points are you hearing from your potential users (developers? end consumers?)

3. Aquareum.app:

instantly usable by millions of users

It sounds like this is more than just a demo app that focuses on demonstrating the features of the node+API. The app would also include content creation feature as well as discovery per your plan. In my view, this target alone will require a dedicated team for 12+ months with 5-10% chance of success if luck is on your side. It goes back to my first point. Who are your targets here?

4. Aquareum Indexer

deliver indexers for Farcaster and Bluesky, working closely with those communities.

Can you talk more about the demand here? What communities in Farcaster and Bluesky would want something like this? What are they intend to do with the indexer api or maybe indexer directly?

[elitmik]

This is the most exciting idea I’ve ever seen proposed in the Livepeer ecosystem. I’ve always been motivated by the idea of connecting developers and end-users directly to the Livepeer network via locally hosted gateways. This creates an easier, more native and efficient developer experience that connects Livepeer into the broader web 3 ecosystem.

I’m very excited about the public goods contribution of integrating Livepeer into an importable C library. Wasm is the magic here that can connect web 2 to web 3 at the end-user level, enabling new use cases and improving UX with the Livepeer network. The blockchain primitives integrations will make Livepeer a must-have component for large existing web 3 applications.

This will solidify Livepeer as decentralized public infrastructure that everyone will want to build on.

Good work, this is impressive!

[iameli]

Thanks for the thoughts, Kuan.

@huangkuan

1. With this proposal, are you trying to win the hearts of end consumers, developers or enterprises? I think it’s very hard to achieve all of them and it would require teams with different DNA to tackle these sectors.
2. Can you talk a little bit about the demand here? What pain points are you hearing from your potential users (developers? end consumers?)

End users and developers.

End users: Prior to Livepeer I worked with a lot of Twitch streamers; they’re furious with the platform and many of them would go elsewhere if there were credible alternatives. YouTubers hate their reliance on Google and many of them live in fear that Google will tweak their recommendation algorithm slightly and completely remove their livelihood. A credible, functional decentralized alternative that empowers users to own their own content library is something that would be very interesting to them. Bundle in a sponsorship layer that allows direct contributions from their audience and bids from advertisers and it’ll start to be financially appealing as well.

We’re not the first people to have the “decentralized YouTube where you own your content” idea, but none of them have gotten very popular. Two reasons where I think we can succeed where others have failed:

1. We have a lot of hard-won video distribution expertise from Studio, compared with a lot of other projects with relatively primitive video stacks
2. Our “indexer” approach allows us to integrate with every decentralized social platform, rather than forcing all users to migrate to a single one. “Move your video collection to whatever decentralized social platform you prefer” is a more appealing message than — to pick on Lens for a moment — “move your video collection to one particular NFT collection on the Polygon network.”

Developers: We’re also very interested in making an extremely developer-accessible project, especially on the front-end side. Here’s how you embed my livestream in a React app:

<Stream address="iame.li" />

And that’s a fully isomorphic component that works on Web (including a PWA), React Native (for iOS and Android) and Electron (for Windows, Mac, Linux). I also want the backend to be similarly embeddable and expressive.

But I’ll also be clear about the differences between what we’re working on so far - we’re not looking to create a generic video developer project in the way Livepeer Studio is; it’s going to be a lot more opinionated than that. Video content is derived from the upstream social networks, there’s no generic “Create Stream” action for use by developers. Closer to Subgraph for Video rather than Stripe for Video. We’re not waiting around for developers to come up with some great idea for video using Livepeer — we think we have the great idea already, and our developer tooling will be focused around making that accessible and allowing people to build on top of it.

Here’s where I’m coming from on this… I spent all of 2022 running Livepeer hackathons and saw dozens of impressive front-end hacks utilizing the Livepeer Studio SDK. But they all were just attached to empty Studio accounts; all of them were empty except for test data that the hackers had uploaded. Very sad! I want to enable the same thing, but those hackers will instead have a rich shared media library that they can built front-end applications for.

@huangkuan

1. Aquareum.app:

instantly usable by millions of users

It sounds like this is more than just a demo app that focuses on demonstrating the features of the node+API. The app would also include content creation feature as well as discovery per your plan. In my view, this target alone will require a dedicated team for 12+ months with 5-10% chance of success if luck is on your side. It goes back to my first point. Who are your targets here?

Yeah, definitely interested in “actually making the app” vs “making a demo and hoping others will make cool things using it later.” And you’re probably right! Two people for eight months is not likely to deliver on the entire vision for this. But think of what we achieve in the meantime:

1. Turn go-livepeer into a library that can be embedded in any application (I started on this already)
2. Turn MistServer into a library that can be embedded in any application (I started on this already too)
3. We will be developing a trustless protocol for replication of self-sovereign video data, finally providing an answer to the question “how does Livepeer handle video distribution?”
4. We will be upgrading the Livepeer Network to use C2PA signatures, turning the output from orchestrators into digital artifacts with coherent provenance chains

I’ll stop there… my point is that we can fall far short of our goals and still create enormous utility for Livepeer. And that’s a case where I’d hope the treasury would be interested in giving us another round of funding.

[iameli]

Thanks, John, glad you like it.

@elitmik

I’m very excited about the public goods contribution of integrating Livepeer into an importable C library. Wasm is the magic here that can connect web 2 to web 3 at the end-user level, enabling new use cases and improving UX with the Livepeer network. The blockchain primitives integrations will make Livepeer a must-have component for large existing web 3 applications.

Yeah, I’m really hoping we can make it stupid obvious for people. “Oh, we want to add video, let’s use Livepeer.” That’s another layer of developer accessibility I didn’t mention above: the interface for specifying a new plugin/indexer should be very, very clear and easy to work with. If your project can provide a social graph and a source of truth for users’ content libraries, Aquareum can index it and provide rich video functionality.

[iameli]

Edited the proposal to remove a reference to “setting up a Livepeer orchestrator” — I forgot that was still in there; I don’t think setting up a production top-100 orchestrator would be a good use of our team’s time. Potentially some of the funding could go toward just enough hardware to develop the go-livepeer software, especially as we move toward the C2PA integration, but we’ve already had volunteers from the O community to collaborate on that stuff so I think we probably won’t have to. Thanks to Brad/ad-astra for calling out that point during the watercooler chat.

[huangkuan]

I am going to be brutally honest here:

I don’t think you two will be able to deliver both a successful consumer product and developer product. They focus on two completely different markets and the efforts required to get some reasonable early adoption for each product is also non trivial. I assume you’d at least do some level of product marketing to prove your thesis instead of just delivering the software and be done with it.

It’s more realistic to focus on one product in this proposal. In my view, a developer product in your case likely has a higher chance of success considering your background. With you current experience and network in the developer community, it wouldn’t be too hard for you to gain some initial tractions.

Building a consumer app can be put into a separate follow up proposal. It would also be easier for people to vote YES after you deliver a successful developer product.

[iameli]

I appreciate the frank feedback, Kuan.

@huangkuan

I don’t think you two will be able to deliver both a successful consumer product and developer product. They focus on two completely different markets and the efforts required to get some reasonable early adoption for each product is also non trivial. I assume you’d at least do some level of product marketing to prove your thesis instead of just delivering the software and be done with it.

I don’t have a thesis for a developer-only Aquareum. I’m just not sure what it would be. It makes me think of those hackathons I was describing — I saw beautiful, polished Twitch and YouTube clones built with Livepeer Studio technology. And yet there was no content, just test videos. What’s the point, if we haven’t built something useful and understandable to content creators? We can build something useful for developers, but who are they building for? The dream of interoperable, freely-distributed content requires content creators to understand the system.

Studio’s focus has been on developers; their approach has been much more “Stripe for Video”. And it’s not a coincidence that over time their important users have become big companies with lots of pre-existing users. Studio’s clients don’t want revolutionary, interoperable video; they want everyone using their app, their website. It’s the content creators that want a credible decentralized alternative. We’re making it for them.

@huangkuan

Building a consumer app can be put into a separate follow up proposal. It would also be easier for people to vote YES after you deliver a successful developer product.

In fairness, Adam and I did deliver a successful developer product; it’s called Livepeer Studio. I built the initial API server and set the foundation for the DevOps infrastructure; more recently I built the multi-node replication system that allows WebRTC to be scaled out horizontally. Adam shipped the first public-facing frontend, designed the first player, and is a huge contributor to the API server. Here’s commits to https://github.com/livepeer/studio:

We’ve done that. Aquareum is us intentionally going a different direction.

[huangkuan]

It is totally fine if you want to focus on the consumer app. And I will have more follow up questions on how you are thinking through the consumer app experience such as

• You thesis on the consumer app, target user profiles
• A set of core features that can prove your initial thesis
• Targeted user insights that you already have
• Some high level distribution questions

I’d feel comfortable to vote YES knowing that even if you putting in all your effort, the app might still fail. That’s a market risk I am willing to take to support your compelling vision. But right now I am feeling the execution risk due to the lack of focus (or maybe clarity of focus?) in this post. The execution risk is that you might end up not serving either market well (developers or consumer) by delivering a mediocre experience on both directions. I have no question about your technical experience. But building a great product that has a shot to win the masses takes more than just technical skills.

[obodur]

I kinda feel this. I think the end product will be awesome but might end up not being used as much as it deserves because…
Because it will need marketing. Fanteasy’s app will need marketing too. Even Livepeer studio still needs marketing too. These are all platforms/apps that serve or will serve superior services for a lot cheaper but mainstream adoption needs marketing.
That’s where I believe that new big client of Livepeer Studio will come in. It will give Livepeer and all it apps/platforms so much visibility. Big companies and smaller ones too, mainstream devs and users, for once, will see the use case of crypto and DePin and Livepeer. I am so looking forward to that.
(Sorry, I went off the topic but I think this is very much related to every app/platform that will be built on Livepeer. Maybe they will become great products but then will need visibility.)

[iameli]

Thank you everyone for helping get this proposal into the best possible state. I’ve put together this Miro establishing our roadmap, and I’ll include a version of this chart with the actual proposal. Including a screenshot here for archival purposes:

Some FAQs from the most recent round of feedback:

How will the money be spent?
Per the chart, I’d like this funding to get us to somewhere between alpha and beta. That will include paying our team, provisioning servers, and maybe contracting as appropriate. The end goal of that is to get the software into a usable and useful state and deliver on the first set of C2PA primitives, integrated with go-livepeer. A working proof-of-concept of a decentralized livestreaming node. From there we can evaluate where we are and decide on the next steps for the team and the project.

Who are your target users?
I got some great questions during the water cooler chat today, one question I got was about who we were targeting as creators on the service. Here are the two I listed:

1. I’ve worked with a lot of Twitch streamers. They’re all furious with the platform, which has exploited their monopoly position to make incredibly user-hostile decisions. To them, our pitch is like this: Don’t move from Twitch to Aquareum, BigStreamer. Move from Twitch to your own personal platform. Have your own website, and eventually your own app in the App Store/Play Store. But because your app is powered by Aquareum, you don’t sacrifice interoperability; you can still host other livestreams and even feature other creators within your app.

2. There’s an emerging audience of dSocial enthusiasts; people who would love to move away from centralized platforms run by megacorporations but the technology just isn’t there yet. They hang out on Bluesky, Farcaster, and ActivityPub servers. These platforms don’t support livestreaming, that kind of content usually it ends up on Twitch or Discord. Aquareum will be an app they can download to livestream on those platforms instantly. From there we can show it off and try and generate some organic usage and follow our users’ leads.

And one more that I didn’t mention in the chat, who will also be instrumental to our success:

3. Node operators, as exemplified by the Livepeer Orchestrator community. The Aquareum primitives and software will handle a lot of the nuts and bolts of video provenance and distribution. With those problems solved, we will need to work with this community of enthusiasts to figure out the next steps around incentivized distribution. Who pays for the bandwidth of a large livestream? If it’s the streamer, how do they pay? If it’s the viewer, how do they pay? Are probabilistic micropayment channels a feasible way to pay for video distribution? Can node operators insert ads into the feed to offset their costs? These are the sorts of things we will be exploring with the community once we establish the foundation set in the Aquareum alpha.

EDIT: Made one more tweak to the proposal, adding this paragraph:

These primitives will be signed by the content creators and include an expressive language for the allowed distribution of their content. Creators will, for example, be able to express licensing information or choose to make content “expire” after a certain amount of time. Aquareum nodes will respect these fields, giving creators control over the lifecycle of their content in the system.

[iameli]

The Aquareum proposal is live!!!

https://explorer.livepeer.org/treasury/74518185892381909671177921640414850443801430499809418110611019961553289709442

Voting will go live in round 3412 tomorrow. Thank you to everyone for helping us get to this point. Please continue to reach out with any questions or concerns, I’ll be around!

[iameli]

And we’ve executed. Thank you so much!

We’re going heads-down now to get ready for the launch. More to come soon, and keep your eye on the timer at https://aquareum.tv/!

[streamm]

hi livepeer - after next to 10 months our farcaster video client is live (using current livepeer infra) - streamm tv
we want to discuss how we can integrate it with your upcoming video layer solution.