If you’re looking to set up multiple Orchestrators/Transcoders in different locations around the globe it may be worth setting geo locating DNS services to improve your latency score.
Best of all you can connect all the geo nodes to a single Orchestrator account and run them anywhere on earth.
Ie. All North America requests go to your Chicago node, all Europe requests go to your London node and all Asia requests go to your Singapore node, all using the same domain.
Here is a guide to help you set up geo-locating DNS using Route 53.
Route 53
Register/Login to Amazon and access the Route 53 | Homepage
Register or transfer your domain into Route 53 | Register Domain
Create a Hosted Zone with your domain name (public) | Hosted Zones
Add “Geolocation rule” and select your desired region
Click “Connect to…” and select “New endpoint”. Enter the destination IP.
NOTE: This IP must be located within the region selected, else Transcoders and Broadcasters will not be able to connect.
Repeat this step by clicking “Add another geo location” until desired locations met
Make sure to have one location set as “Default” pointing one of your IPs. Some requests come from outside any set region or mask their location, all these requests resolve to your default.
Save the new Traffic Policy (you will be prompted to purchase the AWS service here, if you prefer, you can skip this step and come back to it later under the Policy Records tab)
Create a new Policy Record, select the settings you just made.
Recommend leaving the “Policy record DNS name” empty and adjust the “TTL” to 1.
A few configuration reminders. The service address or service URI for your Orchestrator must be published as XXXX.com:port. You cannot forward the port within the DNS.
Run each instance of your Orchestrator at each location all connecting to the same domain and port.
Test that the DNS is resolving correctly by connecting through a VPN within a specified region and ping or trace route the domain. It should resolve to the settings you have configured.
Thank you very much, this is awesome!
To clarify, basically the setup is the following:
One main Orchestrator (with as many LPT staked as possible) to collect requests from broadcasters. If the requests are coming from a different region, they get rerouted to secondary Os within this region?
Those secondary Orchestrators only need to be in the top 100 in terms of stake since all the requests will be coming from the main O anyways?
Hey vires-in-numeris
There is no need for secondary Os.
Run a 2nd, 3rd or Nth instance of the main O on different machines and the traffic gets re routed to the region where the nearest O is located.
There are still some things I have yet to confirm whether they work.
Ticket redemption is one I have not tested as I have not received any winning tickets since I’ve set this up. I believe as long as you have the correct winning ticket info it can be sent as a transaction and accepted from any location of your O.
Streams are sent from and back to Broadcasters directly from/to Nth instance boxes and bypass the main O so no extra hops are added using this configuration, correct?
When running the individual O’s make sure the -serviceAddr is IP:PORT not DOMAIN:PORT
The DOMAIN should only be used for the discovery request and set as the service URI on the blockchain.
Once it hits the O it will continue to transcode the rest of the requests using the -serviceAddr override with the IP and will avoid having to keep getting geo-location rules for each segment. This will greatly improve performance while still sending the discovery requests to the proper locations.
You can also use weighted rules in Route 53 to do load balancing. After the geo-location rule you can add a 50/50 weighted rule (or whatever weight you choose) which will direct the discovery request respectively and can add more transcoding power to different locations within the same region.
Once I have transcoders in other locations, I would setup anycast personally, but it requires you to bring your own IP (easy enough if it’s IPv6 only, not so cheap with IPv4).
It requires some BGP know how, I wrote a post on it a long time ago - Anycasting IPv6 TCP and UDP if anyone’s interested.
You can include Health Checks in your routing which will send traffic to an alternative destination if the Health Check comes back unhealth of a particular node.
Really useful for keeping high success rates especially if you are doing weighted balancing between 2 nodes in the same region.
Only one call will go through. The other O’s will fail with an error like "Orchestrator has already called reward for this round) or something like that.
Thank you for sharing. This is in stark contrast to many other projects… ( for example you will get slahsed in Ethereum if you run two validator nodes with the same key ). But i guess after 2 years of testing this should be safe to follow here
