# Guide: Transcoder monitoring with Prometheus/Grafana

**URL:** <https://forum.livepeer.org/t/guide-transcoder-monitoring-with-prometheus-grafana/1225>\
**Category:** Transcoders\
**Created:** [February 5, 2021, 2:14pm UTC](https://forum.livepeer.org/t/guide-transcoder-monitoring-with-prometheus-grafana/1225 "2021-02-05T14:14:30Z")\
**Posts on this page:** 7\
**Page:** 2

<div class="post-metadata">

**Author:** ![vires-in-numeris](https://yyz2.discourse-cdn.com/flex030/user_avatar/forum.livepeer.org/vires-in-numeris/32/731_2.png) [@vires-in-numeris](https://forum.livepeer.org/u/vires-in-numeris)\
**Post date:** [December 3, 2021, 10:49am UTC](https://forum.livepeer.org/t/guide-transcoder-monitoring-with-prometheus-grafana/1225/23 "2021-12-03T10:49:10Z")

</div>

Hmm are you running it on your Orchestrator? Most metrics are not supported if you’re running it on a Transcoder

---

<div class="post-metadata">

**Author:** ![vires-in-numeris](https://yyz2.discourse-cdn.com/flex030/user_avatar/forum.livepeer.org/vires-in-numeris/32/731_2.png) [@vires-in-numeris](https://forum.livepeer.org/u/vires-in-numeris)\
**Post date:** [December 26, 2021, 3:52pm UTC](https://forum.livepeer.org/t/guide-transcoder-monitoring-with-prometheus-grafana/1225/24 "2021-12-26T15:52:42Z")

</div>

Here’s how you can add a table to your grafana that shows the latest winning tickets:

_Edit: You’ll need to have a winner in the selected “Relative time”, otherwise you won’t see anything. You also need to increase the prometheus storage rentention to something like 1y (default is only 14 days, you can do this by adding --storage.tsdb.retention.time=1y to your startup command/systemd file)._

![image](https://canada1.discourse-cdn.com/flex030/uploads/livepeer/original/1X/57098676962f09f65c7f30c9b15c32e28fce8e2b.png)

First make sure that you select “Table” as your visualization. Then in the query tab, click on the query options and adjust the numbers according to the screenshot.  
The query is:  
**`min((max_over_time(livepeer_winning_tickets_recv[10m]) or livepeer_ticket_value_recv*0) - (max_over_time(livepeer_winning_tickets_recv[10m] offset 10m) or livepeer_ticket_value_recv*0)) > 0`**  
Then click on the “Overrides” tab on the right side, select your query and switch to “Hide in table”.

 ![image](https://canada1.discourse-cdn.com/flex030/uploads/livepeer/original/1X/cb597837ee2206ea46e772a635d418f140633de2.png)

Nerd mode: The part with the `or livepeer_ticket_value_recv*0` might seem weird but it’s needed since the winning\_tickets value is not initialized until you get your first winner - so the query without that wouldn’t work for the first winner after every restart. To counter that we use another value that’s initialized faster and set it to 0. The max\_over\_time query will then take whichever value is bigger.

If you want a bigger “Relative time” aka history, you need to adjust some other values in order to not run into query limits. Here’s an example for 90 days:

 ![image](https://canada1.discourse-cdn.com/flex030/uploads/livepeer/original/1X/cd6cb49bf39a285ecd6fb9806fae50c0e63633e9.png)

---

<div class="post-metadata">

**Author:** ![Authority\_Null](https://yyz2.discourse-cdn.com/flex030/user_avatar/forum.livepeer.org/authority_null/32/1364_2.png) [@Authority\_Null](https://forum.livepeer.org/u/Authority_Null)\
**Post date:** [January 14, 2022, 9:04pm UTC](https://forum.livepeer.org/t/guide-transcoder-monitoring-with-prometheus-grafana/1225/25 "2022-01-14T21:04:54Z")

</div>

Hey! Is there a way to have 2 machines use the same Grafana data? For example I have my main Orchestrator on 1 machine and a backup O on a second machine, howler they both use their own Prometheus instance which is a bit inconvenient.

---

<div class="post-metadata">

**Author:** ![tryingout](https://avatars.discourse-cdn.com/v4/letter/t/4bbf92/32.png) [@tryingout](https://forum.livepeer.org/u/tryingout)\
**Post date:** [January 19, 2022, 8:21am UTC](https://forum.livepeer.org/t/guide-transcoder-monitoring-with-prometheus-grafana/1225/26 "2022-01-19T08:21:21Z")

</div>

hey guys,

thanks OP for the first post/guide. I just want to say that im running everything on windows and can be done using nvidia\_gpu\_exporter ([GitHub - utkuozdemir/nvidia\_gpu\_exporter: Nvidia GPU exporter for prometheus using nvidia-smi binary](https://github.com/utkuozdemir/nvidia_gpu_exporter))

 ![slika](https://canada1.discourse-cdn.com/flex030/uploads/livepeer/original/1X/c91a63281150d5ecae9713fda3b71512c39eb4fb.png)

@Authority_Null as far as i know, grafana doesn’t store data (like kibana) it just display it. So you need one prometheus instance as a source. you could try prometheus agent mode

---

<div class="post-metadata">

**Author:** ![Authority\_Null](https://yyz2.discourse-cdn.com/flex030/user_avatar/forum.livepeer.org/authority_null/32/1364_2.png) [@Authority\_Null](https://forum.livepeer.org/u/Authority_Null)\
**Post date:** [January 19, 2022, 2:12pm UTC](https://forum.livepeer.org/t/guide-transcoder-monitoring-with-prometheus-grafana/1225/27 "2022-01-19T14:12:35Z")

</div>

Thank you! I brought this us on the weekly water cooler chat and it was explained to me that we can open port 9090 and have Grafana point to the correct Prometheus instances. This would require some router firewalling to be secure but it seems easy enough to configure.

---

<div class="post-metadata">

**Author:** ![joejoe](https://avatars.discourse-cdn.com/v4/letter/j/f17d59/32.png) [@joejoe](https://forum.livepeer.org/u/joejoe)\
**Post date:** [February 26, 2022, 10:37pm UTC](https://forum.livepeer.org/t/guide-transcoder-monitoring-with-prometheus-grafana/1225/28 "2022-02-26T22:37:22Z")

</div>

What is the nvida\_exporter used for?

---

<div class="post-metadata">

**Author:** ![papa\_bear](https://yyz2.discourse-cdn.com/flex030/user_avatar/forum.livepeer.org/papa_bear/32/1259_2.png) [@papa\_bear](https://forum.livepeer.org/u/papa_bear)\
**Post date:** [February 27, 2022, 7:25pm UTC](https://forum.livepeer.org/t/guide-transcoder-monitoring-with-prometheus-grafana/1225/29 "2022-02-27T19:25:18Z")

</div>

The nvidia\_exporter allows you to see detailed stats for your GPU

[Previous page](https://forum.livepeer.org/t/guide-transcoder-monitoring-with-prometheus-grafana/1225.md?page=1)
