Netdata is a monitoring tool that is becoming more and more popular, by providing lots of useful information with “zero configuration”, as they promote it and in a “realtime” manner.
For me it will never be the default tool for monitoring, but yet I do use it everywhere as an additional tool
Never the less - the recent “cloud” features they have added like “Machine Learning powered automated anomaly detection” look very promising.
I do prefer all my notifications to be sent to discord, as that’s a few clicks away in the cloud interface, but it also supports all kind of endpoints/methods by setting them in the config files
Installation
Installation of netdata is easy and straight forward, and I’m not going into details here - find your platform/distro section in the site and use it if you haven’t yet.
I’m only using the free tier and so far haven’t found any reason but supporting the team for the pro/business options.
The cloud option is optional, but very useful and time conserving and you could just access the netdata daemon trough it’s web interface, with or without reverse proxies in front of it/them … depending on your setup.
Besides all the builtin metrics, being autodetected there are two options I do configure for the livepeer nodes.
nvidia-smi
Nvidia-smi python collector is easy as edit/uncomment one single line with the builtin edit-config tool :
# cd /etc/netdata/
/etc/netdata# ./edit-config python.d.conf
...
nvidia_smi: yes
nvidia-smi of course should be installed and working before doing it. I do restart netdata after changing the configs. The nvidia-smi section will appear in your interface after a while and should give a you a lot of configurable charts ( the examples are from a host with 4 1070s ):
prometheus metrics
The next step is to add monitoring for the builtin prometheus metrics provided by the livepeer orchestrator. Again, easy as :
/etc/netdata# ./edit-config go.d/prometheus.conf
Editing '/etc/netdata/go.d/prometheus.conf' ...
and adding this at the end of the known prometheus endpoints ( your host/port values may of course be different ) :
- name: livepeer
url: 'http://localhost:7935/metrics'
Again after restarting netdata ( not quite sure it’s needed but … ) a section named prometheus livepeer should appear in your netdata interface :
Of course, you can do a lot of tuning on how your charts look like, add them to dashboards etc, but I’m leaving that for the following posts and hopefully comments by other users that are using or start using netdata after this topic.