Setup Grafana, InfluxDB, and collectd on CentOS 7.x

Introduction

I wanted to monitor things. That's one of the *visions* in IT. Having visibility and transparency is important and you want to have documentation and history of <insert all the things here>. Not only that, it looks awesome (which is always good). Using Grafana and a data source, you can put many statistics tracked into a plotted graph with bars, lines, or points; which ever works best for the particular data set you're working with.

cigstack

The screenshot above is taken from my actual setup here at home so you can get an idea of what a finished product looks like. The entire guide takes place on a newly provisioned CentOS 7.x x64 machine.

Prerequisites and Reference Links

Adding Repositories (or download packages)

You will need to add two repositories: Grafana and InfluxDB. As collectd is natively available already via yum, you should be fine in that aspect. You should be performing the following commands as the root user.

Install Epel-release repository:

yum install epel-release -y

Add repository for InfluxDB:

cat <<EOF | sudo tee /etc/yum.repos.d/influxdb.repo
[influxdb]
name = InfluxDB Repository - RHEL \$releasever
baseurl = https://repos.influxdata.com/rhel/\$releasever/\$basearch/stable
enabled = 1
gpgcheck = 1
gpgkey = https://repos.influxdata.com/influxdb.key
EOF

Add repository for Grafana:

cat <<EOF | sudo tee /etc/yum.repos.d/grafana.repo
[grafana]
name=grafana
baseurl=https://packagecloud.io/grafana/stable/el/6/$basearch
repo_gpgcheck=1
enabled=0
gpgcheck=1
gpgkey=https://packagecloud.io/gpg.key https://grafanarel.s3.amazonaws.com/RPM-GPG-KEY-grafana
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt
EOF

Note: You may (or may not) notice that for Grafana, the repo is set enabled=0. At the time of writing this, the Grafana repository is down. In addition to that, the latest versions of InfluxDB have made some major changes to their API. Because of this, Grafana 3.x+ is the version of Grafana you will want to use which is alright, as version 3.0 is close to being released as a stable version and no longer in beta.

Software Installations

Install Grafana (beta version of 3.x which was just previously mentioned):

wget https://grafanarel.s3.amazonaws.com/builds/grafana-3.0.0-beta21459801392.x86_64.rpm
yum install -y grafana-3.0.0-beta21459801392.x86_64.rpm

Install InfluxDB and collectd:

yum install -y influxdb collectd

Enable services to start at boot:

systemctl enable grafana-server.service
systemctl enable influxdb.service
systemctl enable collectd.service

Setup InfluxDB

  1. Edit the influxdb.conf file (located /etc/influxdb/influxdb.conf) to make some of the following changes and enable collectd:
    [collectd]
      enabled = true # Set to "true" to enable the plugin.
      bind-address = "0.0.0.0:8888" # You can choose any interface IP address here (0.0.0.0 listens on all interfaces).  We will use port 8888 for the sake of keeping it (a little) simple.
      database = "collections" # Choose a database name that we will be creating in a moment.
      # typesdb = ""
    
      # These next lines control how batching works. You should have this enabled
      # otherwise you could get dropped metrics or poor performance. Batching
      # will buffer points in memory if you have many coming in.
    
      batch-size = 1000 # will flush if this many points get buffered
      batch-pending = 5 # number of batches that may be pending in memory
      batch-timeout = "1s" # will flush at least this often even if we haven't hit buffer limit
      read-buffer = 0 # UDP Read buffer size, 0 means OS default. UDP listener will fail if set above OS max.
  2. Access the web GUI to InfluxDB (http://<IPAddress>:8083)
  3. Create new admin user using the InfluxDB admin GUI:
    CREATE USER "admin" WITH PASSWORD 'dbiersmeisawesome!' WITH ALL PRIVILEGES
  4. Create new database with the name we used in the configuration file for influxdb:
    CREATE DATABASE "collections"
  5. Create READ/WRITE user for collectd and assign permissions:
    CREATE USER "collectdrw" WITH PASSWORD 'collectdrwpassword!!!'
    GRANT ALL ON "collections" TO "collectdrw"
  6. Create READ-ONLY user for collectd (stats reader) and grant those permissions:
    CREATE USER "collectdread" WITH PASSWORD 'readstats'
    GRANT READ ON "collections" TO "collectdread"
  7. Edit the configuration for InfluxDB one more time to enable authentication as by default, it is disabled (so you can create an admin user):
    [http]
      enabled = true
      bind-address = ":8086"
      auth-enabled = true
      log-enabled = true
      write-tracing = false
      pprof-enabled = false
      https-enabled = false
      https-certificate = "/etc/ssl/influxdb.pem"
    
  8. Restart InfluxDB so the authentication setting can take effect:
    systemctl restart influxdb.service

Setup collectd to send data to InfluxDB

    1. Edit the collectd config to enable the "network" plugin:
      LoadPlugin network
    2. Add configuration for "network" plugin:
      <Plugin "network">
              <Server "<InfluxDB IP>" "8888">
                      SecurityLevel None
                      Username "collectdrw"
                      Password "collectdrwpassword!!!"
                      Interface "eth0"
              </Server>
      </Plugin>
    3. Restart collectd:
      systemctl restart collectd.service

At this point you should start seeing data flow from collectd and into influxdb database "collections". Since we are *writing* data, we want to use the "collectdrw" user as this user has read/write permissions.

Set up Grafana

By default, starting grafana for the first time will automatically create an admin user with the username/password "admin/admin". Nothing really needs to be changed with the configuration of Grafana, so we will leave it alone for now.

      1. Start Grafana:
        systemctl start grafana-server.service
      2. Access the web GUI for Grafana: (http://<grafana IP>:3000)
      3. Login as "admin" with the password "admin".
      4. Change usernames, add users, change passwords, etc. as desired.
      5. Add a new data source
        • Name: Influx
        • Type: InfluxDB
        • URL: http://<InfluxDB IP>:8086
        • Database: collections
        • User: collectdread
        • Pass: readstats
      6. From here, click "ADD". Also make sure you test the connection.

Conclusion

From that point on you should be able to create your dashboards with the statistics that collectd is collecting and inserting into InfluxDB (which is where Grafana pulls its data).

You may also like...

8 Responses

  1. BikerSco says:

    Your missing the typesdb = "" part of the influxdb plugin configuration

    It should be typesdb = "/usr/share/collectd/types.db" without this influxdb won;t start

    • David says:

      I haven't had to put this in to have influxdb start although it wouldn't hurt to have it included anyways. I believe if it's blank I think it may fall back to the file you've added?

      • BikerSco says:

        Strange it wouldn't let me start the service without this added.

        Actually I'm being a bit silly here, I am using these instructions to install on a RHEL 7 box so it may be slightly different for this setup. Sorry for the confusion mate and thanks for the instructions, these are much simpler than the others i have used online.

  2. Ethan says:

    Is it possible to use something more secure, like SSL?

    • David says:

      Hi Ethan,

      Yes you can set it up with SSL connection without issue. The only extra steps to doing so is to have nginx installed (or your software of choice) and to configure it as a reverse proxy to point to http://$ip:3000 as the backend server. In the nginx.conf config, you would specify the SSL connection, certificates, etc.

  3. dafsic says:

    When i ues influxdb1.2.2,it not work,no data in influxdb database?What`s version you used?

  4. Felip Moll says:

    Hello,
    Authentication in Influxdb (<=1.2) is only provided through HTTP.

    "Note: Authentication only occurs at the HTTP request scope. Plugins do not currently have the ability to authenticate requests and service endpoints (for example, Graphite, collectd, etc.) are not authenticated."

    From: https://docs.influxdata.com/influxdb/v1.2/query_language/authentication_and_authorization/

    Afaik collectd network plugin sends info through UDP, so no HTTP requests are done and therefore authentication is not used in influxdb scope.

    Do you see it correct? If it is, user and password should be removed from your collectd configuration file.

Leave a Reply

Your email address will not be published. Required fields are marked *