Friday, January 18, 2013

Howto Raspberry Pi: Monitor your Raspberry Pi with Observium!






The Goal: 



With Observium associated with Unix agent check_mk the goal will be to monitor any available indicator (CPU, Mem, Traffic interface...) and most of all, specific Raspberry Pi main indicators dynamically allocated when running Overclocked with Turbo mode:
  • CPU Frequency
  • CORE Frequency
  • CORE Voltage
  • BCM2835 Soc Temperature

Corresponding "vgencmd" commands:
# CPU Frequency
vcgencmd measure_clock arm

# CORE Frequency
vcgencmd measure_clock core

# CORE Voltage
vcgencmd measure_volts core

# SoC Temp
vcgencmd measure_temp

There are also other indicators you may want to monitor, even i don't feed it myself useful.
The present article will take care of these 4 indicators.

Take a look here: http://www.elinux.org/RPI_vcgencmd_usage

Global list of indicators available through "vgencmd":
vcgencmd measure_clock arm
vcgencmd measure_clock core
vcgencmd measure_clock h264
vcgencmd measure_clock isp
vcgencmd measure_clock v3d
vcgencmd measure_clock uart
vcgencmd measure_clock pwm
vcgencmd measure_clock emmc
vcgencmd measure_clock pixel
vcgencmd measure_clock vec
vcgencmd measure_clock hdmi
vcgencmd measure_clock dpi
vcgencmd measure_volts core
vcgencmd measure_volts sdram_c
vcgencmd measure_volts sdram_i
vcgencmd measure_volts sdram_p


Installing Observium is out of the scope of this article, Observium installations documentations and well known and easy to read, see above.

Main sources:

I recommend to install Observium and Mysql into a central server which will request our Rpi to generate graphs and so on.

We will use an additional agent called "check_mk" to request the Rpi, system load generated by snmp and Unix agent are very limited which is very great, the Rpi is a small power device and you don't want monitoring to generate high system load!

One time you have Observium up and running, follow this guide to integrate any Raspberry Pi you want to monitor :-)



Summary of steps: 

Step 1: Install and configure snmpd
Step 2: Install check_mk agent (Unix Agent)
Step 3: Add the custom Raspberry agent script
Step 4: Observium custom application configuration
Step 5: Configure your Rpi in Observium, the easy part!

Memorandum


Step 1: Install and configure snmpd

First thing, we will begin by installing the snmpd daemon, to do so:
$ sudo apt-get install snmpd snmp-mibs-downloader
Let's configure some little things:

Edit "/etc/default/snmpd" and:
  • set: export MIBS=UCD-SNMP-MIB
  • Replace the line "SNMPDOPTS=" with the following values to prevent snmpd to log each connection (default behavior):
SNMPDOPTS='-LS 0-4 d -Lf /dev/null -u snmp -g snmp -I -smux -p /var/run/snmpd.pid'

Edit "/etc/snmp/snmpd.conf" and:
  • Comment with "#" the default line "agentaddress udp:127.0.0.1:161" which only allows connections from the localhost itself
  • Comment out the line "agentaddress udp:161,udp6:[::1]:161" to allow remote connections
  • Comment out the line "rocommunity secret <LANSUBNET>" (adapt <LANSUBNET> to the CIDR value of your LAN subnet, example: 192.168.0/24" 

Note: "secret" will the name of the snmp community, only accessible through your local network) 

  • Configure "sysLocation" and "sysContact"
  • Look for the section "EXTENDING THE AGENT" and add the following line:
extend .1.3.6.1.4.1.2021.7890.1 distro /usr/bin/distro
  • Install the "distro" script coming from observium (to recognize the remote OS)
$ sudo wget http://www.observium.org/svn/observer/trunk/scripts/distro -O /usr/bin/distro
$ sudo chmod 755 /usr/bin/distro

Finally restart snmpd daemon:
$ sudo service snmpd restart


Step 2: Install check_mk agent (Unix agent)

We will used the great Unix agent "check_mk" called Unix agent by Observium.

If you want more information about this very cool tool, check its main Web site:
http://mathias-kettner.de/checkmk_monitoring_system.html

Install Xinetd requirement:
$ sudo apt-get install xinetd

Download and install check_mk:
$ wget http://mathias-kettner.com/download/check-mk-agent_1.2.0p3-2_all.deb
$ sudo dpkg -i check-mk-agent_1.2.0p3-2_all.deb

Verify that the package installation generated the xinetd configuration file called "
/etc/xinetd.d/check_mk".

If not (it seems this part fails under Rpi), create the file with the following content:
# +------------------------------------------------------------------+
# |             ____ _               _        __  __ _  __           |
# |            / ___| |__   ___  ___| | __   |  \/  | |/ /           |
# |           | |   | '_ \ / _ \/ __| |/ /   | |\/| | ' /            |
# |           | |___| | | |  __/ (__|   <    | |  | | . \            |
# |            \____|_| |_|\___|\___|_|\_\___|_|  |_|_|\_\           |
# |                                                                  |
# | Copyright Mathias Kettner 2012             mk@mathias-kettner.de |
# +------------------------------------------------------------------+
#
# This file is part of Check_MK.
# The official homepage is at http://mathias-kettner.de/check_mk.
#
# check_mk is free software;  you can redistribute it and/or modify it
# under the  terms of the  GNU General Public License  as published by
# the Free Software Foundation in version 2.  check_mk is  distributed
# in the hope that it will be useful, but WITHOUT ANY WARRANTY;  with-
# out even the implied warranty of  MERCHANTABILITY  or  FITNESS FOR A
# PARTICULAR PURPOSE. See the  GNU General Public License for more de-
# ails.  You should have  received  a copy of the  GNU  General Public
# License along with GNU Make; see the file  COPYING.  If  not,  write
# to the Free Software Foundation, Inc., 51 Franklin St,  Fifth Floor,
# Boston, MA 02110-1301 USA.

service check_mk
{
 type           = UNLISTED
 port           = 6556
 socket_type    = stream
 protocol       = tcp
 wait           = no
 user           = root
 server         = /usr/bin/check_mk_agent

 # If you use fully redundant monitoring and poll the client
 # from more then one monitoring servers in parallel you might
 # want to use the agent cache wrapper:
 #server         = /usr/bin/check_mk_caching_agent

 # configure the IP address(es) of your Nagios server here:
 #only_from      = 127.0.0.1 10.0.20.1 10.0.20.2

 # Don't be too verbose. Don't log every check. This might be
 # commented out for debugging. If this option is commented out
 # the default options will be used for this service.
 log_on_success =

 disable        = no
}


Restart xinetd:

$ sudo service xinetd restart

Finally, ensure your Observium machine willbe authorized to access to the Rpi check_mk service running on port TCP/6556.


Step 3: Add the custom Raspberry agent script


Create a new file "/usr/lib/check_mk_agent/local/raspberry":
#!/bin/bash
#set -x
echo "<<<app-raspberry>>>"
# CPU Frequency
expr `vcgencmd measure_clock arm|cut -f 2 -d "="` / 1000000
# CORE Frequency
expr `vcgencmd measure_clock core|cut -f 2 -d "="` / 1000000
# CORE Voltage
vcgencmd measure_volts core|cut -f 2 -d "="|cut -f 1 -d "V"
# SoC Temp
vcgencmd measure_temp|cut -f 2 -d "="| cut -f 1 -d "'"

Add execution right:
$ sudo chmod a+rx /usr/lib/check_mk_agent/local/raspberry

This script will be called by Observium at each poller time.


Step 4: Observium custom application configuration



Ok now a bigger part, we need to configure Observium to add our custom application has any other.
By this way, we could run this with as many Rpi as you want ;-)

To do so, we need to create and/or modify different configuration files.

Go into your Observium root directory, usually "/opt/observium"

1. "./includes/polling/unix-agent.inc.php" (modify)

Look for the section containing:
      if ($section == "apache") { $sa = "app"; $sb = "apache"; }

And add new one just under :
      if ($section == "raspberry") { $sa = "app"; $sb = "raspberry"; }

2. "./includes/polling/applications/raspberry.inc.php" (create)

Create with following content:
<?php

if (!empty($agent_data['app']['raspberry']))
{
  $raspberry = $agent_data['app']['raspberry'];
}

$raspberry_rrd  = $config['rrd_dir'] . "/" . $device['hostname'] . "/app-raspberry-".$app['app_id'].".rrd";

echo(" raspberry statistics\n");

list($cpufreq, $corefreq, $corevoltage, $soctemp) = explode("\n", $raspberry);
if (!is_file($raspberry_rrd))
{
  rrdtool_create ($raspberry_rrd, "--step 300 \
        DS:cpufreq:GAUGE:600:0:125000000000 \
        DS:corefreq:GAUGE:600:0:125000000000 \
        DS:corevoltage:GAUGE:600:0:125000000000 \
        DS:soctemp:GAUGE:600:0:125000000000 ".$config['rrd_rra']);
}

print "cpufreq: $cpufreq corefreq: $corefreq corevoltage: $corevoltage soctemp: $soctemp";
rrdtool_update($raspberry_rrd, "N:$cpufreq:$corefreq:$corevoltage:$soctemp");

// Unset the variables we set here

unset($raspberry);
unset($raspberry_rrd);
unset($cpufreq);
unset($corefreq);
unset($corevoltage);
unset($soctemp);

?>

3. "./html/includes/graphs/application/raspberry_soctemp.inc.php" (create)

Create with following content:
<?php

$scale_min = 0;

include("includes/graphs/common.inc.php");

$raspberry_rrd   = $config['rrd_dir'] . "/" . $device['hostname'] . "/app-raspberry-".$app['app_id'].".rrd";

if (is_file($raspberry_rrd))
{
  $rrd_filename = $raspberry_rrd;
}

$ds = "soctemp";

$colour_area = "F0E68C";
$colour_line = "FF4500";

$colour_area_max = "FFEE99";

$graph_max = 1;

$unit_text = "°C";

include("includes/graphs/generic_simplex.inc.php");

?>

4. "./html/includes/graphs/application/raspberry_corevoltage.inc.php" (create)

Create with following content:
<?php

$scale_min = 0;

include("includes/graphs/common.inc.php");

$raspberry_rrd   = $config['rrd_dir'] . "/" . $device['hostname'] . "/app-raspberry-".$app['app_id'].".rrd";

if (is_file($raspberry_rrd))
{
  $rrd_filename = $raspberry_rrd;
}

$ds = "corevoltage";

$colour_area = "CDEB8B";
$colour_line = "006600";

$colour_area_max = "FFEE99";

$graph_max = 1;

$unit_text = "Volts";

include("includes/graphs/generic_simplex.inc.php");

?>

5. "./html/includes/graphs/application/raspberry_corefreq.inc.php" (create)

Create with following content:
<?php

$scale_min = 0;

include("includes/graphs/common.inc.php");

$raspberry_rrd   = $config['rrd_dir'] . "/" . $device['hostname'] . "/app-raspberry-".$app['app_id'].".rrd";

if (is_file($raspberry_rrd))
{
  $rrd_filename = $raspberry_rrd;
}

$ds = "corefreq";

$colour_area = "B0C4DE";
$colour_line = "191970";

$colour_area_max = "FFEE99";

$graph_max = 1;

$unit_text = "Mhz";

include("includes/graphs/generic_simplex.inc.php");

?>

6. "./html/includes/graphs/application/raspberry_cpufreq.inc.php" (create)

Create with following content:
<?php

$scale_min = 0;

include("includes/graphs/common.inc.php");

$raspberry_rrd   = $config['rrd_dir'] . "/" . $device['hostname'] . "/app-raspberry-".$app['app_id'].".rrd";

if (is_file($raspberry_rrd))
{
  $rrd_filename = $raspberry_rrd;
}

$ds = "cpufreq";

$colour_area = "B0C4DE";
$colour_line = "191970";

$colour_area_max = "FFEE99";

$graph_max = 1;

$unit_text = "Mhz";

include("includes/graphs/generic_simplex.inc.php");

?>

7. "./html/pages/device/apps/raspberry.inc.php" (create)

Create with following content:
<?php

global $config;

$graphs = array('raspberry_cpufreq' => 'CPU Frequency',
                'raspberry_corefreq' => 'CORE Frequency',
                'raspberry_corevoltage' => 'CORE Voltage',
                'raspberry_soctemp' => 'BCM2835 SoC Temperature',

);

foreach ($graphs as $key => $text)
{

  $graph_array['to']     = $config['time']['now'];
  $graph_array['id']     = $app['app_id'];
  $graph_array['type']   = "application_".$key;

  echo('<h3>'.$text.'</h3>');

  echo("<tr bgcolor='$row_colour'><td colspan=5>");

  include("includes/print-graphrow.inc.php");

  echo("</td></tr>");
}

?>

8. "./html/pages/apps.inc.php" (modify)

Look for the section containing:
$graphs['apache']     = array('bits', 'hits', 'scoreboard', 'cpu');

And add new one just under :
$graphs['raspberry']  = array('cpufreq', 'corefreq', 'corevoltage', 'soctemp');

Ok, we're done!



Step 5: Configure your Rpi in Observium, the easy part!


Now the easiest, add your Rpi into Observium, go to the menu <Devices>, <Add device>.



In our case:
  • Hostname: Enter the hostname or IP of your Rpi
  • snmp Community: secret

Let all the rest by default.


The Rpi shall be detected with sucess, and the Debian logo appears:


Now enter the device and go to device settings:



Go to "Applications" and activate the box corresponding to our Raspberry application:


Then, Go to "Modules" and Activate the Unix agent (disabled by default):



Great, you're done will all configuration parts, wait for a view poller execution (by default Observium proposes a cron task every 5 minutes)

You can run manually the poller under the host running Observium:
$ sudo /opt/observium/poller.php -h all

And if you want to run it into debug mode to get more details:
$ sudo /opt/observium/poller.php -h all -d


In my experience, you have to wait for 10-15 minutes before getting data being graphed.

Some screenshots with application data:

CPU Frequency:




CORE Frequency:



CORE Voltage:




BCM2835 Soc Temperature:




Great :-)



25 comments:

  1. These should all go into the sensors table :)

    ReplyDelete
  2. This comment has been removed by the author.

    ReplyDelete
  3. Thanks for you my hero...

    but in step 5..

    from where I get Menu and add device..? How can I open the GUI window..?

    sorry my dear because I am newbie :(

    ReplyDelete
    Replies
    1. Hi, In the menu "Device" then "Add New Device"

      Delete
  4. Are you wondering how you can get your project written by an expert? At GoAssignmentHelp.com it's the easiest thing in the world! We have the expert assignment writers due to which we can provide the best help for the Assignment help to our customers. GoAssignmentHelp.com has experts who can work super-fast without missing any requirements or hampering the quality of assignments. Our professional assignment helpers are trained to complete superior quality solve my math problem within challenging deadlines. Many companies will provide to do this, but there’s just one that you can trust completely GoAssignmentHelp.com If you choose our company, we will cover all aspects so that you receive remarkable writing in the shortest time. We will share all the perks which you can enjoy from our assignment writing help, homework help service. Pay less and enjoy our wide slew of academic services to hand over a perfect paper to your professors within your deadlines. It is so easy to get in touch with us and, through a real-time chat, phone number or an email and you can be sure all your queries will be solved.

    ReplyDelete
  5. Thank you for sharing the useful post. A reader got a lot of information from this post and utilized it in their research. I also provide independent support for the outlook email. So if you are facing issues with the outlook account then contact me for outlook customer service.
    Also Read: Outlook not connecting to server | Outlook send receive error | outlook cannot connect to server | outlook not receiving emails.

    ReplyDelete
  6. It is really helpful for readers who are looking for online assignment help. I also work in the same area and provide assignment help to students for various subjects. Most of my students are from parts of the UK, USA, India, and Singapore. Our team of professors are from reputed Colleges. Visit our website to know our areas of expertise and let us know if we can be of help. Also Read:
    how many pages is 1000 words,
    how many pages is 2000 words,
    how many pages is 1500 words,
    about us
    informative speech subjects.

    ReplyDelete
  7. Your post is very helpful and information is reliable. I am satisfied with your post. Thank you so much for sharing this wonderful post. i want to share with you some useful source KatMovieHD.

    ReplyDelete
  8. Seek the assistance of our Online Assignment Help in Canada and complete your tasks with utmost precision, without compromising on your studies. When we write your assignments, you get enough time to concentrate on your studies and focus on other academic works.

    ReplyDelete
  9. Do you also want to take online Assignment help? Then don’t worry, we will help you! We offer you the best assignment writing services at a very low price. Browse our website for taking the best assignment writing help.

    ReplyDelete
  10. You have done a great job on this article. It’s very readable and highly intelligent.
    You have even managed to make it understandable and easy to read. You have some real writing talent. Thank you
    amazon quiz
    gk quiz
    general knowledge quiz
    english stories
    bedtime stories
    short stories kids
    english short stories
    short bedtime stories
    english stories collection

    ReplyDelete
  11. Are you unable to get Assignment help in UK? Don’t worry! We offer you the best quality assignment assistance. We have highly qualified writers who will provide you supreme quality help regarding your assignment.

    ReplyDelete
  12. Algebra Homework Help better understand this; thus, we attempt to provide students with unique assignment help to keep them out of trouble. We have specialist assignment helpers on staff, allowing us to deliver the best assignment help to our consumers. Students who want to get good scores can use our assignment writing service and rest, knowing that they will get work that will never let them down.

    ReplyDelete
  13. Surfing the valuable and industry oriented content is my choice. That’s why I am internet savvy to know to dig out the amazing piece to increase our experience. Our Online Assignment Help service chain is doing the best approach to let reflect valuable content effort in their work.

    ReplyDelete
  14. Surfing the valuable and industry oriented content is my choice. That’s why I am internet savvy to know to dig out the amazing piece to increase our experience. Our Assignment Helper service chain is doing the best approach to let reflect valuable content effort in their work.

    ReplyDelete
  15. Great and fast service. custom donut boxes The range and quality of the boxes suitable for all sorts of packing.custom printed donut boxes I will definietly be buying some more.

    ReplyDelete
  16. https://masonintheusa.net/community/profile/sunilsharma/
    https://www.manaaki.io/members/8903378
    https://www.moonlaneink.co.uk/profile/sunilsharmaigp/profile
    https://www.edimprovement.org/profile/sunilsharmaigp/profile
    https://raisondetre.mgen.fr/profiles/sunilsharma/timeline

    ReplyDelete
  17. Clenbuterol has become popular as a weight loss supplement. It has properties similar to those of salbutamol. Order now from official website of mediseller. This medicine available with the name clenbuterol 40mcg.

    ReplyDelete

Please feel free to comment ^^