http://www.developer.com/

Back to article

Play it Cool: Creating Reports from Stored Data


September 20, 2006

Thermal problems have been a major bane of computers since the very first computer was plugged into AC power. Despite server rooms with raised white flooring and high-end Leibert air conditioners, and over-clocked gaming rigs with water-cooling on the chipsets, heat continues to be a problem that computer administrators and technicians can't ignore.

Note: This series of articles uses Linux as the operating system of choice, with scripts written for the shell (Bash) and in Perl. For the tasks outlined within this series, I found Linux to be ideal. If you are using a different programming platform, or if you are unable to replicate these methods on Linux, you should still gain value from these articles. Where possible, I will point out analogous possibilities in Windows and other operating systems.

In Review

In the last few articles, you concentrated on collecting data—primarily from the serial-connected thermometer—and storing the data in various formats. This article shows you how to report on that stored data.

Note: Keep in mind that the concepts outlined in this article can be utilized to display various types of data, not just the data previously collected by the temperature-related scripts.

A CSS-Based Homegrown Solution

Despite the multitude of software available on the Internet, occasionally only a homegrown solution will work for what you need. Fortunately, Web scripting languages and CSS are both robust enough to support a graphing application to trend your data.

Using <div> tags for bar graphs

The <div> (short for "division") HTML tag is used primarily to contain blocks of related text. However, because you can tailor the size and ornamentation of each <div>, it's fairly trivial to use <div> tags to create bars in a bar graph. Consider the HTML document shown in Figure 1. The HTML code I used to create that document is shown in Listing 1.



Click here for a larger image.

Figure 1: <div> tags can be used to create the bars of a graph.

Listing 1: Code used to create the bar graph (sample-graph.html)

<html>
<head>
   <style type="text/css">
      .container { height: 200px;
                   width: 400px;
                   border: thin solid black;
                   padding: 0px;
                   margin: 0px;
      }

      .bar { float: left;
             vertical-align: baseline;
             width: 10px;
             border: none;
             border-left: 1px solid blue;
             border-right: 1px solid blue;
             border-top: 1px solid blue;
             padding: 0px;
             margin: 0px;
             position: relative;
             display: block;
      }

   </style>
</head>
<body>
<div class="container">
   <div class="bar" style="line-height: 100px; top:  95px;"></div>
   <div class="bar" style="line-height: 170px; top:  25px"></div>
   <div class="bar" style="line-height:  75px; top: 120px"></div>
   <div class="bar" style="line-height: 100px; top:  95px;"></div>
   <div class="bar" style="line-height: 170px; top:  25px"></div>
   <div class="bar" style="line-height:  75px; top: 120px"></div>
   <div class="bar" style="line-height: 100px; top:  95px;"></div>
   <div class="bar" style="line-height: 170px; top:  25px"></div>
   <div class="bar" style="line-height:  75px; top: 120px"></div>
   <div class="bar" style="line-height: 100px; top:  95px;"></div>
   <div class="bar" style="line-height: 170px; top:  25px"></div>
   <div class="bar" style="line-height:  75px; top: 120px"></div>
   <div class="bar" style="line-height: 100px; top:  95px;"></div>
   <div class="bar" style="line-height: 170px; top:  25px"></div>
   <div class="bar" style="line-height:  75px; top: 120px"></div>
</div>
</body>
</html>

The concept is straightforward:

  1. Create a container DIV for the graph (to delimit the graph and allow it to be positioned where it's needed on the page).
  2. Create a bar DIV definition to display the bars as 10px wide with a blue border (top and sides only).
  3. Insert the requisite bars in the container, modifying the line-height and top attributes to size each bar correctly and position it along the bottom of the container.

Of course, due to some idiosyncratic behaviors of CSS, you need to do a little fudging here and there to align the bars correctly within the container. Simply supplying a bottom alignment attribute and floating each against the left margin won't do.

Note: An extensive example of using CSS to create bar graphs is available on the Apples to Oranges site: http://applestooranges.com/blog/post/css-for-bar-graphs/?id=55.

Scripting a Dynamic graph

In Part 2 of this series, I showed how the temperature readings could be appended to a simple textual log, in a format including the date, time, and temperature reading:

2006-07-26      15:30      73.7

Parsing that information and creating a simple bar graph is trivial in a scripting language such as PHP. A sample application for displaying a subset of the data (one day's worth) is shown in Listing 2, and a sample of its output is shown in Figure 2.

Listing 2: Graphing one day's worth of temperature data (daygraph.php)

<?php

//Ratio temperature into 200px-high grid

function temp_ratio($temp) {
   // Get amount if temperature is above 50 degrees
   $temp = $temp - 50;
   // Ratio against 50 into 200-high grid
   $ratio = intval($temp * 200 / 50);
   // Return ratio
   return $ratio;
}

// Start page, define styles and grid
print <<<HTML
<html>
<head>
<style type="text/css">

   .container { height: 200px;
                width: 580px;
                border: thin solid black;
                position: absolute;
                display: block;
                margin: 0; padding: 0;
                top: 20px; left: 20px;
                z-index: 0; }

   .hline { width: 576px;
            border: none;
            border-bottom: 1px dotted gray;
            margin: 0px; padding: 0px;
            line-height: 35px; }

   .vline { width: 24px;
            border: none;
            border-right: 1px dotted gray;
            margin: 0px; padding: 0px;
            vertical-align: baseline;
            line-height: 195px;
            float: left; }

   .bar { width: 0px;
          margin: 0px; padding: 0px;
          border: 1px solid blue;
          float: left;
          vertical-align: baseline;
          position: relative;
          display: block;
          z-index: 1; }

</style>
</head>

<body>
<div class="container" style="z-index: 3;">
<!-- Horizontal gridlines (~25 degrees) -->
   <div class="hline" style="line-height: 16px;
         border-bottom: none;"></div>
   <div class="hline"></div>
   <div class="hline"></div>
   <div class="hline"></div>
   <div class="hline"></div>
   <div class="hline"></div>
</div>

<div class="container" style="z-index: 3;">
   <!-- Vertical gridlines (hour) -->
   <div class="vline"></div><div class="vline"></div>
   <div class="vline"></div><div class="vline"></div>
   <div class="vline"></div><div class="vline"></div>
   <div class="vline"></div><div class="vline"></div>
   <div class="vline"></div><div class="vline"></div>
   <div class="vline"></div><div class="vline"></div>
   <div class="vline"></div><div class="vline"></div>
   <div class="vline"></div><div class="vline"></div>
   <div class="vline"></div><div class="vline"></div>
   <div class="vline"></div><div class="vline"></div>
   <div class="vline"></div><div class="vline"></div>
   <div class="vline"></div><div class="vline"></div>
</div>

<!-- Container for graph -->
<div class="container">

HTML;

// Open data file
$fp = fopen("/home/sschafer/todaytemp.log","r");

// Read each line (Format: date time temp)
while (!feof($fp)) {
   // Get next line of log
   $reading = fgets($fp,1024);

   // Third value is temp
   $x = explode(" ",$reading);
   $temp = $x[2];

   // Include a little sanity checking
   // to weed out data artifacts
   if ($temp >= 50 && $temp <= 90) {
      // Ratio the temp into our graph
      $temp = temp_ratio($temp);
      // Adjust for margins, etc.
      $top = 200 - $temp - 5;
      // Print graph bar
      print "<div class="bar" style="line-height: {$temp}px; ";
      print "top: {$top}px; border-color: blue; "></div> n";
   }
}

// Close data file
fclose($fp);

// Close html file
print <<<HTML
</div>
</body>
</html>

HTML;

?>



Click here for a larger image.

Figure 2: A sample of the reporting script's output.

Notice that this example assumes that one day's worth of data is stored in todaytemp.log—the simplest case to show the basics of how such a script would work. A filtering algorithm could be added to the script to parse a larger file with more data. Also, a pager routine could be implemented to move between portions of data. Of course, if the data were stored in a database, you could use targeted queries to obtain just the data necessary to display the desired graph.

The graphs output by this script also make use of CSS layers to superimpose gridlines (vline- and hline-classed DIVs) over the graph to aid in reading it.

Using Packaged Graphing Applications

Several prepackaged graphing programs are available on the Internet, especially in the open source arena. I'll cover a couple of the more useful options.

Graphing data with MRTG

One popular application, Multi Router Traffic Grapher (MRTG), traditionally is used to graph bandwidth usage flowing across routers on local LANs and the Internet. Many a system administrator uses MRTG to keep an eye on server bandwidth usage.

Note: MRTG is available from the main MRTG site, http://oss.oetiker.ch/mrtg/, and has been packaged for most Linux distributions.

MRTG uses a set of Perl programs to collect, store, and display graphical data. A set of fast, specialized C routines consolidate the data and generate the graphs, which then are embedded in Web pages for display. MRTG can read a specially formatted text file for its data, or can use basic SNMP queries to gather the data. Whereas SNMP queries are better suited for collecting data from network peripherals (such as routers), a text file is readily accessible via your temperature-reading infrastructure.

MRTG runs as an executable and takes its data collection and reporting cues from a monolithic configuration file. The file I use for my temperature collection is shown in Listing 3.

Listing 3: My mrtg.cfg file for temperature-data reporting

Workdir:  /var/www/mrtg
Logdir:   /var/www/mrtg/logs
Interval: 5

Target[office.temp]: 'awk '{print $1; print $1;
                            print "quite some time";
                            print "temp sensor"}'
   </home/sschafer/officetemp/currtemp.log'
Title[office.temp]: Office Temperature
PageTop[office.temp]: <H1>Office Temp Stats</H1>
MaxBytes[office.temp]: 100
Unscaled[office.temp]: ymwd
Options[office.temp]: nopercent,gauge,integer
LegendI[office.temp]: &nbsp;temp:
LegendO[office.temp]: &nbsp;temp:
Ylegend[office.temp]: deg
ShortLegend[office.temp]: deg
Legend1[office.temp]: Temp
Legend2[office.temp]: Temp
Legend3[office.temp]: Temp
Legend4[office.temp]: Temp

Notice that the configuration file delimits targets (in this case, just one, office.temp) by using bracketed identifiers for each configuration option. The Target line specifies the protocol and location of the device to monitor, or specifies the external data-gathering script that will supply the data. The target is also used to name the generated reports—my temperature report is office.temp.html.

In the case of an external script, which is used here, the data returned must consist of four lines:

  • First variable (usually incoming bytes)
  • Second variable (usually outgoing bytes)
  • Uptime of target
  • Name of the target
Note: To simplify matters, a handful of shell scripting lines are used in place of calling an external script containing those lines. I chose this method to avoid maintaining yet another script outside the MRTG framework. However, these lines rely upon the file "currtemp.log" being present and containing the last temperature read by the reading script.

As previously mentioned, MRTG is used primarily to track bandwidth across a router—hence the separate incoming and outgoing variables. However, you only need one variable, so your external script simply repeats the temperature value twice. A typical line of data from the shell scripting code resembles the following:

77
77
quite some time
temp sensor

The rest of the configuration directives are mostly subversions of the MRTG system to display legends and markers on the MRTG page. MRTG should be run periodically from a standard scheduling service, such as cron on Linux; the data is collected and new graphs (and pages to house them) are generated on each run.

For example, the following line in my system's crontab file runs MRTG with the appropriate configuration file every five minutes, to correspond to the five-minute interval specified in the configuration file:

*/5 * * * *     mrtg /var/www/mrtg/mrtg.cfg

Figure 3 shows a sample of what the resulting MRTG document looks like on my server.



Click here for a larger image.

Figure 3: The MRTG temperature-reporting page on my server (office.temp.html).

More detail, but more complexity: RRDtool and Cricket

Users who want more detail and flexibility in their reports might want to check into using RRDtool, a popular SNMP data-collection tool; and Cricket, a framework for presenting the data collected by RRDtool.

Figure 4 shows a sample page of the reporting generated by these tools.



Click here for a larger image.

Figure 4: My temperature reporting via RRDtool and Cricket.

The infrastructure for using RRDtool with Cricket is not trivial to set up, but can be worth the effort if you need a powerful, flexible SNMP data infrastructure. Both applications are packaged for most Linux distributions, or can be found at the following locations on the Internet:

One advantage of this infrastructure is that the reports are generated in real time when the reporting page is loaded. Although the temperature readings aren't time sensitive, other data-monitoring applications may need to have up-to-the-minute data on tap.

For my temperature needs, I co-opted a Cricket configuration file from the contribution section of the Cricket Web site—one that was specifically designed to report the temperature of Sun servers. It was much easier to implement that feature than it would have been to create a configuration from scratch.

Conclusion

This series of articles demonstrated several technologies and methods for collecting and reporting data from a variety of sources. Although the series used examples from a particular application, a thermometer connected to a serial port, the techniques used in that application can be applied to any situation in which data collection and reporting is a necessary component.

About the Author

Freelance consultant Steve Schafer has written multiple technology books and articles. In the business world, he most recently worked in-house as COO/CFO of Progeny Linux Systems in Indianapolis. Serving as president and CEO in the company's startup years, Steve led Progeny's business and financial turnaround during the tech crash of the early 2000s. Prior to joining Progeny, he was a senior title manager for Macmillan Digital USA, overseeing the operating system and entertainment software products and tripling Macmillan's Linux product line revenue. He partnered Macmillan with Mandrake, bringing Mandrake's Linux distribution to the competitive retail market.

Sitemap | Contact Us

Thanks for your registration, follow us on our social networks to keep up-to-date