Valid HTML 4.01 Strict!
Valid CSS!

Describing the past, not predicting the future

Index

Intro
Describe your problem
Work around this inconvenience
Wrapup

Intro

RRDtool stores rates during time intervals. It can only do so once the rate is known. Usually this means there are two moments in time, two counter values, do some calculations and then store the result.

This is why RRDtool describes the past. This may seem confusing at first but once you think about it, it makes much sense. Usually you can be certain about a value only once an event has happened.

Describe your problem

Before setting up your database, it is wise to take a moment to think what it is you are trying to achieve. There are lots of examples, not only on this site, that deal with traffic. But that's not all RRDtool can do. You will have to keep in mind though how RRDtool was designed (and why) in order to get best results.

Suppose I'd like to monitor the content of my wallet. I buy something, and update my RRDtool database. I want to have the actual number in my database so I choose the GAUGE data type. So far, so good. But what number goes into the database?

I bet a significant number of people would look at the content of the wallet after the purchase, and put that number in the database. This will seem to work. I start with an empty wallet. Let's look at some numbers:

TimeActionNew content of wallet
08:00Get money from the ATM$75.00
09:00Do some shopping and pay$60.00
10:00Get coffee and pie$55.00
12:00Buy lunch$45.00
18:00Taxi home$25.00
19:00Pay for pizza$10.00
24:00Save $10 for dvd recorder$00.00

This could be done using the following RRDtool commands:

rrdtool create wallet.rrd                       \
        --start 1130799600                      \
        --step 3600                             \
        DS:content:GAUGE:86400:0:U              \
        RRA:AVERAGE:0:1:48
rrdtool update wallet.rrd 1130828400:75.00
rrdtool update wallet.rrd 1130832000:60.00
rrdtool update wallet.rrd 1130835600:55.00
rrdtool update wallet.rrd 1130842800:45.00
rrdtool update wallet.rrd 1130864400:25.00
rrdtool update wallet.rrd 1130868000:10.00
rrdtool update wallet.rrd 1130886000:0.00
rrdtool graph wallet.png                        \
        --start 1130796000                      \
        --end start+26h                         \
        --height 160                            \
        --upper-limit 80                        \
        --lower-limit 0                         \
        --y-grid 5:2                            \
        --x-grid HOUR:1:HOUR:1:HOUR:2:0:%H      \
        DEF:content=wallet.rrd:content:AVERAGE  \
        AREA:content#0000FF
content of wallet

Nice. We have a graph, but do we have information? Have a look at the descriptive table above and compare against the graph. At 24:00 hours (meaning: midnight at the end of the day) I removed $10.00 from my wallet to save for that new dvd recorder I want to buy. But according to the graph my wallet was empty at 19:00 hours. And only at 08:00 I got some money, before that my wallet was empty, remember? The graph doesn't show this either. After carefull examination, all the amounts are right but the moment in time is not.

What has happened is that I did not describe the past. Let's try again. The create and graph commands stay the same, only updates differ:

rrdtool update wallet.rrd 1130828400:0.00
rrdtool update wallet.rrd 1130832000:75.00
rrdtool update wallet.rrd 1130835600:60.00
rrdtool update wallet.rrd 1130842800:55.00
rrdtool update wallet.rrd 1130864400:45.00
rrdtool update wallet.rrd 1130868000:25.00
rrdtool update wallet.rrd 1130886000:10.00
corrected content of wallet

MUCH better! As you can see, I shifted all amounts up by one update. This is correct: When I take money out, I tell RRDtool what was in the wallet upto that moment. From that moment on, the wallet contains a new amount of money and of course I do know what amount it is. This is a case where you actually do know part of the future. But you still don't know how long it's going to be valid.

As demonstrated, it not only matters what numbers you give to RRDtool but also when you do this. If you fail to do this right, you need to start over again or -even worse- you are basing decisions on false information.

The moral of the story:
"what is ..." is the wrong question for RRDtool
"what was ..." is the right question for RRDtool

Work around this inconvenience

Knowing what was in the wallet before is easy. Either look at the content before you take the money out, or look at the new content and add the amount you put in / took out. But this may not always be easy to do.

There is a trick. Suppose you know, for certain, that you are going to look at some value every whole hour. Of course you do not know in advance what the result will be, but you will know at what time the next update is going to happen. If you need to describe the near future, then in stead of remembering the current value for a later update, you could also modify the timestamp. If you know, at 08:00, that a rate is going to be valid from 08:00 to at least 09:00, you could set the timestamp to 09:00 even if you do the update at 08:00. When nothing happens at 09:00, you update the database again (using 10:00 for the timestamp) and so on. This would result in the following updates for my example:

rrdtool update wallet.rrd 1130803200:0.00  # update at 00:00 with timestamp 01:00
rrdtool update wallet.rrd 1130806800:0.00  # update at 01:00 with timestamp 02:00
rrdtool update wallet.rrd 1130810400:0.00  # update at 02:00 with timestamp 03:00
[...and so on ...]
rrdtool update wallet.rrd 1130828400:0.00  # update at 07:00 with timestamp 08:00
rrdtool update wallet.rrd 1130832000:75.00 # update at 08:00 with timestamp 09:00
rrdtool update wallet.rrd 1130835600:60.00 # update at 09:00 with timestamp 10:00
rrdtool update wallet.rrd 1130839200:55.00 # update at 10:00 with timestamp 11:00
rrdtool update wallet.rrd 1130842800:55.00 # update at 11:00 with timestamp 12:00
rrdtool update wallet.rrd 1130864400:45.00 # update at 12:00 with timestamp 13:00
rrdtool update wallet.rrd 1130868000:45.00 # update at 13:00 with timestamp 14:00
rrdtool update wallet.rrd 1130871600:45.00 # update at 14:00 with timestamp 15:00
[...and so on ...]

Wrapup

This concludes this explanation of how rrdtool works with data from the past. Always make sure you describe the past (from rrdtool's point of view) and not the future. It may seem to work, but you will most likely encounter problems.

Do you like this information? Tell others! Don't you? Tell me!

This page was created by Alex van den Bogaerdt, an independent IT consultant. If you want to provide feedback or if you want to hire me please see contact alex.
Back to the top of this page. You can also go to the index