In the last year, customers often complained about our bad performance. We had just fixed a small memory leak (which crashed our publishing instances about every hour or so), so we were quite interested in getting reliable data to confirm or deny their anger. That time I thought that we need to have a possibility to get a quick overview of the performance of our CQ instances. One look to see “Ok, it must be the network, our systems perform great!”
So I dug out my perl knowhow and wrote a little script which parses through a request.log and prints out data which which is understood by gnuplot. And gnuplot draws then some nice graphs of it. It displays the number of requests per minute and also the average request duration for these requests.
(Click on the image for a larger version.)
These images proved themselves as pretty useful, because you show them to your manager (“Look, the average response went down from 800 miliseconds to 600 although the number of requests went up by 30%.”) and they help you in daily bussiness, because you can spot problems quite well. When at a certain time the response times go up, you better had a look at the system and find the reason for it.
Because this scripts is quite fast (it parses 300 megabytes of request.log in about 15 seconds on a fast Opteron-based machine), we usually render these images online and integrate the resulting images in a small web application (no CQ but a small hacked up PHP script). For some more interactivity I added the possibility to display only the requests which matches a certain string (click on the image to view a larger version). So it’s very easy to answer questions such “Is the performance of my landing page that bad as customer report?”
You can download this little perl script here. Run it with “–help” first and it will display a little help screen. Give a number of request.log files as parameter to it, pipe the output directly into gnuplot (I tested with version 4.0, but will probably also work with newer versions) and it will output a png file. Adjust the scripts to your needs and contribute back, I released it under GPL version 2.
(For the hackers: Some things can probably be performed better and I also have some new functionality already prepared in it, but not active. Patches are welcome :-))
Things on a content management system 
Hi there, i tried to get the script running on my windows box but i’ve not been very lucky until now
You pipe the scriptoutput right to gnuplot right? like: perl graph-request-log.pl | wgnuplot.exe
Hi Bastian
Yes, I pipe it directly into gnuplot. If my DOS knowledge is right, it should also work on Windows. In case it doesn’t, you can redirect the output into a file and then process the file with gnuplot.
Hmm i dug deeper into the script now it gives me following output
#!/usr/bin/gnuplot -persist
set timefmt "%d.%m.%Y %H:%M"
set format x "%H:%M"
set xdata time
set title "(no title given) "
set xlabel "Time"
set ylabel "answered requests [1/min]"
set y2label "delivery time per request [ms]"
set term png small size 770,400
set y2tics
set logscale y2
set grid
#set logscale y2 2
#set y2range [:5000]
plot "-" using 1:4 axes x1y2 title "average delivery time per request " , \
"-" using 1:3 title "number of delivered requests"
e
e
when i uncomment line 116 it prints me the duration (wich is right) but it wont appear in the sniplet wich will be piped to gnuplot. Any idea?