Twitter change and why Leemba is awesome7/14/2010
Ok, so my last post on Leemba and Twitter wasn't entirely complete. Everything is showing a major change, even the number of alerts generated:
The original image shows a big drop in response time. But if you look closely at the options there, you'll see I set the "sample method" to minimum. This is the aggregate Leemba is using to trim the number of points. Twitter is being checked every minute and the time period shown is about 10 days, so that would be around 14400 points to graph.
If Leemba displayed all of those points, then a chart like this would look like a big block of color. It'd be impossible to see a trend in all of that noise.
Here's the original:
Most monitors will simply display averages for this kind of historical data, but Leemba offers a number of tools to help analyze charts and data. Here's the same period, but with the average sample method. This is generally what you see in other monitoring solutions.
This looks roughly the same (although the scale has changed). It doesn't tell us too much we didn't already know.
This chart has another option, however. By checking the "Show Standard Deviation" checkbox, we can get a sense of the variability of the graph. Here's the same data again, this time with the standard deviation displayed (right axis).
With the standard deviation displayed, we can see we've got an awful lot of variation, even increased variation when the minimum response times have gotten faster.
Interestingly, the standard deviation (stddev) never really drops below a second before the change, but after there are periods with much less. What this might have been caused by was the opening of a data center closer to the monitor, which might lower response times but and error rates. Or it could have been a change in the geographic load balancing.
The variance (sometimes a request is fast, sometimes it's slow) could only be really explained by somebody very familiar with the Twitter architecture, which is why you need tools like Leemba that can display data in different ways. If we wanted, Leemba can zoom into any of these time periods on the fly, which makes it fantastic for testing performance changes in real time or evaluating the impact of past changes.