Analyzing Trends with Timechart in Splunk

Posted by David Veuve - 2011-07-27 11:59:51
One of the most useful theories to get when using timechart is generalizing data to a certain level of granularity, and then tracking changes over time. A good example of this is looking at the hourly amount by Splunk, but viewed on a per day basis. Splunk is going to toss many events per hour. You need to first find the hourly sum for every hour over the last 7 days, and then look at the daily statistics around that. Try this query: index=_internal source=*metrics.log splunk_server="*" group="per_index_thruput" earliest=-7d@d latest=@d

     | eval MB=kb/1024

     | timechart span=1h sum(MB) as HourlySumMB

     | timechart span=1d avg(HourlySumMB) as "Hourly Average", max(HourlySumMB) as "Max in an Hour", min(HourlySumMB) as "Min in an Hour", stdev(HourlySumMB) as "Standard Deviation of Hourly Average"

In practice, though, you're not usually going to want to use two back-to-back timecharts. You'll have more flexibility if you use timechart at the end, and switch earlier commands to stats, due to timechart's renaming of fields. Get a lot more detail on that (and an extension of our example here) at Timechart Versus Stats.

Ready for more? Check out: