Search Engine Stats

I added a couple of new charts to the Stats section ... in this case, both charts contain some search engine stats.  Once again, visual data tells a really cool story -- it's great to see about 14 months of data summarized in a visual way.

In these charts, I decided to focus on Google, MSN, and Yahoo search engine bots and referrers.  There's quite a few other bots, but these three make up the vast majority, and the even when the other bots hit, I'm not seeing any interesting traffic generated from their respective portal.  The clearest and easiest solution was to simply omit them for the time being, but they can be added when/if needed.

What's really interesting is watching the ramp up of the various engines as they index the site over the first few months (September 2004).  The number of requests have a direct relationship with the bandwidth usage of the bots, and it's no secret that MSNBot is quite, um, err, a bandwidth hog.  There, I admitted it.  

What I don't know is how and if the behavior is indicative of site maturity and content versus the changing of the algorithms the search engine bots use.  For example, there's a huge spike in September 2005 (mainly from MSN, but also from the others) -- this may be caused by an increase in blogging activity (notice a correlation with the Traffic Analysis data).  At best, though, it's too early to tell since October has just begun and those numbers will undoubtedly gain quickly on September's data. 

The Search Engine Referrals chart shows the number of referrals I'm getting from the various sites.  I'm not surprised at all to see Google well above the others; to me, this illustrates Google's popularity and what is likely a more effective indexing strategy -- Google is certainly taking less bandwidth.

Both charts seem to indicate that October will be a very busy month, since the numbers for every data point have surpassed September for the same time frame, except for the number of hits from MSN's bot.  Going forward, I'd like to do drill down into the data to see the frequency and duration of the crawls -- unfortunately, I do not have enough data to construct much of a detailed history, but it will be there going forward.
Comments are closed

My Apps

Dark Skies Astrophotography Journal Vol 1 Explore The Moon
Mars Explorer Moons of Jupiter Messier Object Explorer
Brew Finder Earthquake Explorer Venus Explorer  

My Worldmap

Month List