Best traffic / performance / usage monitoring modu

Question

Are there any open source (or I guess commercial) packages that you can plug into your site for monitoring purposes? I'd like something that we can hook up to our ASP.NET site and use to provide reporting on things like:

• performance over time
• current load
• page traffic
• SQL performance
• PU time monitoring

Ideally in c# :)

With some sexy graphs.

Edit: I'd also be happy with a package that I can feed statistics and views of data to, and it would analyse trends, spot abnormal behaviour (e.g. "no one has logged in for the last hour. is this Ok?", "high traffic levels detected", "low number of API calls detected") and generally be very useful indeed. Does such a thing exist?

At my last office we had a big screen which showed us loads and loads of performance counters over a couple of time ranges, and we could spot weird stuff happening, but the data was not stored and there was no way to report on it. Its a package for doing this that I'm after.

Answer 1

It should be noted that google analytics is not an accurate representation of web site usage. This is because the web beacon (web bug) used on the page does not always load for these reasons:

1. Google analytics servers are called by millions of pages every second and can not always process the requests in a timely fashion.
2. Users often browse away from a page before the full page has loaded and thus there is not enough time to load Googles web beacon to record a hit.
3. Google analytics require javascript to be installed which can be disabled.
4. Quite a few (but not substantial amount) of people block google-analytics.com from their browsers, myself included.

The physical log files are the best 'real' representation of site usage as they record every request. Alternatively there are far better 'professional' packages, of which Omniture is my favourite, which have much better response times, alternative methods for recording actions and more functionality.

Answer 2

One option is to use external monitoring tools, which will monitor the web performance from outside the firewall by simulating end user activities.

Catchpoint Systems has an interesting approach that requires very little coding and gives you the performance stats from outside the datacenter and from inside the asp.net (like processing time, etc)

http://www.catchpoint.com/products.html

Answer 3

If you're after things like server data, would RRDTool be something you're after? It's not really a webserver type stats program though, I have no idea how it would scale.

Edit:

I've also just found Splunk Swarm, if you're interested in something that looks "cool".

Answer 4

http://www.serverdensity.com/

Answer 5

Google Analytics is quick to set up and provides more sexy graphs than you can shake a stick at.

http://www.google.com/analytics/

Answer 6

@Ian

Looks like they've raised the limit. Not very surprising, it is google after all ;)

This free version is limited to 5 million pageviews a month - however, users with an active Google AdWords account are given unlimited pageview tracking.

http://www.google.com/support/googleanalytics/bin/answer.py?hl=en&answer=55543