Woodstone Servers Alive®

Servers Alive is a monitoring and alerting software from  Woodstone that is capable of quite a few things at an affordable price. Though this software has a bunch of integrated test methods it allows for expansion and customization via an ActiveX COM interface. See  this document on how to customize and get the most out of this software (including an easy way to write flexible plugins to extend the functionality).

LogParser

Microsoft offers an incredible tool for evaluating log files that helps you extracting only the important bits. See this  info on LogParser for a general introduction of the tool and what you can do with it.

Windows, Performance Counters and SNMP

while it was possible to integrate Performance Counters in SNMP on Windows NT 4.0 it is officially not supported anymore. Good news that someone re-implemented the functionality for Windows 2003. Have a look at  Erwan's SNMPTools where you can download the SNMP extension DLL. You can use this SNMP extension to integrate it into ServersAlive. Defining your own OIDs on the target system you can query any performance counter or start your own programs to do some checks. The result will can then be reported back to ServersAlive where you get your statistics and alarms as usual.
 Note   Contrary to documentation it also supports int64 counters (not only int32).

Web Performance Testing

Yet another tool to do some web based performance tests is WCAT. I always found it not too intuitive to set up so I compiled some text to make is easier. Have a look at  WCAT.