for amateurs, GoPro 2 or 3 is the best out there in terms of bang for the buck.
IMO, the downsides of GoPros are:
1. that it can get quite expensive when you add up all the accessories, and these accessories almost NEVER goes on sale
2. software is pretty shitty. I have a Hero 2 HD, and for some stupid reason, the only way to get the wifi bacpac to update and sync is to use a downgraded version of their desktop software, when you use a Windows 7 PC. The Hero 3 solves this problem as it comes with built-in wifi. I'm not sure if Hero 3 users have had software update issues.
If you want to go cheap, Vivitar offers an option, they also have cheap acesssories online and at Target:
and for some people (i don't know reasons why), Sony is marketing their new cams hard, for about the same price:
On amazon, there's quite a few comparison reviews of the GoPro 3 vs Sony
One option if you're on a budget, is look for the remaining GoPro Hero 2 HDs. If you're lucky you can find new unopened ones for less than $150.