...you should go for the 2005 M3 with competition packet and save the money, if you cannot decide in which car you will feel best. Unless you will show off with the latest, of course. (I assume you have tried both cars.)
I am driving a 2003 M3 myself and it is definitely a keeper !
I can buy a E92 M3 anyday if I want to, but there is no need !
Besides...,you will always have the possibility to replace the car for a E92 M3 later without much loss.
My opinion is that the E46 M3 looks and feels more like a sportscar than the E92 M3 and therefore suits me best (at a lower cost.)
The E46 M3 drinks much less petrol
The Competition Packet M-track system also lets you drive pretty hard before the electronics comes to rescue
The E36 M3 is a cheaper car to own (insurance, parts)
The E92 M3 is not that much faster.
Take care.