I have both a ZCP 06' E46 M3 and a E90 M3. I drive both quite a bit and like most people will say the E46 is more of a 'raw' experience and just feels more of an event every time your behind the wheel of it. I've had mine for 6 years and its been great with hardly any issues however I would be lying to you if I said I don't constantly worry about things going wrong with it since I am north of 100k miles. If I was looking for another M3 today it wouldn't be another E46 just because I think they are way overpriced now.
My advice is if you buy the E46, stay away from the forums because you will think you'll need to spend more money maintaining it than you actually need too