I can appreciate your dilemma as I was just there myself. I used to drive a 330 ci and thought the E46 M3 would be an extension (you say "hopped up") of this driving experience. I was very wrong. The M is just a different driving experience, maybe not better as it depends on your use and needs, but it is very very different. The cost of the E92 M3 is a really concern (think about the tire replacement costs with this new monster too!). I suggest you check out a pre-owned E46 M3 (as I stumbled into) as a good compromise. Still, I hear the 335 is great drive too, if don't want that "tight" M drive (I like it but others find it a bit much for daily drives). I have pre-ordered a E92 M3 but I enjoy the E46 M3 so much I no longer care that it may take a couple of years (given my spot on the list and the options I want).
And, the money you save you put into some good mods if you want to add to the appearance and or performance.
Hope this helps and good luck with your decision.