OP, were the M3's you test drove broken in? We're both in MD so if you want I can give you a test drive - see what the M3 would feel like 1-2 years from now.
I was in a similar dilemma, choosing between the 135i and M3 after a 2 year stint in a JB-modified 335i. The 135i should have been perfect in theory - smaller and lighter and cheaper and with the same engine I loved. But the 135i felt like a tippy and understeery 335i, where the best way to describe the M3 was in the words of an online magazine writer: a discounted supercar.
It felt expensive, bespoke, fast, roll-free, understeer-free and with an incredible throttle response. On track, it's all of the above except even more so - a symphony and you are the conductor.
Get the special car, there's always time for the merely competent when you're older.
And if you go for Euro Delivery, I don't have to tell you where you need to take it - or do I?