This is my first post here (member of too many forums, go through a lot of cars!).
If you've had fun with the M3 I say do it. You only live once and if you absolutely hate the Z it's not like you can't get back into the seat of an M. BMWs and Z06s live very well together. I previously had an 2004 M3 and the wife currently has an 2007 335xi. I have a 2009 Z06 and a 2010 Cobalt SS (DD) with a turbo upgrade (kind of boy racer for a grown man but it keeps me happy).
I would say go used. You can probably find a very low mileage 2009 in the low 50's that's still within the original bumper to bumper coverage. Two things you have to do though 1) get a GM Protection Plan warranty (can find them through dealers online for about $1000 for a 4 year); and 2) get it thoroughly inspected by someone who can detect if the car was ever tuned. GM will void your entire warranty in a heartbeat even if the car was tuned by the former owner and put back to stock.
I went into it knowing the car's limitations and trade-offs. We all know the interior is krap, stock speakers are krap, and the plastic fantastic body panels are not quite up to par with a BMW. BUT, that's what you trade to spank an F430 with a $50K used car.
There are little things you can do that completely transform the entire car. Halltech CAI (truly adds 15+ whp), DRM custom valved Bilstein shocks (eliminates rear end snap due to bumps, plants the car), and Michelin PSS tires. I was one of the first people to get PSS tires on mine. I've gotten the car to launch in 1st down to 45 degrees F with ~1 foot rollout. Also did some extensive rain testing and was able to use about 80-90% throttle with sharp steering changes in cold rain. A guy in AZ was able to pull 1.1 lateral g with them on the skidpad.
Looking good together [that's what red ones actually look like if you take care of them; they usually photograph terribly but I used a DSLR]:
Rain = no problem!