The E46 is more refined and a better daily driver.
The Z4 is more sporty with a much harsher ride and a better weekender.
It sounds like you eventually want a weekend fun car, in which case you should get the Z4 M and put up with the hard ride and lack of space/seats over the few months it will be your daily driver.
I would choose the E46 (I am actually buying an E92 M3) but my circumstances are different to yours as this will be my only car and I would prefer the practicality of four proper seats and good boot space.
If I had another car though I would not hesitate to buy a 2-seater, having said that it would be a Cayman S not a Z4 M.