I just last week picked up an '09 E90 M3 DCT in AW. I have not had the chance to put more than about 600kms on it yet. I traded in a '08 535xi with 21,000kms on it.
My initial impressions fit with most of the comments above. The M3 is just a "special" car, in handling, tightness, "feel", high end grunt (can't get way up there yet as still on break-in), sounds, and "emotional feel". Initially, I would say the 5er was a touch more comfortable, as it should be as larger, heavier car with "comfort seats" option. The torque of the TT 3.0L was nice, and was appreciated in around town driving.
I am older, the "M" was a bit of a bucket list thing for me, and I'm sure I will enjoy for 3 or 4 years, then probably go back to something a bit more docile (like a 335d...maybe?). Every auto enthusiast should have the opporturnity of drive a BMW M car for a while......
