I actually think an M3 is a really good car for this - assuming it is in a non flashy (i.e. not one of the Lime Rock orange ones) color. To most non-car people, it looks like any other 3 series. The other car enthusiasts know that it is a bit more special but that's OK, if they are car people, they probably won't judge. It's also not crazy expensive (e.g., not like an exotic) and while prices have come up quite a bit since the E36 version, I don't think the M3 has a reputation for being that expensive even for those who know what it is.
I've given this topic a bit of thought because I drive my M3 to work specifically because I am concerned about the image of driving my Porsches to work (which I avoid doing). But with the M3, I am pretty comfortable.