I recently traded in an M3 for an S4. Here's my perspective (which I posted on an Audi board):
I recently traded in a 2009 BMW M3 sedan for a 2013 Audi S4 and I couldn't be happier with the decision. Since I live in Washington, DC where most of my driving has 35 MPH speed limits, I think the S4 suits my needs better. For most of my driving, I would say I like the S4 much more. I only have 1 car at a time and I just don't think the M3 is a great "only car." The S4 just seems to be a great total package...I love it! Sure, maybe it's not as fast as the M3 (or C63, etc.), but it's fast enough for me. Maybe I'm getting older, but I don't feel as much of a need to get one of the fastest cars out there anymore.
Pictures of both rides attached...
Here's my take on the S4 versus M3 (in no particular order):
1) Gas consumption (S4 hands down): I averaged 12.7 MPG with the M3 in the city and I didn't drive very aggressively. I'm averaging about 18 MPG with the S4. I know the M3's V8 is thirsty, but the tank isn't very big and gassing up all the time annoyed me.
2) Daily driving (S4 hands down): Maybe not the best comaparison since the M3 has a 6-speed, but the S4's dual-clutch transmission is awesome and super smooth. In the city, the S4 is better with more low-end torque. I find it easier to pull away at a stoplight with the S4. M3 is fast for sure, but you need to really rev the engine and it gets loud... Quick story: 3 weeks after getting the M3, I got pulled over. I was at a stoplight and gunned it when the light turned green. I get pulled over by a US Park Service or Secret Service Cop (can't remember which one...wasn't a DC cop) because he said he heard me accelerating although he didn't clock me. He said I was probably doing 50 MPH in a 30 MPH zone...get off with a warning, but the bottom line is that the M3 gets loud at high RPM. The M3 engine sounds great, but the loudness did get somewhat annoying if I was just running errands.
3) Spirited driving: somewhat of a toss-up (I'm no weekend racer and don't go to the track). While the handling/feel of the M3 seems superior, I love the Audi transmission and quattro. There's no doubt that the M3 would probably crush the S4, but in terms of "driving satisfaction," they're both really fun to drive. M3 is faster and pulls harder, but you need to really rev the M3 hard (and on public roads, you can't do that much)
4) Exterior styling (toss-up): both beautiful in their own ways, although I had the Metallic Jerez Black with the M3, which was gorgeous. S4's grey is somewhat common, but I do like all the LED lights.
5) Interior styling (S4 hands down). M3 was pretty basic...
6) Features (S4, but not a fair comparison): my M3 didn't have too many options, while the S4 has the NAV, back-up camera, etc. Living in DC, it's awesome having a rear back-up camera and that's not even an option with the current M3.