Drive both and make your own judgement. A friend of mine owned a S5, I loved the looks of the interior and exterior over the M3 but the M3 shined more in the performance area, now the RS5 has more hp so it is indeed better competition for the M3. The all wheel drive is a good selling point if you need to drive it in incliment weather. Both are great choices, just get the one that meets your needs and style.
ESS VT-2 625 silverstone E90 /// M3 - 6spd - AD08 - Spacers - H&R - AA exhaust - Primary cat delete - Custom test pipes - ESS tune - Lighted ZHP.