The RS4 is a fast, luxurious, safe sports sedan. The AWD really puts the power down well in all conditions. However, for more talented drivers who know how to drive a rear-wheel drive car well, the M3 will be far more satisfying. It will let you have oversteer when you want it (for fun or for track racing) and will offer better steering feel because no power is going through the front wheels. It will also be lighter (by at least 300 lbs) and thus more "tossable."
If you live in a snowy/icy/excessively rainy area and want to drive the car year round the RS4 is the way to go. For anyone else the M3 will be faster, a better driver's car, and probably a bit less expensive.
M3 > RS4