M3 GTS is really a wrothy competitior of the 911 GT3 RS, but I would still compare it to 911 GT3. 911 GT3 RS rival should be the M3 GTS-R. And still think about this: 911 is a pure sports car, M3 just a 3 Series.
These are simply the best street legal track cars in the world. My personal preference:
1. Ferrari 430 Scuderia
2. Porsche 911 GT3 (RS)
3. BMW M3 GTS(-R)
But I would take the M3 because I am not a good driver, and because I am biased, the fact that the M3 is rare, it will still be enough respected on track.