To me it's not a matter of mdm slowing you down, but but rather each time you're at the track you're training your brain with dsc in play. It covers a lot of very bad habits - mistakes where without dsc you would simply spin. This gets magnified even more when you approach the limits of the car.
If you're having fun and don't want to risk trashing your nice toy, by all means leave it on.
But if you're possibly pursuing a racing career then you should never have dsc on.
This is coming from a guy who leaves mdm on at the track