Been down the same road myself not that long ago. Tried to source an M3 through my local dealership and they went out of their way to get me a drive even though they didn't have a car in stock. Built up a good relationship with the salesman, but when I told him I could source nearly new for £15k below what they were asking he told me to go for it, he couldn't match.
I found the car via the BMW website, bit the bullet. Sold my Audi to the dealership in Crewe as PX for the M3 and haven't looked back. The best experience I've had in buying and selling. Got great price for my Audi, found the nearly new M3 had only 15 miles on the clock after being advertised as 500. I'd recommend anyone to give it a go.
Besides, your existing dealer can pick up the servicing post sale anyway, so they get something in return anyway.
If you're getting the car in the spec you want at the right price don't feel guilty about it....rejoice!