I think I have figured out the F2F issue with my compact. After reading a thorough investigation, it appears it's a factory issue. The slide lock has a small piece of plastic that grabs the slide when you're out of ammo. That piece of plastic is too close to the rounds, and since the XD uses a left/right feed technique, the rounds on the left are catching.
The remedy is to file down that plastic tip. Looking at pictures he posted it looks like mine may be suffering that, but it doesn't look quite as bad as the picture. I'm also reluctant to shave this tip down.
Edit:This is what he saw:
My gun, with two rounds in:
With three rounds in