I've been playing forza 2 lately (never bought 3) and I think it's pretty challenging. I play with all the aids off, and a few of my cars are simply not drivable anywhere near their limits. Don't spend much time in factory-spec cars in these games.
Also a drastic difference in driving dynamics between cars.
Anyway, looking forward to F4. Pretty excited to see how the kinect will feel while driving.