Nice lookin' Evo!
I sold my Evo IX MR SE, so that I could get into an M3 also. I have to admit that I do miss the Evo. It has a more raw and unfiltered driving experience compared to my new M3. Also, the violent rush after 3k rpm is unforgettable.
If you're looking for a more refined car without losing most (if any) of the Evo's performance, then you're heading in the right direction. I feel the M3 looks better too.
At the end the Evo is all about performance, and it happens to be practical with the 4 doors as well. I don't regret going with my M3, but the Evo has that 'one of a kind' driving experience that most performance cars will never, ever be able to touch. I would keep it if I were you.