Mine looks "good" almost all the time. We garage our cars at night and I wash every two weeks and polish/wax every 6 months. From ~20 feet it looks great always, but the white does show road dust, bugs, ect. The AW doesn't show light scratchs, ect much at all. The cf roof shows water spots like crazy.
I personally think grays and silvers look the best the majority of the time given the same amount of dirty-ness as other colors.
But, if you are a person who is really concerned about the appearance of a car (like me), they all look f'd up after a couple days from the wash. Dirt and grime is dirt and grime.
I just got done with a 3 hour wash and light detail of my wife's black S550 and I actually think black hides a lot (just not dust). Its much harder to polish and wax dark colors (as you said) because the swirls, ect show much more.
Sorry for the long response.