I don't pull the wheels, but I definitely clean the barrels every time I clean my wheels. I use some quick detailer and microfiber cloths after every 2-3 drives. Keeps dirt from accumulating and makes cleaning easy. I also clean the inside of my exhaust tips and routinely use a metal polish on the muffler cans and tips. About once a month, I jack the car up to clean and dress the wheel wells and clean off the suspension components.
My wife understands. She is the same way about our kitchen. We are a true OCD couple.
Precision tool: 2011 BMW ///M3
Sledgehammer: 2014 Audi S6