You're right in that it's all about comfort level. My preferred mode of walking around with my rig is to have the camera in some sort of bag and only pull it out when I need it. That obviously wouldn't work for you because you need to be ready to grab a shot of a bird at a second's notice. But my targets are different and I don't feel right having the camera around my neck, especially in a city environment. In fact, neither of my bodies has a neck strap. They both have wrist straps. So for me, ease of movement in and out of a bag is important. And again, as stated above, I tend to shoot at closer range, but not always "indoor" close. When I shoot street shots, I like to be at "standoff range" so the subjects of my shots don't notice me. There again, a less inconspicuous lens helps.
That said, I've been questioning the need for my current collection of smaller primes and have started thinking about downsizing my collection. I've been reading David duChemin lately and he suggests working with one lens exclusively for a week or so to really get to know the characteristics of the lens. I'm thinking this would also be a good way to determine the utility of the lens. Do I really need a 35mm f2 when my default lens (24-105) will do 35mm at f4? I do like my 85mm f1.8 and I tend to choose that over the 50mm f1.4 unless I really need the shorter FL. Note that speed is not a consideration there. So the 50mm f1.4 could go as well. If I also sell the T1i with the kit lens and the 55-250, plus some extra stuff that only works with the T1i, I could easily afford a 135mm F2L and have funds left over to put towards a FF down the road. That would give me 10-22, 24-105, 135, and if I really want to travel light, the 85. Add a 1.4x TC and I'd have some long range if/when I need it.
Starting this weekend I'm going to use one prime per week exclusively (unless something out of the ordinary requires otherwise). It'll be interesting to see if I can come up with images from a given prime that I could not have produced with the 24-105.
"I know rocks. Rocks are hard."