I had this issue with my AP BBK and I had to order the spring-kit mentioned and it helped keep the pads back and reduce that rubbing/growning sound your talking about at low speed (3~5mph). If your BBK didn't come with it you'll have to order it. It helped the problem. Its nothing bad or damaging what you feel but it can be a bit bothersom.
Good luck and let us know what you resolve with it.