The worst of your noise was likely the one @ .020, with the other three "chattering in" @ .010 from time to time. It really is amazing how much variance and slop you can find in the stock cast support plates.
As you increase cam lift, the amount of lateral force goes up as well due to the fact that the pushrods are angleswith respect to the rocker assembly.
"It's been my experience" that rockers in the TP boxes, S/E forged rocker supports, and the new Bob Wood support plate have much tighter fit and likely require no shimming at all ~ which is best.
Congrats on getting things quieted down!