Mmmmmm. I don't have that problem. Have you checked tyre pressures? If you are heavy, riding 2-up or with heavy luggage consider increasing tyre pressures, especially on the rear. Harley ignore tyre pressures IMHO where adding a passenger plus luggage is concerned and I use higher pressures (on my Glide) than recommended, to overcome handling problems. Well worth pursuing this further, as there may be a cheap and simple solution.
Check your tire pressure in both tires. The correct pressure psi for each tire is in your front down tubes.
Note: Any recommended tire pressures found on the bike or in the Owners Manual, apply only to the stock tires that came on the bike. When running different tires other than stock (brand OR model) refer to the tire manufacturers website for the proper recommended inflation pressure.