For $100 its about the cheapest part you can buy for your Harley.. Do what works for you - Replace it when it goes bad. Skip 1 dinner with the wife a year and you will pay for it.
What kinda of oil are you running ? The wrong oil could get too thick in lower tempatures and give the engine a hard time turning over. That would also explain why the bike starts fine in the garage with the bike being warmer the oil thins out