Have the riding jacket and pants. Have a neck warmer (made out of an old wool hat with the top cut out). Decent pair of boots. Throw in a 3/4 helmet (or if it's real bad my full face modular) and I am usually good to go (in the high 40's or low 50's)...
Except for the Hands!
Cold hands is not a good thing if you want to take a ride for a couple of hours (not even doable for really long rides). You can find yourself losing the ability to control your ride.
I had seen ATV's with the handlebar "Mittens" and snowmobiles as well. didn't know if a good fit was available for the Fatboy.
Searched here & found "Hippo Hands"... but couldn't find any current seller or site for them.
Then I tried Amazon.com (I am not affiliated with any vendor or site selling any motorcycle product; or any kind of product for that matter).
Amazon had some things called "Scooter Mittens" (not my name!).
I took a shot & ordered them.
$27 plus $7 for shipping.
They are made with quality materials.
They went on the bike in 40 seconds (two adjustable secure Velcro fastening straps).
They go around the mirrors & the brake reservoir.
They held their shape at speeds up to 90 mph.
My hands found their way to the buttons & levers without any problems.
They have a clear plastic window to see the controls labels on the buttons.
Getting my hands in & out at stop lights or to "Wave" at passing riders wasn't an issue at all.

It was 45 degrees when I started my ride I did two hours on back roads at speeds from 40 to 90. I was wearing my summer riding gloves (a cheap pair of "Mechanics Gloves" from Discount Auto Parts). Hands were never cold! Not even a little bit...

I stopped for gas at the "Hess Station" on Cortez Blvd. in Hernando County FL. (usually find a couple of hundred riders there on Saturdays & Sundays); I could have sold twenty pair of them...

Heavy gloves even good ones make you give up some feel.
Heated grips may make the palm side of your hands comfortable but they don't make the tops of your hands much warmer.
Wind shields on the grips (like on a BMW I saw there) are more of a permanent installation; these go off and on in seconds and provide more protection.

For colder climates there is plenty of room to add those "Hunter's hand warming packets".

For rain you could spray the outside of the "Mittens" with a good water repellent spray.

Bottom line is these things worked great; I was pleased & thought I would share.

Search Amazon.com for "Scooter Mittens" (if cold hands while riding is an issue for you)

 

I dunno, those look a tad bulky for me. I've gotten good service out of a pair of padded and lined fingered ski gloves I bought at Bass Pro for like $14.

 

Thanks for the info. I ride down to 20 degrees and my hands are my only problem area.

 

This is perverse joke

 

Looks like a decent idea to keep the wind off.
At 40 degrees I wear my thin leather gauntlet gloves with jersey gloves for liners (About $20 for the leather gloves and $1 for the jerseys).
I also have Insulated leather gloves ($20). The insulated gloves are warmer but a bit bulkier.
Are you using those without any gloves?

 

"Could, but i used them with the thin Mechanics Gloves i wear most of the year when riding"...

 

When it gets cold enough, I first slip on a pair of UnderArmor glove liners, then my leather riding gloves. I found the liners at a ski shop. This has worked at temps that were in the upper 20's.

 

The only solution I've found for cold hands were heated gloves. I fought this for years and this last year I bought a set of heated gloves and a jacket liner (Gerbrings). Also bought a set for my wife. I was always concerned about the wiring aspect but its not as bad as I thought. I wished I would have done this long ago but like I said was too worried about the wiring issue. Believe me they're worth the price if you ride for long periods in cold weather.

 

I wear a pair of white gloves from Walgreens under ski gloves and I'm good to go down to about 25 degrees.

 

I stumbled on this idea last year riding dirt bikes in 15 degree weather and it's great - I bought some of those little packet "hand warmers" that you see in the outdoor stores this time of year - the kind that heat up when you expose them to air. My buddies had some too and put them in the palm of their gloves but I had a different idea: Since the main blood flow going to the hands is exposed on the top of your hand (look at the veins!), I dropped a packet in the top of my hand inside the glove and I was toasty all day. My buddies were all complaining about how cold their hands were and how dangerous it was to lose feeling - they tried this idea and it worked great for everyone. I don't ride frequently at 40 degrees or below, but this is my solution for my hands...