I have a 96 wideglide that has saddlebag brackets on it, there are 2 Allen bolts on the outside but I can't see or feel a nut on the inside of the fender

 

It's not a nut, it's a bracket kind of thing that both front and back bolts go into, it supports the fender.
Look up your bike here and look at "rear fender supports" for a diagram.
http://partsfinder.onlinemicrofiche....DER%20SUPPORTS

It's #15,,

 

Thanks for the info, but I cant get the bolt out. The inside thing just spins and I can't grab it with anything. It just spins

 

Try some Visegrips on the back side to hold that fastener, or be prepared to either cut or drill the heads off the allen bolts.

 

It's basically a T Nut welded onto that bracket. Weld must have broke.

Like said grab it with pliers or vice grips.