Andrew and Hugo strengthed as they approached land along the Southeast coast - and both went over the Gulf Stream as Matthew will be doing soon. That is the wild card, and it is not a good one.
The last water temp chart I saw showed the Gulf Stream quite warm, relative to the surrounding water.
And, you are correct, it hasn’t hit it yet.
Hurricanes draw their strength from warm water, they are heat engines.