"Nobody believed the balloons could have come directly from Japan. It was thought that the balloons must be coming from North American beaches, launched by landing parties from submarines. "
From Fire Balloons
I thought you might find the link in reply#8 interesting.