They knew the earth was round because they could observe ships leaving port and see less and less of the mast until it finally disappeared into the horizon. They could also observe that the moon and sun are round. I don’t believe that there was a widespread belief that the earth was flat.
Only among uneducated landlubbers, and religious extremists. The reason there was a lookout in the 'crows nest' was so they could see a ship's mast coming over the horizon sooner than someone on the deck......................
The ancient Greeks (and others) knew that the earth is round. Eratosthenes of Cyrene, about 200 B.C., calculated the circumference of the earth fairly accurately (far closer to the right figure than Columbus was).