What part of the invasion force would have arrived safely if it hadn't been for the Royal Navy escorts ? Germany had mainly a fleet-in-being (except for the submarines), but Italy had a sizable and good quality navy, with state-of-the-art battleships.
Furthermore, the Vichy French navy at Toulon (which fielded good battleships and two aircraft carriers) would have already been seized by the Germans - they only refrained from doing so since Hitler did not want the ships to sail for England and enhance Great Britain's fighting ability.
Take England out of the war and you take more than half of the ships, men, planes, and logistic support available in 1942.
The US initially lost quite a bit of its merchant ships to German U-Boat attacks but saved its escorts for troop movements. By May of 1942 over a 150,000 troops were shipped via the American navy. That's not to take anything away from the British navy, next to the US's and probably after Pearl Harbor there's was second to none.
Here's some great reading on the U-Boat attacks early in the war: