Not every military base around the world in those days had it’s own hospital.
A local hospital where births occurred were declared to be ‘Military bases’ to cover such instances.
I have no uncertainty about McCain’s status to be eligible to be President based upon his birth.
In those days not all births occurred in hospitals, in fact most did not. However Coco Solo did have a Navy doctor assigned, so I imagine they at least had a dispensary. However the announcement in the local English language newspaper, the Panamanian American, said the birth was in the "Submarine Base Hospital". McCain's mother said he was born in the base hospital.