You are correct babies do need passports. An angle that has not been brought up before.
(I keep gettin' that durn' red line under falutin'. Is it 1 L or 2?)
The law assumes the child is NOT a citizen of the US. Else there'd be no need to add the child to a visa since US citizens don't need a visa to be inside the US.
Foreign rules regarding Americans who give birth abroad are comparable.
There are also "status of forces" agreements in many countries and military personnel may be governed differently (regarding the visa deal).
Just quickly looking through the State Department's rules, it doesn't say children have to have a passport (when accompanied by a parent or guardian).