Standing on a street and photographing anything visible from that street is perfectly legal.
Shooting over a fence, through a window, or anything that is not visible from the public way can be illegal under local or state regulations.
Of course, very private people dont always agree with these rules.
Until Homeland Security Shows up and whisks you away in a black windowless van.
The Patriot Act changed a few things.