Yup, it's safe.
Do you live in San Jose perhaps? They think San Fran is safe because the only time they go there is to use the airport, and it's pretty safe in your car (relatively speaking).
It's not like Fairfax County VA, though ~ we have a very low crime rate.
We take the kids to the zoo every other Saturday, never had a problem, what is the problem on Sundays you suggest.