Not easily. I walked the streets of downtown San Francisco with my antique sword a few years ago, sheathed and slung over my back. I was taking it to a Japanese sword expert in Japantown to have it inspected and appraised. Nearest parking that day was about six blocks away. No one batted an eye, just got curious smiles. Even when I shopped for a few groceries in a Japanese food market. You can't really hide something that long.
CA's open carry laws allow to carry a fixed blade knife (sword) openly on the belt. I don't know if carrying it over the back is prohibited; it's probably something a LEO would have to decide on the spot.
But some cities have stricter rules, and people were stopped for carrying swords in a way that is legal in the state but not in this particular municipality.
One easy way to avoid the problem (unless you want to test the LEO response) is to put the sword into a long box, or to wrap it. I understand that it is not good to give up on doing certain legal things because of the pressure, but everyone picks their own battles. It's also legal to stand in the street and sell powdered sugar in small plastic bags, but a rational person will not be doing that.