Personally, I think there should be equal responsibility. When I see someone in a crosswalk, paused, waiting to cross, I stop.
I have an issue with someone just appears there and steps out forcing you to stop from a higher than normal speed without even looking.