Wall didn’t go up until 1961, it would have been very simple to cross into East Berlin at the time. In most cases, as easy as crossing the street.
If you see the movie "One, Two, Three" which took place about a year before the Wall went up, it showed people coming in and out of East Berlin rather easily....(Just as long as you don't have a balloon saying "Russki Go Home" when riding your motorcycle through East Berlin ;))