The "23" in the term, does indeed come from 23rd Street; however, my gran told me that at one time there were "undersireables" ( her word...but meaning the worst sorts ) that would hang around the Flatiron building, the cops would tell them ( not nicely! ) to "MOVE ALONG". This then became a slang term, for them, which they would say when they saw cops approaching, which then spread to others, meaning lets get out of here.
"SKIDDOO", alone, actually means : "LEAVE QUICKLY!" :-)
I love the English language and have many books that deal with it, euphemisms, idioms, and such. A really cute book to uses as reference/to read just for fun is one from the LET'S BRING BACK series, titled :THE LOST LANGUAGE EDITION, by Lesley M.M. Blume.
HL Mencken wrote some great books about the language.