They were headed to Baltimore, a new port, and would have created general havoc there. A major battle? Not necessarily, if they burned everything and left.
Then they were taking the scenic route. Gettysburg and central Pennsylvania are nowhere near Baltimore, and Lee wasn't even heading in the general direction.