I was reading some of the accounts from Carlisle, for example, and it seems the Confederates were rather restrained when they were in Pennsylvania....until they were fired upon from within towns.
How about Hunter's troops in the Shenandoah Valley, too? While the burn marks on Virginia Military Institute might be acceptable, as evidence of an attack on a military target, it's the civilian destruction that garnered complaints.
You might want to read some of the more recent books on the Gettysburg campaign. Both Noah Andre Trudeau and Stephen Sears wrote books within the last year or two, and both detail the confederate behavior in the North, usually in the words of the soldiers themselves. Farms looted, property taken, blacks abducted and sent south to slavery. Lee was forced to issue a general order condemning such behavior and ordering a stop to it, though it continued unabated.