It must have really s*cked to be one of those 40 guys. But even assuming you're right about the scale of the damage. Lee's army was in Pennsylvania for less than two weeks. Northern armies had been in the South for periods of that length without doing any serious damage. If Lee had been in Pennsylvania longer, if there had been more fighting over a longer period of time, who knows whether Southern troops would have been able to maintain discipline.
They went out of their way to slaughter, pillage and destroy everything in their path from Atlanta to the sea, a scale of destruction unmatched since the Thirty Year's War, some two centuries earlier.
Sure, if you want to overlook the Wars of the French Revolution and Napoleon. The Spanish and Russian campaigns and the war of Frenchman against Frenchman were quite destructive. Even our Indian wars were disastrous for those involved.
You want to back that whopper up? CS Cavalry was patrolling PA in May of 1863, 2 months before the first day.