Lee never should have let him do it.
That's one good side of the argument. The other is that Stuart went much farther in his ride than Lee had ever suggested. That was the two-edged nature of Lee's command style: he let his subordinates have a lot of discretion in how they carried out his orders.