I read that Lee waited for a while in Gettysburg in a defensive position, thinking Meade might come after him. Also, Pickett was devastated and never forgave Lee for what happened to his men. Longstreet also wanted to get away from Lee at that point.
Other way around. Lee began his return to Virginia the next day, with the first of his wagons beginning the journey south on the afternoon of July 4. You're correct in that Pickett never forgave Lee.
Point of trivia. Pickett received his appointment to West Point from an Illinois congressman. Pickett's uncle and this congressman were political enemies and the uncle enlisted the help of a third party to act as go between and assist in obtaining the appointment. That go-between was Abraham Lincoln. Nice story, even if it may not be entirely accurate.