I think it would be pretty powerful to be reminded that if you send your nephew out of your home, you are virtually executing him. I can see a reminder like that hitting Petunia pretty hard!
I think, for Petunia, it was less guilt over sending her nephew out than it was fear of knowing what could happen.
Yes, but she knew it was from Dumbledore -- even Harry, who knew his voice well, didn't know that.
It's a minor point, really -- but Rowling did address the question of why Dumbledore referred to "his last," which again implies "not the only correspondence we've had." Ultimately, it ties back to Petunia's childhood, and helps to explain how she became the person she did.
Interestingly, the letter also sets up the final goodbyes at Number 4 Privet Drive. Petunia and Vernon had been building a wall between the muggle and wizard worlds. Learning that Petunia had actually corresponded with Dumbledore irretrievably smashes the barrier between the wizard and muggle worlds. The Dursley's begin to appreciate the danger posed to them by Voldemort, and in Book 7, Petunia and Dudley finally recognize the protection that had been given to them over Harry's life.
Unsaid, of course, is that Vernon's take on things is right, too: Dumbledore knowingly and without their permission put them in mortal danger by putting Harry on their doorstep. Given Dumbledore's manipulativeness, of which we learn in the last book, one wonders if he mentioned that little item in his last letter to Petunia.
At the same time, though, they begin to see that the wizard world (the breeding dementors) is beginning to intrude on them anyway ... and Dudley, realizing this, finally humbles himself and is, I suppose, saved. Petunia, too.
I've always wondered, BTW, what nastiness baby Harry might have magically inflicted on baby Dudley. The tantrum of a magical 2-year old might do some serious damage. Perhaps Vernon and Petunia had a more personal reason to detesting Harry's magical abilities, and perhaps Dudley's dementor experience is explained as well.
And if Dudley suffered spell damage as a result of Harry's tantrums ... mightn't Petunia have to have asked Dumbledore's help in setting things right again?
Interesting....