First, would they be driving on roads upon which there would be no cars with human drivers? If not, how do you program the car to deal with unexpected things that a human-driven non-computerized vehicle might do?
What happens if the GPS or the road sensors don't work, or have a glitch?
What if someone decides this is a great terrorist target? Just disrupt the GPS or road sensor signals and watch the results..
Even programming in construction detours would be a challenge.
While a lot of drivers out there are pretty scary behind the wheel, they retain the potential to react appropriately in sudden situations despite ordinary inattentiveness or poor skills.
A computer might not be able to do so, especially if it is installing one of bazillions of updates and security patches, etc.
Of course, compromise such a system with evil intent, and the potential for carnage is phenomenal, not to mention the potential for the 'removal' of individuals, while blaming a 'computer glitch'.
It'd make a great sci-fi plot, but I can't honestly say I see where it would be an improvement.