I disagree with you on this. Driving is indeed a right. The right to use the public ways goes back to Roman times.
"The right of the citizen to travel upon the public highways and to transport his property thereon, either by carriage or by automobile, is not a mere privilege which a city may prohibit or permit at will, but a common right which he has under the right to life, liberty, and the pursuit of happiness."- Thompson v Smith 154 SE 579.
You have the right to ride in a vehicle on the public roads, but piloting one is a privilege.