Being a minority of one does tend to make a person feel lonely. I've been there, done that. But when you know you're right, you feel a little sorry for those who for whatever reason do not understand fully the issue they voted on.
There's a brief section in Barry Goldwater's book, The Conscience of a Conservative, where Sen. Goldwater tells of being the only vote against a change to labor law that was supposed to reduce the power of corrupt unions. He was concerned that a half-way fix was worse than no fix at all, since it would remove the pressure and it was unlikely they would revisit the issue at a later time and do it right.
When the bill went to the House, it was rejected and the Landrum-Griffin bill substituted in its place.
Whatever may be said about Goldwater going liberal in his later years, he was a man of great integrity, as is Dr. Paul.
(Google books, I found, provides the very pages I summarized above from Goldwater's book. Go here to read it, and the rest of the chapter on labor. Fascinating stuff, and still relevant today.)
Goldwater never went liberal. It his later years, as abortion became a more heated political issue, he made it clear he was pro-choice, which to him was consistent with the idea of favoring personal liberty. He presumably always held this view, but it wasn’t really part of the political discourse back when he ran for President. I’m not necessarily endorsing this view, but I don’t think it makes him liberal.