That isn't his position. It's not about eliminating copyrights. It's about bringing it back to the limited granted monopoly concept of the Constitution instead of the distorted semi-permanent rights system we have now that was bought by corporate lobbyists and puts the Constitution subordinate to international treaties.
Jefferson was so worried about the situation we have today developing that he didn't want any copyright or patent to exist at all. Government-granted monopolies (that is what copyrights and patents are) and their extreme penchant for abuse scared the hell out of him, and rightfully so. Madison was a bit naive and replied to Jefferson that the will of the people would keep abuses under control. We can see how well that worked out.
So, if you write a book, it’s OK for me to just run copies and sell them myself? That’s pretty cool. There’s a lot of good books I can make a lot of money on.