You miss several points which contradict your statement. First of all, college athletes are free to choose what college they wish to play for, and people do pay to see them compete against each other. You may argue that college athletics are not "a business," but many would disagree.
So lets move onto professional athletes. There is now a "free agency" system in most professional sports, which a "draft" precludes for entering players, no matter what the level of their skills. So many professional athletes are indeed free to choose their employer and place of employment and people indeed pay to watch them to compete against one another.
Historically, professional athletes DID have the right to choose their employer and place of employment, even on entry. No sports "draft" was existent until c. 1935 when the NFL invented one - primarily to save on incoming players' salaries. Similarly, major league baseball didn't have a "draft" for previously amateur incoming players until 1965. So ALL incoming professional baseball players had the right to choose their employer and (to a certain extent) their place of employment for close to a century before then. Even today, such incoming baseball players from outside the US, Canada, and Puerto Rico still have that right. And professional hockey didn't have a "draft" until the 1970s(?), so hockey players had such rights before then.
Dallas trades and picks up CB Mike Jenkins.