In many (all?) sports, you can’t go to the Games if you don’t qualify beforehand.
That, plus the US qualifies in many team sports while China does not (as one example). Basketball has 24 athletes who only are eligible to win two medals (men’s and women’s team). The men’s and women’s teams also qualified in water polo (28 athletes for two more medals) and rugby (not sure how many athletes there but at least 14 for two medals). That is at least 66 athletes who combined can only win 6 medals. That will necessarily drag down the medals per athlete measure