In the 1950s it was.
Baseball was the biggest American sport for a hundred years between the Civil War and the Civil Rights Act, and maybe longer. Any claim that boxing was bigger than baseball the 1950s is simply incorrect.