The show is also fixed, a lot of time record companies dump artists they signed who didn’t work out on the show in the hope of re-launching their careers.
Remember also that Cowell and the others are not at the original auditions where thousands and thousands attend. They show up later and listen to maybe a few dozen.
Their people purposely find BAD singers and good ones to send to them.