Re a: There doesn't have to be a place containing prime numbers. To believe so would be taking an extreme Platonist position. Neither does one need infinite space to discuss infinite things. One only makes finitely derived statements about infinite things. There is no "infinite list of primes" only a proof that given any prime, one can produce a larger prime. These methods are discussed in books about "foundations of mathematics" and similar topics.
Re b: We reason about mathematics with more certainty because the whole subject is man-made.
Re c: There's a book called something like "The Unreasonable Effeciveness of Mathematics." We do invent things in math to describe the real world. It's often surprising that math works so well. We make a mathematical system do describe one thing (electrons, for example) then extrapolate the math to other things (positrons) and ofter, the other things exist physically. Pythagoras suggested "everything is number."