It seems pretty straightforward to me, there's nothing that says that the most basic physical rules were broken in anyway, right? So therefore, the conservation of energy holds. So, the matter went...nowhere...it was always there...in the form of energy? Matter? Chances are, though, it was extremely compact. Until we pierce the infrared background, we can only make models using the IR background as a boundary condition.
In short, it did NOT come from nowhere. Nor is it a miracle. I propose a corollary to Clarke's theorem, "Any sufficiently advanced technology is indistinguishable from magic", which is: "Any physical phenomenon that we do not understand is NOT a miracle."
Doesn't that fall into "a miracle happened" because your idea violates the known laws of physics?
I think that actually means today's miracles are tomorrow's technology (I don't agree with your spin)