One of the assumptions are the laws of physics are the same across time and space. Although there are people working on what is called Modified Newtonian Dynamics that claim the gravitational constant itself can change.
I think this is exactly what I was wondering. If the gravitational constant were not so constant, a great many things could be better explained (Black Holes, Big Bang, Neutron Star masses, etc.)
But it really screws a lot of thing up too. I have also wondered what makes us so sure the speed of light is constant throughout time and space?