I think you are getting close to understanding what they are trying to get at, but rejecting the point by referencing the weakness of standard Quantum Mechanics to deal with this question as the reason this solution can't work when the contrary is exactly the point.
This Invariant Set Postulate concept is that QM is ignorant of the existence of the "reality" that is the Invariant Set and thus predicts possibilities that are within this Set and those which are outside of this Set, but only those which are within the set are manifest in "reality"
To go to your basic point about smooth functions and a smooth predictable volume within the Universe, this idea sees this volume as a much more complex space wherein many functions may have described the Volume but only a select few indeed do. The insight is that the Quantum Mechanical states possible are restricted to a specific set of states that are only a small group of the states that would otherwise be possible and this set of real possibilities is what we term as physical reality.
What I see missing from this discussion is the fact that the "Invariant Set" must by force be collapsing and thus I am unsure about the name Invariant which would imply that all events throughout all time are already pre-ordained and though potentially not predictable are yet inevitable.
My best guess is that this is not the meaning Palmer has meant to project by using the term Invariant, but that is the limit of this description.
This statement is wildly false. what you are describing is in fact what QM already DOES DO.
To go to your basic point about smooth functions and a smooth predictable volume within the Universe, this idea sees this volume as a much more complex space wherein many functions may have described the Volume but only a select few indeed do.
You missed my point completely. The analogy deals with mathematical simile and has nothing to do with this putative theory, AT ALL.
The insight is that the Quantum Mechanical states possible are restricted to a specific set of states that are only a small group of the states that would otherwise be possible and this set of real possibilities is what we term as physical reality.
The part of what you say here that is actually correct is not an insight at all. Dirac commented on this informally in the 1930's and it was made part of a serious theory by Feynman. The other part of your statement is, once again, wildly false. If you have a background in physics, you have forgotten what you learned in grad school. If you don't, please Google "Feynman Path Integral." It is a key observation of the theoretical basis of QM that even physically disallowed paths through phase space must contribute to the ultimate trajectory of a state through phase space. This principle is thoroughly established in so many theoretical and experimental results that a theory that does not incorporate this concept CANNOT be correct -- and incidentally, this article does not claim that the "new theory" disallows "unreal" Lagrangian functionals to contribute to a trajectory; it simply says they are not manifest in the Invariant Set... Well, again, SO WHAT? QM already says EXACTLY the same thing: that is the point of the Feynman Path Integral.