Did these people actually contend in Court that the issue is "trivial"?!
I don't think so. I think you're missing the larger context of this decision (which, it should be noted, Van Irion's side lost). As I read it--and I'm not a lawyer--Irion's side first filed the challenge as a state issue. The defendants had the case moved to federal court. This was an attempt by the plaintiffs--Irion's side--to have it moved back to state court. The attempt was denied.
The judge says there are three reasons a state case might get moved to federal court: "(1) the plaintiffs cause of action is created by federal law; (2) the well-pleaded state-law claim has as a necessary element a substantial, disputed question of federal law; or (3) the claim pleaded is in fact one of federal law." He then goes through these criteria to show this case meets them. Basically, he concludes that of course it has a question of federal law as a necessary element, because the definition of natural-born citizen is a matter of federal law; and of course it's a disputed issue, because a lot of people are disputing it.
The conclusion is just that therefore the case stays in federal court. Reading any more into it is wishful thinking.