For example, the government was allowed to claim in Wickard v. Filburn that the growing of private grain materially affected interstate commerce, without any chance being given to challenge that assertion before a jury.
Other issues that need to be brought before juries include the reasonableness of searches, existence of probable cause, the appropriateness of punishments for particular criminal acts, etc. While it's good to have judges act as a first-line barrier against government abuse of such issues, a jury should act as a backstop. A judge generally won't throw out the results of a search unless it was patently unreasonable; a jury can and should apply a somewhat looser standard and disregard evidence if they find that the search was not reasonable (even if it's not so patently unreasonable that no honest person could find it reasonable).
To be sure, today's jurors often aren't the best and brightest, but it's pretty clear that the reason prosecutors and judges don't allow such matters to be brought before them is that they know they'd lose a lot of cases if jurors knew the truth.
Filburn wasn't a defendant.