Neither are mentioned in the Constitution, so, they should BOTH be STATE Issues, having NOTHING to do with the FEDERAL GOVERNMENT.
Actually, until the late thirties, your view was the dominant view of the rights of the states vs. the feds. Then eventually FDR got enough justices on the court to essentially decide that the commerce clause empowered the federal government to do anything it damn well pleases. It has become the classic case of an exception swallowing the rule.