According to the Constitution, there are only three federal crimes: treason, piracy and counterfeiting. Everything else is outside the Constitution as written.
That’s not entirely accurate. Article I, Section 8 states Congress has the power to (1) provide for the punishment of counterfeiting the securities and current coin of the United States; and (2) define and punish piracies and felonies committed on the high seas, and offenses against the law of nations. Article III, Section 3 delegates the power to declare the punishment of treason. But no where does it state Congress cannot define other crimes and their punishments. Also, Congress had the power, but were not required, to provide those punishments. If they took no action, there would be no punishment for those crimes. They could have made them punishable with fines only or short prison sentences.
Under your interpretation, defrauding the federal government, destroying federal property, committing perjury in federal court, or committing murder in a federal jurisdiction cannot be crimes.