Don't replace with federal agents at all. There are aspects of law enforcement which should be retained on a federal level, such as the crime lab, communications, but all the actual field work should be carried out by the LEOs of the various states. It would be a sort of domestic Interpol, but with no arrest powers, existing for the sole purpose of coordinating LEOs, some of which could be designated with investigation and arrest powers across state lines. A state issued special agent badge recognized across the nation, but ultimately accountable to the state issuing it.
The head of the federal coordinating office could be a sheriff or state police commander designated by the state legislature to serve as one of a pool of 50 from which to choose in a random selection every 3 years to serve in that position for same.
After thinking about it, I agree with you.