In every sphere - many things cost what they cost. The government cannot "save money" on many of those things, but it can appear to do so by shifting part of the cost somewhere else.
If too many criminals are let back onto the street too soon, I imagine all the "savings" in direct tax supported expenditures for incarcerating criminals, and more, will be shifted to the cost of property and lives lost, higher local police administration costs and higher insurance premiums.
Either way we must PAY for fighting crime and the best approach is usually the most direct approach - do the maximum job we can up front.
There is only one area where "half measures" have proven useful, and that is in "juvenile justice" where active measures are sometimes taken to turn the first-time juvenile offender away from becoming a career criminal. They don't always work with every juvenile either, but they have a higher rate of success than they do with repeat adult offenders.
Nice comment - I agree - money to keep a kid from becoming a career criminal is money well spent - - even if the odds aren't that great...