Depends on what kind of water heater you had installed as well. A hybrid electric unit would be about $1,400 from the orange store. I could see that costing around $2400 to install once you calculate the plumbing re-routing, code/inspection BS.
But if he installed a 5 year 40 gal orange store special, those things are like $300, you got overcharged.
I did one myself, 50 gal electic, AO Smith 15 year warranty model, was about $700 from Lowes, after I bought copper tubing, replaced 3 valves that were also bad, solder, flux, and all the other goodies to do the job right, I was under $850. I bought a higher end unit because they generally last longer.
If your plumber installed a high end unit with a long warranty and he had to replace a lot of pipe or re-route a lot of pipe, I could see it costing that. (And he sweat the pipe, none of that shark bite BS) I could see it costing that much.
If you're paying a plumber then yeah, go with sweat fittings, but Shark Bite fittings work great in accessible areas for the do-it-yourself-er. I have Shark Bite fittings that are going on 40 years now with no problem.
“Depends on what kind of water heater you had installed as well.”
Absolutely. Oil fueled hot water heaters are also expensive.