"Fiscally conservative" means that you don't run a budget deficit if you can help it, and you don't spend money on unneccessary services.
New York, by its nature, needs a lot of services: police, fire, schools, garbage, sewerage, etc. Let any one of them break down through neglect or lack of funding and you get, well, New York circa 1980, which was a city mired in crime and spinning out of control.
You have to spend money, a LOT of money, on basic infrastructure and government services for a place like New York City to be able to exist at all. The underfunded, underserviced version of New York would be someplace like Detroit, and the comparison is like heaven and hell.
So yeah, Bloomberg presides over an immense budget and an immense payroll. He's a fiscal conservative because he doesn't waste money on crap, but having identified what is NEEDED to keep New York City the Greatest City in the World, he tallies up the costs and insists that realistic taxation be applied which covers those costs, so that the city is not drowning in red ink and living on borrowed time.
That's fiscal conservatism.
I would suggest that your definition is way too narrow. What you describe might be more appropriately defined as "fiscally responsible," the antonym being "fiscally reckless". By your definition, a "fiscal liberal" would be someone who promotes deficit spending. Frankly, I consider anyone in that category as simply unqualified, not liberal nor conservative.