Taking something with a grain of salt refers to an ancient belief that salt neutralized many poisons.
To say you need more than a grain of salt implies that there is a lot of poison in something.
Well, that's a clever extrapolation but:
I'll take it with a grain of salt.