I am not sure why you refer to Banach-Tarski. These things, as far as I understand stem from the Axiom of Choice and should not be THAT surprising: once you accept it, you are stuck, for instance, with non-measurable sets.
You are correct sir, e.g., tensor analysis for general relativity, and functional analysis for quantum mechanics.
I'm not sure about any point with B-T. It was an example, (in your favor) of a well-established mathematical fact which you would never expect to describe a physical phenomenon, but, in fact may. And, yes, the sets involved are not Lebesgue-measureable, but if (and that's a big if), there is a physical process that is modeled this way, well that's pretty funky if you ask me...