That depends on your notion of "distance." Take the set of natural numbers (or real numbers, if you prefer) and append a "positive infinity" point to it. Extend the Euclidean metric on your original set by allowing "infinity" to be a possible distance. It's no longer a proper metric, but something has to give when you work with infinity. Now we obtain the following: the distance from any natural (real) number to positive infinity is...drumroll, please!...INFINITY! (thankyouverymuch).
And thus infinity is the same (infinite) distance from every natural (real) number. No number is any closer to infinity than another. Interesting theological implications, perhaps...
Of course, you may redefine your metric so as to realize your desire to make 1,000,000 be closer to infinity than 1,000. However, you then lose the truth that the distance from 1 to 2 is the same as the distance from 101 to 102. Is it worth it?
Granted, from 1,000's point of view, 1,000,000 would appear to be closer to infinity, given that 1,000,000 is between 1,000 and infinity. But how much closer? You can't nail down a translation-invariant notion of distance that will answer this. For this reason I would respectfully suggest that "closer" is an ill-advised choice of word.
Don't you guys still believe in mathematics? If not, what do you use these days?
And yes, I do use mathematics, but my formal schooling only extends to college algebra; everything else I pick up pretty much from conversations like this one {;^)>