Yes! That's it exactly.
For those of you who reading don't follow what VadeRetro just said, let me try to explain.
Let's suppose you're sitting on the surface of the Earth, and disagreeable aliens suddenly enclose the Earth in a very big, very massive, spherical shell. What would you feel? In any direction away from you, you would be pulled by some part of the shell. Which part would win? In what direction would you be pulled?
Let's draw a cone of some angle, in some direction, with its vertex (the pointy part) right at your center of mass, and consider the part of the sphere that falls within the cone. It lies a certain distance from you, it has a certain size, and its mass is proportional to that size. Its pull is counterbalanced by some other piece of the sphere, lying in the opposite direction, which has some other size, mass and distance. The size (mass) of each piece increases as the square of its distance from you, but the gravitational pull decreases as the square of the distance. These factors cancel, so the pulls from these two pieces cancel exactly.
This same trick works in every direction, and at every point within the sphere, so you--and everything else within the sphere--feel no net gravitational pull from it. All you feel is the pull of the Earth, just as before.
Hmmm... Then if I drill down 100 miles, then as far as the net gravitational pull on me goes, it would be like I was standing on a planet whose diameter was 200 miles smaller than Earth's? Because all the mass located farther out from the center than myself would be like that massive sphere in your example?