At the hammer and feather scale it is so minute to be almost ridiculous, but think of it this way. If you could stop the Earth and Moon from rotating and place a feather the same distance from Earth as the Moon, I guarantee that we will collide with the Moon first.
Let's try math, instead of handwaving. We wish to determine the acceleration due to gravity acting upon an object as a function of its proximity to some other object. The two objects are (let us say) a hammer, and the moon. The each have mass Mh and Mm. According to Newton, a force acting on a mass M causes acceleration.
F=m*a
The gravitational force between two massive objects can be computed
Fg = G*(M1*M2)/ R2
where "R" is the distance between them. So, the gravitational acceleration of a hammer falling on the moon may be calculated:
Fg = G*(Mm*Mh)/ R2 = Mh*a
Note that Mh cancels out of this equation. Acceleration of an object (a hammer) due to the gravitational attraction of another object (the moon) is not a function of the first object's (the hammer) mass.