No, smaller pieces come down slower because they are more affected by the drag of the atmosphere all the way down. Of course, it depends on the shape of each piece. For the same weight, a sphere falls fast: a thin piece flutters down slowly.
But the orbit will decay more quickly the lighter the piece.
I think the meaning was which will “de-orbit” faster, not fall faster.