Some of the antiship missiles and some of the drones can fly at about the same speeds. There are loitering antiship missiles these days, just like there are loitering drones, and they can be roughly the same size and radar return. It’s not just a matter of “the faster ones are the lethal missiles” - additionally, some of the drones are large enough to be loaded with anti-armor warheads and used as suicide drones, turning them into small antiship missiles as well.
Keep in mind that some of the Houthi drones are not “child’s toy” but a significant fraction of the size of a real aircraft, or the size of a conventional missile, so they can carry a decent payload or warhead. It’s how they’re sinking merchant ships.
What is the difference between a loitering missile and a loitering drone? Is it that the latter has real-time control from a base?