Found it!
Or All the Seas with Oysters (short story 1958, novel 1962) by Avram Davidson
I read the short story, didn't even know there was a novel, will have to look it up...
I know Dick did write a short story where aliens of some type disguised themselves as common items like towels, etc. But they were hostile, and some guy gets strangled by them. The people end up leaving the planet, and can take nothing with them, so they run to the rocket ship totally nude.
At least they think it's their rocket ship...