Ideally it would be, but it will pick up dust and other impurities from the coils it is condensing on. And since dehumidifier water isn't meant to be drunk the manufacturer doesn't have to follow the same rules it would for food-safe usage so can do thing like use lead solder which it couldn't on water pipes.
I wonder if I could use it in my iron? I buy distilled water so I don’t get gunk built up in there.