I had a good link on this (but I can't seem to find it now...) - apparently this problem was only thoroughly & quantitatively solved within the past couple years - you're totally right - the problem lies in the fact that the doorway itself has to have a particle selection mechanism that itself must obey physical laws.
Gibbs' Paradox is another strange thermo. problem - it involves the amount of entropy in a system being dependent on the way particles are labelled; i.e. unless the quantum mechanical principle of truly identical particles is introduced, entropy is a subjective quantity. (I always enjoyed reading about this one)
If the demon is truly magic and runs by magic, then entropy can be reversed. But the demon can't just be a futuristic nanomachine or anything running by physical principles. Otherwise, the entropy increase happens inside the demon, who basically melts down with his frantic activity.