Are you saying that implicitly?
Under the circumstances as discribed the man had no legal right to refuse the officers entrance.
First off he was acting badly and followed his wife in to the house. If the officers felt she was in danger, which his later behaviour indicates, they could enter in order to protect her. She also implied consent when she told her husband to let them in.
All the officers need to say is they felt she was in danger.
Now if they come to your house and just to come in without a warrent that is of course different.