Where did the 'virginal husband' come from? There isn't any reference to whether Joseph was a virginal husband, is there?
I think there is no good reason to make up things that we really do not know about.