I think the fact that Margaret became a degenerate after the Queen didn't allow her to marry a divorced guy (or whatever was the reason) was why the Queen hesitated to tell Harry "no." That, and there would have been world-wide tabloids screaming racism.
It was long before that. A few weeks after Elizabeth secured her title she got a law passed to make sure her younger sister could never take her place as queen. This royal family is not for sissies.