Here is a copy of the appropriate part of the Lemuel Penn ruling that pertains to this matter. i've underlined the appropriate comments.
In addition, the Court observed that the statute’s wording does not require that the conspiracy be racially motivated. The Court further held that, since conspiracies can have multiple goals, it is not necessary to prove that the sole goal of interfering with a particular civil right is the main goal of the conspiracy. With regard to the federal case relating to Lemuel Penn, the Supreme Court found that the trial court erred in dismissing the part of the indictment that alleged an interference with Penn’s right to interstate travel and use the instrumentalities of interstate commerce such as highways. Subsequent federal cases found rights covered by the statute are the rights not to be killed without due process, to testify at a federal trial, to vote, and to exercise housing rights.50
States, especially a state with a large population such as New York have much more to loose than would New Hampshire in this particular case. The number of New York residents going to New Hampshire for what ever reason is far larger than the number of New Hampshire residents going to New York...remember, reciprical recognition.
So the states cannot refuse out-of-state drivers licenses, but the local 7-11 can refuse to serve you alcohol if you're out of state.