If it is left up to the states, then it is up to the states to honor other states' marriages or not. The problem is the Constitution,. for good reasons, requires all states to honor each other's contracts. So it is not up to the individual states.
If first cousins couldn't marry in your state, that didn't invalidate a marriage of first cousins in another state. The problem with the whole states rights thing is that we're not in any way shape of fashion the same nation we were in the late 18th century. We're a very fluid population. In the founders time, most people lived and died within 50 miles of their birth. Today it's not unusual to live in 3 or 4 states in your lifetime. The problem with your argument there, is that we've never let that bother us before.