Context, man. Context.
The point of contention is whether the laws were appropriately overturned. The Justices in question participated in activities which contradict those laws prior to hearing the appeal on those laws.
That’s pure grounds for recusal.
Well, you might have a point if they had performed the weddings in states where bans had been overturned by the federal courts. In this case, however, the marriages took place in MD and DC, both of which have chosen to allow gay marriage.
I don't think anyone is suggesting that the federal courts would ban gay marriages in the states that want to allow them, so I don't see the controversy in presiding over a wedding in one of those states.