If it is legal, what would be the remedy since he's already been sworn in? Would it essentially be a blown call with no booth review allowed?
No expertise, per se, but all you need to do is read the Constitution, Article I, Section 5 - Membership, Rules, Journals, Adjournment:
Each House shall be the Judge of the Elections, Returns and Qualifications of its own Members ...
Regardless how people think concession *doesn't count*, the House seated the projected winner of the race, if it becomes contested, the House will do a re-count. Sam Gjedenson won re-election up in Connecticut by TWO VOTES some years ago, there is also precedent of the Democrats cheating, leaving votes in the box and seating their candidate over the obvious GOP winner, see Panetta, Gingrich, and what Michel did to anger his conference to allow Gingrich to move from back-bencher to Minority Whip.