File a criminal complaint of breaking and entry, trespassing, theft, etc. Break down the door, move in and force him to get you out.
As I said up above ...
Not that I agree with what has been done here - but in regards to what you say - someone who was smart and slick about it (in taking over the house) would always have two or three people around as witnesses, be armed himself ... and ... if any altercation happened, call the police immediately, who would then arrive, look at the legal paperwork, and say, “It’s a civil matter and you’ll have to fight it out in court.”
If the previous homeowner tried to “do anything physical” about it, without going to court - then he would be subject to being arrested.