Well, Harry isn’t actually Charles’ son anyway, so no big loss. Maybe Charles should just demand a DNA test and rip off the band-aid once and for all.
Apparently Charles said something along the line of ‘maybe I’m not really Charles, etc’ when Harry confronted his dad about that. Charles was trying to joke/blow it off. Harry complained about that too.