I agree, but I also agree with Scalia that the intent of the drafters is not what determines the meaning of a law or constitutional provision. The drafters, as such, have no constitutional authority or status at all. If they have such an intent, then it is there responsibility to put that intent into writing so that the people who are tasked with enacting it into law can see the meaning on its face.
The folks who turn a piece of paper into a law/constitutional amendment are the members who pass it, and even more importantly, the states that ratify it. So, you go by the ordinary meaning of the words as they would have been understood at the time by the people who voted for/ratified it.
The ordinary meaning of the words used is not limited just to former slaves, as it makes no mention of them. The best argument I've heard about "subject to the jurisdiction thereof" is that it excused Indians living on tribal reservations not fully subject to U.S. law. Perhaps you could stretch that to folks who are here illegally as well. But if the 14th was intended only to apply to freed slaves, why didn't they simply say that expressly in the Amendment itself rather than drafting it so broadly?
If Scalia believes that, then my opinion of his legal mind has dropped considerably. Words change meanings over time and it is imperative that in interpreting a statute of a provision of the Constitution that the reviewing court must determine as best as possible the intent of the drafters utilizing the contemporaneous discussions surrounding the drafting and the passage of the provisions and further they must use the definitions of words contemporaneous with the general definitions of the words at the time the provision was passed.
Words must be given their original intent and any attempt to give a modern definition to an archaic word or expression would result in a misinterpretation of the statute or provision in question.
Do you have any specific reference where Scalia made such an idiotic remark?