It depends on the official position of the person with diplomtic immunity.
An ambassador, for example, has full, complete and unlimited immunity from prosecution or being called as a witness, but an ambassador can be expelled.
Wikipedia has a nice chart showing the limits of diplomatic immunity:
https://en.m.wikipedia.org/wiki/Diplomatic_immunity#Diplomatic_immunity_in_the_United_States
A couple of examples:
In July 2013, Joshua Walde, an American diplomat in Nairobi, Kenya, crashed into a mini-bus, killing one man and seriously injuring eight others, who were left with no financial assistance to pay for hospital bills. United States embassy officials took the diplomat and his family out of Kenya the following day. The United States government was concerned about the impact the accident could have on bilateral relations with Kenya.Walde gave a statement to police, but was not detained due to his diplomatic immunity. Kenyan police say the case remains under investigation.
On 24 April 2008, in New Orleans, Mexican press attaché Rafael Quintero Curiel was seen stealing BlackBerry PDA units from a White House press meeting room. Quintero made it all the way to the airport before members of the United States Secret Service caught up with him. He initially denied taking the devices, but after being confronted with security video, Quintero claimed it was purely accidental, gave the devices back, claimed diplomatic immunity and left New Orleans with the Mexican delegation. He was eventually fired for the incident.
RE: but an ambassador can be expelled.
Yet he is not subject to the jurisdiction of the United States. And his children born here are not either.
So, why should an illegal alien who can also be expelled ( AKA deported ) have his children considered subject to our jurisdiction? Shouldn’t they be considered subject to the jurisdiction of the country their parents come from?