"Privilege": "Middle English: via Old French from Latin privilegium bill or law affecting an individual, from privus private + lex, leg- law."
Privilege means being subject to a different set of rules than regular people; being able to do things that regular people would get in trouble for; being given special advantages under law.
In the US, in 2018, white people are not privileged. If anything, it is the people given special status under law, via "affirmative action" and "minority set-asides" who are privileged.
You make a solid point. Privilege, from an objective viewpoint, in a country of laws, only deals with how the law is applied to the individual. The rest is just subjective excuse making for why people have stuff you don’t have.
And yes, regarding “Privelege”, from a legal standpoint, there is no white privilege, but there most definitely is (or at least was) black privilege with things like Affirmative action.