Check and see if they are in black every day.
I haven’t seen White House security, but at Presidential events, they normally dress in black. It may be to identify them differently than military who are often pulled in for some security