I think everybody has their own potato salad recipe. Mine has your ingredients, plus eggs, sweet relish, a little apple cider vinegar, a tiny but of sugar, (both of those to counteract the blandness of the mayo) and CELERY SEED (for a huge bowl, I use about half a tablespoon). The celery seed is vitally important. It gives it a subtle complexity. Here in Oklahoma, I think they use more mustard than mayo. It’s bright yellow, and practically inedible. My husband likes mustard, so I put in more than I would for just myself, but not that much. Never tried cayenne. Are you related to Chef John? ;)
Yes, I left out the vinegar.
It drives me crazy that potato salad that we buy in the store here seems to have No Mustard! It’s just this pallid, white mess, with undercooked taters...