Here in New York State we don’t have the right to vote in primaries, unless we’re enrolled as a Democrat or Republican. I changed my registration once to Independent, only to find out that I wouldn’t be able to vote in any of the primaries...and quickly changed it back to Republican.
One year my parents pulled a Democrat ballot and forgot to change their declaration back to unenrolled. Messed them up for a couple of years.
Any of the parties can hold a primary, but since it can be expensive, only the largest parties do it. So de facto, you are right. It makes no sense here to register as unaffiliated.
It’s the Independence Party you might have enrolled in.