My wife is a member of NCAE here in NC, which in turn sends a certain amount to the NEA, IIRC. I doubt she would be able to recoup any money since she is not a member of the NEA directly, though they call her all the time and she gets their rag each month. I may be wrong about her membership, so if there are any NC teachers here, please feel free to correct me. Regardless, my wife's dues are automatically taken out of her paycheck. (sound familiar?) She doesn't make waves because the teaching community here is very cut throat. She goes out of her way NOT to get political because she does fear retribution. She wanted to refuse to "donate" $5.00 to the United Way, which is the brainchild of her local school's assistant principal, and was essentially told (by this person) that it "would NOT be a good idea" not to contribute.
I will tell you she is a member of NCAE ONLY because they will send lawyers to back up teachers regardless of the charges. In some cases, they are a teacher's only defense. My mother-in-law had a student's mother sue her once and the school & school district ignored it. NCAE lawyers exposed the mom for a fraud and got it tossed. My mother-in-law never had to pay out anything, to the mom, the lawyers no one. As much as I hate their stances on things, NCAE has come through. Now, as for the NEA, they are a worthless pile of cow dung. And that is putting it politely.
I believe you can retain the legal benefits because you will still by paying union dues for those benefits. You need to demand that they do an accounting of how much is used for political purposes. You have a right to get that back.