If you get a chance, you might want to google latex-banana allergies...it's eye-opening.
One of the things that you might want to talk to your doctor about. Allergies like these can become more serious as minor exposures take place, even if you don't notice the reaction much at the time, can occasionally lead to anaphylaxis.
Good ideas. I'll definitely look into it now that I've been alerted to the likelihood that this is what's going on.