If "the wafer thing" is how you think of it, then it would be better if you didn't participate. Practicing Catholicism isn't like signing up with Carnival Cruises for a five-day getaway to St. Bart's.
I don't remember the priest saying that you had to be Catholic to do it. How does this work in practice - does the priest ask people who is Catholic before certain services begin?
No. You're catechized and baptized into the faith if you're an adult, and you are further catechized, confessed, shriven, and given a penance of prayers to perform. Once this is accomplished, you may then share in Holy Communion. Baptism, confession, and Holy Communion are the three principal sacraments of the Roman Catholic Church, and they don't play around with them.