Your minister sounds like a real snake-oil salesman, who is in the business for self-glorification and satisfaction of his own perversion. Get away from this person ASAP and have no more contact.
As to your own sexuality, you are clearly conflicted. We all are subject to various types of lust - so as much as you have been manipulated by others (your neighbor, your minister) you also have an equal part to play in the sin. Focus on overcoming those desires in yourself.
Thanks, PGR88, good comments.