Jesus admonished us that when we give, we are not supposed to let anyone know. No one should know if you tithe or not. That’s none of anyone’s business.
Maybe you should tell your pastor to mind his own business and to stop gossipping about the giving habits of people other than himself.
Tithing was not one of the continued portions of the NT in Acts 15.
Jesus said in Luke 11:42 that we should be most concerned with practicing justice and the love of God, but that we also should not forget about tithing, and that is not a bad approach at all. However, there is no solid NT reiteration of tithing for Christians.
Truthfully, though, there really is nothing in the Old Testament of what we consider tithing in our era. One tenth of income really is not reflective of the intent, practice, and purpose of OT tithing.