Honestly, your situation is what the issue is.
Your second marriage is (forgive me for assuming) stable and you have five children. For you to leave that second marriage for your first wife would be (again forgive me for assuming) rather horrific at this point. Not just for you and your wife, but the children involved.
So how does a church and/or pastor handle that? In the Catholic church they typically do the annulment thing, in most non Catholic churches they ignore it.
Her husband would most assuredly be a bit put out...