Great, that eliminates the possibility of the lender requiring you to pay for mortgage insurance. There is no need to re-finance in a rush.
Do you have any high interest rate credit card or other debt? You sound disciplined enough to borrow only what is required to pay the amount owned on the older loan. However, you can borrow a bit more, don’t get carried away, and pay off other debts but you will be paying interest on the amount for the duration of the new loan. You can refinance for 30 years and still pay it off at a faster rate. Make sure any new loan is assumable should you find the need to sell.
“Do you have any high interest rate credit card or other debt?”
Nope. Credit card debt is pure 100% poison.
I owe on a used car. My wife bought a new car about 8 months ago. That’s it. I am not wealthy, but I try to do the most with what I have. That’s why I’m looking at a refinance.