Matthew 8:14 — And when Jesus was come into Peter’s house, he saw his wife’s mother laid, and sick of a fever.
You don’t have to leave your wife and family to follow Jesus.
No you don’t, but his apostles did. He didn’t say “come with me, and bring your family”. He said just the opposite. He told them to leave everything and follow him.