Most Lutheran churches hold that it was Joseph's children from a previous marriage. The reason given for John being tasked by Jesus to take care of Mary was that His other brothers didn't follow Him at that time.
Then most Lutheran churches would be wrong. There are no previous marriages mentioned in the Bible. The Bible plainly says that Jesus had brothers. Read the Bible the natural way.