God's Word was Luther's authority.
wikipedia
Martin Luther believed that Mary did not have other children, and did not have any marital relations with Joseph,[25] maintaining, that the brothers mentioned were cousins.[26] This is consistent with his lifelong acceptance of the idea of the perpetual virginity of Mary.