"And what prior Father stated that [unleavened bread] was a major matter?"
Implicitly, I would say St. James the Brother of the Lord appears to have done so, thus the universal practice of the East, excepting Armenia. It would be a part of what the Orthodox call the oral Tradition.
As pointed out, there are no extant instructions from him on the subject. St. James certainly would have known that both leavened and unleavened bread are just that, bread, and so both are valid matter. Implicitly, therefore, his lack of instructions to use only leavened bread shows that it doesn't matter.