Of course, but "firstborn" is indication of status, and does not imply that there was also second-born. You are, of course, correct that if one wants to read the scripture the way Full Court does, or the way Don Brown does, one can, if he disregards the Tradition of the Church.
Certainly not as much as "Who is My mother and who are My brothers?" (Matt 12:48)