It is used nine times in the NT...six in reference to Jesus.
John 1:14; 1:18; 3:16; 3:18; Hebrews 11:17; 1 John 4:9.
Contrast with how Luke describes Jesus.
πρωτότοκος: 4416 prōtótokos (from 4413 /prṓtos, "first, pre-eminent" and 5088 /tíktō, "bring forth") – properly, first in time (Mt 1:25; Lk 2:7); hence, pre-eminent (Col 1:15; Rev 1:5).
αὐτῆς τὸν πρωτότοκον καὶ ἐσπαργάνωσεν ...and she gave birth to her firstborn son...
The word prototokos allows for other children.
The Catholic, I am sure, will dismiss this. They seem to want the text to say Joseph and Mary had sex on Thursday and nine months later, James was born and so on and so on.
That verbiage is not necessary based on the abundance of texts we have all establishing the brothers and sisters being in Him family...with Joseph and Mary being the biological dad and mom of His brothers and sisters.
1) You are ignoring Hebrew idiomatic usage which pervades the Greek NT
2) Nowhere in Scripture is anyone called a Son of Mary except Jesus Christ.