It is when you do so because you don't like the clear and obvious meaning of the original passage. There is no reason to go beyond Matthew 16 to understand Matthew 16 unless you are trying to make it say something other than what it says.
I mean really...the rhetorical device is so obvious it survives translation into another language!
If anything, James would seem to have been ‘highest’ among the Apostles in scripture. Peter wouldn’t fear the followers of James if he was above James in rank.
James was certainly the local Bishop, and as such chose the local implementation, but note the controversy was settled after Peter stood up and spoke, not after James ruled.
Discuss the issues all you want, but do not make it personal.
And note that Peter was afraid of the delegation from James, and that Paul called James (first in the list), Peter and John ‘so-called pillars’. And that his statement is found in writings the Catholic Church agrees is inspired by God.
Thus, I reject the ‘establishing the Papacy’ interpretation for the ones I mentioned previously.
The Catholic interpretation cannot be shown false, but it seems highly unlikely, given the statements and history found elsewhere in the Bible.