Lemme try again.
As I understand your argument, we can’t call something heretical because Rome and Orthodoxy disagree on a few points.
Well, aside from those points, if all the ancient Churches agree that something is heretical, isn’t that a good indication that it is?
For example, you and I both agree that denial of the Trinity is heretical, right?
If schism is troubling to you, and fuller communion with various Orthodox be desired, then drop 'Papacy' as that had become over-inflated into being.
It is the Church of Rome who is the most egregious schismatic of them all. Nobody should surrender themselves to an error.
Here's a useful wiki page, having a range of information in one place;