My Lutheran pastor told us during confirmation classes when I was a kid that the reason he would not accept invitations to meet with various so called religious organizations was that they were political and subversive. He said the WCC and NCC were not what they sounded like but were communist fronts. The reason it came up was that some congregation members wanted to know why he wasn’t “getting along” with other churches in the community.
“Getting along” is somewhere on the “fellow traveler” continuum.
Your Pastor was VERY wise!