Where is that in the passage? It speaks of "every man", not the pastors; it speaks of burning off stubble, not of "lesser rewards". Read what is written.
First off, I do believe I said Pastors and leaders in the church. The entire chapter is talking about those who “teach”. Refer back to the “milk” and “meat” statements. The entire chapter is talking about those who are strong (meat fed) and weaker (milk fed) Christians. Those who are weak will not hold fast through the fire of trials and will be “burned up”
Here again he calls himself and the others who are “teachers” labourers. Those who are building the church. That would be those who are either teaching the “milk” or the “meat”.
1 Corinthians 3:9 For we are labourers together with God: ye are God's husbandry, ye are God's building.
Not once in that entire chapter does he talk about anyone other than those who are “labourers” or teachers in the church. He doesn’t switch when he says “every man”. He’s still talking about those leaders and teachers within the church. Even if you were to stretch that into somehow meaning “all men” it would still refer to “any man” who teaches either “milk” or “meat” of scripture. It would still be talking about whether those who were taught were strong in the faith and when “tried by the fire of temptation” remained strong or those who were taught only the “milk” of scripture and were weak and were lost through those temptations.