You didn't miss anything, no such "group" exists. It is a derogatory reference to anyone that may either have a connection with pubic schools or doesn't always agree with the idea that home schooling is the only way children should be educated.
How is that "derogatory"?