The removal of the Confederate flag from Amherst County's official seal has upset Southern heritage groups, who contend residents weren't told of the change. County officials acknowledge the image was quietly removed in August 2004 to avoid an uproar.