No, “ginger-haired” or ginger-headed has been around for many generations. Such as Malcom X would have been called “that ginger-headed fellow...”
But calling someone “a ginger”?