Except that hair/fur would protect against the elements better, like frostbite, and sunburn, and skin cancer. With the amount of time and energy people have expended over the years to keep warm, you'd think that hair would be an advantage. More energy could be channeled into feeding and raising young.
If hair kept them warm, they wouldn't need clothing and so would not likely think it up and start to wear it. The need would have only arisen AFTER the loss of hair and the need for other protection.
My guess is that one layer of fur (our own) wan't enough to fully protect against severe weather. Once animal fur was discovered, probably immediately after eating said animal, it was learned that thicker clothing works better. So, I could see a desire to wear furs even if you had your own. More is better when its freezing out. Also, in hot climates/seasons it's better to have the option to remove the warm clothing.