When I read that passage it suggests to me that Abraham's father Terah served other gods, but I don't think you can draw the necessary inference that Abraham himself served other Gods. The passages in Genesis that deal specifically with Abraham do not make the inference and this is merely the blanket statement that the fathers of the Children of Israel served other Gods, but unless Abraham is specifically numbered in the passage, then a statement of fact that "Abraham was an idolater" is reading something into the bible which isn't there. He very well may have been at some point in his life an idolater, but clearly we can't state it as a fact. At least not from that passage.
Are there any other passages which make the inference?
I may be wrong but this was the first time I ever saw that assertion and it kinda took me by surprise.
Thanks
Marlowe