I, too, was put off by the local school for gifted children. Arrogance is not a virtue. I prefer this beautiful goofy (sometimes intense) little boy who comes in so filthy from playing in the garden that all you can see is the whites of his eyes!
I think you're a very wise lady. I was never genius level, but I was smarter than most kids my age, and definitely a few years more advanced than they were. My parents, by homeschooling, helped keep me from getting either lazy or arrogant - I know I have tendencies toward each, but I have to ask myself what I would be like if I had attended a school where almost all the other kids were slower, and the material didn't challenge me at all. I was ready for college at 16, and fortunately had the emotional maturity to handle it. And better yet, I had a childhood. Sounds like your son will be the same way (but then I already know you're a great mom!)