I have watched every episode as well. What is amazing is the breadth of his knowledge. I'm pretty good at Jeopardy, and I can often keep up with many of the contestants. James, however, seems to know something about everything and leaves me in the dust. The fact that he took time off to study for a long time does explain a lot. He mentioned once on the show that he read a lot of books in the childrens' section of the library. That way, he probably knows at least the one thing you need to know about something. He either has an eidetic memory or he studied those books like flash cards until he memorized them.
I've seen at least a few shows in which the other contestants just stand there. If the question is something simple like "he was known as Tom Sawyer's friend", I know they know the answer. In some cases, they don't even seem to try to push their buttons. It's like they are dazed by James and forget to play.
I think he is just plain smart.
Like you, I think I know about as much as the average contestant but am sure I couldn’t beat him.
I also think his knowledge of odds really helps him. I was once trying to find a gambler. I was talking to the chief of a department in the city the guy was from.
The cops were happy that we were after him. One thing in his file was that he had been caught playing craps in an alley. he also made high dollar bets of games etc.
The fact is, being a gambler, it really helps to know a lot. You really need to be smart.