The first book is serious, yes.
Humor is interspersed throughout the rest of the series.
And some stuff is unintentionally funny, like the were-wolf in Prince Caspian at the mound of Aslans Howe.
(I found what he said to be quite funny.. but that's me.)
Reepicheep is a well written character, and Lewsis description of him is good.
I've managed to finished reading other of Lewis' works, but never the Chronicles....now, if I can just remember which box they're in.....
Reepicheep is a great character. The proverbial mouse that roared. I think some of the humor in the characters is that we can see parts of ourselves and others, the good, the bad and the absurd, in them. We can relate to their weaknesses and aspire to their nobility.