Ideas transmitted by books actually increases intellectual growth depending on the type of ideas you are consuming.
TV watching—especially by young children is detrimental to intellectual development and makes the brain lazy. The ability for children to understand complex ideas in books took maturity and concentration and memory work and real life experience —before they could decode and digest the book.
It is an exercise of the brain. TV shuts the brain down to a large extent and does all the work for the child, where books allow imagination and uniqueness.
The view was that there are some ideas you don't want the population to grasp. Until the early 1900’s, literacy was not viewed as a good thing in much of Europe. Look at Spain for example (pre revolutionary Spain). Now, I love books. My children will inherit that. But they are a shrinking medium, and the things you are saying were said against books not that long ago. Imagination was frowned on, which is why some of the Oz books were banned.
Whether tv or books challenge your mind is a separate issue from what I was commenting on. Both are, or rather can be, a means to escape from reality.