Since I deal in used and rare books I have access to a bunch of books that I pick up for pennies so I can indulge my vice cheaply where I would have to pay through the nose for electronic books.
Speaking of used/rare books... One of these days I'm going to get serious about picking up a 1910 Brittanica set. If you're interested in some ebooks that won't break the bank and are into science fiction, check out the Baen Free Library. They have a bunch of them available for free download in just about any format imaginable. Beware though, the evil bastards at Baen understand the concept of "the first one's free". The 1632 series is especially good, and it has expanded from a single novel into about 15 books and over 70 "gazettes" of short stories set in the 1632 universe. I also recommend anything by John Ringo. I'd read a grocery list if he put it together.