You are censoring somebody else from reading the book you are stealing.
If that books remains in the public library, it’s going to disappear at some point and may be needed by future generation. I’d rather have that book available to them. The library can always replace it, if they are so inclined. The typical library replaces hundreds of books every year, so it’s not that big of a deal in that sense.