When Shannon coined the terms information and bit and entropy back in 1948 - he defined them very clearly, very mathematically. One cannot be speaking to his model and slip in some common usage definition for the terms and hope to assert a credible argument.
The word complexity is even more explosive because there are hard, mathematically oriented, definitions of various types of complexity. We can't be throwing the term around here and hope to be communicating if we don't first stop and define what we mean.
If I had my way, I wouldn't use "information," "complexity" or any other word that has an alternate meaning in English. The possibility of an inadvertent slip into some less precise meaning, either on the part of the writer or the reader, is just too great. Perhaps we could, for our own discussions, consider adopting a symbol that has no other meaning.