Yeah, we had an interesting discussion about this a few weeks ago. Our kids were highly amused and appalled at how the market can be SO manipulated!
You apparently are not aware of the many, many practical uses of diamonds that such as rubies cannot touch. Showpiece-quality diamonds are probably extremely rare. Diamonds are the "hardest thing in existence" and also extremely conductive, and as such are in huge demand by various industries.
Given all that, they are probably not rip-offs.