That’s how it often works at that income level. Those who can afford to spend $30 million on a house will often buy the house for the land and then spend $50 million tearing down the house and building a replacement that they prefer.
Two things come to mind.
Location, Location, Location.
And don’t overbuild for your neighborhood.
In the Seattle area I’ve done jobs at beautiful homes built in the 1970’s on large waterfront lots. These beautiful homes get torn down. Sometimes a nice house gets built, but often it is something like Jordan’s with no character (IMHO).