You are right about all of that. I sold my home of 20+ years and moved into a home that is smaller but paid for and much less in property taxes(14,000 vs 4800). In terms of memories you are right, those are right where they belong, in my soul. NO need to be attached to materials.
FYI If you’re in California and 55 or over.
You can sell your home and buy one for less and transfer your property taxes, if you move within or too one of the southern Cal coastal counties, ex: San Diego.
Propositions 60 and 90.