You make a bunch of excellent points. However, there is another psychological mechanism that you did not mention. The ability to nibble all day without thinking about it or remembering it later.
Almost every person I know who has been forced by their doctors to keep a daily log of every morsel and drink that they put into their mouths has found that they typically have been eating and drinking far more than they realized. And most of what they eat is snack food of questionable nutritional value.
Most people have snack food and drink available to them every minute of the day and eat most of their calories without even thinking about it. In this past year where we live the availability of restaurant food has been reduced, but people just switched to “fast food” where they typically get more empty calories for less money.
Sorry for meandering a bit.
I know you want to believe but...when I was in medical school 40 years ago I believed as you do now.
I bought lean cuisine (or equivalent brand) and ate one for lunch & one for dinner and had a boiled egg for breakfast.
I jogged in center city philly through the smog & car exhaust. I drank only seltzer & ice tea w/o sugar.
I had so little sugar that when I mistakenly took a sip of tonic water the sugar was shocking. I know now that there is sugar in tonic water but til that point I couldn’t taste it and didn’t know.
After a solid month with rigid adherence to 1000 cal/day I had gained 5 lb. What I know now having lost 60lb once and 100 lb after the 60 returned in 5 years, is that for me carbs are the perfect food, digested efficiently & transferred as a layer of fat. On my month of <1000 calories a day I chose the pasta calorie counter meals.
The problem with keto diets is not that they don’t work it’s that after 3-4 years on one my body rebels and I don’t feel well. I have proved two ways through controlled eating that for me at least a calorie is not a calorie and that we can only accomplish so much with our genetics. Most scientists working on obesity are investigating which forms of which enzymes are the most important for how food energy is utilized, stored or wasted. There will be a more nuanced understanding & perhaps even ways to regulate food metabolism.