Of course I’m not suggesting to eat 4000 calories. A sedentary person probably won’t need more than 1200-1500 calories.
It depends what you mean by the body adapting. Outside of Centenarians,half of whom live extremely unhealthy lifestyles but have the genes to compensate, sedentary life is a shorter life. The only debate is by how much.