That's a good question, and the answer is that if you've had a mild or asymptomatic case of COVID-19, the levels of neutralizing antibodies in your body tends to be fairly low and don't last very long (https://www.nature.com/articles/s41467-020-20247-4). Meanwhile, the vaccines induce a much stronger immune response than a mild or asymptomatic case, leading to better protection that last longer (https://www.nature.com/articles/s41586-021-03324-6).
This becomes more important when we talk about variants like P.1, which has slight structural changes to the S protein such that the neutralizing antibodies based on the April 2020 variant (or the original Wuhan SARS-CoV-2 for that matter) don't bind as effectively. Sort of like trying to unlock a lock with a key that's been altered slightly. It can still work, but it requires a lot more time, effort, and luck.
The city of Manaus, Brazil is a stark example of this. Widespread serology (antibody) testing showed that 70% of the city had been infected previously. Then P.1 showed up and suddenly massive waves of COVID-19 struck the city again. They had already been hit incredibly hard before. They should have had immunity from infection recovery. P.1 didn't really care. (Source)
So, the only truly immunity comes from death?
Makes sense.