Aluminum phosphate is used in fertilizer:
https://farmonaut.com/mining/aluminum-phosphate-tin-phosphate-phosphate-mineral-uses
That is correct. As AlPO4. It’s the PO4 (phosphate) that acts as the fertilizer. And if your pH is correct then phosphate gets released for your plants to take up and the aluminum ion (in correct pH) is incorporated into minerals in the soil before the plant can absorb them.
If your pH is too low, the whether you add this fertilizer or not aluminum will leach out of minerals in the soil and your plants take it up. So if you’re a good farmer you maintain proper pH in your soil.