I just asked Perplexity.ai which NFC division the AFC West will play in 2026. It actually claimed that the AFC West played the NFC West in 2024, when it played the NFC South. That was a VERY straightforward question to botch.
LLMs by themselves can’t answer many of those questions, they have to have data fed into it, either by getting information from the web, or from a vector store database, or a tool.