Replies

I do have some questions about the methodology in the study, but despite the accuracy of citations for matched text (the purpose of the study), it found that Perplexity was correct more often than the other AIs (despite also being more incorrect in the citations to the text).

My question about the methodology is that the study is testing a very rudimentary function, that is, matching a source material to the actual citation. I'm more interested in its reasoning model and its ability to answer related questions based on sources and prior answers. In this area, I find Perplexity quite satisfying in its reasoning and its presentation.

I have found some early inconsistencies, such as when asking about obituaries where the AI mixes up family members across different obituaries of people with similar names. That's why it is important for the querant to have some basic knowledge of what they're querying in order to tell if the AI is making any fundamental mistakes in its answers ("confidently wrong" as the article calls it).

Ragarding interpretation of the news rather than just textually matching to sourced citations, I have posted several dialogs with Perplexity AI in the past few weeks. I must add that in the previous weeks of experimenting with AIs, I feel like Captain Kirk talking to Nomad.

Discussion about Ken Paxton, John Cornyn, and Texas politics.

Judge Boasberg and District Court activism.

Dropbox PDF of my discussion about Letitia James's legal troubles.

Dropbox PDF of my discussion about Kamala Harris and her true abilities.

Note that discussions like this with an AI requires a lot of predecate questions to set up the frame of the converstion that the AI will reference back to.

-PJ