Replies

This is the best layman explanation of how a generative model works:

It starts with every piece of training data being broken down into very small tokens. In the case of books or written works, these tokens could be the individual words, numbers, or punctuation marks..These individual tokens are then stored, along with the probability of their presence and their likely relationship to other tokens…once an AI model has been trained, when it is prompted for an output, the prompts provided are used to navigate around this multidimensional model to find tokens that are likely to be present in the output. There is no attempt to truly process the meaning of the prompts. They are simply used to hone in on the collection of tokens that are relevant for the context the user has set. Tokens with a high probability of featuring in the requested output are then stitched together where probabilities are also used to get the sequencing right.

These models seem like magic.

But when you know how they work, it takes them down a peg or three.