Thank you for the link! Yes, there have been recent amazing advances made in local hardware capable of running heavy duty LLMs locally. I have a fairly capable home lab setup and have been running LLMs locally, but with the current pricing structure, it does not make a lot of sense for many people.
If the data center cabal had not driven the price of memory, storage, and GPUs up by 3 times by hogging all the means of production. These machines would already be available at reasonable prices and the need for AI Data Centers would already be starting to wane dramatically.
This harkens back to the beginning of the PC era where mainframes were still being used by big business for serious endeavors. PCs took over for many good reasons. This AI data center nonsense is echoing a past distant enough that most here do not remember it. There will still be uses for AI Data centers, but for various reasons including latency, hybrid solutions which I briefly touched on in post #24 will be the wave of the future.
I think in 5-10 years you will be able to run those kinds of models on your iPhone.
That’s why I predict in the long-run, Apple will be the big winner in AI.
Hunt the Wumpus