Hacker News new | past | comments | ask | show | jobs | submit
Viewing LLMS as compression is flawed because it shouldn't be possible to solve new novel problems. Zipping every single file in existence and then asking Winzip to produce a novel working zip file clearly doesn't work. Clearly LLMs are learning from data it is fed and a lot of what we do before LLMs also involved "plagirism". Even the greatest writers and artists would copy each other to master their styles and extract the dimensions. I think framing LLMs as compression is not only silly it demonstrates a poor understanding of the inner workings of LLMs
Except a statistically salient search context is in proximity to something similar, and manual content curators drop their thumb on the scales weights to boost desirable outcomes in the quantized model.

https://www.youtube.com/watch?v=LPZh9BOjkQs&list=PLZHQObOWTQ...

The encoding of the LLM is almost always lossy compression, but we agree it is not real intelligence.

As a sentient turnip, I likely wouldn't understand such things, but do recognize copyright bleed-through when it includes known obscure works. =3

https://en.wikipedia.org/wiki/Mechanical_Turk

Nothing in your video or sourced wikipedia support any of your claims, what is your exact argument that compression can solve novel problems without forcing readers to comb through the links that seemingly has nothing to do with the subject at hand ?
Did you actually know what the G.P.T. stands for in chatGPT?

Indeed, we shouldn't kink shame peoples cognitive dildo. =3

https://en.wikipedia.org/wiki/Sealioning