Sota LLMs like GPT-4o can natively understand b64 encoded text. Now we have algorithms that can decode and encode b64 text. Is that what GPT-4o is doing ? Did training learn that algorithm ? Clearly not or at least not completely because typos in b64 that would destroy any chance of extracting meaning in the original text for our algorithms are barely an inconvenience for 4o.
So how is it decoding b64 then ? We have no idea.
We don't built Neural Networks. Not really. We build architectures and then train them. Whatever they learn is outside the scope of human action beyond supplying the training data.
What they learn is largely unknown beyond trivial toy examples.
We know connections form, we can see the weights, we can even see the matrices multiplying. We don't know what any of those calculations are doing. We don't know what they mean.
Would an alien understand C Code just because he could see it executing ?
Our DNA didn't build our brain. Not really. Our DNA coded for a loose trainable architecture with a lot of features that result from emergent design, constraints of congenital development, et cetera. Even if you include our full exome, a bunch of environmental factors in your simulation, and are examining a human with obscenely detailed tools at autopsy, you're never going to be able to tell me with any authenticity whether a given subject possesses the skill 'skateboarding'.
loading story #41523199
Base64 encoding is very simple - it's just taking each 6-bits of the input and encoding (replacing) it as one of the 64 (2^6) characters A-Za-z0-9+/. If the input is 8-bit ASCII text, then each 3 input characters will be encoded as 4 Base64 characters (3 * 8 = 24 bits = 4 * 6-bit Base64 chunks).
So, this is very similar to an LLM having to deal with tokenized input, but instead of sequences of tokens representing words you've got sequences of Base64 characters representing words.
loading story #41522127