Indeed, specification of what "understanding" even is is one of the hard problems that would have to be solved before answering such a question! Nobody disagrees, for instance, that an LLM has no understanding, awareness, or even access to the underlying numerical weights of its neural network, but still there is much debate whether it understands anything at all, even subjects that it is clearly superhuman at performing in. Perhaps "understanding" will only have a meaningful definition to us if it is tied with conscious awareness, but that's hardly better.

Another very relevant Borges story is "On Exactitude in Science." If we could precisely model every physical law and even the precise distribution of matter in the universe, but this model conveyed to us no intuition or usefulness on a human level, can it even be called understanding? Instead we seek teleological anthropomorphic explanations in everything, and it's baked into how we discuss things. "Big cats developed powerful muscles to run fast and catch prey," "our immune system exists to keep us from getting sick," "rooks are worth more than knights," "the universe favors states with the highest possible entropy," "de Rham cohomology detects topological holes."