What we can do is simulate very simple brains by simulating relatively few neurons as they appear in worms. In this sense we are multiple magnitudes away where the increasing complexity implies exponential increasing difficulty.
I would think we are so far away that there will be unknown unknowns we encounter on the way.
https://www.quantamagazine.org/ai-is-nothing-like-a-brain-an... https://pmc.ncbi.nlm.nih.gov/articles/PMC9665914/
This is why making more neuromorphic NNs is still an active area of research, although they typically all focus on another extremely simplified model (spiking neural networks).
And it seems that, given enough input/outputs/compute, it is possible to train the necessary function.
Details of how the building bricks look like (matmul, electromagnetism or quantum effects) are not that relevant in the broader picture.
What is missing right now, is the fact that the function in question changes over time in biomachines, while our LLMs are static at inference time.
a) The brain might have an entropy source (then it can't be modeled as a function). Trivially to fix, and in some sense, with diffusion models starting from random numbers, AI has done so.
b) The hidden function might be not computable. I would have no idea how that would work, but I think this is what it boils down to if people say "the human brain is more than a machine".
b) well, it can be the case that, say, certain kinds of computation are either too inefficient or outright impossible within the current model.
Who knows...
Yeah, but it's hard to explain this to people, especially AI-pro people. Too many are convinced that all we are doing is a cut-down version of the human brain, and it's hard to explain to them that, no actually, we aren't modeling the human brain to the level of granularity you think we are.