I don't think the underlying implementation matters that much actually. In computability theory, any system of data-manipulation rules, be it a computer's instruction set or a cellular automaton, that's Turing-complete is considered to be computationally universal. If what our minds do can be expressed as computation, then this process can be implemented on a different substrate.

The key in my view is that the system has to create an internal simulation of the physical world as the basis for its predictive engine, and then map language to this internal model. It's still a stochastic system in the end, but the difference is that the language becomes a means to compress and transmit a snapshot of a far more nuanced internal state. Currently, the language is all that the model knows, and it's just doing statistical correlation between tokens. It's fundamentally no different from a Markov chain.