I can't speak for the current state of the art, like the ANN's used by LLM's like ChatGPT, but I have some ideas about the future of neural networks. I think there's a future in networks based on Predictive Coding, which are more versatile, and also in SNN's implemented on neuromorphic hardware.
"Opportunities for neuromorphic computing algorithms and applications" by Schuman et al. provides a good overview of neuromorphic computing.
For an overview of Predictive Coding see "Predictive Coding: a Theoretical and Experimental Review" by Millidge et al.