I heard back than that the open source driver is pretty much just a layer of abstraction and that most actual code runs in the firmware which is proprietary

The driver you're likely referring to is NVK, which is also not developed by Nvidia, check out the annoucement post by Collabora, it says:

As said above, NVK is a new open-source Vulkan driver for NVIDIA hardware in Mesa. It's been written almost entirely from scratch using the new official headers from NVIDIA. We occasionally reference the existing nouveau OpenGL driver [...]

And also

a few months ago, NVIDIA released an open-source version of their kernel driver. While this isn't quite documentation, it does give us something to reference to see how NVIDIA drives their hardware. The code drop from NVIDIA isn't a good fit for upstream Linux, but it does give us the opportunity to rework the upstream driver situation and do it right.

So they're developing a driver based off headers made available by Nvidia and some of the reverse engineered code from regular Nouveau. In fact, it seems to be a branch of Nouveau as it stands:

Trying out NVK is no different than any other Mesa driver. Just pull the branch nvk/main branch from the nouveau/mesa project, build it, and give it a try

So the "OSS drivers from Nvidia" aren't what makes it work, it's the whole community effort to build NVK from scratch.

Regardless, it only supports the most recent cards using Turing architecture. From mesa3d docs:

NVK currently supports Turing (RTX 20XX and GTX 16XX) and later GPUs. Eventually, we plan to support as far back as Kepler (GeForce 600 and 700 series) GPUs but anything pre-Turing is currently disabled by default.

They released them a while ago and they've gotten pretty good lately.

Based on what?