3D doesn't necessarily mean 3D. Web browsers and video players (including those inside web browsers) will often use the 3D pipeline to write 2D rectangles to the screen. Other software may do the same sorts of thing.
And even if you're not actually viewing anything in particular, software might be loading things that don't show obvious on-screen changes but which still might pre-calculate via the GPU.
As for how to reconfigure GPU behaviour, that's heavily dependent on the software. I know Firefox has things in about:config for it. Can't speak to Chrome or other browsers, but I assume something similar exists. Other programs may or may not have any settings for it.
Given the only moving parts on a graphics card tend to be the fan, maybe there's another fan on there you haven't accounted for?
At your own risk you could try gently stopping fans - on the graphics card and otherwise - with your finger. On the hub, preferably. Most will handle this and spin right back up again. If not give it a flick in the right direction. If the grinding noise continues, the noise probably isn't coming from the fan you're stopping. (FWIW, I have an old NVIDIA card whose fan sometimes makes noise at low speeds, which is kind of the opposite problem. I manually 'reset' that fan at least a hundred times with no issues, but I imagine it hasn't been great for the motor.)
Obviously, don't hold a fan stopped for any significant length of time. It's there for a reason.
Another possibility is sympathetic vibration to a fan or fans at certain speeds. My last PC case loved to sing along with the CPU fan during moderate use. I cured that with shims of cardboard and a few bits of old packing sponge in the most vibrational parts. (Not enough to hamper airflow though.)