What you are looking for is gpu pass through. You can do it with integrated graphics as well as dedicated. There is a lot of good documentation out there for proxmox to do this.
I would totally start with a hypervisor. The ability to spin up a new VM for an experiment or project far outweighs any downside. I use proxmox, but that’s mainly because it was the thing to do when I was starting out. There are a few options out there, including installing Debian and running kvm/qemu, which is basically what proxmox is. They all have the same functionality, just different UI/features. Lxc containers in proxmox help a lot as well, when you have a small service you want to run but don’t want to take up a VMs worth or resources. These also helped a lot.
Good luck