2001:db8::/32
Going IPv6-only with what you use the internet for daily will break things.
For time being the recommended approach is a dual-stack setup with NAT64 + DNS64 in the mix.
NAT64 you'll need Jool on Linux and DNS64 you can just easily use Google or Cloudflare's public DNS64 servers or run your own.
I think the best way to do this is to setup a Wireguard VPN server on the router itself or on a machine in the LAN. The router firewall will block everything inbound(tcp/udp) except to the inbound VPN udp/IP/port.
Then on the client side you setup a Wireguard client that connects to your Wireguard server remotely and access the LAN resources from there.