Since you're on PC, using cheats or potentially mods can expand the list of games you have available, since you might be able to just ignore certain aspects of a game, like combat, money or resources in general.

The more recent Assassin's Creed games have something called a Discovery Tour, where you can just explore the world, without combat. They are partially meant for educational purposes and can teach about the different regions and history, and offer guided tours. I don't know if there's a real free roam, where you can just do whatever though.

You mentioned buying Goat Simulator, so maybe other of these "Simulator" games might be for you.

I don't know how much you can go into houses, but Farming Simulator could be worth a look. Your kid can probably just explore the map and maybe even try some of the farming.

As someone else mentioned, Slime Rancher might be fun just to explore the world with the cute creatures all around, but there's not really much or any buildings as far as I know (don't know if the sequel has this stuff).

A Short Hike could be worth a look. You're on a mountainous island / national park type deal and your main objective is to get to the peak of the mountain, but you can just explore, do some side quests, solve puzzles, etc. I feel like the screenshots on the Steam Store page for the game look really ugly, since it's so pixelated, but that's just a filter, that you can change in game (from smooth to really pixely).