How would the distro tell if the tar.gz is a program or just a bunch of compressed files? I tend to use tar files for compression rather than for distributing or installing software

Cause .tar.gz is not an executable binary in itself. It is more like a zip of the binary and its support files. Since it would be really difficult to distribute and maintain native binaries for every type of distro, application publishers used to just distribute for distros that has a larger userbase, like a .deb for Debian/Ubuntu and a .rpm for Fedora/RHEL/Suse, and then they would provide a tarball for any other distro. Flatpak and Snap have been introduced to eliminate this overhead from application developers, but they still offer tarballs as the newer solutions are still far from standardization.

I mean, what else would double-clicking a .tar.gz file or an appimage do than install it (yes, I know, look into the archive, but really - how often is that the desired thing to do)?

Personally, I open tarballs quite often. Why? Just wanna know what's inside them.

Double click translating to install on tar.gz is not wise since anything can be packed in a tarball, a gzip archive or a gzipped tarball.

And then you have a similar problem with tar.xz (it's becoming more and more popular from what I can see). It doesn't use gzip to compress the archive, it uses LZMA, so you have to use xz instead of gzip to decompress the tarball.

Basically, it boils down to how UNIX used to work. Mind you, back in the day there were no package managers, it was all done with .tar.zip or make install. This really was hell to be honest... keeping track of what app installed what libraries and versions of it... yes it really was a PITA. This is the real reason why package managers were invented (thank god!) and why having only one package manager on a system is the preferred way to handle apps/software. Otherwise, you'll soon be in dependency hell. Shared libraries are at the core of any UNIX based OS. Yes, Flatpaks/Snaps/AppImages and package managers like nix circumvent this problem, but in no way is the problem gone. It's still there, we just don't mess around with it.

So, basically the idea of having an "installer" came quite late into the game. Plus, having to check on all dependencies and making an install script that worked on every single distro out there was just so complicated, that no one would ever want to go through with it. Sure, there are install.sh scripts in some pacakges out there, but they basically check nothig, it's more or less "copy this here, that there" which of course could be done by hand anyway... and then run the app and pray it has all of the dependencies it needs 😬... which it never does 😂.

Do you not know what a tar.gz is? it's like a zip file. how are you supposed to install it automatically? the structure inside is unknown.

also, it's commonly used for things like backups.

A .tar.gz is a regular archive, the file manager shouldn't assume it's a program. Also, how do you expect it to work? There's no general rule for archives. As for appimages, one of their selling points is that you can run them portably, so it shouldn't install by default, but run. However, a right click install option would be nice.