Open source alternative to executables (self.opensource)

submitted 1 month ago by obbeel to c/opensource@lemmy.ml

26 comments fedilink hide all child comments

Today I had to downgrade fastapi from 0.114.0 to 0.112.4 to make a software work. And it just hit me - what if pip didn't support 0.112.4 anymore? We would lose a good piece of software just because of that.

Of course, we can "freeze" the packages into an executable that will run for as long as the OS supports it. Which is a lot longer. But the executable is closed source. We can't see the code that is run from an executable.

Therefore, there is a need for an alternative to which we still have access to the packages even after the program is built. That would make it safely unnecessary for pip to store all versions of all packages forever more.

Any ideas?

all 27 comments

sorted by: hot top controversial new old

[-] CameronDev@programming.dev 39 points 1 month ago

If its an open source project, the answer is to rebuild from the tagged source.

Eg: https://github.com/fastapi/fastapi/tree/0.112.4

With the right repo setup, you can pip install git+https://github.com/fastapi/fastapi.git@0.112.4 (example only, not sure it works), so pypi doesn't need to keep all previous wheels, its just easier for it to do so.

[-] Solumbran@lemmy.world 6 points 1 month ago

There are hellish things such as guix for that.

Or ideally the software needs to be upgraded.

[-] Klaymore@sh.itjust.works 4 points 1 month ago

Nix as well

[-] gomp@lemmy.ml 1 points 1 month ago

btw :)

[-] 31337@sh.itjust.works 6 points 1 month ago

There's plenty of open source projects that distribute executables (i.e. all that use compiled languages). The projects just provide checksums, ensure their builds are reproducible, or provide some other method to verify.

In practice, you're going to wind up in dependency hell before pypi stops hosting the package. E.g. you need to use package A and package B, but package A depends on v1 of package C, and package B depends on v2 of package C.

And you don't need to use pypi or pip at all. You could just download the code and directly from tbe repo, import it into your project (possibly needing to build if it has binary components). However, if it was on pypi before, then the source repo likely had all the code pip needs to install it (i.e. contains setup.py and any related files).

[-] pol@infosec.pub 4 points 1 month ago

Nix

[-] remram@lemmy.ml 1 points 1 month ago

what if pip didn’t support 0.112.4 anymore?

What do you mean by that? If new versions of Python didn't run that version of fastapi? If PyPI removed it?

[-] obbeel 0 points 1 month ago

If prior versions were not support by pip anymore, so yes, if it were removed. There are cases of packages not being supported by the platforms, aren't there? I've run into cases where the package was fully deprecated and not useable or downloadable anymore.

[-] remram@lemmy.ml 1 points 1 month ago

What do you mean "not supported by the platforms"? And do you mean that or "removed"?

[-] obbeel 0 points 1 month ago

I couldn't download it even if I wanted to. That's what I mean. It returns a message saying it isn't supported.

[-] remram@lemmy.ml 2 points 1 month ago

"It" being the PyPI server not finding it? Pip not supporting the API? Or it downloads correctly but the setup.py prints that error?

[-] obbeel 1 points 1 month ago

Anyway, more access to the open source packages can't be bad.

[-] obbeel 0 points 1 month ago

I think it said it's deprecated or something? I'm not sure, I just know I had problems downloading packages before.

I don't think it was setup.py . I think I tried to download it directly through pip install xx==0.4.0 or something (the version was required by the program) and it said the package doesn't exist.

[-] jlow@beehaw.org 0 points 1 month ago

Nixpackages or Flatpaks/Appimages(/Snaps 🙈) are a general solution to this (not in this specific case probably), no?

[-] obbeel 0 points 1 month ago

But do Appimages make the dependencies code available? They pack everything into one working program, but what about the packages?

[+] iamnickops@fosstodon.org -2 points 1 month ago

[deleted]

[-] obbeel 1 points 1 month ago

What do you mean?

I just find that if pip did not support that version anymore, the software would be lost. As that is covered by making executables, as I mentioned them. But what if I wanted to have access to the libraries that were used in the program? That wouldn't be possible. Because all we get in the source code is the dependency fetching, not the dependencies themselves.

It would be good to have an alternative where you get all that you need to compile the code again, not depending on fetching them from websites that might not even have them anymore.

This mentality of ephemeral code just adheres to the way big tech would like to do things, with programmed obsolescence.

An alternative to that way of doing things would be nice and would make sure we get access to the same working open source program in 30 or 40 years.

[-] CameronDev@programming.dev 7 points 1 month ago

I'm not sure there is a "mentality of ephemeral code" in open source projects. The source is literally available on github or similar, and anyone can mirror it as they like.

If it is popular enough, then the project is probably backed up in the github artic vault as well.

[-] 2xsaiko@discuss.tchncs.de 3 points 1 month ago

https://www.softwareheritage.org/

[-] N0x0n@lemmy.ml 3 points 1 month ago

While I don't know of a solution I totally get what you mean... A lot of cool projects don't work anymore because of that...

Something similar to the container technology, where everything is packed into an image with all it's dependencies to the correct version. The app would probably be a security nightmare, but could still work in it's own contained system.

This sounds like a very neat technology that Linux is missing !

[-] badcommandorfilename@lemmy.world -3 points 1 month ago

Isn't this the whole idea behind flatpak but everyone seems to hate it

[-] pastermil@sh.itjust.works 10 points 1 month ago

Who hate flatpak?

[-] N0x0n@lemmy.ml 8 points 1 month ago* (last edited 1 month ago)

I hate them ! How can something be flat and a package at the same time ? Those flatpackers are the flatearther conspirators of IT ! Just burn them on hot pile of logs like we are in the 17th century !!! Burrrrn !!!

[-] pastermil@sh.itjust.works 6 points 1 month ago

i knew i should've gone with roundpack instead :(

[-] gomp@lemmy.ml 5 points 1 month ago

I hate them (seriously).

It's basically a second distro inside your distro (try du -chs /var/lib/flatpak/) and if something breaks (eg. last year mesa with my graphics card) it isn't easy to identify were the problem is (because all libs update at the same time), plus you can't just try a newer (or older) version of some lib as you would in your distro.

Moreover, you can't flatpak CLI tools (also servers and OS components, but I guess the ubuntu folks are the only ones who care about those).

[-] moonpiedumplings@programming.dev 1 points 1 month ago

So, I'm not gonna pretend flatpak doesn't use more space then normal apps, but due to deduplication (and sometimes filesystem compression), flatpaks often use less space than people think.

[nix-shell:~/Playables/chronosphere]$ sudo /nix/store/xdrhfj0c64pzn7gf33axlyjnizyq727v-compsize-1.5/bin/compsize -x /var/lib/flatpak/
Processed 49225 files, 21778 regular extents (46533 refs), 22188 inline.
Type       Perc     Disk Usage   Uncompressed Referenced
TOTAL       53%      898M         1.6G         3.6G
none       100%      499M         499M         1.0G
zstd        34%      399M         1.1G         2.6G

[nix-shell:~/Playables/chronosphere]$ du -sh /var/lib/flatpak/
1.7G    /var/lib/flatpak/

I only have one flatpak app installed, and du says that takes up 1.7 GB of space... but actually, when using a tool that takes up BTRFS transparent compression into account, only half of that space is used on my disk.

I recommend using compsize for a BTRFS compression aware version of du and flatpak-dedup-checker for a flatpak filesystem deduplication aware checker of space used.

I think flatpak absolutely does use up more space, because yes, it is another linux distro in your distro. But I think that's a tradeoff people accept in order to have a universal package manager for graphical apps.

Also, you can flatpak cli tools. They are just difficult to run at first because you have to do the flatpak run org.orgname.appname thing, but you can alias that to a short command. Here is a flatpak of micro, a terminal based text editor.

(I prefer nix for cli tools though, and docker/podman/containers for services).

[-] beyond@linkage.ds8.zone 1 points 1 month ago

This is why the marketing around flatpak bothers me. It's touted as some kind of "universal Linux package manager" but Linux is just a kernel - all the stuff that apps depend on comes with the distro. So, in order for flatpaks to be "distro independent" they basically have to supply all the stuff that normally comes from the distro - effectively building a second distro on top of your existing one.

Nix and Guix are the same but at least I think they're more up front that they are effectively distros that can run on top of your existing distro or as a standalone operating system directly on top of Linux.