57

I want to have a local mirror/proxy for some repos I'm using.

The idea is having something I can point my reads to so that I'm free to migrate my upstream repositories whenever I want and also so that my stuff doesn't stop working if some of the jankiest third-party repos I use disappears.

I know the various forjego/gitea/gitlab/... (well, at least some of them - I didn't check the specifics) have pull mirroring, but I'm looking for something simpler... ideally something with a single config file where I list what to mirror and how often to update and which then allows anonymous read access over the network.

Does anything come to mind?

top 14 comments
sorted by: hot top controversial new old
[-] bjoern_tantau@swg-empire.de 21 points 2 months ago* (last edited 2 months ago)

A bash script would probably be easiest to write and pluck into cron.

Edit: Clone all repos you want into one directory and then loop with a script over all cloned dirs and issue git fetch. Done. If you want to add a repo you clone another.

[-] shnizmuffin@lemmy.inbutts.lol 15 points 2 months ago* (last edited 2 months ago)

This can be made even simpler by installing all the repos you want to mirror as submodules of the parent directory's git repository. Instead of many git pull or git fetch, you blast a single git submodule update --recursive --remote and go about your day.


Bonus: This has the added benefit of generating a git history for your automated process if you script in a commit message with a timestamp, making your mirrors reversible.

[-] Findmysec@infosec.pub 1 points 2 months ago

I need to try this, thanks

[-] intensely_human@lemm.ee 19 points 2 months ago
[-] possiblylinux127@lemmy.zip 5 points 2 months ago* (last edited 2 months ago)

If only it were decentralized. We need a federated alternative.

Oh wait...

[-] i_uuuh_what@lemmy.world 9 points 2 months ago

Hey, that's almost what I made Gire for. It doesn't support completely anonymous reads though (requires SSH keys for auth).

git-sync looks like it does at you're looking for.

[-] Linuturk@lemmy.world 8 points 2 months ago

I have an instance of Gitea that I use to mirror stuff.

[-] eskuero@lemmy.fromshado.ws 7 points 2 months ago* (last edited 2 months ago)

Assuming you have all of them under a folder, I just run this lol

for f in *; do
    echo "$f";
    git -C "$f" pull;
    git -C "$f" submodule update --recursive --remote;
    echo "";
    echo "#########################################################################";
    echo "";
done
[-] butitsnotme@lemmy.world 4 points 2 months ago* (last edited 2 months ago)

Bash and a dedicated user should work with very little effort. Basically, create a user on your VM (maybe called git), set up passwordless (and keyless) ssh for this user but force the command to be the git-shell. Next a simple bash script which iterates directories in this user’s home directory and runs git fetch —all. Set cron to run this script periodically (every hour?). To add a new repository, just ssh as your regular user and su to the git user, then clone the new repository into the home directory. To change the upstream, do the same but simply update the remote.

This could probably be packaged as a dockerfile pretty easily, if you don’t mind either needing to specify the port, or losing the machine’s port 22.

EDIT: I found this after posting, might be the easiest way to serve the repositories, in combination with the update script. There’s a bunch more info in the Git Book too, the next section covers setting up HTTP…

[-] vsis@feddit.cl 2 points 2 months ago

git is already a decentralized version control software. Your local git repos are mirrors by themselves.

Put some git fetch in a server crontab, and you're done. You can access them via ssh if your user have permissions.

[-] possiblylinux127@lemmy.zip 2 points 2 months ago

You need a bot that waits for a change and then pushes

[-] friend_of_satan@lemmy.world 2 points 2 months ago

I'd personally do what others are suggesting and use bash, but you could also go with http://myrepos.branchable.com/

[-] it_a_me@literature.cafe 1 points 2 months ago* (last edited 2 months ago)

I'd look into the git-maintenance's prefetch task. From what I understand, that is more or less what you are looking for. Then just run any old http(s) server and clone them from that https://git-scm.com/docs/git-maintenance

this post was submitted on 16 Aug 2024
57 points (100.0% liked)

Selfhosted

39937 readers
403 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS