setting up a mirror

[Graymalk]

One of the more frustrating things with my Pis is that whenever I apt-get update etc., the mirrors don't always behave well (I update several Pis with identical installations and get a different experience on each of them, every time, ranging from an update going perfectly to an update erroring out and refusing to continue because it can't find a file or communicate with a server or something like that). So I have decided to try making one. I'm not yet sure if I'll make it publicly available, though. Mainly, I have questions I need answered first.

I have found the Wiki, and I have also been looking at other Wikis for setting up mirrors in general, but they sort of skip over basic stuff, like which user should be owning the partition the mirror is on and what the permissions ought to be. Since I'm going to keep it local for now, I decided to try it on the pi user, so I created my file system and set the owner to root and the group to users, and gave it rw permission for the group. So any user can mount it and read/write, or in other words, a hopefully pretty normal filesystem. I then copy/pasted the rsync command off the Raspbian wiki, ran it, and noticed it attempted to chown the mount point. For fun, I cancelled and ran it as sudo to see what it would change it to, and it changed it to pi:pi. I'm guessing it really just changes it to <current user>:<current user> so that if you run it as sudo it will.....I don't know? Anyway, I find this confusing. What is it trying to do?

What user:group do people here recommend that a mirror exist as? If I decide to make it publicly available, what permissions do I need to put on it? Would it be safe to just give it to www? (I am in fact running Apache so I already have that all set up)

Can I put the big long rsync command in my crontab? I assume that's what most people would do? Or is there some other magical thing I don't know about? I sadly must admit this is my first time using rsync.

There's a good chance I might make it available in the future, if it works well for my own usage with my own Pis. My country only has one mirror. I have a data cap so this might be a huge mistake, but I can always give it a try for a month and see how bad it is. Well I may be capped but I'm also slow (yes of course I'm using a slow little Pi to do this, what makes you ask?) so that might help a bunch, lol. edit: Oh wow, I have no data cap. OK, I may become my country's second mirror.

[Graymalk]

OK, I think I have answered my own questions and will share them here for anyone else who had the same questions.

First, I had forgotten that anything my Pi serves up via the web is actually owned by root (user, not owner, is www). The idea behind that is if someone breaks into the account, they have to also be able to elevate their privileges before they can do anything at all. So that was easy - I just left the mirror owned by root and made a symbolic link to it in my html folder. And that was basically all I had to do. I can point my various Pis at it now and they update from it easily.

Small hitch, though... Long ago I forced all traffic over to HTTPS for a project I'm slowly working on, so before I could switch my Pis over to my own mirror, I had to apt-get install apt-transport-https. Hosting a mirror this way is probably pretty pointless so for that reason alone, I doubt I'll ask to have it listed in the official mirrors list. Still, works great after that little tool is installed. NO MORE ERRORS!

Oh I think I asked about putting it in cron? I did that. Seems fine. I did this (this is one entry):

Code: Select all

0 0 * * * rsync --archive --verbose --delete --delete-delay --delay-updates \archive.raspbian.org::archive /<mirror location> && date -u >> /<timestamps location>/mirrortimestamps.txt

I'm more of a middle-end cron user so I'm not sure if it's possible to have one entry that runs things multiple times per day so I made different entries for different times of day to update at.

And that's about it. This has, at least for now, 100% solved my mirror problem. The cost to me is a 500 GB partition on a 2 TB drive I had sitting around already anyway.

Possible future project: set up a second Pi without https, maybe? Put the mirror on that? Then get listed. I think I can accomplish this on the first Pi as well. I'm running Apache, which I think was pretty much made for this situation: direct http traffic only to the mirror and everything else to https... So much to learn! (but for now, OMG YAY so happy I have eliminated my apt-get update errors! )