... and also because sometimes it's just easier to have a local cache
I've a mirror machine sitting next door: it's a private mirror of CentOS, Debian, Debian CDs, EPEL [extra packages for Enterprise Linux from Fedora], Ubuntu releases, and Ubuntu CDs. I've run this intermittently for a few years for myself: it's grown recently because I added CentOS and EPEL.
A 6TB LVM volume is now at about 62% full, so there's plenty of space.
Layout
My machine has these mirrors placed under /srv. The mirrors directory is root owned, the individual directories are owned by a non-root user which also runs the scripts. So you end up with /srv/mirrors/debian or /srv/mirrors/ubuntureleases, for example, where the final directory is owned by a mirror user and the mirror user owns the scripts and the crontab that runs them.
Package used for mirroring
The ftpsync Debian package is pretty much all you need: each individual download source has its own ftpsync.conf script which sits under /etc/ftpsync
The configuration files are essentially identical for Debian and
Ubuntu - the only difference is in the source from which you mirror.
Configuring the ftpsync.conf script
The changes needed from the example configuration script in /usr/share/doc/ftpsync are minimal and straightforward: edit The TO directory and the source from which you mirror.
My ftpsync-debian.conf has the following lines changed from the supplied config file, for example.
TO="/srv/mirrors/debian/"
RSYNC_HOST="debian.hands.com"
RSYNC_PATH="debian"
Calling and usage:
Calling is simple: ftpsync sync:archive:debian which references the ftpsync-debian.conf above. Each ftpsync call must have a corresponding configuration file or the ftpsync script complains.
Usage is from mirror's crontab - excerpt below
13 03 * * * /usr/bin/ftpsync sync:archive:debian
37 03 * * * /usr/bin/ftpsync sync:archive:centos
Hope this helps someone: will add a couple of bits on configuring Apache to serve this in another post.
[EDIT - changed rsync host listed above at request of jcristau]
Subscribe to:
Post Comments (Atom)
I have a question on the choice of path.
ReplyDeleteI have a server set up mirroring Ubuntu at the moment, and I want to add Debian mirroring to it.
The drive set up currently is:
/dev/sda2 is mounted as /media/8TB
/dev/sda3 is mounted as /media/8.1TB
/dev/sda4 is mounted as /media/8.2TB
Within /media/8TB I have ubuntu-cdimage/ and ubuntu-mirror/. The drive is 1/3 full.
Within /media/8.1TB I have ubuntu-oldimage/. It's 75 percent full, so it's complete.
Within /media/8.2TB I have nothing. It's empty.
I'm anticipating debian-cd/ and debian/ (packages) going into /media/8TB, but I'm not clear where to point ftpsync for the TO variable.
Do I have enough space on /dev/sda1 and /dev/sda3 to mirror *everything* Debian?
The top lines of my config for debian-cd are
ReplyDeleteMIRRORNAME=`hostname -f`
TO="/srv/mirrors/debian-cd/"
MAILTO="$LOGNAME"
# HUB=false
for example. So wherever the mount point is
that you have for /dev/sda4 so /media/8.2TB/debian-cd , maybe?
The backup config for apache in Debian has a prebuilt entry to display everything under /srv as discussed.
debian-cd is 203G
[That's the latest Debian media images so
those for Debian 11 Bullseye for all architectures - so 11.0 as released on
release day].
debian is 1.4T - All current distributions, all current architectures - so sid -> bookworm -> bullseye -> buster -> stretch -> jessie