Monday, 22 June 2020

And a nice quick plug - Apple WWDC keynote

Demonstrating virtualisation on the new Apple to be powered by Apple silicon and ARM. Parallels virtualisation: Linux gets mentioned - two clear shots, one of Debian 9 and one of Debian 10. That's good publicity, any which way you cut it, with a worldwide tech audience hanging on every word from the presenters.

Now - developer hardware is due out this week - Apple A12 chip inside Mac mini format. Can we run Debian natively there?


Saturday, 9 May 2020

CD / DVD testing for Buster release 4 - 202005092130 - Slowing down a bit - but still going.

Last few architectures are being built in the background. Schweer has just confirmed successful testing of all the Debian Edu images - thanks to him, as ever, and to all involved. We're slowing up a bit - it's been a long, hot day and it's not quite over yet. The images release looks to be well on course. As ever, the point release incorporates security fixes and some packages have been removed. The release announcement at https://www.debian.org/News/2020/20200509 gives the details.

CD image testing for Buster release 4 - 202005091950 - Most install images checking out well

Lots of hard work going on. schweer has just validated all of the Debian Edu images.  Most of the normal install images have gone through tests with only a few minor hitches. Now moving on to the Live images. These take longer to download and test but we're working through them gradually.

As ever: a point release doesn't mean that the Debian you have is now obsolete - an apt-get / aptitude update will bring you up to the latest release very quickly. If you are updating regularly, you will have most of these files anyway. One small thing: the tools may report that the release version has changed. This is quite normal - base files have changed to reflect the new point release and this causes the notification. The notification is a small warning so that you are not taken by complete surprise but it is quite normal in the circumstances of a Debian point release.

Thanks to the other folk doing the hard work: 10+ hours and continuing.

CD image testing for Buster release 4 - 202005091538 - Coming along nicely

We're most of the way through testing the various installs, with good success. Waiting on a few things to be built. It's a good way to pass an afternoon / evening.

CD / DVD image testing happening now - 202005091330

The usual suspects very much involved - Isy, myself, RattusRattus, schweer and Sledge. Image testing going on to good effect. Update pulses are starting to hit mirrors: just done a dist-upgrade on one of the machines here. All good

And again - CD release for Buster release 4 is going to happen over the weekend ...

And for our regular readers - it's happening again. I'm sitting here with two laptops, a desktop, a connection to IRC - and friends in Cambridge and elsewhere involved in this too.

All good fun, as ever :)
 

Sunday, 9 February 2020

CD release time post 3 20202029 1653 - Release for Oldstable / Squeeze / 9.12 has happened

Just tidying up on the last of my tests. The release to the mirrors for 9.12 has happened.

Much sterling work by  amacater, Isy, RattusRattus, Sledge and Schweer (in alphabetical order). This one seemed to have turned up fewer bugs.

Lots of fun, the usual bouts of problems between chair and keyboard and a feeling for me of something worthwhile to share in.

My private CD mirror is now up to date with 10.3 and a few machines here have already been upgraded :) On a day when the weather is cold, wet, windy and rainy, it's been a good excuse to sit in the warm with a laptop.

Here's to the next one, then.

Saturday, 8 February 2020

CD release time post 2 20200208 2152 - lots of install CD/DVD/BD image testing going on

It's been a busy day. Lots of installs, a couple of bugs found, mostly going very well. The main 10.3 Debian update has been pushed to the mirror network and I've updated two of my machines so far. All good.

Hofstadter's Law states that "It always takes longer than you think, even taking account of Hofstadter's Law" ( the Douglas Hofstadter who wrote Goedel Escher Bach) and these long days always disappear into the memory until the next time.

There's also an update planned for the images for oldstable - so we'll be busy for a fair while yet.

UPDATE: The release looks good: some CDs for the smaller arches remain to be built but it's done for now.

Oldstable testing will happen tomorrow

As has become traditional - another blog post round about CD release time

Just waiting to start: Isy, Sledge, RattusRattus and Schweer are all standing by on IRC

Debian 10.3 update is out - the CDs will begin being built and tested this weekend. Would much rather be in Cambridge, as ever :)

Also dotting between various machines doing general updates and stuff: this is being written from my newer laptop and is the first update from this one.
As ever, it should be fun.

Sunday, 2 February 2020

Rebuilding a mirror - Fedora EPEL mirroring and Apache config

This follows on from the previous post: Fedora does things differently. EPEL is a sub-project: software packaged for CentOS/Red Hat Enterprise Linux by the Fedora Project maintainers.

Mirroring Fedora appears to be unusual if you're not a major (Tier 1) mirror.

The Fedora Wiki page at https://fedoraproject.org/wiki/Infrastructure/Mirroring
is very useful.

Public mirrors are required to have minimum bandwidth and, ideally, to include everything. Fedora is large: fedora-buffet is the whole thing under /pub to download for those who have many TB to spare. fedora-enchilada is everything under /pub/fedora (including the Fedora distribution itself), fedora-epel is EPEL alone.

Public mirrors are required to register with the Fedora Account System [FAS]: those undertaking private mirroring are requested to mirror from Tier1 mirrors or below. The script below relies on mirrors which include fedora-buffet. These are used to achieve rsync speed-ups (by accessing cached bundles of timestamps to speed up differential transfers based on the time files changed).

Since I'm not running a public mirror and not reporting back to the public mirror network stats using mirror-manager, I needed to find a Tier1 mirror that would permit me to mirror privately and my nearest is Hoch Schule Esslingen at ftp-stud.hs-esslingen.de

The suggested mirroring script is quick-fedora-mirror available from Fedora's git repositories at https://pagure.io/quick-fedora-mirror This is licensed under Creative Commons CC0.

The script is written in zsh (with features not included in bash or other shells) and Python3 so I had to do a quick apt-get install zsh. There is a quick-fedora-mirror.conf.dist file included as a template for editing. The suggestion for the completed quick-fedora-mirror.conf file is to move it to /etc

The script requires space somewhere to store a couple of files: timefile and timefile.prev which record the times that the script was last run. I found it easier to just put these into /home/mirror and I copied the quick-fedora-mirror script itself into /usr/local/bin/

One easy mistake to make: this script should be configured to write into the top level directory - so into /srv/mirrors rather than /srv/mirrors/epel or you end up with /srv/mirrors/epel/epel

Apache2 configuration

This was relatively easy - set the document root to /srv/mirrors by uncommenting the relevant lines for /srv in /etc/apache2/apache.conf and amend the /etc/apache2/000-default to show /srv/mirrors as the new DocumentRoot

This results in the six separate directories all being appropriately served under http://localhost as /centos, /debian, /debian-cd, /epel, /ubuntu and /ubuntu-releases.


Rebuilding a mirror - software mirroring of Linux distributions for fun

... and also because sometimes it's just easier to have a local cache

I've a mirror machine sitting next door: it's a private mirror of CentOS, Debian, Debian CDs, EPEL [extra packages for Enterprise Linux from Fedora], Ubuntu releases, and Ubuntu CDs. I've run this intermittently for a few years for myself: it's grown recently because I added CentOS and EPEL.

A 6TB LVM volume is now at about 62% full, so there's plenty of space.

Layout

My machine has these mirrors placed under /srv. The mirrors directory is root owned, the individual directories are owned by a non-root user which also runs the scripts. So you end up with /srv/mirrors/debian or /srv/mirrors/ubuntureleases, for example, where the final directory is owned by a mirror user and the mirror user owns the scripts and the crontab that runs them.

Package used for mirroring

The ftpsync Debian package is pretty much all you need: each individual download source has its own ftpsync.conf script which sits under /etc/ftpsync

The configuration files are essentially identical for Debian and Ubuntu - the only difference is in the source from which you mirror.

Configuring the ftpsync.conf script

The changes needed from the example configuration script in /usr/share/doc/ftpsync are minimal and straightforward: edit The TO directory and the source from which you mirror.

My ftpsync-debian.conf has the following lines changed from the supplied config file, for example.

TO="/srv/mirrors/debian/"

RSYNC_HOST="debian.hands.com"
RSYNC_PATH="debian"

Calling and usage:


Calling is simple: ftpsync sync:archive:debian which references the ftpsync-debian.conf above. Each ftpsync call must have a corresponding configuration file or the ftpsync script complains.

Usage is from mirror's crontab - excerpt below

13 03 * * * /usr/bin/ftpsync sync:archive:debian
37 03 * * * /usr/bin/ftpsync sync:archive:centos


Hope this helps someone: will add a couple of bits on configuring Apache to serve this in another post.

[EDIT - changed rsync host listed above at request of jcristau]