MiniDebConf Cambridge - 26th November 2023 - Afternoon sessions

That's all folks ...

Sadly, nothing too much to report.

I delivered a very quick three slides lightning talk on Accessibility, WCAG [Web Content Accessibility Guidelines] version 2.2 and a request for Debian to do better

WCAG 2.2:  WCAG 2.2 Abstract

Debian-accessibility mailing list link: debian-accessibility

I watched the other lightning talks but then left at 1500 - missing three good talks - to drive home at least partly in daylight.

A great four days - the chance to put some names to faces and to recharge in Debian spaces.

Thanks to all involved and especially ...

Thanks to Cambridge Debian folk for helping arrange evening meals, lifts and so on and especially to those who also happen be ARM employees who were badging us in and out through the four days

Thanks to those who staffed Front Desk on both days and, especially, also to the ARM security guards who let us into site at 0745 on all four days and to Mark who did the weekend shift inside the building for Saturday and Sunday.

Thanks to ARM for  excellent facilities, food, coffee, hosting us and coffee, to Codethink for sponsoring - and a lecture from Sudip and some interesting hardware - and Pexip for Pexip sponsorship (and employee attendance). 

Here's to the next opportunity, whenever that may be.

Back at ARM for MiniDebConf day 2 - Morning sessions 26th November 2023

 Quick recap of slides and safety information for the day from Steve McIntyre

Now into the Release Team questions following a release team overview.

A roomful of people all asking questions which are focused and provoke more questions - how unlike a Debian session :)

May just have talked myself into giving a lightning talk this afternoon :)

Now about to have a talk about from Sudip about OpenQA, kernel testing and automation

Afternoon talks - MiniDebConf ARM Cambridge - Day 1

A great talk on SteamOS progress to effective boot loaders for atomic OS updates.

How to produce something that will allow instant updates and instant fallbacks when updating a whole OS image - lots of explanation - and it's good when three or four people who are directly interested in problems and solutions round, for example, Secure Boot are in the room.

Jessica Clarke on CHERI, Morello and security protections in hardware, software and programming hardware which has verifiable pointers and routines. A couple of flourishes which had the room breaking out in applause.

Roberto Sanchez and Santiago Rincon on suggestions for LTS and ways forward. The presentation very clearly set out what LTS is, is not, and maybe should be.

Last presentation of the day was from Ian Jackson on a potential change to git based working and tagging. 

Then lots of chasing around to get people out of the building. Thanks very much to the Arm personnel, especially the security staff who have been helpful throughout the day with getting us all in and out

Thanks to all involved with Arm, Codethink and Pexip for hosting and sponsorship without which this would not have been possible.

 

Lightning talks - MiniDebConf ARM Cambridge - Day 1

 A quick one slide presentation from Helmut on how to use Debian without sudo - Sudo Apt Purge Sudo

A presentation on upcoming Ph.D research on Digital Obsolescence - from Eda

Antarctic and Arctic research from Carlos Pina i Estany 

* Amazing * what you can get into three well chosen slides.

Ten minutes until the afternoon's talks

Laptop with ARM, mobile phone BoF - MiniDebConf Cambridge day 1

 So following Emanuele's talk on a Lenovo X13s, we're now at the Debian on Mobile BoF (Birds of a feather) discussion session from Arnaud Ferraris

Discussion and questions on how best to support many variants of mobile phones: the short answer seems to be "it's still *hard* - too many devices around to add individual tweaks for every phone and manufacturer.

One thing that may not have been audible in the video soundtrack - lots of laughter in the room prompted as someone's device said, audibly "You are not allowed to do that without unlocking your device"

Upstream and downstream packages for hardware enablement are also hard: basic support is sometimes easy but that might even include non-support for charging, for example.

Much discussion around the numbers of kernels and kernel image proliferation there could be. Debian tends to prefer *one* way of doing things with kernels.

Abstracting hardware is the hardest thing but leads to huge kernels - there's no easy trade-off. Simple/feasible in multiple end user devices/supportable  - pick one ☺

ARM lecture theatre - MiniDebConf Cambridge day 1

And we're here - a couple of lectures in. Welcome from one Steve, deep internals of ARM from another Steve. A room filling with people - and now a lecture I really need to listen to on a machine I'd like to own.

 As ever, the hallway track is interesting - and you find people who know you from IRC or mailing lists. 

Four screens and a lecture theatre layout. Here we go.

Video team doing a great job, as ever - and our brand new talkmeister is doing a sterling job. 

Mini-DebCamp ARM Cambridge day 2

 Another really good day at ARM. Still lots of coffee and good food - supplemented by a cooked breakfast if you were early enough :)

Lots of small groups of people working earnestly in the main lecture theatre and a couple of meeting rooms and the soft seating area: various folk arriving  ready for tomorrow. Video team setting up in the afternoon and running up servers and cabling - all ready for a full schedule tomorrow and Sunday.

Many thanks to our sponsors - and especially the helpful staff at ARM who were helping us in and out, sorting out meeting rooms and generally coping with a Debian invasion. More people tomorrow for the weekend.

20231123 - UEFI install on a Raspberry Pi 4 - step by step instructions to a modified d-i

Motivation

 Andy (RattusRattus) and I have been formalising instructions for using Pete Batard's version of Tianocore (and therefore UEFI booting) for the Raspberry Pi 4 together with a Debian arm64 netinst to make a modified Debian installer on a USB stick which "just works" for a Raspberry Pi 4.

Thanks also to Steve McIntyre for initial notes that got this working for us and also to Emmanuele Rocca for putting up some useful instructions for copying.

Recipe

Plug in a USB stick - use dmesg or your favourite method to see how it is identified.

Make a couple of mount points under /mnt - /mnt/data and /mnt/cdrom

1. Grab a USB stick, Partition using MBR. Make a single VFAT
   partition, type 0xEF (i.e. EFI System Partition)

For a USB stick (identified as sdX) below:

$ sudo parted --script /dev/sdX mklabel msdos 

$ sudo parted --script /dev/sdX mkpart primary fat32 0% 100% 

$ sudo mkfs.vfat /dev/sdX1 $ sudo mount /dev/sdX1 /mnt/data/

Download an arm64 netinst.iso

https://cdimage.debian.org/debian-cd/current/arm64/iso-cd/debian-12.2.0-arm64-netinst.iso

2. Copy the complete contents of partition *1* from a Debian arm64
   installer image into the filesystem (partition 1 is the installer
   stuff itself) on the USB stick, in /

$ sudo kpartx -v -a debian-12.2.0-arm64-netinst.iso 

# Mount the first partition on the ISO and copy its contents to the stick 

$ sudo mount /dev/mapper/loop0p1 /mnt/cdrom/ 

$ sudo rsync -av /mnt/cdrom/ /mnt/data/ 

$ sudo umount /mnt/cdrom

3. Copy the complete contents of partition *2* from that Debian arm64
  installer image into that filesystem (partition 2 is the ESP) on
   the USB stick, in /

# Same story with the second partition on the ISO

$ sudo mount /dev/mapper/loop0p2 /mnt/cdrom/

$ sudo rsync -av /mnt/cdrom/ /mnt/data/ $ sudo umount /mnt/cdrom

$ sudo kpartx -d debian-testing-amd64-netinst.iso $ sudo umount /mnt/data

4. Grab the rpi edk2 build from https://github.com/pftf/RPi4/releases
   (I used 1.35) and extract it. I copied the files there into *2*
   places for now on the USB stick:

   /      (so the Pi will boot using it)
   /rpi4  (so we can find the files again later)

5. Add the preseed.cfg file (attached) into *both* of the two initrd
   files on the USB stick

   - /install.a64/initrd.gz and
   - /install.a64/gtk/initrd.gz

   cpio is an awful tool to use :-(. In each case:

$ cp /path/to/initrd.gz .
$ gunzip initrd.gz
$ echo preseed.cfg | cpio -H newc -o -A -F initrd 

$ gzip -9v initrd

$ cp initrd.gz /path/to/initrd.gz

If you look at the preseed file, it will do a few things:

   - Use an early_command to unmount /media (to work around Debian bug
     #1051964)

   - Register a late_command call for /cdrom/finish-rpi (the next
     file - see below) to run at the end of the installation.

   - Force grub installation also to the EFI removable media path,
     needed as the rpi doesn't store EFI boot variables.

   - Stop the installer asking for firmware from removable media (as
     the rpi4 will ask for broadcom bluetooth fw that we can't
     ship. Can be ignored safely.)

6. Copy the finish-rpi script (attached) into / on the USB stick. It
   will be run at the end of the installation, triggered via the
   preseed. It does a couple of things:

   - Copy the edk2 firmware files into the ESP on the system that's
     just been installer

   - Remove shim-signed from the installed systems, as there's a bug
     that causes it to fail on rpi4. I need to dig into this to see
     what the issue is.

That's it! Run the installer as normal, all should Just Work (TM).

BlueTooth didn't quite work : raspberrypi-firmware didn't install until adding a symlink for boot/efi to /boot/firmware

20231127 - This may not be necessary because raspberrypi-firmware path has been fixed

Preseed.cfg

# The preseed file itself causes a problem - the installer medium is
# left mounted on /medis so things break in cdrom-detect. Let's see if
# we can fix that!
d-i preseed/early_command string umount /media || true

# Run our command to do rpi setup before reboot
d-i preseed/late_command string /cdrom/finish-rpi

# Force grub installation to the RM path
grub-efi-arm64  grub2/force_efi_extra_removable boolean true

# Don't prompt for missing firmware from removable media,
# e.g. broadcom bluetooth on the rpi.
d-i hw-detect/load_firmware boolean false

Finish.rpi

 !/bin/sh

set -x

grep -q -a RPI4 /sys/firmware/acpi/tables/CSRT
if [ $? -ne 0 ]; then
    echo "Not running on a Pi 4, exit!"
    exit 0
fi

# Copy the rpi4 firmware binaries onto the installed system.
# Assumes the installer media is mounted on /cdrom.
cp -vr /cdrom/rpi4/. /target/boot/efi/.

# shim-signed doesn't seem happy on rpi4, so remove it
mount --bind /sys /target/sys
mount --bind /proc /target/proc
mount --bind /dev /target/dev

in-target apt-get remove --purge --autoremove -y shim-signed

Arm Cambridge - mini-Debcamp 23 November 2023

 At Arm for two days before the mini-Debconf this weekend.

First time at Arm for a few years: huge new buildings, shiny lecture theatre.

Arm have made us very welcome. A superb buffet lunch and unlimited coffee plus soft drinks - I think they know what Debian folk are like.
Not enough power blocks laid out at the beginning - only one per table - but we soon fixed that 😀

The room is full of Debian folk: some I know, some new faces. Reminiscing about meeting some of them from 25 years ago - and the chance to thank people for help over a long time.

Andy (RattusRattus) and I have been working out the bugs on an install script using UEFI for a Raspberry Pi 4. More on that in the next post, maybe.

As ever, it's the sort of place where "I can't get into the wiki" is sorted by walking three metres across the room or where an "I can't find where to get X for Raspberry Pi" can be solved by asking the person who builds Raspbian. "Did you try and sign up to the Debian wiki last week - you didn't follow the instructions to mail wiki@ - I _know_ you didn't because I didn't see the mail ... "

My kind of place and my kind of people, as ever.

Thanks again to Arm who are one of our primary sponsors for this mini-Debconf.

Point release weekend for Debian: two releases this weekend: 202311071653

Over in Cambridge with RattusRattus, Sledge, egw and Isy. Andy is very kindly putting us up.

We're almost all of the way through testing 12.2 and some of the way through testing 11.8.

It's a LONG day - heads down into laptops and relatively quiet - I think we're all tired and we've a way to go yet.

Building a mirror of various Red Hat oriented "stuff"

Building a mirror for rpm-based distributions.

I've already described in brief how I built a mirror that currently mirrors Debian and Ubuntu on a daily basis. That was relatively straightforward given that I know how to install Debian and configure a basic system without a GUI and the ftpsync scripts are well maintained, I can pull some archives and get one pushed to me such that I've always got up to date copies of Debian and Ubuntu.

I wanted to do something similar using Rocky Linux to pull in archives for Almalinux, Rocky Linux, CentOS, CentOS Stream and (optionally) Fedora.

(This was originally set up using Red Hat Enterprise Linux on a developer's subscription and rebuilt using Rocky Linux so that the machine could be passed on to someone else if necessary. Red Hat 9.1 has moved to x86_64v2 - on the machine I have (HP Microserver gen 8) 9.1 it fails immediately. It has been rebuilt to use Rocky 8.8).

This is a minimal install of Rocky as console only - the machine it's on only has 4G of memory so won't run a GUI reliably. It will run Cockpit so can  be remotely administered. One user to run everything - mirror.

Minimal install of Rocky 8.7 from DVD .iso. SELinux is enabled, SSH works for remote access. SELinux had to be tweaked to allow /srv/ the appropriate permissions to be served by nginx. /srv is a large LVM volume rather than a RAID 6 - I didn't have enough disks

Adding nginx, enabling Cockpit and editing the Rocky Linux mirroring scripts resulted in something straightforward to reproduce.

nginx

I cheated and stole large parts of my Debian config. The crucial part to remember is that there is no autoindexing by default and I had to dig to find the correct configuration snippet.

  # Load configuration files for the default server block.

        include /etc/nginx/default.d/*.conf;

        location / {
                autoindex on;
                autoindex_exact_size off;
                autoindex_format html;
                autoindex_localtime off;
                # First attempt to serve request as file, then
                # as directory, then fall back to displaying a 404.
                try_files $uri $uri/ =404;
        }

 Rocky Linux mirroring scripts

Systemd unit file for service

[Unit]
Description=Rocky Linux Mirroring script

[Service]
Type=simple
User=mirror
Group=mirror
ExecStart=/usr/local/bin/rockylinux

[Install]
WantedBy=multi-user.target

 Rocky linux  timer file

[Unit]
Description=Run Rocky Linux mirroring script daily

[Timer]
OnCalendar=*-*-* 08:13:00
OnCalendar=*-*-* 22:13:00
Persistent=true

[Install]
WantedBy=timers.target

Mirror script

#!/bin/env bash
#
# mirrorsync - Synchronize a Rocky Linux mirror
# By: Dennis Koerner <koerner@netzwerge.de>
#
# The latest version of this script can be found at:
# https://github.com/rocky-linux/rocky-tools
#
# Please read https://docs.rockylinux.org/en/rocky/8/guides/add_mirror_manager
# for further information on setting up a Rocky mirror.
#
# Copyright (c) 2021 Rocky Enterprise Software Foundation

This is a very long script in total.

Crucial parts I changed only listed the mirror to pull from and the place to put it.

# A complete list of mirrors can be found at
# https://mirrors.rockylinux.org/mirrormanager/mirrors/Rocky
src="mirrors.vinters.com::rocky"

# Your local path. Change to whatever fits your system.
# $mirrormodule is also used in syslog output.
mirrormodule="rocky-linux"
dst="/srv/${mirrormodule}"

filelistfile="fullfiletimelist-rocky"
lockfile="/home/mirror/rocky.lockfile"
logfile="/home/mirror/rocky.log"

 Logfile looks something like this: the single time spec file is used to check whether another rsync needs to be run

deleting 9.1/plus/x86_64/os/repodata/3585b8b5-90e0-4856-9df2-95f646bc62c7-PRIMARY.xml.gz

sent 606,565 bytes  received 38,808,194,155 bytes  44,839,746.64 bytes/sec
total size is 1,072,593,052,385  speedup is 27.64
End: Fri 27 Jan 2023 08:27:49 GMT
fullfiletimelist-rocky unchanged. Not updating at Fri 27 Jan 2023 22:13:16 GMT
fullfiletimelist-rocky unchanged. Not updating at Sat 28 Jan 2023 08:13:16 GMT

It was essentially easier to store fullfiletimelist-rocky in /home/mirror than anywhere else.

Very similar small modifications to the Rocky mirroring scripts  were used to mirror the other distributions I'm mirroring. (Almalinux, CentOS, CentOS Stream, EPEL and Rocky Linux).

 

 

 

20230828 - OMGWTFBBQ - Breakfast is happening more or less

 And nothing changes: rediscovered from past Andrew at his first Cambridge BBQ and almost the first blog post here:

"House full of people I knew only from email, some very old friends. Wires and leads filling the front room floor - laptops _everywhere_ .

...

Thirty second rule on sofa space - if you left for more than about 30 seconds you had to sit on the floor when you got back (I jammed myself onto a corner of the sofa once I realised I'd barely get through the crush :) )
[Forget students in a mini / UK telephone box - how many DDs can you fit into a very narrow kitchen :) ]

It's a huge, dysfunctional family with its own rules, geeky humour and in-jokes but it's MINE - it's the people I want to hang out with and, as perverse as it sounds, just being there gave me a whole new reaffirmed sense of identity and a large amount of determination to carry on "wasting my time with Linux" and Debian"

The *frightening* thing - this is from August 31st 2009 ... where have the years gone in between.

20230826 - OMGWTFBBQ - BBQ still in full swing

 There's been a very successful barbeque running in the garden: burgers, sausages, beer, vegetarian dishes and then ice cream.

The chance to catch up with people you only meet in IRC. Talking and laughter - and probably a couple of games of Mao.

Thanks also to our sponsors - Collabora, Codethink and RattusRattus for contributions to food and drink.

20230826 OMGWTFBBQ - Cambridge is waking up

 The meat has been fetched: those of us in the house are about to get bacon sandwiches. Pepper the dog is in the garden. Time for the mayhem to start, I think.

Various folk are travelling here so it will soon be crowded: the weather is sunny but cool and it looks good for a three day weekend.

This is a huge effort that falls to Steve and Jo and a huge disruption for them each year - for which many thanks, as ever. [And, as is traditional on this blog, the posts only ever seem to appear from Cambridge].

20230722 1804 UTC - All signed, all pushed, all done.

So it's there on  Download on the Debian web site.

Thanks to the folk who've done this point release testing. No new major bugs found: a couple of pre-existing ones may still be there.

Thanks very much indeed to the new people who have been in IRC on debian-cd, downloading, testing, editing the wiki page.

This release has gone very well indeed - I'll see some of the same folk at the BBQ in late August but otherwise we'll be back in September or so for the next point release for Bookworm (and probably a release for Bullseye as well).

Now trying various tricks with Raspberry Pi 4s to get them to boot with UEFI and the standard Debian installer.

20230722 1436 - Happily chunking through release tests

Lots of tests being tried and passing. The usual bits of finger trouble / misreading the wiki.

 People are settling into the rhythm. Almost all of the tests for the standard .iso files are done and we're well into testing live images.

 All's good

20230722 - Releasing Debian testing for Debian Bookworm point release 12.1

And so we're back at Steve's in Cambridge for release testing.

 A few testers here: we're now well into testing the various iso images.

So we have Sledge, RattusRattus, Isy, smcv and myself. We've also been joined here by Helen who has just done her first install.  

Online we've got a couple of new folk - luna, The_Blode. Thanks to everyone involved - we always need all the help we can get.

Cold and grey outside: usual warmth in here.  Also the usual snake of cables to trip over across the floor.

 

 

202306101949 - Release of install media - scripts running now

People are working quietly, cross-checking, reading back steps and running individual steps - we're really almost there for the install media.

Just had a friendly, humorous meal out by the barbeque in Sledge's garden. It's been quite a long day but we're just finished.

All this and then we'll probably have the first point release for Bookworm 12.1 in about a month. That will contain some few fixes which came in at the last minute and any other issues we've found today.

BOOKWORM IS HERE!!

202306101353 - Release testing of media in full swing

 Most of the install images for Debian media have now been tested.

Various folk are now testing the live media.

We have been joined by a couple of people in IRC who have also done a few tests.

Useful things to note :)

The release name is Bookworm *not* Bookwork.

Debian 13 will be Trixie when it gets here: testing will be re-enabled shortly.

The release notes detail the changes in /etc/apt/sources.list to accommodate the changes to non-free-firmware but also see also Sources List on the Debian wiki.

202306101010 - Debian release preparations and boot media testing in Cambridge

 We've all met up in Cambridge - so there's an egw_, amacater, kibi who has travelled over to join us, Isy, RattusRattus and Sledge mostly sat round a table. The usual number of laptops, three monitors, Rattus' tower machine.

Network running well and we're all ready to go, I think - there's normally a flurry of activity to get things started then a wait for a while for the first images

Coffee and tea at the ready - bacon sandwiches are on the way

[And the build process is under way - and smcv has joined us]

And it's now after 2100 - so the unexpurgated version

 We're all but done - a couple of bugs sorted. All testing complete.

Release testing for 11.7 - pictures (with appropriate identity protection)

 

Debian Bullseye 11.7 release - testing going on - 202304291427

 We've been joined by Simon (smcv) - lots of chat bouncing backwards and forwards. Laptops appearing out of backpacks suddenly being repurposed.

Settling very much into a rhythm and routine.

Working with two laptops on your lap ends up being quite heavy :)

Debian Bullseye 11.7 release weekend 202304291215UTC

 A switch failure early on: some quick changes of wiring and we're off. The room is very quiet for a few minutes then a burst of chatter, then on we go.

Just the noise of keyboards and quiet concentration

Back in Cambridge - Debian point release for Debian Bullseye due this weekend - 11.7

Back in Cambridge for a point release weekend. Lots of people turning up - it comes to something when large monitors, a desktop machine require two or three trips to the car and there's still a crate of leads to go.

As ever, lots of banter - computer renovations and updates were done yesterday - if they hadn't been, I'd have had at least another expert engineer on hand.

This is *definitely* the place to be rather than at the other end of an IRC chat.

This is *not* the release for Debian Bookworm: that will be on June 10th/11th - likely to be the same personnel, same place. There was a release of Debian Bookworm RC2  media yesterday incorporating fixes up to date.