• About
• Latest Posts

Purism

Beautiful, Secure, Privacy-Respecting Laptops, Tablets, PCs, and Phones

Latest posts by Purism (see all)

• PureOS Crimson Development Report: October 2024 - November 14, 2024
• PureOS Crimson Development Report: September 2024 - October 16, 2024
• Quantum Safe Communication with Purism - September 23, 2024

I’m convinced that the Librem 14 is the best laptop for Qubes and our customers seem to agree. Originally, customers who selected Qubes with their order would have to install it themselves with a USB thumb drive we added to the order. More recently we started offering Qubes as a pre-installed option, all set up on the computer before it ships. This required a number of changes to the Qubes OEM project Nitrokey created for their own use so that we could allow customers to reset their disk unlock passphrase on first boot.

Originally we shipped Qubes 4.0.4 and when Qubes 4.1 was released, many of our customers asked when we would switch to the newer version. Updating our OEM installer for Qubes 4.1 took a lot more effort than we expected, but we are happy to announce that our Qubes 4.1 OEM installer is now complete and we are starting to use it for new orders. This is a project I worked on personally, and in this blog post I’ll do a brief technical dive into what’s involved in making the Qubes OEM installer and why upgrading to Qubes 4.1 was more complicated than expected.

Qubes OEM Architecture

We maintain our Qubes OEM installer in our Purism code repository as a fork of Nitrokey’s own OEM project. The base project takes advantage of the fact that Qubes uses Fedora’s Anaconda installation software (as dom0 is based on Fedora), and automates a default install using the standard Kickstart tool. Kickstart works by providing the installer a configuration file that contains answers to all the questions the installer asks during an interactive install. Kickstart has been used for years by sysadmin to automate server installs and its syntax is well-documented and behavior is relatively well-defined.

The OEM installer takes the official Qubes ISO and copies its files to a new ISO that adds the Kickstart configuration, supporting files the installer needs, and a modified boot config that activates Kickstart. The base Nitrokey project was mostly focused on automating the install itself so it could run unattended (handy for OEMs like Nitrokey and Purism) as well as pre-install a few Nitrokey tools. This base installer enabled disk encryption, but with a pre-set “changeme” style password, presumably with the understanding that the end-user would be able to open a command line and change the LUKS passphrase themselves when they booted the computer.

Before we pre-installed Qubes on Purism computers, I wanted the process of changing the LUKS passphrase to be simpler for our customers, and closer to how we handle it in PureOS. In PureOS, a first boot wizard walks the user through setting up their account, setting their password, and setting a disk encryption passphrase all with an easy-to-use GUI. Qubes actually has a similar first-boot wizard it uses to set a user account and Qubes-specific settings already, but it didn’t handle disk encryption.

To add support for changing disk encryption passphrases to the Qubes 4.0.4 install, I reviewed the Anaconda scripts (written in Python, appropriately enough) and decided that while I could write an entirely new Anaconda module for this purpose, the easiest approach would be to modify the existing user module instead. Due to how Anaconda organizes things, this meant making changes to quite a few different files because the layout of these windows is defined separately from the script, and the script that performs the actions the user module sets is separate from the user module itself.

The final touches were to add my modified versions of the files to the ISO itself, and add a few commands to the Kickstart post-install section to copy those files over the top of the default versions. That way when the Qubes first boot script launched the Anaconda user module, it would launch my modified version, which would prompt the user for their disk passphrase alongside their user password. You can see the full set of changes in my pull request to the upstream Nitrokey project. Now when a customer orders Qubes as their OS, at first boot they get presented with a prompt to enter their disk password, they press Enter (the default password is blank), and then they move to the first boot wizard to replace that blank password with one of their choosing.

Qubes 4.1

When Qubes 4.1 was first released, we immediately had some customers ask if their laptop would get it by default. It made sense because this release has a number of nice new features and new versions of its OS templates. I have been using Qubes for many years and sometimes in the past, major point releases like this have suffered from stability issues the first few weeks after release that result in a quick point release. It’s no fault of Qubes as they have a rigorous release candidate process, it’s simply that a wider set of users would be testing the official stable release and possibly finding new bugs. Even though we had been testing the Qubes 4.1 release candidates and they installed fine on our Librem 14, I decided before we shipped it to our customers, I would take some downtime on my work laptop and do a proper backup-full install-restore process and test Qubes 4.1 as my daily driver for a few weeks.

It didn’t take long for me to appreciate the many improvements in Qubes 4.1. It wasn’t any particular large change, but more that the combination of newer software across the board, and a number of small improvements all added up to make a great experience, especially on a laptop with CPU and RAM to spare like the Librem 14. It is definitely an improvement over 4.0.4.

Fighting an Anaconda

After a week or two of testing without hitting any issues, and seeing that the public release was also going smoothly, I decided the time was right to switch customers over to Qubes 4.1. Nitrokey had already updated their own repo with only a few minimal Kickstart-related changes, so I figured it would just be a matter of inspecting any changes they made to the Kickstart file, adapting them to mine if needed and then rebuilding. I was wrong.

The automated install itself worked fine, but the problem came at first boot, where the first boot wizard would crash and leave me at a login prompt with no password set. This is something I had run into in the past and typically meant an error somewhere in the Anaconda scripts. Since I couldn’t login as a user (no password!) I had to reboot into a rescue disk and mount the Qubes root disk to pore through logs for hints at what could be wrong. A lot was wrong, it turns out.

The Qubes 4.1 upgrade also upgraded the base Fedora OS used for dom0 to Fedora 32. It so happens that this update also brought along a rather significant change to Anaconda as it transitioned to Python 3.8. While the overall location of the various scripts was mostly intact (but not entirely!), the scripts themselves had been completely rewritten in most cases, to the point that functions, variables, and most other major parts of the script were different, even if the underlying processes the scripts performed were mostly the same.

It was clear at this point that my existing scripts simply would not work. Instead, I would need to adapt the changes I made to the previous scripts to this complete redesign. Ultimately this meant making copies of all of the 4.0.4 and 4.1 Anaconda scripts, performing a diff of my scripts with the 4.0.4 versions, and then painstakingly identifying how to adapt those changes to these new scripts, one script at a time. In a few cases the previous files were no longer present, either having been moved to a new location or removed entirely and replaced with logic in a new script.

After quite a bit of detective work, and a lot of failed Qubes installs, I was ultimately able to adapt all of my changes to the new layout and build a working OEM install for Qubes 4.1 that allows you to change your disk encryption passphrase at boot. You can see the complete set of my changes here. If you just want to download and test out the OEM install yourself, you can find the latest versions of that here. Just note that this is designed for OEM use. Once booted, it will immediately erase anything on the disk and replace it with a fresh Qubes install.

I’m excited that our customers will be able to use Qubes 4.1 right out of the box when they order Qubes with their hardware. I’ve been using it on my own Librem 14 for quite some time now without issue, and can’t wait for everyone else to try it out.

Purism Products and Availability Chart

	Model	Status	Lead Time
	Librem Key (Made in USA)	In Stock ($59+)	10 business days	Learn More Buy Now
	Librem 5	In Stock ($699+) 3GB/32GB	10 business days	Learn More Buy Now
	Librem 5 COMSEC Bundle	In Stock ($1299+) Qty 2; 3GB/32GB	10 business days	Learn More Buy Now
	Liberty Phone (Made in USA Electronics)	Backorder ($1,999+) 4GB/128GB	Estimated fulfillment February	Learn More Buy Now
	Librem 5 + SIMple (3 GB Data)	In Stock ($99/mo)	10 business days	Learn More Buy Now
	Librem 5 + SIMple Plus (5 GB Data)	In Stock ($129/mo)	10 business days	Learn More Buy Now
	Librem 5 + AweSIM (Unlimited Data)	In Stock ($169/mo)	10 business days	Learn More Buy Now
	Librem 11	In Stock ($999+) 8GB/1TB	10 business days	Learn More Buy Now
	Librem 14	Backorder ($1,370+)	Estimated fulfillment December	Learn More Buy Now
	Librem Mini	Backorder ($799+)	10 business days	Learn More Buy Now
	Librem Server	In Stock ($2,999+)	45 business days	Learn More Buy Now