Tag: Advanced readers

In-depth technical news/tutorials, aimed at advanced readers

Coreboot and Skylake, part 2: A Beautiful Game!

Hi everyone,

While most of you are probably excited about the possibilities of the recently announced “Librem 5” phone, today I am sharing a technical progress report about our existing laptops, particularly findings about getting coreboot to be “production-ready” on the Skylake-based Librem 13 and 15, where you will see one of the primary reasons we experienced a delay in shipping last month (and how we solved the issue).

TL;DR: Shortly we began shipping from inventory the coreboot port was considered done, but we found some weird SATA issues at the last minute, and those needed to be fixed before shipping those orders.

  • The bug was sometimes preventing booting any operating system, which is why it became a blocker for shipments.
  • I didn’t find the “perfect” fix yet, I simply worked around the problem; the workaround corrects the behavior without any major consequences for users, other than warnings showing up during boot with the Linux kernel, which allowed us to resume shipments.
  • Once I come up with the proper/perfect fix, an update will be made available for users to update their coreboot install post-facto. So, for now, do not worry if you see ATA errors during boot (or in dmesg) in your new Librem laptops shipped this summer: it is normal, harmless, and hopefully will be fixed soon.

Read more

A shipping update for Qubes-related orders

In light of our shipping-from-inventory fulfillment model having recently begun and in order to answer questions we have been getting related to pre-loaded versions of Qubes OS, we would like to announce that we will be fulfilling Qubes-related pending orders made until now by offering the Qubes install as a complimentary USB key included at no additional charge in those shipments.

  • Orders made with Qubes OS as the chosen operating system are expected to be fulfilled using the USB key accessory within approximately two weeks. Although the technical method of delivery is different, the $100 licensing fee Purism pays to Qubes will still be honored, for all Qubes-related orders made until August 2nd, 2017.
  • Should you prefer to forego the USB key accessory (containing the latest publicly available Qubes OS installer), please contact ops@puri.sm to adjust your order, which will then begin shipment procedures immediately alongside PureOS-related orders.
  • We apologize for not being able to pre-load Qubes OS onto your machine’s main solid state storage drive, as we have been unable to obtain an automated OEM image (suitable for secure and efficient mass-preloading in our fulfillment center) through our existing agreement with the Qubes Project/Invisible Things Lab s.c.

The current “free Qubes installer USB key” offer will also remain valid for orders placed between today (July 4th) and August 2nd, 2017, at which point our agreement with Qubes shall no longer be in effect. After this date, we may, at our discretion and as a convenience option for customers, offer a USB key accessory containing either a customer-requested free operating system (subject to availability), or a blank USB key, for a minor fee covering the cost of materials and labor.

Some customers have asked us about the recent removal of the Librem 13 “v1” from the certified hardware page on Qubes website. While we were not notified in advance of this change to the contents of the Qubes website, we understand this de-listing to be due to the fact that the Librem 13 v1 is no longer being manufactured, as it has now been succeeded by the Librem 13 version 2 in 2017. The “version 2” has not been sent through the certification process.

Moreover, while our current and previous hardware has been found to work with Qubes OS, we currently no longer have plans to undergo the official certification process for the Librem 13 v2 nor Librem 15 v3. The costs involved, requiring a supplementary technical consulting contract with Qubes/ITL (as per their new Commercial Hardware Goals proposal document), are not financially justifiable for us.

We have no doubts that Qubes users will be able to independently report their successes with Qubes OS on the Librem laptops within the next few weeks.

We wish Qubes and Invisible Things Lab s.c. the best of luck in establishing commercial partnerships that match their new expectations.

Coreboot on the Librem 13 v2, part 1

Hello everyone! I am very happy to announce that the coreboot port to the Librem 13 v2 as well as the Librem 15 v3 is done! Wow, what an adventure! The entire thing took about 2 weeks of hard work, and an additional week of testing, fixing small issues that kept popping up, and cleaning up the code/commits.

It was truly an adventure, and I would have liked to stop and take the time to write 10 blog posts during that time, one for every major bump in the road or milestone, but I was under a strict deadline because we needed to finish the port before we started shipping the new Librem 13 v2 hardware (from now on referred to as ‘the v2’), so it could be shipping with coreboot pre-installed from day one. Now that the port is finished, I can finally start writing the first chapter in the story.

TL;DR: in the process of porting the Skylake-based Librem 13 v2 to coreboot, I have implemented a new debugging method (“flashconsole”) and added it to coreboot. It has been reviewed and merged upstream. Read more

Your own music studio with JACK, Ardour and Yoshimi

Last week, after flashing coreboot on my Librem 13 (as a beta tester of the new coreboot install script), I came across a few problems with my heavily tweaked PureOS install, so I decided I would do a full, fresh install of PureOS 3.0 beta so my environment would be much closer to what a new user would expect.

While re-installing all my creative environment, I decided that I would do a quick tutorial on installing and using Jack as it is not straight forward and that there are not so many tutorials about it on the Internet.

What is JACK?

JACK stands for “JACK Audio Connection Kit”. It is a free software that lets you handle audio input and output between different applications.

You can see it as a set of audio jacks that you will be able  to plug between different programs.

For example, you can use it to connect a software synthezizer (Yoshimi, ZynAddSubFX) to a multitrack sequencer (Ardour, LMMS).
You can use it to connect an audio editing software (Audacity) to a video editing software (Blender).

Many applications have Jack support. Here is a list from the JACK’s website.

As an example for this tutorial, I will show you how to use Yoshimi with Ardour.

Install the applications

First of all, we need to install all the required applications

sudo apt install qjackctl ardour yoshimi

Enable real time scheduling

Real time scheduling is a feature of all Linux based operating systems that enables an application to meet timing deadlines more reliably. It is also considered to be a potential source of system lock up if your hardware resources are not sufficient so, most of the time, it is not enabled by default.

As mentioned on the JACK’s website, JACK requires real time scheduling privileges for reliable, dropout-free operation.

There is a well detailed tutorial from the JACK’s team that describes how to enable real time scheduling on your system. I will go through the main steps here. It works for me on PureOS but should also work without problem on many other GNU/Linux distributions.

First of all, create a group called “realtime” and add your user to this group (replace USERNAME with your current login) :

sudo groupadd realtime
sudo usermod -a -G realtime USERNAME

You can check that “realtime” is now part of the user’s groups by running the following command :

id USERNAME

Also, make sure that the user is part of the audio group. If not, just add it :

sudo usermod -a -G audio USERNAME

On PureOS (and Debian), you should have a folder called /etc/security/limits.d. If so, just create and edit the file /etc/security/limits.d/99-realtime.conf with your favorite editor. (If you don’t see this folder, you need to edit /etc/security/limits.conf).

Add the following lines and save the file :

@realtime   -  rtprio     99
@realtime   -  memlock    unlimited

You need to logout and login again for the changes to take effect.

WARNING : You should only add new or existing users to the “realtime” group only if an application that they use (like JACK) requires it . By doing so, you give them pretty high privileges to interact with the process priorities, and this may affect the whole usability of the computer.

Run JACK

Before being able to connect anything with JACK, we need to set it up and start its deamon. For that matter, we will use QJackCtl which is a graphical application that controls JACK’s inputs and ouputs.

We will first make sure that JACK is setup correctly. Press the “Setup…” button.

I am not an expert with audio hardware and configurations and this setup is working perfectly on my Librem :

  • Driver: alsa
  • Realtime : yes
  • Interface : hw:PCH
  • Sample Rate : 44100
  • Frames/Period : 128
  • Periods/Buffer : 2

 

 

Save your settings and, on the main QJackCtl controls window, press the “Start” button. After a few seconds, you should see the “Connections” window popping up. This is where all the connections take place.

Connect Yoshimi to Ardour

Now, we are ready to connect our virtual jacks. It is time to open Ardour and create a new session. You should now see a lot more connections in the JACK’s connections window. It shows how Ardour interacts with the system’s audio inputs and outputs.

Let’s add a new track to Ardour. Click the menu “Track”->”Add Track, Bus or VCA…”. Call your new track “Drums” and set it as stereo.

Now you see 2 more Ardour inputs in the JACK’s connections window. They show the name of the audio track that we just created and they are currently connected to the default system’s capture device (the microphone). That is is not what we want so we will disconnect them.

Right click on one of them (Drums/audio_in 1) and chose “Disconnect”. It will disconnect the audio capture device. We will now connect our track to Yoshimi.

Open Yoshimi and wait for it to be fully loaded. You should now see the Yoshimi’s output appear on the JACK’s connections window. In order to connect the Yoshimi’s output to the Ardour’s input, just drag one on top of the other (make sure to respect the vertical order).

 

You are now ready to enjoy your fully operational free software powered professional music studio! 🙂

Please, feel free to comment this post or ask any question in our forums.

Have fun! 😉

RAW footage with Magic Lantern & MLRawViewer

A picture of my post production studio.

Software freedom is amazing! Used with the right hardware, it becomes limitless. Being part of the Purism team as well as the Ethic Cinema project, makes me do a lot of research in term of freedom in visual creations.

Everyday, I realize a bit more, how powerful my free software based studio is when it comes to handling a professional film making workflow. And of course, as a film maker coming from the old school of proprietary technologies, I am so glad to know that now, I am in control.

Getting the best out of your video footage

On a previews post, talking about A/V formats, I said that I didn’t know any camera that lets you be in full control of your footage. Especially if you are on a budget. Most of the time, you will have to deal with footage in a compressed, proprietary format. This can be a problem in a post production workflow because if you re-encode your footage based on an already compressed one, it will start to degrade. If you chose to keep your original footage, you will have to deal with the limitations that come from the proprietary nature of the formats.

This may be true but there is a way to bypass the problem.

RAW files

Thankfully the amazing people from Magic Lantern came to the rescue!

Magic Lantern is a video camera firmware that is released under the GPL license and runs on most Canon DSLRs (Are there some equivalents for other cameras ?). This firmware extends the functionality of the camera and most of all, lets you record your footage as RAW files.

RAW files are brute data coming straight from the sensor. It is like a film negative that would have had no treatment yet.

Based on this RAW file, we are free to export our original footage to the format that we wish. This step is what would happen in the camera in order to generate the footage. The camera would apply your color presets to the RAW data coming from the sensor end encode it to a usable video format. Usually an H.264 format within a MOV container.

With Magic Lantern we have access to the RAW file, so we are in control.

Handling RAW files

Once the RAW file is stored in the computer, it is time to generate our original footage.

MLRawViewer is an amazing free software, made in python and based on FFmpeg. It lets you preview, color grade and encode your RAW footage.

In its latests version (1.4), MLRawViewer only encodes as Apple ProRes (.mov) or Adobe Digital Negative (.dng) formats. Unfortunately, both formats are proprietary, so as part of the Ethic Cinema project, we have decided to contribute to MLRawViewer. We have added the free lossless Huffyuv (.mkv) format to the list as well as the ability to rotate the encoded video. We sometimes film with the camera upside-down when doing camera movements close to the ground with our steadycam, so rotating our footage during this process is very useful.

While our changes are not merged into the original project, you can test it from our repository.

Having the footage being encoded from RAW to a lossless format makes it keep its full quality (which wouldn’t be the case when using the built in H.264 compressed format). Gradients and details are perfectly preserved. It also gives us the ability to use the highest dynamic range available from the camera, or to use a custom LUT (picture style) that would suit our needs.

Please, note that this step is not to be confused with the actual color grading process that takes place at the end of the post production, when the editing is complete. The goal here, is to prepare the footage to allow as much flexibility as possible during the color grading phase. Usually, we chose a very flat picture style at this stage, in order to make sure that we keep as much details as possible from dark to bright.

All in all, the footage we get through this process is at the best possible quality and very close to what one could get from a very high end cinema camera.

This was the missing bit of my workflow. I have now achieved full control and freedom over the whole post production workflow.

Installing MLRawViewer

Note, I have updated this part on 07/04/2017 after noticing some problems with different configurations running python3 along with python2

I plan to make an Appimage build of MLRawViewer, but it is not done yet, so you will have to compile it yourself.

Don’t worry, it is not very difficult and here are the instructions for PureOS and other Debian based systems (It should be very similar on others systems) :

First of all, you need to install git and python, along with pip. (I installed pyaudio with apt because for some reason it failed installing with pip).

Note that you need to install version 2.7 of python as version 3 is not supported by MLRawViewer.

sudo apt install git python2.7 python2.7-dev python-pip python-pyaudio libglfw3-wayland
# Use ‘libglfw3’ if you are not on wayland

Then, you need to install the required dependencies making sure that you use the right versions (which may not be the latests)

pip2 install scandir
pip2 install -I PyOpenGL==3.0.2
pip2 install numpy==1.9.1
pip2 install glfw
pip2 install Pillow==2.1.0

Then, you need to get the source code of MLrawViewer from the Ethic Cinema repository for the updates

git clone https://github.com/ethiccinema/mlrawviewer.git

or from the original repository

git clone https://bitbucket.org/baldand/mlrawviewer.git

It is now time to build the application.

cd mlrawviewer
python2 setup.py build
cp build/lib.linux-x86_64-2.7/bitunpack.so bitunpack.so

And run it

./mlrawviewer.py

Don’t hesitate to ask any question in the forums if you have any trouble  or if you wish me to post any tutorial related to multimedia manipulation with free software. You can use the PureOS area in the forums. I am very happy to help!

 

What’s next?

Well, how cool would it be to shoot with an Axiom camera ? … along with the new Librem 15 v3 !

 

 

Todd’s Purism Librem 13 experience with coreboot and a neutralized ME

A few days ago, I got to experience the efforts of a culmination of free software supporters; from Purism team members, ME hackers, coreboot developers, and a lot of other individuals. I am very pleased to run a Librem 13 with coreboot, running a neutralized Intel Management Engine, and no microcode update. I used that setup to type this blog post! Read more

Neutralizing the Intel Management Engine on Librem Laptops

In my last blog post, I have spoken of the completion of the Purism coreboot port for the Librem 13 v1 and mentioned that I had some good news about the Intel Management Engine disablement efforts (to go further than our existing quarantine) and to “stay tuned” for more information. Since then I got a little side-tracked with some more work on coreboot (more below), but now it’s time to share with you the good news! Read more

The Librem 13 v1 coreboot port is now complete

Here are the news you’ve been waiting for: the coreboot port for the Librem 13 v1 is 100% done! I fixed all of the remaining issues, it is now fully working and is stable, ready for others to enjoy. I fixed the instability problem with the M.2 SATA port, finished running all the tests to ensure coreboot is working correctly, fixed the headphone jack that was not working, made the boot prettier, and started investigating the Intel Management Engine issue. Read on for details. Read more