Category: Miscellaneous

In memory of James M. Rufer

Purism has lost an important voice. We are deeply saddened to learn that our colleague James M. Rufer has passed away on September 27th, 2018, at the age of 39. He is survived by his loving wife Mary; sons Xander and Paxton; mother-in-law Sylvia; and friends he made outside and inside Purism.

James has been with Purism since its early days, being the very first team member to join and has remained a dedicated, honest, hardworking chap who brought enjoyment to our lives daily.

James meeting with us on his last birthday

Purism is a team of people located in all corners of the world, and those of us who have had the privilege to meet James in person know that his human qualities were a model to us all. James was one of the warmest, pure-hearted and kind people you can find, and had a passion for bringing ethical technology within the reach of the general public. I wish he lived on to see the long-term impact of his efforts on society.

François and James casually chatting

James was the customer’s advocate as director of product, and also the cheerful voice of Purism on social media from 2016 to this day. If you have interacted with Purism on Twitter, Mastodon, Google+, Facebook and other outlets, know that in 99% of cases, you had been discussing with James. He was also the main writer behind the “Frequently Asked Questions” area of our website, and the shift in tone in our social media interactions when he took over in 2016 was largely thanks to him and his humane, down-to-earth approach.

In recent months, James had been deeply involved in defining requirements and improvements to our current and future products.

More importantly, James was not just a close collaborator, he had quickly become a personal friend of mine. We shared the joys, the laughter and the burdens of moving mountains to fulfill our mission as a social purpose corporation taking on the world against all odds.

I and various Purism team members will be travelling to attend James’ memorial service this Saturday.

In honor of James, we hope to engrave his name into the interior of the Librem 5 phone. We are working with his close family to support them in these difficult times. Additionally, we are giving James a special send-off and honoring his wife’s request by having James’ remains sent to space in 2019 for his kids. It will be a farewell for him, and he will return to earth and burn up as a “shooting star”.

Goodbye James, you will be sorely missed.
— Jeff

Many purists join me in mourning him, and some here wanted to add below a personal public statement to this effect.


Mladen Pejaković said:

Farewell James, I’ll miss our meetings and the fun and laugh we had. Rest in peace, dear colleague.

David Seaward said:

I finally got to meet James in person this year after working with him remotely. I had always admired his affability, warmth and respect for people, but was really blown away when I got to experience those qualities from him first-hand. I remember how lucky I felt to get to know such a great guy better. How lucky I felt that there was someone at work so capable of bringing those attributes which are all too often missing in our industry. I wish I could preserve that feeling and that memory in a bottle and leave it here for his children so that they could know what a humble and humane man their father was. I hope his family and friends know how much his colleagues appreciated him.

Everyone in our finance team said:

All is silent with James. Rest in peace, brother. Your colleagues here are dedicated to carrying our shared ideals to the four corners of the globe. A tribute to you and the life you led…

Todd Weaver said:

The universe is less bright without James in it and I will miss him immensely. He was a funny, thoughtful, and caring person who brightened up everything just by being present.

Nicole Færber said:

During a lifetime one meets a lot of people. Some you like to forget instantaneously again, some you are fine to work with but you don’t want them to be around more than necessary, some are good pals for a party, a hangout or a special occasion, some are great to talk to.

And then there are some people who you simply enjoy having around you whenever and wherever, who fill any room with kindness, understanding and empathy, who make you feel welcome, understood and accepted, unconditionally—this is what James was to me.

I had the honor and pleasure to work with James, mostly remote over the Internet, but still his positive energy could always be felt, even through the wires. Just a few weeks ago we finally had the pleasure to meet in person and enjoyed a wonderful time together with the team, which even further increased our bond.

James leaves a big hole in our hearts.
Our thoughts are with his family now, his wife and children.

Farewell James, may peace be with you, we miss you. We will try our best to keep your spirit alive, but it will never be the same without you.

François Téchené a dit:

Thank you James. Thank you for having been such a great colleague, such a nice and bright person. Thank you for having been my friend. I am very proud to have been working with you, for the last three years, on making the world a better place. It feels empty without you. I will sincerely miss you. May you know peace forever.

Zlatan Todorić said:

James and I have been working together for nearly three years, so we had time to become friends. Even though we never met in person, I can say that James was a rare genuinely good, kind-hearted person. His presence in our chats, and his voice in our meetings were soothing to me. He will be dearly missed but he will live through-out our memories of him. His beliefs in a better world will continue to live in me forever. Thank you my friend for having presence in my life and may you watch over us wherever you are now.

Heather Ellsworth said:

James was always a pleasure to work with and had a wonderful way of making his team members around the world laugh. I feel fortunate to have been his teammate and he will always have a fond place in my thoughts. My deepest sympathies go out to the family in this difficult time and may they too find peace in their cherished memories.

Kyle Rankin said:

People in technology have a tendency to solve problems in a way that suits only their own selfish needs. But not James. The word advocate gets overused these days, to the point that it sometimes has lost its meaning. Yet James was a real and true advocate for technology working for everyone, not out of some cynical drive to grow market share, but because that’s where his heart was. He believed that technology should be accessible and intuitive, and he consistently pushed to improve everything we did with that in mind. Those fingerprints are in everything we have made so far. If only we had more people with a heart like James in technology, we would all be much better off.

Chris Lamb said:

Whilst I never had the privilege of meeting James in-person I will not quickly forget the many thoughtful conversations and interactions I had with him. That he appeared to make an immediate impact on everyone who was fortunate enough to have come across him, even through the oft-distancing medium of faceless and impersonal technology, is truly a credit to his character. My deepest sympathies towards his wife and his children.

Theodotos Andreou wrote:

Farewell beloved colleague. Purism will miss you.

Librem 5 general development report – Updated August 6, 2018

The Librem 5 team has been a busy group with GUADEC along with lots of exciting development changes. Here’s a summary of what has been going on with the Librem 5 team the last few weeks.

GUADEC

Recently most of the Librem 5 team members attended GUADEC. Some of the Librem 5 attendees gave talks as well. Since many of the talks are still being edited, here are a few of them for your viewing pleasure:

There were also talks given on security items and implementing phone UIs with GTK+. We’ll link to those talks when the editing is complete.

Design

The Librem 5 will look beautiful but that doesn’t come without effort. Lately, our design team has been hard at work on a new icon style for GNOME 3.30 that will be used by the phone. They have also been working on expanding the mockups for the cellular settings panel with more advanced features as well as more detailed work on the shell.

 

Software Work

Images

The images were taking a long time to build so we were able to cut down the total build time by making a few tweaks. This makes development and testing a little nicer. There was also the first prototype aarch64 build with PureOS wich worked well. The images haven’t completely shifted over to PureOS as a base (instead of Debian buster) but that is coming real soon. If you’ve been running the x86_64 VM, then you’ll be happy to know that recent images allow you to resize your rootfs to fill a larger (31GB) space. This will make development much easier since previously the image was only 3.6GB. There is still work to be done on resizing the rootfs but it’s in a usable state now.

Phosh

Phosh has seen many under-the-hood improvements in the form of bug fixes, like several potential crashers and missing initializations. Also the brightness slider was fixed to behave properly when moved.

Wayland global handling was moved into a separate GObject. A lockscreen-manager object was introduced to unclutter things (it also picks up the timeout form GSettings now) and all remaining Layersurfaces were converted into PhoshLayerSurfaces.

It’s these code clean ups that really pave the way for community contributions because the more organized code is, the easier it is to understand and contribute.

The integrity of phosh is critical since it is the phone shell so a gitlab smoketest has been added to run phosh under Valgrind. Also it’s just a start to our eventual language support, but Spanish and German translations have been added.

And there were some odds and ends fixed up in phosh too. As we mentioned in the last progress report blog post, there was some ongoing redshift work. This code was merged to master including a fix for race conditions when listing video modes. Phosh/wlroots now starts via gnome-session and a (software) home button was added to the bottom of the screen.

Wlroots

In the land of wlroots there were also plenty of under-the-hood changes. Some custom video mode patches were merged upstream to allow any custom video mode to be defined. Since the Librem 5 will ultimately not include X support, we needed to remove the dependency on xwayland. So now wlroots is built both with and without xwayland support. A wlroots freeze has also been found to be caused by one logging out of the ssh session that started wlroots and there it is awaiting some upstream discussion on how to handle this.

Keyboard

The keyboard (virtboard) will ultimately benefit from the great text-input protocol work that has been recently worked on. The text-input-v3 patch has also been updated and sent upstream to Wayland for review. An implementation of the text-input protocol for GTK3 was submitted upstream and is a work in progress. For wlroots support, a recreated implementation has been drafted, soon to be submitted for review.

Calls

In order to integrate Calls with the gnome-settings-daemon and gnome-control-center, it became clear from discussions at GUADEC that the best path forward was to switch to using ModemManager instead of ofono. So even though the testing thus far has been with an ofono implementation, this was an unavoidable necessary change. The initial implementation of ModemManager backend of Calls was completed and has started undergoing tests.

The UI of Calls has made some strides to look like the mockups from the design team. Below you can see the implementation (left) next to the mockup (right).

Libhandy

Some more bugs were fixed in libhandy too as well as more preparation for GTK+4. One of the issues found and fixed was memory leak was found and fixed. Also there was a bug found and fixed in HdyColumn where the wrong width was being used for column height calculation.

If you are following the librem-5-dev email list, then you may have seen that libhandy v0.0.2 has been released too!

Epiphany/GNOME Web

Nobody likes unsolicited ads so Better ad blocking was suggested to upstream to be used with epiphany and is undergoing discussion.

Messaging

The phone will ship with an SMS app which also has E2EE messaging. We are working with the Fractal project upstream to get encryption implemented, but it’s not clear whether the Fractal 1-1 successor app (GNOME Messages) will have all the things we need by launch. For a more detailed analysis of Fractal’s role on the Librem 5 read the Banquets and Barbecues blog post.

This is why “Chatty”  (code name) is being developed by Purism, a new chat application using a libpurple backend, which will contain E2EE of XMPP messages via OMEMO from day 1 (when Librem 5 phones are shipped in January), as well as non-encrypted SMS. Since the revelation that ModemManager is needed instead of ofono for the Calls application, a D-BUS handler was created for the ModemManager backend of the messaging app. With this ModemManager setup, sending and receiving SMS is working so far!

Security

Security is one of our favorite things (maybe you’ve noticed) so some research was done on TrustZone, TPM and other related topics. There have also been some internal discussions about tamper-resistant boot, Heads, and alternate USB modes for video output. So we’re really starting to think hard about implementing security measures for the Librem 5.

Kernel

It is no small feat to get a working kernel and drivers ready for the i.MX8M board. With lots of hard work, we now have ethernet working. As a requirement for DRM and graphics support, the PCIe has been forward ported. The second SD port is now working but not SDIO yet (the SDIO board is powered by USB so need to get USB working first) so working on getting the designware USB core working. More i2c devices have been enabled. The board will also need some sort of battery charger and the one being tested now is the BQ25896 from TI, but a power supply driver had to be added and submitted upstream.

There is still a long road ahead towards getting the kernel and all of the drivers in working order, especially on the graphics front. If there are any graphics driver experts out there willing to lend a hand, please reach out to us in the Matrix chat rooms.

Hardware Work

We’re still working with our potential manufacturer of the development boards to review the schematic developed by the Librem 5 hardware engineers and make suggested changes. However, many things are set in stone for the development boards and many parts have been ordered so here are some components you can count on being on your development board:

Community Outreach

There is a new FAQ up on the developer documentation site, just based on some repeat questions we’ve seen in the community/librem-5 matrix channel. We are not aiming to answer ALL questions here that you can think of because that would require too much time but rather we’re adding questions that are just commonly asked.

A big Thanks goes out to all of the external teams that have helped review and merge changes into upstream projects. Everyone’s time and contribution is much appreciated!

That’s all for now folks. Stay tuned for more exciting updates to come!

Over $1.6 million raised for the Librem 5 — What this means for you

This Monday, 14 days early, we have crossed a historic milestone. By helping us reach our $1.5M goal early, you have secured your future and freed yourself from the chains of privacy-stripping mobile platforms and allowed us to continue upholding your digital rights with a convenient product made “by the people and for the people”; you have proven that there is a market demand for in-depth security & privacy-focused smartphones that can withstand the test of credibility, by virtue of true community ownership and auditability of the code.

With this milestone comes not only rejoicing about our collective achievement (and the potential of an even greater achievement in weeks to come, as contributions continue to add-up), but also the assurance that the Librem 5 phone project, as a product, will happen. The dreams of a generation will finally come to reality with a convenient smartphone hardware offering that you can truly own and control.

The $1.5 million milestone allows us to do a couple of things as it relates to the production of the physical product:

  • Immediately resume negotiations with component suppliers, with a much stronger hand (with money on the table to enter contractual relationships)
  • Produce more complete prototypes to evaluate, in order to begin development now
  • Move into hardware production as soon as possible, for the development kit
  • Begin developing the base software platform with the help of the community (fully in the open, upstream-first approach) to bring the product’s software to first stage “usable state” for early adopters.
  • Move into hardware production for finalized hardware products, begin order fulfillment for those who want their devices early (and are ready to help us smooth out the rough edges from the software side, in the beginning).

This will also allow us to seek additional partnerships and investment in parallel to amplify and speed-up our project.

…let’s go above and beyond: to stretch goals!

The goals above already represent a groundbreaking step for users around the world who have been clamoring—for years—for a mobile platform they can truly trust and own. But it’s only the beginning! As we are writing this, we are already at $1.6 million and counting, but we need to push further to accomplish more.

Indeed, to make this hardware product an even more compelling offer beyond early-adopters, we should go beyond the “base platform” and make it into an “awesome user experience”, as much as possible. This is something we hope to achieve by reaching a number of stretch goals in this campaign:

  1. $4m = VoIP phone number, call-in, call-out features: what this means is that we need to reach the $4 million milestone to hire the Matrix team to implement calls to/from the POTS/PSTN, to complement the existing VoIP features.
  2. $6m = Reverse engineering faster WiFi/Bluetooth firmware
  3. $8m = Free encrypted VPN tunnel service for all backers for 1 year
  4. $10m = Run Android applications in isolation on the Librem 5

Let’s do this!

YouTube streaming with less interruptions and more privacy

In this short tutorial, I will show you how to watch your favorite YouTube videos without being annoyed by the ads or those random visuals popping around (like “annotations”). It will also improve your privacy by avoiding storing some history and cookies from watching those videos within your browser.

As a film maker, I think that displaying any kind of visual artifact (ads, comments/annotations…) on top of the video is degrading the artwork. It is like going to a museum and seeing Post-Its and stickers pasted all over the sculptures and paintings. How would a museum could justify such a business model? Of course, YouTube is not a museum and I don’t want to discuss ethics or business models here (maybe on another post?). YouTube is also a great source of inspiration and learning for me—I simply want a better viewer experience.

The solution to improve your watching experience is called GNOME MPV. It is a video player that lets you watch any video from your computer as well as remote videos like the ones from Youtube.

GNOME MPV is based on FFmpeg and is able to read almost any video format. It has a very simple interface and it is very fast. It has become my main video player.

Install it

I don’t think that GNOME MPV is currently the default video player in PureOS, so you may need to install it. It is very easy: open the GNOME software center (“Software”) and search for “GNOME MPV”. From there, click on the “Install” button. When done, just launch it.

Watching a YouTube video

On GNOME MPV, click on the “+” button on the top left of the window and select “Open Location”. A small dialog will appear.

In the text field, paste your Youtube video link and click “Open”. You can try with this example (A song from Free Music Archive): youtube.com/watch?v=4M9Puanhdac

Of course, I cannot guarantee that it will always work. Be aware that Youtube remains master of their videos and can decide which level of restrictions they apply to them. Also make sure that your system is up to date when problems occur. New versions with corrections may be available.

Play an entire YouTube playlist

You can also play an entire playlist. This time,  just paste a YouTube playlist URL.

Note that for it to work, I had to remove the video id from the URL and only leave the “list” attribute.

You can test with this example: youtube.com/watch?list=PLzCxunOM5WFJ3B0F5AnUCwMBTlyq64vKP

From there, you may go to the menu button, on the top right of the window (the 3 horizontal lines) and select “Toggle Playlist”

I use Youtube as an example in this tutorial because it is the streaming service that I use the most, but GNOME MPV also works with Vimeo and many other online streaming services. Just give them a try!

Your own music studio with JACK, Ardour and Yoshimi

Last week, after flashing coreboot on my Librem 13 (as a beta tester of the new coreboot install script), I came across a few problems with my heavily tweaked PureOS install, so I decided I would do a full, fresh install of PureOS 3.0 beta so my environment would be much closer to what a new user would expect.

While re-installing all my creative environment, I decided that I would do a quick tutorial on installing and using Jack as it is not straight forward and that there are not so many tutorials about it on the Internet.

What is JACK?

JACK stands for “JACK Audio Connection Kit”. It is a free software that lets you handle audio input and output between different applications.

You can see it as a set of audio jacks that you will be able to plug between different programs.

For example, you can use it to connect a software synthezizer (Yoshimi, ZynAddSubFX) to a multitrack sequencer (Ardour, LMMS).
You can use it to connect an audio editing software (Audacity) to a video editing software (Blender).

Many applications have Jack support. Here is a list from the JACK’s website.

As an example for this tutorial, I will show you how to use Yoshimi with Ardour.

Install the applications

First of all, we need to install all the required applications

sudo apt install qjackctl ardour yoshimi

Enable real time scheduling

Real time scheduling is a feature of all Linux based operating systems that enables an application to meet timing deadlines more reliably. It is also considered to be a potential source of system lock up if your hardware resources are not sufficient so, most of the time, it is not enabled by default.

As mentioned on the JACK’s website, JACK requires real time scheduling privileges for reliable, dropout-free operation.

There is a well detailed tutorial from the JACK’s team that describes how to enable real time scheduling on your system. I will go through the main steps here. It works for me on PureOS but should also work without problem on many other GNU/Linux distributions.

First of all, create a group called “realtime” and add your user to this group (replace USERNAME with your current login) :

sudo groupadd realtime
sudo usermod -a -G realtime USERNAME

You can check that “realtime” is now part of the user’s groups by running the following command :

id USERNAME

Also, make sure that the user is part of the audio group. If not, just add it :

sudo usermod -a -G audio USERNAME

On PureOS (and Debian), you should have a folder called /etc/security/limits.d. If so, just create and edit the file /etc/security/limits.d/99-realtime.conf with your favorite editor. (If you don’t see this folder, you need to edit /etc/security/limits.conf).

sudo vi /etc/security/limits.d/99-realtime.conf

Add the following lines and save the file :

@realtime   -  rtprio     99
@realtime   -  memlock    unlimited

You need to logout and login again for the changes to take effect.

WARNING : You should only add new or existing users to the “realtime” group only if an application that they use (like JACK) requires it . By doing so, you give them pretty high privileges to interact with the process priorities, and this may affect the whole usability of the computer.

Run JACK

Before being able to connect anything with JACK, we need to set it up and start its deamon. For that matter, we will use QJackCtl which is a graphical application that controls JACK’s inputs and ouputs.

We will first make sure that JACK is setup correctly. Press the “Setup…” button.

I am not an expert with audio hardware and configurations and this setup is working perfectly on my Librem :

  • Driver: alsa
  • Realtime : yes
  • Interface : hw:PCH
  • Sample Rate : 44100
  • Frames/Period : 128
  • Periods/Buffer : 2

 

 

Save your settings and, on the main QJackCtl controls window, press the “Start” button. After a few seconds, you should see the “Connections” window popping up. This is where all the connections take place.

Connect Yoshimi to Ardour

Now, we are ready to connect our virtual jacks. It is time to open Ardour and create a new session. You should now see a lot more connections in the JACK’s connections window. It shows how Ardour interacts with the system’s audio inputs and outputs.

Let’s add a new track to Ardour. Click the menu “Track”->”Add Track, Bus or VCA…”. Call your new track “Drums” and set it as stereo.

Now you see 2 more Ardour inputs in the JACK’s connections window. They show the name of the audio track that we just created and they are currently connected to the default system’s capture device (the microphone). That is is not what we want so we will disconnect them.

Right click on one of them (Drums/audio_in 1) and chose “Disconnect”. It will disconnect the audio capture device. We will now connect our track to Yoshimi.

Open Yoshimi and wait for it to be fully loaded. You should now see the Yoshimi’s output appear on the JACK’s connections window. In order to connect the Yoshimi’s output to the Ardour’s input, just drag one on top of the other (make sure to respect the vertical order).

 

You are now ready to enjoy your fully operational free software powered professional music studio! 🙂

Please, feel free to comment this post or ask any question in our forums.

Have fun! 😉

Yet Another EFI/UEFI Exploit, this one Utilizing NVRAM and Persistent Storage

Continuing on our previous post on this topic, another EFI/UEFI BIOS exploit theoretically known–and even proven to work by Trammel hudson some years ago–that resurfaced through the Vault 7 documents, is the EFI/UEFI exploit that can write to NVRAM or persistent storage. This means that this exploit cannot be detected from hard drive inspection, and can survive through a complete OS reinstall if you’re using EFI/UEFI (which is not a problem for Purism users running coreboot).

The CIA documents describe it best:

“These variables present interesting opportunities for our tools since they will survive a OS reinstall and are invisible to a forensic image of the hard drive. What’s also interesting is that there is no way to enumerate NVRAM variables from the OS… you have to know the exact GUID and name of the variable to even determine that it exists.” — the CIA, as leaked through the Vault 7 Persistent Storage Document

This line also summarizes intent for the exploit:

“This might be a good place to put either implants or encryption keys. If every implant deployment used a different GUID/name pair, it would make the variables a bit more difficult to discover.” — the CIA, from the Vault 7 Persistent Storage Document

This continues to reinforce that our philosophy and beliefs are the only way to have long-term products that respects users’ digital rights.

Proving the Known, EFI/UEFI Exploited for BIOS Level Attacks

We’re continuing with a second report (many more coming!) on the “Vault 7” Documents we started digesting recently. There is an extensive section dedicated to EFI/UEFI exploitations. While this threat has been known from a theoretical standpoint from the moment the non-free BIOS replacement–EFI/UEFI–came into existence, the Vault 7 documents published recently now confirm that these threats are real and these weaknesses are actively being exploited.

One interesting read we’re focusing on today is the EFI/UEFI “ExitBootServices Hooking” exploit and sample copy-and-paste code to inject a hook into the last execute state of the EFI/UEFI process (the “ExitBootServices”).

Copy-and-paste code was included in the leaks which allow for the exploitation of UEFI-based boot systems by altering the operating system’s kernel which is loaded into memory before exiting the UEFI boot sequence. The copy-and-paste code allows for an attacker to insert a custom hook which can be used to arbitrarily alter the operating system’s kernel in memory immediately before execution control is handed to the kernel. — Wikipedia’s summary.

It is trivial to utilize this exploit:

Because the ExitBootServices service can be found by getting its pointer from the global EFI_BOOT_SERVICES table, hooking the ExitBootServices call is trivial. […] When you’re running in UEFI, that EFI_BOOT_SERVICES table isn’t protected by anything, so you can just write directly to it. — Vault 7 ExitBootServices Hooking

The result is that the entire system is compromised. As the page highlights, “At this point, you can do whatever you want.”

This type of exploit once-again highlights that security is a game of depth. This exploit is one level below the kernel, which means it has complete control of every level above it, such as the kernel, the entire operating system, any and all applications, network traffic, web application usage, and all user interaction.

The good news is, Purism recently completed the port of coreboot to the Librem 13 v1 (with more ports to come for the rest of our devices), providing a free/libre and open source replacement for EFI/UEFI which avoids all of the exploits mentioned within the documents.

The only long-term approach to protect oneself is to have complete control of the device. Control is the key word, and there is no other way to have complete control than to have as much of the software released under free software licenses where the source code is available to confirm it operates in your best interest and not that of criminals, spies, bad hackers, nations, or thieves.

Confirming that EFI/UEFI has a known and trivial exploit that is built into the standard also confirms that there is no depth too deep to exploit, and the only defense against unwanted stripping of a users’ digital rights is to use hardware and software that you control. Purism does just that by releasing all software under a free software license where the source code is available to be audited, reviewed, and scrutinized making a user control their device not the device controlling the user.

What the US Senate Vote Barring the FCC from Protecting the Privacy of Customers Means

On March 23rd, 2017 the US Congress disapproved the rule submitted by the Federal Communications Commission relating to “Protecting the Privacy of Customers of Broadband and Other Telecommunications Services”, and so that rule shall have no force or effect.

This means the FCC does not have the legal authority to protect the privacy of customers from ISPs gobbling up all the data they want to. The ISPs own the connection from your router to the Internet at large. ISPs have access to everything that passes over the connection including any non-encrypted content such as, every webpage you visit, every email you send, every photo you share, every document you deliver, and any social media post you make. Utilizing SSL helps guard against this threat of ISPs selling your head-end usage data, which is why Purism integrates EFF‘s HTTPS Everywhere in PureOS by default. In the future Purism will also be including SSL tunneling by default to help users stop ISPs from the privacy invading fire-hose of everything you do online.

RAW footage with Magic Lantern & MLRawViewer

A picture of my post production studio.

Software freedom is amazing! Used with the right hardware, it becomes limitless. Being part of the Purism team as well as the Ethic Cinema project, makes me do a lot of research in term of freedom in visual creations.

Everyday, I realize a bit more, how powerful my free software based studio is when it comes to handling a professional film making workflow. And of course, as a film maker coming from the old school of proprietary technologies, I am so glad to know that now, I am in control.

Getting the best out of your video footage

On a previews post, talking about A/V formats, I said that I didn’t know any camera that lets you be in full control of your footage. Especially if you are on a budget. Most of the time, you will have to deal with footage in a compressed, proprietary format. This can be a problem in a post production workflow because if you re-encode your footage based on an already compressed one, it will start to degrade. If you chose to keep your original footage, you will have to deal with the limitations that come from the proprietary nature of the formats.

This may be true but there is a way to bypass the problem.

RAW files

Thankfully the amazing people from Magic Lantern came to the rescue!

Magic Lantern is a video camera firmware that is released under the GPL license and runs on most Canon DSLRs (Are there some equivalents for other cameras ?). This firmware extends the functionality of the camera and most of all, lets you record your footage as RAW files.

RAW files are brute data coming straight from the sensor. It is like a film negative that would have had no treatment yet.

Based on this RAW file, we are free to export our original footage to the format that we wish. This step is what would happen in the camera in order to generate the footage. The camera would apply your color presets to the RAW data coming from the sensor end encode it to a usable video format. Usually an H.264 format within a MOV container.

With Magic Lantern we have access to the RAW file, so we are in control.

Handling RAW files

Once the RAW file is stored in the computer, it is time to generate our original footage.

MLRawViewer is an amazing free software, made in python and based on FFmpeg. It lets you preview, color grade and encode your RAW footage.

In its latests version (1.4), MLRawViewer only encodes as Apple ProRes (.mov) or Adobe Digital Negative (.dng) formats. Unfortunately, both formats are proprietary, so as part of the Ethic Cinema project, we have decided to contribute to MLRawViewer. We have added the free lossless Huffyuv (.mkv) format to the list as well as the ability to rotate the encoded video. We sometimes film with the camera upside-down when doing camera movements close to the ground with our steadycam, so rotating our footage during this process is very useful.

While our changes are not merged into the original project, you can test it from our repository.

Having the footage being encoded from RAW to a lossless format makes it keep its full quality (which wouldn’t be the case when using the built in H.264 compressed format). Gradients and details are perfectly preserved. It also gives us the ability to use the highest dynamic range available from the camera, or to use a custom LUT (picture style) that would suit our needs.

Please, note that this step is not to be confused with the actual color grading process that takes place at the end of the post production, when the editing is complete. The goal here, is to prepare the footage to allow as much flexibility as possible during the color grading phase. Usually, we chose a very flat picture style at this stage, in order to make sure that we keep as much details as possible from dark to bright.

All in all, the footage we get through this process is at the best possible quality and very close to what one could get from a very high end cinema camera.

This was the missing bit of my workflow. I have now achieved full control and freedom over the whole post production workflow.

Installing MLRawViewer

Note, I have updated this part on 07/04/2017 after noticing some problems with different configurations running python3 along with python2

I plan to make an Appimage build of MLRawViewer, but it is not done yet, so you will have to compile it yourself.

Don’t worry, it is not very difficult and here are the instructions for PureOS and other Debian based systems (It should be very similar on others systems) :

First of all, you need to install git and python, along with pip. (I installed pyaudio with apt because for some reason it failed installing with pip).

Note that you need to install version 2.7 of python as version 3 is not supported by MLRawViewer.

sudo apt install git python2.7 python2.7-dev python-pip python-pyaudio libglfw3-wayland
# Use ‘libglfw3’ if you are not on wayland

Then, you need to install the required dependencies making sure that you use the right versions (which may not be the latests)

pip2 install scandir
pip2 install -I PyOpenGL==3.0.2
pip2 install numpy==1.9.1
pip2 install glfw
pip2 install Pillow==2.1.0

Then, you need to get the source code of MLrawViewer from the Ethic Cinema repository for the updates

git clone https://github.com/ethiccinema/mlrawviewer.git

or from the original repository

git clone https://bitbucket.org/baldand/mlrawviewer.git

It is now time to build the application.

cd mlrawviewer
python2 setup.py build
cp build/lib.linux-x86_64-2.7/bitunpack.so bitunpack.so

And run it

./mlrawviewer.py

Don’t hesitate to ask any question in the forums if you have any trouble  or if you wish me to post any tutorial related to multimedia manipulation with free software. You can use the PureOS area in the forums. I am very happy to help!

 

What’s next?

Well, how cool would it be to shoot with an Axiom camera ? … along with the new Librem 15 v3 !