Shipping Security Updates

Nicolas B. Pierron
NixCon 2015

Security Matters

Exploits don't wait for updates:

Metasploit: http://www.rapid7.com/...
US-CERT: https://search.us-cert.gov/...

Trust, Who?

Can we even trust software makers?

Do they care about you?
Are issues fixed quickly?
Are fixes released quickly?

Examples

Android?
Firefox?
NixOS?

Android Example

Ok, Andoird. This graph represent the Android API versions which are tied to major and minor versions of Android.

First, let's focus on the bottom on this figure. We see that the Android developpers are making quite a lot of release. What is not shown is the fact that they have multiple "patch" version with the same API. Also, after 3.0, they made no more fixes to old versions. This that if you have an Android phone, then you are probably safe if you follow the latest version.

Now, let's look at the rest of the graph. This graph represents the percent of which versions at a given date. As we see they are a lot of phones which are lagging behind. This means, that as of today 90% of the Android phones are running versions which are no longer receiving security updates.

The problem here is related to the fact that network operators, and phone manufacturers have to pay to get each version validated. This validation process takes months and cost a lot. Thus when facing the choice of updating software, they prefer to let user use old version instead of making updates.

Thus, can probably you trust the Android developers, but probably not the phone manufacturers for releasing the security issue fixes.

          https://commons.wikimedia.org/wiki/File:Android_historical_version_distribution_-_vector.svg
          Author: Erikrespo (user on wikipedia.org)
          This file is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license.

Firefox Example

Ok, let's look at Firefox now.

As we can see in this graph of Firefox releases, we see that Firefox releases about 10 major versions per year, and about 20 "patch" releases.

One of the question we might ask, is how do they manage to do so many "patch" releases?

          https://fr.wikipedia.org/wiki/Historique_des_sorties_de_Mozilla_Firefox
          This generated file is licensed under the Creative Commons
          Attribution-Share Alike 3.0 Unported license.

Making Security Updates

Exceptional implies stress, stress implies mistakes:

Typos.
Copy & Pasta.

Chemspill

Make a process

Reduce mistakes.
Improve the speed.

The way Mozilla do is that they have a release process that we call "chemspill" for "chemical spillage". The idea of having such process is to make sure that we can do things with a minimum of mistakes and makes sure that we don't spin while trying to find what is the next thing to do.

        Illustration made by Hervé Renault[1] base on one made by lisaamybeth[2].
        [1] https://twitter.com/HerveRenault
        [2] https://openclipart.org/detail/152197/chemical-symbol

Chemspill Timeline

This graph highlights how builds are going and how they are released to Firefox users. So, as Mozilla is able to provide frequent fixes, they are also capable of releasing these within a few hours to Linux users, and within a day to windows users.

So, let's look how we propose to release updates of sofware such as Firefox

          http://www.aosabook.org/en/ffreleng.html
          This work is made available under the Creative Commons Attribution
          3.0 Unported license.

NixOS Example

Suggested ways:

Default channel: Long critical path. (when it builds)
Small channel: small ciritcal path (with local rebuilds).
Do it yourself pkgs.replaceDependency or system.replaceRuntimeDependencies.

So far, we have multiple ways to handle such releases.

NixOS Example

Dear user, please proceed as follow:

system.replaceRuntimeDependencies = with pkgs; [
  ({ original = openssl;
     replacement = callPackage /some/path/nixpkgs/pkgs/development/libraries/openssl {
       fetchurl = fetchurlBoot;
       cryptodevHeaders = linuxPackages.cryptodev.override {
         fetchurl = fetchurlBoot;
         onlyHeaders = true;
       };
     };
  })
];

.

        https://nixos.org/wiki/Security_Updates

NixOS Requirements

Require:

User awareness.
User actions.
ABI compatible versions.
Security updates are the exception. (not the rule)

Otherwise: Normal updates are shipped in a month.

Trust NixOS?

Yes-ish
which means
No

        http://heartbleed.com/
        Heartbleed logo is free to use, rights waived via CC0.

Security Issues

        Futurama

pkgs.replaceDependency

Substitute references inside package outputs:

Maintained by hand.
Realize packages during the evaluation.
Realize the packages to apply the fixes.
Lack support for statically linked packages.

One problem at a time

By order of importance:

Make it transparent for users.
Make it fast for Hydra.
Make it easy for package managers.
Make it sane. (no build during evaluation)

Transparent

$ nix-channel --update
$ nix-env -u firefox

Should become:

$ nix-channel --update
$ nix-env -u firefox

Transparent (Ideal)

$ nix-channel --update
$ nix-env -u firefox

Should become:

Implicitcations

Fixed packages should have the same layout as Nixpkgs.

Fast

Break the critical path in parallel compilation.

Do not recompile when a dependency change.

Fast & Sane

Implicitcations

Generate the same set of packages.

Implicitcations

Generate the same set of packages with one extra no-op iteration.

Implicitcations

Recompile fixed packages based on fixpoint of stable packages.

Implicitcations

Partially recompile fixed packages based on fixed packages.

Implicitcations

Entirelly recompile fixed packages based on fixed packages.

Implicitcations

Use the recompiled-info as a hint for patching with dependencies.

Easy

3 lists of packages:

Latest compiled branch.
ABI compatible changes branch.
Security channel. (merges both)

Release Management

Packagers options:

Push to master & Cherry-pick ABI compatible changes.
Push to ABI changes & Cherry-pick in master.

On new stable channel update:

Merge master into the ABI branch.
Reset the ABI branch.

Remaining Issues

What is left for improvements:

Use runtime dependencies generated by Hydra.
Automated updates for registered packages.
Support for statically linked packages.

Questions?

          NASA - STS-31 - Hubble launch.