CS SYD - Nix, Docker and Haskell

I have slowly been converting my services to being deployed using Nix and Docker and found that the resources on the topic are not quite as extensive as I would have wanted when I started. Traditionally, tutorials seem to tell you what to do, but not why that is what you do, so I thought I would walk you through the process from start to finish with all the obstacles included.

Haskell

Let's say you want to run a Haskell service. You have written the source code, it compiles, and it works well enough when run locally. When you run the service locally, and you point your browser at http://localhost:8000, you see your web service. So far so good.

Nginx

You rent a virtual private server (VPS), you buy a domain, and you point the DNS records at the static IP address of your VPS. So far so good. The first thing you may try is to build your Haskell service with stack build, and then copy over the resulting binary to your VPS using scp. The first problem you will run into is that when you browse to your domain name, your service will not show up at all. You need to explain to your web server that traffic for your domain should end up at your service. But you are smart and you know about virtual hosts, so you write an nginx configuration file for your service (and you do not forget to restart nginx).

server { 
    server_name my.domain.name;

    location / {
        proxy_pass http://127.0.0.1:8000;
    }
}

Docker

Some time passes, you update your operating system and at some point after that, you deploy a new version of your service. The next problem you run into is that your service no longer boots on the VPS. You try to figure out why by looking at the logs, and you see something like the following.

root@0b5ea9a8d84c:/www/my-service/bin# ./my-service:
/lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.27' not found

Now, the specifics could be different for your service, but the point is that you learn that the dynamically linked dependencies of your service need to have the same version on your VPS and your development machine. After a bit of googling, you decide to dockerise your service to make sure that the dependency versions are the same wherever you run a certain container. You think to yourself 'Oh, this is quite smart! Now I can even run multiple services on the same server without worrying about how they will interact'. (At this time, you do not realise how naive that is, but it is good enough for now.) You write a docker file that looks something like this:

FROM ubuntu:18.04
RUN apt-get update && apt-get install -y libgmp10

ADD my-service /www/my-service/bin/my-service
RUN chmod +x /www/my-service/bin/my-service

CMD /www/my-service/bin/my-service serve

You notice that these docker images tend to be quite big. Hundreds of megabytes, in fact. You decide that this is not a big enough deal to spend time on, but it still leaves a nagging feeling at the back of your head.

This works for a while, but you upgrade your operating system some more, and you see similar problems with dynamic libraries that you thought you had solved. You figure out that you need to make sure that the base image of the docker build keeps up with your development machine.

At this point you may throw up your hands in frustration and outsource the entire problem to a DevOps consultant. However, you are frugal and you are very proud to run your services all on a single 7$/month VPS so paying someone else to do it is not an option.

Nix

You have been hearing about Nix for a while now, and it looks like it should solve the problems you have. So you spend a few minutes looking into Nix documentation and decide that maybe this whole digital computers thing is maybe not your thing after all. You decide to give up on the problem for now until at some point, someone sits down with you and together you write a Nix expression for your Haskell service. Now you can build your service with nix-build -A haskellPackages.myService. In your excitement, you think: this will solve all my problems! You are completely oblivious to the pain that you are in for.

You try to deploy your service by adding it to a docker image and you notice that, instead of just one dynamic link being broken, now all the dynamic links are broken. You face-palm and say to yourself: Of course that doesn't work, I need to put all the dependencies in my docker containers as well.

At this point you find out that Nix can create docker images as well. So you write a nix expression as follows:

{
  my-service-static = justStaticExecutables haskellPackages.my-service;
  dockerImage =
    dockerTools.buildImage {
      name = "my_service";
      tag = "latest";
      contents = "${final.my-service-static}/bin";
      config = {
        Cmd = [
          "${final.my-service-static}/bin/my-service"
          "serve"
        ];
      };
    };
}

You think: Oh, I don't even need a from image because nix just puts everything I need in the image via the contents field. This solution has solved both the size of the container problem (as long as you do not use pandoc) and the dynamic libraries problem.

At this point, you think all your problems are solved. The service runs reliably, the images are small and the build is reproducible.

About a day later, another problem pops up. Your service performs some HTTP requests as part of its normal workings. You see an error like the following:

getProtocolByName: does not exist (no such protocol name: tcp)

It turns out that /etc/protocols does not exist in the docker container that Nix builds using the above expression. There is one more change you need to make: You need to use a minimal base image for the docker image. Alpine should suffice.

{
  my-service-static = justStaticExecutables haskellPackages.my-service;
  dockerImage =
    dockerTools.buildImage {
      name = "my_service";
      tag = "latest";
      fromImage = pullImage {
          imageName = "alpine";
          imageDigest = "sha256:e1871801d30885a610511c867de0d6baca7ed4e6a2573d506bbec7fd3b03873f";
          sha256 = "05wcg38vsygjzf59cspfbb7cq98c7x18kz2yym6rbdgx960a0kyq";
        };
      contents = "${final.my-service-static}/bin";
      config = {
        Cmd = [
          "${final.my-service-static}/bin/my-service"
          "serve"
        ];
      };
    };
}

At this point, everything works as intended, for now...

Conclusion

Use Haskell to write your web service.
Use Nix to build your executable
Use Docker to fit multiple services on the same VPS. (If you do not need this, try NixOS.)
Use Nix to build a docker image