Meet Containerization

Hi, my name is Michael and today, I’m happy to introduce a new open source framework.

Containerization is a Swift framework for building containerized applications with a native feel. We will explore how Containerization’s focus on security, privacy, and performance helped shape its design and implementation of Linux containers.

Before jumping into the Containerization framework, we’ll take some time to learn about containers. Then, we’ll look at APIs that Containerization provides. Finally, we’ll explore a command line tool for building, running, and managing containers. Now, let’s go over what a container is and some of the common use cases that they solve.

Linux containers have become a standard for how server-side applications are built, tested, and deployed.

When deploying server-side workloads at scale, these workloads must run in many different environments. Containers allow us to isolate these workloads from the host machine as well as isolate workloads from one another. Containers also allow us to replicate our deployment environments on our local machine. This lets us develop and test our applications locally, as if they were running in production. Containers do this by allowing developers to package their application and the dependencies of the application. Dependencies can include binaries, dynamic libraries, and assets. Having the application and dependencies packaged together allows containers to have different versions of those dependencies from the host machine as well as other containers.

The result is packaged into a single deployable unit, a container.

Containers also provide additional runtime isolation. Containers typically have an isolated network stack from the host machine. For privacy, processes running in one container cannot view or inspect the processes running on the host or other containers. And containers are able to be sized and scaled independently of one another. Resources like CPU, memory, and disk can be allocated based on the workload’s needs.

In order to run a Linux container on macOS, we need to virtualize the Linux environment. The historical solution is to spawn a large virtual machine to host all of the running containers. Resources are allocated to this virtual machine, and as containers are added, those resources are divvied up where needed.

When you need to share additional directories and files from your Mac, these are first shared with the virtual machine, before being provided to the specific container that requested the data.

When we were looking at ways to bring Linux containers to macOS, we had a few goals that we wanted to achieve.

For security, our goal is to provide each container with the same level of isolation the large virtual machines use today. We also want to reduce the need for core utilities and dynamic libraries inside of these virtual machines. This reduces the attack surface and maintenance cost of keeping these up to date. For privacy, limiting the access of directories should be done on a per container basis. Only the container requesting the directory should have access to those contents.

And we want to provide a performant experience while respecting the user’s resources.

We kept these design goals in mind when building Containerization. Containerization is an open source Swift framework. It provides APIs for image management, container execution, and a powerful init system built in Swift. Let’s explore some of the APIs for how a container is created. First, let’s look at how Containerization handles image management.

Most containers are created from an image. An image is a distribution artifact that contains filesystem contents and a default configuration. Images act as a template when creating a new container.

Containerization provides APIs to fetch the image’s filesystem contents and configuration. This operation consists of requests to a Registry, a service that handles the storage and distribution of images, and then writing the Registry’s response to the local filesystem for use.

Once an image exists locally, the image’s configuration can be used as a starting point for a new container.

Image configuration can contain the default process to execute, the working directory for where the process is run, as well as what user to run as. The filesystem contents of an image includes the files and directories for the application.

For performant access to the image’s contents, we expose the filesystem as a block device. A block device consists of creating a large file and formatting it with a filesystem. For our Linux containers to consume this block device, it must be formatted with a filesystem that Linux can understand. EXT4 is a widely used Linux filesystem and Containerization provides a Swift package that allows you to format, create a directory structure, and populate an EXT4 filesystem directly from Swift. After a container has been created from an image, a Linux virtual machine must be started to run the container.

For security, our goal was to provide the same level of isolation used by the large virtual machines and apply that to each and every container that is started. Containerization does this by running each container inside of its own lightweight virtual machine while still providing sub-second start times.

This also provides the benefit that each container has its own dedicated IP address. The dedicated IP address provides performant network access to each container and removes the need to map individual ports when you want to access the services the containers provide.

When sharing directories and files, only the container requesting the directory has access to the contents. And resources like CPU and memory; if no containers are running, no resources will be allocated.

Once started, the next steps happen inside of the virtual machine. Let’s look at how Containerization sets up the runtime environment before starting the container.

After the virtual machine is started, an initial process is spawned. The binary for this process is provided by a minimal file system that’s part of Containerization. This filesystem contains a binary called vminitd. vminitd is an init system built in Swift and runs as the first process inside of the virtual machine.

Running as the initial process comes with many different responsibilities before and during container execution. vminitd is responsible for assigning IP addresses to the network interfaces, mounting filesystems, such as the image’s contents that we expose with the block device.

It is responsible for the launch and supervision of all processes running inside the virtual machine.

vminitd has an API that allows the processes to be spawned and managed from the host.

Historically, when using a large virtual machine, they were booted as a full system. The filesystem of these large virtual machines will include things like a libc implementation, dynamic libraries, and core utilities like cd, cp, and ls. For security, we want to reduce the attack surface of our containers. The file system provided by Containerization has no core utilities. It contains no dynamic libraries and no libc implementation.

In order for vminitd to run in this constrained environment where there are no libraries to link to, we need to compile vminitd as a static executable.

For this, we use Swift’s Static Linux SDK, which allows us to cross-compile static Linux binaries, directly from our Mac. We are also able to use musl, which is a libc implementation that has excellent support for static linking. With this, we produce vminitd as a static linux executable cross-compiled from our Mac and able to execute in this isolated environment. Containerization combines all of these core components, offering a powerful API for building solutions around Linux containers. Now let’s look at a command line tool which provides a simple and robust way to run containers.

The container tool consists of a CLI and XPC services built using the Containerization APIs. These services provide support for Storage, image management, and Network services that allocate IP addresses to containers and handle DNS requests. And finally the management and runtime of containers. Let’s start by jumping into our terminal and use container to pull an image to our local machine.

Now that we are in the terminal, we can start by typing container image pull.

And then we provide an image name. For this demo, we’ll use alpine:latest.

As we run this command, container is pulling the image contents and configuration locally before creating the block file for us to use. Great.

Now that an image exists locally, let's run a container based on that image.

The tool will create a container from the image’s file system contents and configuration. Then it will use Containerization APIs to start a lightweight virtual machine for running the container.

Let’s jump back into our terminal to see this in action.

We can start our run command by typing container run, and we want to run an interactive shell. So we will add a terminal device with -t, an interactive input with the -i flag. Finally, we provide the image name and the command we want to run, which will be a sh, a shell. Within a few hundred milliseconds, we're dropped into an interactive shell. We can then inspect the runtime environment of our container running uname -a shows us that we are now inside of a Linux environment. And because of the isolation that containers provide, when we run commands like ps aux, we only see our shell and the ps process. No other processes running on the host or other containers are visible. So that's the container command line tool available on GitHub for you to check out.

Using the Containerization APIs, it’s able to offer a secure, private, and performant experience for containers.

We invite you to join us as we bring Containerization primitives to macOS. If you are interested in building a project that integrates Linux containers, check out the Containerization framework on GitHub. View the source code for how lightweight virtual machines are started. Explore all the cross-platform Swift packages that were created for vminitd. and check out the example projects. If you want to get started running containers, take the container tool for a spin and join the conversation on GitHub. You can view the source code, submit issues, and create pull requests.

We're excited to see what you create next.