Hello nerdctl
Please note: this post is not intended to diminish the Docker team’s work to promote the usage of containers in modernizing application deployments. Instead, highlight alternative strategies that can be used. Docker is still a viable solution for many.
A while ago, the Kubernetes (K8s) team removed parts of Docker from their product and placed in containerd as a substitute. The switch was due to a lack of compatibility with Docker and CRI-O, readers interested in more details can refer to the blog post https://kubernetes.io/blog/2020/12/02/dont-panic-kubernetes-and-docker/. The compatibility issue could eventually impair K8s innovation and interoperability with other technologies. One of the drawbacks of containerd was the lack of building containers. However, since Docker does produce OCI (Open Container Initiative) compliant container images, K8s could still run the containers.
The initial reaction was confusion among the community, which was also addressed by the K8s team citing that there are other alternative products to build OCI (Open Container Initiative) compatible images. For example, podman, kaniko and many others. Workflows did have to change to accommodate the deployment of Docker generated containers to a K8s cluster. Previous strategies allowed workflows to build containers directly on K8s nodes because Docker was installed on them. The removal of Docker required alternative strategies to be developed to get built images onto K8s without the need of registries for dev testing and QA.
One strategy was to reconfigure the CICD pipeline to build the container using Docker and push the image to the K8s cluster using SSH. The example below is a sample code that demonstrates creating a Docker container and pushing it to a K8s node.
$ docker build --rm -t mainwebdev:0.2.1 .
$ docker save mainwebdev:0.2.1 | bzip2 | ssh -i /home/salim/ssh_key/k8s.key salim@k8s-main.local ‘bunzip2 | sudo ctr images import -’Docker itself serves a specific purpose governed by its organization. containerd also serves a specific purpose governed by its organization and backing from the Cloud Native Computing Foundation (CNCF). containerd aims to continuously create innovative solutions that work in line with Kubernetes. One notable product from containerd is nerdctl. nerdctl seeks to be a direct drop-in solution for Docker. However, nerdctl is treated as a non-core sub-project to the containerd project, meaning that nerdctl is not a dependency to run containerd.
So why would you want to swap Docker for nerdctl even though nerdctl is a separate installation? First, it provides an alternative to container management. Having alternative products that offer the same services helps safeguard your company if one of the essential products is no longer available. Second, having multiple products helps advance the field of container management. Various companies/communities can create solutions to address concerns seen within their community and apply them to the container management technologies. For example, nerdctl allows developers to encrypt their container images (https://github.com/containerd/nerdctl/blob/master/docs/ocicrypt.md)and split up pushes/pulls of containers based on their specific underline platforms. More information on the additions to the standard Docker CLI can be found on nerdctl’s website (https://github.com/containerd/nerdctl#command-reference).
I was interested to see how much work was required in swapping out Docker for nerdctl. My dev setup is currently a Raspberry Pi 4 w/ 8GB of RAM running GitLab and K3s. In the past, I would also install Docker to help build containers. [Yeah, I know Kaniko can do this, but at the time, someone needed the solution to involve Docker.]
After downloading nerdctl from its site (https://github.com/containerd/nerdctl#install), I gave nerdctl a quick spin. Unfortunately, I was greeted with the following error.
Well, good thing that this community actually gives proper error messages. After downloading the cni-plugins from the URL provided in the error message and plopping the binaries into /opt/cni/bin … re-running the nerdctl command resulted in success.
Now, we have roughly got nerdctl working. Let’s see if we can build our containers on the server. An example of a basic Dockerfile will be used.
Well, we are in luck. A better description of the error and where to go download the needed tools to get through this part. Adding the buildctl to /usr/bin (while you are at it toss in nerdctl here as well), and running buildkitd with nohup (nohup sudo ./buildkitd — containerd-worker-addr /run/k3s/containerd/containerd.sock &). We can now retry and see what happens.
Looks like success. We can verify the image is there by running the nerdctl images command and…
we are almost good.. let’s try to run the container and get inside the container.
So apparently, I made a major mistake when I was doing this. nerdctl images was showing the built image in the listing on the local machine. However, every time I ran the nerdctl run command, it would go to the Docker Hub repo to look for local/alpine:0.1 and bailout because it wasn’t there. Since I was running Docker-ce on this server previously, it left containerd.io when I removed the Docker-ce packages. Nerdctl got confused and wasn’t referring to k3s’ containerd. Once I removed containerd.io and started to use
$ nerdctl --address unix:///run/k3s/containerd/containerd.sock, then everything seemed to work well. [Now you are all probably swearing I waited this long to reveal this while you were probably facing the same issues earlier following along. I too was also stumped and confused up to this point as well].
In the next post, we will try to see if K3s can pick up the containers now built on this server.
