Mirroring content in a local registry is always a good idea to prevent relying on upstream services. Tools like Artifactory or Nexus let you do that easily with upstream mirrors. But what if your resources are limited and you only want to run a single Docker Registry?

An upstream mirror

In a production environment, you want to be able to continue operations at all times. When you're using container images to deploy software, you want to be sure that you can build new images. But what if your Dockerfile contains

FROM python:3.9.10

and the Dockerhub is down? When there's no local copy of that image on your build server, you're out of luck and you can only continue building and releasing your software when the Dockerhub is back in business.

We can solve this problem by mirroring images to a container registry that we deploy in our own infrastructure. The Dockerfile would then look like this:

FROM registry.local/python:3.9.10

In an enterprise world, companies often choose for an all-in-one package like Artifactory. These applications have built-in options to mirror container registries by using a pull-through registry. This mode will check the local cache of the server to see if the image is already present. If it is, then it's served from the cache. If it's not, then it will be pulled from the upstream registry and added to the cache.

In a more restricted environment, an application like Artifactory might be overkill, resource-wise and function-wise.

Enter Docker Registry

Docker Registry is a lightweight and barebones container registry. It doesn't have a built-in UI for example. Other projects build on the Docker Registry and add a UI or better authentication methods.

You can configure a Docker Registry as a pull-through registry, but then you can't push your private images into it. This would mean you need to have multiple instances of the Docker registry deployed. One for your private images, and one per upstream registry you're mirroring from.

I didn't want to host multiple versions in my Homelab environment, so I decided to solve this in another way.

Introducing Skopeo

Skopeo is tool to work with local and remote container images. It has an option to copy an image from one registry to another. This is what we need to mirror images. It's as easy as this:

skopeo copy docker://docker.io/python:3.9.10 docker://registry.local/python:3.9.10

Automating the mirroring

In my Homelab, I'm running a Drone server, so I'm going to use this to mirror the images in an automated way.

I have two simple .txt files containing all images I want to mirror. One for amd64 images and one for arm64 images. A bash script loops over the files to copy the images one by one.

#!/usr/bin/env bash

arch=$1

# Read every line in image_list_$arch.txt
while IFS= read -r line; do
  # Split on first '/' and copy with Skopeo
  IFS='/' read -r registry image <<< "$line"
  if ! skopeo inspect "docker://registry.local/$image"; then
    skopeo copy "docker://$registry/$image" "docker://registry.local/$image"
  else
    echo "Image already exists in registry.local"
  fi
done < image_list_"$arch".txt

An example .drone.yml file can look like this:

---
kind: pipeline
type: docker
name: container_image_mirroring_amd64

platform:
  os: linux
  arch: amd64

steps:
  - name: Mirror amd64 images
    image: quay.io/skopeo/stable:v1.4.1
    commands:
      - ./mirror_image.sh amd64

---
kind: pipeline
type: docker
name: container_image_mirroring_arm64

platform:
  os: linux
  arch: arm64

steps:
  - name: Mirror arm64 images
    image: quay.io/skopeo/stable:v1.4.1
    commands:
      - ./mirror_image.sh arm64

This is all stored in a git repository. Every new commit will trigger a pipeline run and mirror the new images to our local Docker registry.

The main disadvantage to a pull-through registry is the need to add a new image manually to a list. But at the same time, this can be a nice way of documenting the different images used in your environment.

An additional step could be to delete all the unused images in your Docker registry.