Starting from OpenShift version 4.2, airgap/disconnected installation is supported. This paper established an opinionated way of setting up a local registry to install the OCP 4.2 for an airgap environment.

The following chart describes the steps at a high level. Setup a container registry with an internet connection, mirror the images for OCP, export those images into files. Then in the isolated environment, set up a container registry, import those image files thus to finish the local registry for the installation purpose.

Updates: The traditional save and load approach is not working. The image digest is changed. Instead, we need to use the skopeo tool to export the image into a directory on the source registry, then import the directory into the target registry.

Prepare the internet-facing VM

First, let's prepare a VM with internet access. Assume we are using a Ubuntu VM, install the necessary packages, podman, skopeo, htpasswd , as below,

sudo apt-get install software-properties-common -y

sudo add-apt-repository -y ppa:projectatomic/ppa

sudo apt update

sudo apt install -y podman skopeo apache2-utils

Secondly, prepare the certificates to be used by the docker registry. I use cfssl tool to create the self-sign CA, server certificate with both the short and fully qualified hostname of this VM.

Assuming we have the CA cert, server cert, server key saved as myca.pem, registry.pem, registry-key.pem files respectively. Upload these files to the VM in the specific directory.

sudo mkdir -p /opt/registry

sudo chown -R ubuntu:ubuntu /opt/registry mkdir -p /opt/registry/certs

cp registry.pem registry-key.pem /opt/registry/certs

Make the self-signed CA trusted by the host. (Notice the CA has to be named with the extension as crt)

sudo cp myca.pem /usr/local/share/ca-certificates/myca.crt

sudo update-ca-certificates

Create the authentication files,

mkdir -p /opt/registry/auth

htpasswd -bBc /opt/registry/auth/htpasswd admin password

Third, run the registry container,

mkdir -p /opt/registry/data podman run --name mirror-registry -p 5000:5000 -v /opt/registry/data:/var/lib/registry:z -v /opt/registry/auth:/auth:z -v /opt/registry/certs:/certs:z -e REGISTRY_AUTH=htpasswd -e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/registry.pem -e REGISTRY_HTTP_TLS_KEY=/certs/registry-key.pem -d docker.io/library/registry:2



Validate the docker registry is running properly. Notice I am using the fully qualified name of the host because of the certificate.

Mirror the container images

Download the OpenShift client tool. The version must be matching with the OpenShift version you intend to set up later. I am using Golang template below to show the commands, replace the {{ .ver }} based on the actual version, for example, 4.2.12.

curl -LO https://mirror.openshift.com/pub/openshift-v4/clients/ocp/latest/openshift-client-linux-{{ .ver }}.tar.gz tar zxvf openshift-client-linux-{{ .ver }}.tar.gz sudo mv oc kubectl /usr/local/bin

Update the pull secret json file, pull_secret.json , with the mirrored registry,

{

"auths": {

"cloud.openshift.com": {

"auth": "...",

"email": "..."

},

"quay.io": {

"auth": "...",

"email": "..."

},

"registry.connect.redhat.com": {

"auth": "...",

"email": "..."

},

"registry.redhat.io": {

"auth": "...",

"email": "..."

},

"mirror-ocp-registry.ibmcloud.io.cpak:5000": {

"auth": "{{ .base64 }}",

"email": "..."

}

}

}

The {{ .base64 }} needs to be replaced by the base64 encoding of the text

username:password, for an example,

echo admin:password | base64

Now start to download and mirror the images with the command below,

export OCP_RELEASE=4.2.12

export LOCAL_REGISTRY='mirror-ocp-registry.ibmcloud.io.cpak:5000'

export LOCAL_REPOSITORY='ocp4/openshift4'

export PRODUCT_REPO='openshift-release-dev'

export LOCAL_SECRET_JSON='/home/ubuntu/pull_secret.json'

export RELEASE_NAME="ocp-release" oc adm -a ${LOCAL_SECRET_JSON} release mirror --from=quay.io/${PRODUCT_REPO}/${RELEASE_NAME}:${OCP_RELEASE} --to=${LOCAL_REGISTRY}/${LOCAL_REPOSITORY} --to-release-image=${LOCAL_REGISTRY}/${LOCAL_REPOSITORY}:${OCP_RELEASE}

Copy and save the output for later usage,

To use the new mirrored repository to install, add the following section to the install-config.yaml: imageContentSources:

- mirrors:

- mirror-ocp-registry.ibmcloud.io.cpak:5000/ocp4/openshift4

source: quay.io/openshift-release-dev/ocp-release

- mirrors:

- mirror-ocp-registry.ibmcloud.io.cpak:5000/ocp4/openshift4

source: quay.io/openshift-release-dev/ocp-v4.0-art-dev

Export images

List the image tags with the following command, (jq needs to be installed)

We get the following file

[

"baremetal-installer",

"mdns-publisher",

"cluster-openshift-apiserver-operator",

"ironic-rhcos-downloader",

"cluster-autoscaler",

"openstack-machine-controllers",

"grafana",

"jenkins",

"kube-proxy",



...skipped many lines... ]

The conventional approach, for each of the images save into tar file, and then load the tar file on the target registry is tested not working! During this process, the digest of the image is changed and the installer will not be able to find the image from the local registry.

For each of the tags as listed in the JSON file, use the skopeo tool to export the image into a directory

mkdir -p images/{{ .tag }} skopeo copy --src-creds admin:password docker://{{ .reg }}/ocp4/openshift4:{{ .tag }} dir:{{ .dir }}/{{ .tag }}

Copy the whole directory images/ into a portable medium to be imported into the next VM.

Prepare the local VM and setup the registry

Create the local VM that resides in the isolated environment, similarly as the internet-facing VM, run the following steps

Install podman, skopeo, httpasswd tools

Create new certs for this server

Create httpasswd authentication file

Run the docker registry container with podman.

For each of the tags as listed in the JSON file, use skopeo import it to the registry running on this server.

skopeo copy --dest-creds admin:password dir:images/{{ .tag }} docker://local-ocp-registry.ibmcloud.io.cpak:5000/ocp4/openshift4:{{ .tag }}

Validate the images are pushed into the registry,

Installation config

Now update the OCP install-config.yaml file as below,

apiVersion: v1

baseDomain: ibmcloud.io.cpak

compute: ...Skipped... pullSecret: '{

"auths": {

"local-ocp-registry.ibmcloud.io.cpak:5000": {

"auth": "base64 encoded string",

"email": "email address"

}

}

}'

imageContentSources:

- mirrors:

- local-ocp-registry.ibmcloud.io.cpak:5000/ocp4/openshift4

source: quay.io/openshift-release-dev/ocp-release

- mirrors:

- local-ocp-registry.ibmcloud.io.cpak:5000/ocp4/openshift4

source: quay.io/openshift-release-dev/ocp-v4.0-art-dev

additionalTrustBundle: |

-----BEGIN CERTIFICATE-----

...skipped...

-----END CERTIFICATE-----

In the downloaded pull secret file, all the internet registry entries can be discarded since its disconnected. Add the registry server, encode the “admin:password” with base64 as the value of the auth key.

Insert the imageConentSources copied from the mirroring result, but replace the server’s name as the one in the airgap environment.

It is important that the DNS name of the registry server must be registered in the DNS server used for the OCP installation.

Lastly, since we are using a self-signed cert, its signing CA needs be inserted into the CoreOS host. Add the additionalTrustBundle entry with the CA cert content indented.

We are ready for the airgap/disconnected installation for OCP 4.2.