Harbor is a container image registry developed by VMware. It was recently handed over to the Cloud Native Computing Foundation, and its development is now driven by the open-source community. Harbor includes a couple of other open-source projects, like CoreOS/RedHat Clair which allows to scan images for security issues, or Notary which allows to sign your container images.
It also delivers a very nice web interface in which you can manage the various projects you are working on, as well as the permissions associated with these projects. In order to manage your users, it is also possible to link Harbor to your existing LDAP or Active Directory.

In this lab, we will configure Kubernetes to access a Harbor registry and deploy private container images to our Kubernetes cluster.

Requirements

You need a Kubernetes cluster using Docker as container engine. We will also install Harbor on a Ubuntu server, and your Kubernetes worker nodes need to be able to communicate with this server. In my case, the Kubernetes worker nodes are 10.10.40.40, 10.10.40.41, and 10.10.40.42. The Harbor virtual machine will be 10.10.40.4.

If you don't have a Kubernetes cluster already, you can refer to the Install and configure a multi-master Kubernetes cluster with kubeadm article, or to the Install and manage automatically a Kubernetes cluster on VMware vSphere with Terraform and Kubespray article if you are using VMware vSphere.

You will also need a client machine with kubectl configured to access your Kubernetes cluster, as well as a Docker engine installed on it.

Installation of Harbor

Install Ubuntu 16.04

1- Create a new virtual machine.

2- Name the virtual machine "harbor".

3- Choose the placement of the virtual machine.

4- Select the VMware vSphere compatibility.

5- Select Ubuntu as guest OS.

6- Change the number of CPU to 2.

7- Change the amount of RAM to 4 GB.

8- Change the size of the disk to 100 GB.

9- Configure on which network the virtual machine will be plugged.

10- Select your Ubuntu server 16.04 ISO.

11- Connect the CD drive at boot.

12- Create the virtual machine.

13- Power on the virtual machine.

14- Open the virtual console.

15- Choose the language of the Ubuntu installer.

16- Choose to install Ubuntu server.

17- Choose your language.

18- Configure the mapping of your keyboard.

19- The configuration of the network is failing as we don't use DHCP.

20- Configure the network manually.

21- Configure an IP address.

22- Configure the network mask.

23- Configure the gateway.

24- Configure the DNS server.

25- Configure the hostname.

26- Configure the domain if you have one.

27- Configure your username.

28- Choose a password.

29- Choose to encrypt or not your disk.

30- Validate your time zone.

31- Configure your disk.

32- Configure a proxy if you have one.

33- Choose to install the security updates automatically.

34- Select OpenSSH server.

35- Install GRUB.

36- Reboot the server.

Install Docker and docker-compose

1- SSH to your new Ubuntu 16.04 server.

$ ssh sguyennet@10.10.40.4

2- Add the Docker GPG key.

$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | \
sudo apt-key add -

3- Add the Docker repository.

$ sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"

4- Install Docker.

$ sudo apt-get update

$ sudo apt-get install docker-ce

5- Allow your user to use Docker without administrator privileges.

$ sudo usermod -aG docker $USER

6- Exit the SSH session.

$ exit

7- Log back in.

$ ssh sguyennet@10.10.40.4

8- Check that your user can use Docker.

$ docker info

9- Install docker-compose.

$ sudo apt-get install docker-compose

Generate self-signed certificates

1- Create a certificate authority.

$ openssl req \
-newkey rsa:4096 -nodes -sha256 -keyout ca.key \
-x509 -days 3650 -out ca.crt

2- Generate a certificate signing request.

$ openssl req \
-newkey rsa:4096 -nodes -sha256 -keyout harbor.inkubate.io.key \
-out harbor.inkubate.io.csr

3- Create a configuration file for the Subject Alternative Name.

$ vim extfile.cnf
subjectAltName = IP:10.10.40.4

4- Generate a certificate.

$ openssl x509 -req -days 3650 \
-in harbor.inkubate.io.csr -CA ca.crt -CAkey ca.key -CAcreateserial \
-extfile extfile.cnf \
-out harbor.inkubate.io.crt

5- Copy the certificate to /etc/ssl/certs.

$ sudo cp *.crt *.key /etc/ssl/certs

Install harbor

1- Download the Harbor online installer.

$ wget https://storage.googleapis.com/harbor-releases/harbor-online-installer-v1.5.2.tgz

2- Untar the installer.

$ tar xvzf harbor-online-installer-v1.5.2.tgz

3- Go to the Harbor directory.

$ cd harbor

4- Edit the Harbor configuration and change the following options in the file.

$ vim harbor.cfg
hostname = 10.10.40.4
ui_url_protocol = https
ssl_cert = /etc/ssl/certs/harbor.inkubate.io.crt
ssl_cert_key = /etc/ssl/certs/harbor.inkubate.io.key
harbor_admin_password = [your_harbor_admin_password]
db_password = [your_db_password]
clair_db_password = [your_clair_db_password]

5- Install Harbor.

$ sudo ./install.sh --with-notary --with-clair

Configuring the Docker daemon of the Kubernetes worker nodes

The following steps have to be repeated for each of your Kubernetes worker nodes.

1- Copy the certificate authority from the Harbor machine to your Kubernetes worker node.

$ scp ../ca.crt sguyennet@10.10.40.40:~

2- SSH to your Kubernetes worker nodes.

$ ssh sguyennet@10.10.40.40

3- Create a directory for the certificate authority.

$ sudo mkdir -p /etc/docker/certs.d/10.10.40.4

4- Move the certificate authority to the new directory.

$ sudo mv ca.crt /etc/docker/certs.d/10.10.40.4

5- Restart the Docker daemon.

$ sudo systemctl restart docker

Configuring Kubernetes

1- From your client machine, create a Kubernetes secret object for Harbor.

$ kubectl create secret docker-registry harbor \
--docker-server=https://10.10.40.4 \
--docker-username=admin \
--docker-email=sguyennet@inkubate.io \
--docker-password='[your_admin_harbor_password]'

Deploying a private container image

Configure the client machine Docker daemon

1- Download the certificate authority from the Harbor machine.

$ scp sguyennet@10.10.40.4:~/ca.crt .

2- Create a directory for the certificate authority.

$ sudo mkdir /etc/docker/certs.d/10.10.40.4

3- Move the certificate authority to the new directory.

$ sudo mv ca.crt /etc/docker/certs.d/10.10.40.4

4- Restart the Docker daemon.

$ sudo systemctl restart docker

Create a private image

1- Access the Harbor web interface, browse to https://10.10.40.4 and login with the admin user.

2- Create a new project.

3- Call it private and leave the public checkbox unchecked.

4- Download the public image from Kubernetes Up & Running book.

$ docker pull gcr.io/kuar-demo/kuard-amd64:1

5- Tag the image to use your Harbor private registry.

$ docker tag gcr.io/kuar-demo/kuard-amd64:1 10.10.40.4/private/kuard:v1

6- Login to the Harbor private registry.

$ docker login 10.10.40.4

7- Upload the image to the private Harbor registry.

$ docker push 10.10.40.4/private/kuard:v1

8- Check that the image has been properly uploaded to the Harbor private registry.

Deploy the private image on the Kubernetes cluster

1- Create a manifest for the deployment.

$ vim kuard-deployment.yaml
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  labels:
    run: kuard
  name: kuard
spec:
  replicas: 1
  selector:
    matchLabels:
      run: kuard
  template:
    metadata:
      labels:
        run: kuard
    spec:
      containers:
      - image: 10.10.40.4/private/kuard:v1
        name: kuard
      imagePullSecrets:
      - name: harbor

2- Launch the deployment.

$ kubectl apply -f kuard-deployment.yaml

3- Check that Kubernetes was able to download the private kuard image.

$ kubectl get pods