How to install and use VMware Harbor private registry with Kubernetes
words - read.

How to install and use VMware Harbor private registry with Kubernetes

Harbor is a container image registry developed by VMware. It was recently handed over to the Cloud Native Computing Foundation, and its development is now driven by the open-source community. Harbor includes a couple of other open-source projects, like CoreOS/RedHat Clair which allows to scan images for security issues, or Notary which allows to sign your container images.
It also delivers a very nice web interface in which you can manage the various projects you are working on, as well as the permissions associated with these projects. In order to manage your users, it is also possible to link Harbor to your existing LDAP or Active Directory.

In this lab, we will configure Kubernetes to access a Harbor registry and deploy private container images to our Kubernetes cluster.

harbor logo

Requirements

You need a Kubernetes cluster using Docker as container engine. We will also install Harbor on a Ubuntu server, and your Kubernetes worker nodes need to be able to communicate with this server. In my case, the Kubernetes worker nodes are 10.10.40.40, 10.10.40.41, and 10.10.40.42. The Harbor virtual machine will be 10.10.40.4.

If you don't have a Kubernetes cluster already, you can refer to the Install and configure a multi-master Kubernetes cluster with kubeadm article, or to the Install and manage automatically a Kubernetes cluster on VMware vSphere with Terraform and Kubespray article if you are using VMware vSphere.

You will also need a client machine with kubectl configured to access your Kubernetes cluster, as well as a Docker engine installed on it.

Installation of Harbor

Install Ubuntu 16.04

1- Create a new virtual machine.

Install Ubuntu

Install Ubuntu

2- Name the virtual machine "harbor".

Install Ubuntu

3- Choose the placement of the virtual machine.

Install Ubuntu

Install Ubuntu

4- Select the VMware vSphere compatibility.

Install Ubuntu

5- Select Ubuntu as guest OS.

Install Ubuntu

6- Change the number of CPU to 2.

Install Ubuntu

7- Change the amount of RAM to 4 GB.

Install Ubuntu

8- Change the size of the disk to 100 GB.

Install Ubuntu

9- Configure on which network the virtual machine will be plugged.

Install Ubuntu

Install Ubuntu

10- Select your Ubuntu server 16.04 ISO.

Install Ubuntu

Install Ubuntu

11- Connect the CD drive at boot.

Install Ubuntu

12- Create the virtual machine.

Install Ubuntu

Install Ubuntu

13- Power on the virtual machine.

Install Ubuntu

14- Open the virtual console.

Install Ubuntu

15- Choose the language of the Ubuntu installer.

Install Ubuntu

16- Choose to install Ubuntu server.

Install Ubuntu

17- Choose your language.

Install Ubuntu

Install Ubuntu

18- Configure the mapping of your keyboard.

Install Ubuntu

Install Ubuntu

Install Ubuntu

19- The configuration of the network is failing as we don't use DHCP.

Install Ubuntu

20- Configure the network manually.

Install Ubuntu

21- Configure an IP address.

Install Ubuntu

22- Configure the network mask.

Install Ubuntu

23- Configure the gateway.

Install Ubuntu

24- Configure the DNS server.

Install Ubuntu

25- Configure the hostname.

Install Ubuntu

26- Configure the domain if you have one.

Install Ubuntu

27- Configure your username.

Install Ubuntu

Install Ubuntu

28- Choose a password.

Install Ubuntu

Install Ubuntu

29- Choose to encrypt or not your disk.

Install Ubuntu

30- Validate your time zone.

Install Ubuntu

31- Configure your disk.

Install Ubuntu

Install Ubuntu

Install Ubuntu

Install Ubuntu

Install Ubuntu

32- Configure a proxy if you have one.

Install Ubuntu

33- Choose to install the security updates automatically.

Install Ubuntu

34- Select OpenSSH server.

Install Ubuntu

35- Install GRUB.

Install Ubuntu

36- Reboot the server.

Install Ubuntu

Install Docker and docker-compose

1- SSH to your new Ubuntu 16.04 server.

$ ssh sguyennet@10.10.40.4

2- Add the Docker GPG key.

$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | \
sudo apt-key add -

3- Add the Docker repository.

$ sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"

4- Install Docker.

$ sudo apt-get update

$ sudo apt-get install docker-ce

5- Allow your user to use Docker without administrator privileges.

$ sudo usermod -aG docker $USER

6- Exit the SSH session.

$ exit

7- Log back in.

$ ssh sguyennet@10.10.40.4

8- Check that your user can use Docker.

$ docker info

9- Install docker-compose.

$ sudo apt-get install docker-compose

Generate self-signed certificates

1- Create a certificate authority.

$ openssl req \
-newkey rsa:4096 -nodes -sha256 -keyout ca.key \
-x509 -days 3650 -out ca.crt

2- Generate a certificate signing request.

$ openssl req \
-newkey rsa:4096 -nodes -sha256 -keyout harbor.inkubate.io.key \
-out harbor.inkubate.io.csr

3- Create a configuration file for the Subject Alternative Name.

$ vim extfile.cnf
subjectAltName = IP:10.10.40.4

4- Generate a certificate.

$ openssl x509 -req -days 3650 \
-in harbor.inkubate.io.csr -CA ca.crt -CAkey ca.key -CAcreateserial \
-extfile extfile.cnf \
-out harbor.inkubate.io.crt

5- Copy the certificate to /etc/ssl/certs.

$ sudo cp *.crt *.key /etc/ssl/certs

Install harbor

1- Download the Harbor online installer.

$ wget https://storage.googleapis.com/harbor-releases/harbor-online-installer-v1.5.2.tgz

2- Untar the installer.

$ tar xvzf harbor-online-installer-v1.5.2.tgz

3- Go to the Harbor directory.

$ cd harbor

4- Edit the Harbor configuration and change the following options in the file.

$ vim harbor.cfg
hostname = 10.10.40.4
ui_url_protocol = https
ssl_cert = /etc/ssl/certs/harbor.inkubate.io.crt
ssl_cert_key = /etc/ssl/certs/harbor.inkubate.io.key
harbor_admin_password = [your_harbor_admin_password]
db_password = [your_db_password]
clair_db_password = [your_clair_db_password]

5- Install Harbor.

$ sudo ./install.sh --with-notary --with-clair

Configuring the Docker daemon of the Kubernetes worker nodes

The following steps have to be repeated for each of your Kubernetes worker nodes.

1- Copy the certificate authority from the Harbor machine to your Kubernetes worker node.

$ scp ../ca.crt sguyennet@10.10.40.40:~

2- SSH to your Kubernetes worker nodes.

$ ssh sguyennet@10.10.40.40

3- Create a directory for the certificate authority.

$ sudo mkdir -p /etc/docker/certs.d/10.10.40.4

4- Move the certificate authority to the new directory.

$ sudo mv ca.crt /etc/docker/certs.d/10.10.40.4

5- Restart the Docker daemon.

$ sudo systemctl restart docker

Configuring Kubernetes

1- From your client machine, create a Kubernetes secret object for Harbor.

$ kubectl create secret docker-registry harbor \
--docker-server=https://10.10.40.4 \
--docker-username=admin \
--docker-email=sguyennet@inkubate.io \
--docker-password='[your_admin_harbor_password]'

Deploying a private container image

Configure the client machine Docker daemon

1- Download the certificate authority from the Harbor machine.

$ scp sguyennet@10.10.40.4:~/ca.crt .

2- Create a directory for the certificate authority.

$ sudo mkdir /etc/docker/certs.d/10.10.40.4

3- Move the certificate authority to the new directory.

$ sudo mv ca.crt /etc/docker/certs.d/10.10.40.4

4- Restart the Docker daemon.

$ sudo systemctl restart docker

Create a private image

1- Access the Harbor web interface, browse to https://10.10.40.4 and login with the admin user.

Habor registry

2- Create a new project.

Habor registry

3- Call it private and leave the public checkbox unchecked.

Habor registry

4- Download the public image from Kubernetes Up & Running book.

$ docker pull gcr.io/kuar-demo/kuard-amd64:1

5- Tag the image to use your Harbor private registry.

$ docker tag gcr.io/kuar-demo/kuard-amd64:1 10.10.40.4/private/kuard:v1

6- Login to the Harbor private registry.

$ docker login 10.10.40.4

7- Upload the image to the private Harbor registry.

$ docker push 10.10.40.4/private/kuard:v1

8- Check that the image has been properly uploaded to the Harbor private registry.

Harbor registry

Harbor registry

Deploy the private image on the Kubernetes cluster

1- Create a manifest for the deployment.

$ vim kuard-deployment.yaml
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  labels:
    run: kuard
  name: kuard
spec:
  replicas: 1
  selector:
    matchLabels:
      run: kuard
  template:
    metadata:
      labels:
        run: kuard
    spec:
      containers:
      - image: 10.10.40.4/private/kuard:v1
        name: kuard
      imagePullSecrets:
      - name: harbor

2- Launch the deployment.

$ kubectl apply -f kuard-deployment.yaml

3- Check that Kubernetes was able to download the private kuard image.

$ kubectl get pods

Comments

comments powered by Disqus