A secure Docker-registry behind a Kong Reverse Proxy - Part II

In this second part of our write-up we are going to deploy a secure docker-registry behind a Kong API Manager (of course, everything on a kubernetes cluster). However, this time we’ll be using some of the nice new features of Kong: stream_listen and TCPIngress.

In case you missed Part I you can go back and read that first to learn a bit more about the context, as well as see already one possible way to solve this. To recap, the scenario covered here should solve a docker registry that is reachable on yourdomain.com:5000 (when http://yourdomain.com/ and https://yourdomain.com/ are already in use).

Same disclaimer as for Part I: we assume a working knowledge on docker, kubernetes, helm (charts) and Kong (helm chart).

Overview of the Solution

Let’s look at the proposed solution:

What’s different from the first solution ? First of all, the URI - evidently. On top of that, we don’t have ‘normal’ https traffic through Kong, but ‘raw’ TCP (which will be tls flavour in our case, but Kong doesn’t care). The main change in the overview is that the “KongIngress” Custom Resource Definition (CRD) paired with the kubernetes Ingress resource have been replaced by the Kong CRD “TCPIngress”.

Step by Step

Besides a normal setup (default docker-registry helm chart deploy on the kubernetes with Kong already set up), there are a number of steps required to get this flow working.

Note that you have to use a recent helm chart for Kong, as we’re using pretty recent additions. We have been using version 1.7.0: https://github.com/Kong/charts/releases/tag/kong-1.7.0

The steps:

1. Set up a tcp stream in Kong itself (kong listens to a new port, and passes on the traffic)

2. Define a TCPIngress crd that links the kong stream (internal port) to the correct service

3. Ensure docker-registry has the appropriate certificate to use https (same as in part I)

Step 1: Set up a tcp stream

In the Kong chart (values.yaml file) there is an entire section for the proxy configuration. One of the recent additions is that you can define something that is called a “stream”.

An extract from https://github.com/Kong/charts/blob/master/charts/kong/values.yaml :

In our case (assuming proxy ingress is already enabled and tls set up correctly) the following lines need to be added:

proxy:
  # [..]
  stream:
    - containerPort: 5000
      servicePort: 5000

Step 2: TCPIngress definition

Also one of the ‘new-ish’ features is the CRD TCPIngress. This couples nicely onto the stream definitions. If you installed everything correctly (Kong, that is), it should have installed these CRDs. Check if they exist with a simple:

kubectl get crd

If you don’t feel like typing these commands, check out this blog for some handy tools.

The definition:

apiVersion: configuration.konghq.com/v1beta1
kind: TCPIngress
metadata:
  name: registry-ing
spec:
  rules:
  - backend:
      serviceName: registry-docker-registry
      servicePort: 5000
    port: 5000

Step 3: Setup TLS for Docker-registry

This step was also needed in the solution we presented in part I. For completeness we’ll reiterate it here:

# add this line to your custom values.yaml for docker-registry

tlsSecretName: yourdomain.le.tls

That’s it, you should be cooking now.

You’ll never guess what happens next

In conclusion, two ways in which you can set up docker-registry behind a Kong proxy, without using termination.

Hopefully this write-up provides some better understanding of the workings of Kong, Kubernetes and docker-registry. If you have further questions or remarks, leave us a comment below.

And the plot twist ? We had to do this specific setup for a customer, but typically we wouldn’t use docker-registry helm chart. Our goto solution, which abstracts away from many of the problems covered in Part I and II, is to use harbor … bet you didn’t see that one coming.