A secure Docker-registry behind a Kong Reverse Proxy - Part II
In this second part of our write-up we are going to deploy a secure docker-registry behind a Kong API Manager (of course, everything on a kubernetes cluster). However, this time we’ll be using some of the nice new features of Kong: stream_listen and TCPIngress.
In case you missed Part I you can go back and read that first to learn a bit more about the context, as well as see already one possible way to solve this. To recap, the scenario covered here should solve a docker registry that is reachable on yourdomain.com:5000 (when http://yourdomain.com/ and https://yourdomain.com/ are already in use).
Same disclaimer as for Part I: we assume a working knowledge on docker, kubernetes, helm (charts) and Kong (helm chart).
Overview of the Solution
Let’s look at the proposed solution:
What’s different from the first solution ? First of all, the URI - evidently. On top of that, we don’t have ‘normal’ https traffic through Kong, but ‘raw’ TCP (which will be tls flavour in our case, but Kong doesn’t care). The main change in the overview is that the “KongIngress” Custom Resource Definition (CRD) paired with the kubernetes Ingress resource have been replaced by the Kong CRD “TCPIngress”.
Step by Step
Besides a normal setup (default docker-registry helm chart deploy on the kubernetes with Kong already set up), there are a number of steps required to get this flow working.
Note that you have to use a recent helm chart for Kong, as we’re using pretty recent additions. We have been using version 1.7.0: https://github.com/Kong/charts/releases/tag/kong-1.7.0
Set up a tcp stream in Kong itself (kong listens to a new port, and passes on the traffic)
Define a TCPIngress crd that links the kong stream (internal port) to the correct service
Ensure docker-registry has the appropriate certificate to use https (same as in part I)
Step 1: Set up a tcp stream
In the Kong chart (values.yaml file) there is an entire section for the proxy configuration. One of the recent additions is that you can define something that is called a “stream”.
An extract from https://github.com/Kong/charts/blob/master/charts/kong/values.yaml :
In our case (assuming proxy ingress is already enabled and tls set up correctly) the following lines need to be added:
proxy: # [..] stream: - containerPort: 5000 servicePort: 5000
Step 2: TCPIngress definition
Also one of the ‘new-ish’ features is the CRD TCPIngress. This couples nicely onto the stream definitions. If you installed everything correctly (Kong, that is), it should have installed these CRDs. Check if they exist with a simple:
kubectl get crd
If you don’t feel like typing these commands, check out this blog for some handy tools.
apiVersion: configuration.konghq.com/v1beta1 kind: TCPIngress metadata: name: registry-ing spec: rules: - backend: serviceName: registry-docker-registry servicePort: 5000 port: 5000
Step 3: Setup TLS for Docker-registry
This step was also needed in the solution we presented in part I. For completeness we’ll reiterate it here:
# add this line to your custom values.yaml for docker-registry tlsSecretName: yourdomain.le.tls
That’s it, you should be cooking now.
You’ll never guess what happens next
In conclusion, two ways in which you can set up docker-registry behind a Kong proxy, without using termination.
Hopefully this write-up provides some better understanding of the workings of Kong, Kubernetes and docker-registry. If you have further questions or remarks, leave us a comment below.
And the plot twist ? We had to do this specific setup for a customer, but typically we wouldn’t use docker-registry helm chart. Our goto solution, which abstracts away from many of the problems covered in Part I and II, is to use harbor … bet you didn’t see that one coming.