We want to host a nice and secure container registry - to push docker images there and our kubernetes cluster would pull them from this registry. So came the idea to use the gitea over ssl. Part of Developer Tools: The Complete Guide to Modern Development Workflows 1) Gitea already has container registry 2) Apache as TLS terminating proxy will add HTTPS to our Gitea. 3) That how it's started... Rot CA, self-signed certificates... When we need the registry in k8s Unfortunately registry in kubernetes cluster created by kubespray doesn't work for me. to push into it we need to create temporary tunnel via kube-proxy after pushing into it mu current version of new cluster could not pull images from this internal registry After spending some awesome two evenings trying to fix it, I have decided just to use Gitea's internal container registry, just need to configure HTTPS access to it. Will make it public anyway, so docker and k8s doesn't have to do any docker logins. Maybe. Let's see how we go. Testing if it's working To test if container registry suits us we want to be able to push image there and to create deployment in k8s from that image sudo docker pull alpine:3.12.0 sudo docker images sudo docker tag a24bb4013296 localhost:5000/rg/alpine:version.3.12.0 sudo docker push localhost:5000/rg/alpine:version.3.12.0 Now xed alp1.yaml or nano alp1.yaml depending on what mood you are in and apiVersion: apps/v1 kind: Deployment metadata: name: alp-registry-test spec: replicas: 1 selector: matchLabels: app: alp-registry-test template: metadata: labels: app: alp-registry-test spec: containers: - name: alpine-test image: localhost:5000/rg/lpine:version.3.12.0 imagePullSecrets: - name: registry-secret Here is this file available for download Then creating this deployment kubectl create -f alp1.yaml kubectl get pods kubectl describe po alp-registry-test-5f5cb94b97-njsp2 # or whatever pod was created or wasn't created Yes I know about this bit imagePullSecrets: - name: registry-secret registry-secret is the name of the secret kubespray created. Cleanup kubectl delete -f alp1.yaml How A bunch of documentation is available on Gitea site: https-setup And on that page: https://docs.gitea.com/administration/reverse-proxies Step 1 - Install Apache and create simple test site Install Apache sudo apt install apache2 Check what we have in the firewall sudo ufw status If firewall is active think which port you want to expose via https and allow it. Standart apache configs are sudo ufw app list We might see something like Available applications: Apache Apache Full Apache Secure OpenSSH and to enable only port 443 we execute sudo ufw allow 'Apache Secure' ok. now check the service status sudo systemctl status apache2 Next - create simple virtual server to test the Apache sudo mkdir /var/www/reg.homelab sudo chown -R $USER:$USER /var/www/reg.homelab sudo chmod -R 755 /var/www/reg.homelab sudo nano /var/www/reg.homelab/index.html Put there <html> <head> <title>Welcome to reg.homelab!</title> </head> <body> <h1>Success! The reg.homelab virtual host is working!</h1> </body> </html> then sudo nano /etc/apache2/sites-available/reg.homelab.conf and put there <VirtualHost *:3080> ServerAdmin webmaster@localhost ServerName reg.homelab ServerAlias www.reg.homelab DocumentRoot /var/www/reg.homelab ErrorLog ${APACHE_LOG_DIR}/error.log CustomLog ${APACHE_LOG_DIR}/access.log combined </VirtualHost> then disable default site, enable this one and check how we are doing sudo a2ensite reg.homelab.conf sudo a2dissite 000-default.conf sudo apache2ctl configtest see something like this? AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName' directive globally to suppress this message then sudo nano /etc/apache2/apache2.conf add to the end: ServerName reg.homelab And this is still not the end! now need to remove port 80 binding attempt sudo nano /etc/apache2/ports.conf put there Listen 3030 ... Listen 443 And now sudo systemctl restart apache2 sudo systemctl status apache2 journalctl -xeu apache2.service curl localhost:3080 OK! Now navigate to :3080 Step 2 - Converting this site to insecure reverse proxy :) to Gitea sudo nano /etc/apache2/sites-available/reg.homelab.conf putting there <VirtualHost *:443> ServerAdmin webmaster@localhost ServerName reg.homelab ServerAlias www.reg.homelab ErrorLog ${APACHE_LOG_DIR}/error.log CustomLog ${APACHE_LOG_DIR}/access.log combined ProxyPreserveHost On ProxyRequests off AllowEncodedSlashes NoDecode ProxyPass / http://localhost:3000/ nocanon </VirtualHost> Do some config test sudo apache2ctl configtest Add some apache modules and restart Apache sudo a2enmod proxy proxy_http ssl sudo systemctl restart apache2 sudo systemctl status apache2 OK, now navigate to or curl # Yes, it is still a http, but on the 443 port curl http://localhost:443 http://<Server_IP_Address>:443/ Step 3 Self - signed root CA and site cert SweetHome-RootCA. CANAME=MostImportant-RootCA # optional, create a directory mkdir $CANAME cd $CANAME # generate aes encrypted private key openssl genrsa -aes256 -out $CANAME.key 4096 # create certificate, 1826 days = 5 years openssl req -x509 -new -nodes -key $CANAME.key -sha256 -days 1826 -out $CANAME.crt -subj '/CN=My Root CA/C=AT/ST=Vienna/L=Vienna/O=MyOrganisation' # create certificate for service MYCERT=reg.homelab openssl req -new -nodes -out $MYCERT.csr -newkey rsa:4096 -keyout $MYCERT.key -subj '/CN=My Firewall/C=AT/ST=Vienna/L=Vienna/O=MyOrganisation' # create a v3 ext file for SAN properties cat > $MYCERT.v3.ext << EOF authorityKeyIdentifier=keyid,issuer basicConstraints=CA:FALSE keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment subjectAltName = @alt_names [alt_names] DNS.1 = reg.homelablab DNS.2 = gitea.homelablab IP.1 = 192.168.0.10 IP.2 = 192.168.0.11 EOF openssl x509 -req -in $MYCERT.csr -CA $CANAME.crt -CAkey $CANAME.key -CAcreateserial -out $MYCERT.crt -days 730 -sha256 -extfile $MYCERT.v3.ext On machines connecting to gitea / registry Register root certificate on linux: sudo cp MostImportant-RootCA.crt /usr/local/share/ca-certificates sudo update-ca-certificates Registre root certificate on windows: Doubleclick file MostImportant-RootCA.crt Import to local user Select Truster Root CA. When prompted about importing untrusted certificate - click yes When git pull on windows says about Unable to resolve "unable to get local issuer certificate... you can tell git to use windows networking layer by git config --global http.sslbackend schannel Step 4 - Securing proxy with self-signed certificate https://httpd.apache.org/docs/2.4/ssl/ssl_howto.html Create a self-signed certificate IF you didn't in step 3 sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 \ -subj "/CN=reg.homelab" \ -addext "subjectAltName = DNS:reg.homelab" \ -keyout /etc/ssl/private/apache-selfsigned-reg.homelab.key \ -out /etc/ssl/certs/apache-selfsigned-reg.homelab.crt Or just take it from previous step sudo cp reg.homelab.crt /etc/ssl/certs/apache-selfsigned-reg.homelab.crt sudo cp reg.homelab.key /etc/ssl/private/apache-selfsigned-reg.homelab.key Again, open virtual host config sudo nano /etc/apache2/sites-available/reg.homelab.conf Add there to the bottom the SSL section with certs <VirtualHost *:443> ServerAdmin webmaster@localhost ServerName reg.homelab ServerAlias www.reg.homelab ErrorLog ${APACHE_LOG_DIR}/error.log CustomLog ${APACHE_LOG_DIR}/access.log combined ProxyPreserveHost On ProxyRequests off AllowEncodedSlashes NoDecode ProxyPass / http://localhost:3000/ nocanon SSLEngine on SSLCertificateFile /etc/ssl/certs/apache-selfsigned-reg.homelab.crt SSLCertificateKeyFile /etc/ssl/private/apache-selfsigned-reg.homelab.key </VirtualHost> Check config, restart server, check status and navigate to our Gitea over SSL sudo apache2ctl configtest sudo systemctl restart apache2 sudo systemctl status apache2 # goto: http://<Server_IP_Address>:443/ # or curl -k -v https://localhost the browser will warn about the self-signed certificate Your connection is not private Attackers might be trying to steal your information from reg.homelab (for example, passwords, messages or credit cards). Learn more NET::ERR_CERT_AUTHORITY_INVALID But we will ignore it, for now, till we use Let's Encrypt. Note: Once your Gitea SSL setup is working, consider backing up your server. See Backup and Restore Gitea server for detailed instructions. Now k8s test Do some DNSing... On each k8s node: sudo nano /etc/hosts and add there 192.168.18.200 gitea.homelab 192.168.18.200 reg.homelab Root CA On each k8s node: sudo cp SweetHome-RootCA.crt /usr/local/share/ca-certificates sudo update-ca-certificates And restart now Create secret with registry credentials https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/ sudo docker login reg.homelab kubectl create secret generic regcred --from-file=.dockerconfigjson=/home/rg/.docker/config.json --type=kubernetes.io/dockerconfigjson or kubectl create secret docker-registry regcred --docker-server=your-registry-server --docker-username=your-name --docker-password=your-pword --docker-email=your-email New docker image and k8s deployment sudo docker pull alpine:3.12.0 sudo docker images sudo docker tag a24bb4013296 reg.homelab/rg/alpine:version.3.12.0 sudo docker push reg.homelab/rg/alpine:version.3.12.0 Now nano alp2.yaml, file is [avaliable]avaliable apiVersion: apps/v1 kind: Deployment metadata: name: alp-registry-test spec: replicas: 1 selector: matchLabels: app: alp-registry-test template: metadata: labels: app: alp-registry-test spec: containers: - name: alpine-test image: reg.homelab/rg/alpine:version.3.12.0 imagePullSecrets: - name: regcred Here is this file available for download: Then creating this deployment kubectl create -f alp2.yaml kubectl get pods kubectl describe po alp... Cleanup kubectl delete -f alp2.yaml Useful links https://docs.gitea.com/administration/reverse-proxies https://httpd.apache.org/docs/2.4/ssl/ssl_howto.html https://docs.gitea.com/packages/packages/container Gitflow Explained: Steps, Alternatives, Pros, and Cons GIT Cheatsheet: Most useful GIT commands