diff --git a/README.md b/README.md index 0583a6e..c3092e0 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,4 @@ + # AWX on Single Node K3s An example implementation of AWX on single node K3s using AWX Operator, with easy-to-use simplified configuration with ownership of data and passwords. @@ -7,25 +8,25 @@ An example implementation of AWX on single node K3s using AWX Operator, with eas - Fixed (configurable) passwords for AWX and PostgreSQL - Fixed (configurable) versions of AWX and PostgreSQL + ## Table of Contents -- [AWX on Single Node K3s](#awx-on-single-node-k3s) - - [Table of Contents](#table-of-contents) - - [Environment](#environment) - - [References](#references) - - [Procedure](#procedure) - - [Prepare CentOS 8 host](#prepare-centos-8-host) - - [Install K3s](#install-k3s) - - [Install AWX Operator](#install-awx-operator) - - [Prepare required files](#prepare-required-files) - - [Deploy AWX](#deploy-awx) - - [Backing up and Restoring using AWX Operator](#backing-up-and-restoring-using-awx-operator) - - [Backing up using AWX Operator](#backing-up-using-awx-operator) - - [Prepare for Backup](#prepare-for-backup) - - [Invoke Manual Backup](#invoke-manual-backup) - - [Restoring using AWX Operator](#restoring-using-awx-operator) - - [Prepare for Restore](#prepare-for-restore) - - [Invoke Manual Restore](#invoke-manual-restore) +- [Environment](#environment) +- [References](#references) +- [Procedure](#procedure) + - [Prepare CentOS 8 host](#prepare-centos-8-host) + - [Install K3s](#install-k3s) + - [Install AWX Operator](#install-awx-operator) + - [Prepare required files](#prepare-required-files) + - [Deploy AWX](#deploy-awx) +- [Backing up and Restoring using AWX Operator](#backing-up-and-restoring-using-awx-operator) + - [Backing up using AWX Operator](#backing-up-using-awx-operator) + - [Prepare for Backup](#prepare-for-backup) + - [Invoke Manual Backup](#invoke-manual-backup) + - [Restoring using AWX Operator](#restoring-using-awx-operator) + - [Prepare for Restore](#prepare-for-restore) + - [Invoke Manual Restore](#invoke-manual-restore) +- [Deploy Private Container Registry](#deploy-private-container-registry) ## Environment @@ -325,3 +326,11 @@ Then restore the Secret for TLS manually (or create newly using original certifi ```bash kubectl apply -f awx-secret-tls.yaml ``` + +## Deploy Private Container Registry + +To use Execution Environments in AWX (AWX-EE), we have to push the container image built with `ansible-builder` to the container registry. + +If we don't want to push our container images to Docker Hub or other cloud services, we can deploy a private container registry on K3s. + +See [📝 `registry/README.md`](registry/README.md) for instructions. diff --git a/registry/README.md b/registry/README.md new file mode 100644 index 0000000..d66b0a3 --- /dev/null +++ b/registry/README.md @@ -0,0 +1,230 @@ + +# Deploy Private Container Registry + +Deploying your private container registry on your K3s to use with AWX. + + +## Table of Contents + +- [Procedure](#procedure) + - [Prepare required files](#prepare-required-files) + - [Deploy Private Container Registry](#deploy-private-container-registry) +- [Quick Testing](#quick-testing) + - [Testing with Docker](#testing-with-docker) + - [Digging into the Registry](#digging-into-the-registry) +- [(Optional) Use as Private Container Registry for K3s](#optional-use-as-private-container-registry-for-k3s) + - [Procedure](#procedure-1) + - [Testing](#testing) + +## Procedure + +### Prepare required files + +Generate a Self-Signed Certificate. Note that IP address can't be specified. + +```bash +REGISTRY_HOST="registry.example.com" +openssl req -x509 -nodes -days 3650 -newkey rsa:2048 -out ./registry/tls.crt -keyout ./registry/tls.key -subj "/CN=${REGISTRY_HOST}/O=${REGISTRY_HOST}" -addext "subjectAltName = DNS:${REGISTRY_HOST}" +``` + +Modify `hosts` and `host` in `registry/ingress.yaml`. + +```yaml +... + - hosts: + - registry.example.com 👈👈👈 + secretName: registry-secret-tls + rules: + - host: registry.example.com 👈👈👈 +... +``` + +Generate `htpasswd` string by your own username and password to use as the user for the container registry. + +```bash +$ kubectl run htpasswd -it --restart=Never --image httpd:2.4 --rm -- htpasswd -nbB reguser Registry123! +reguser:$2y$05$VLMvcWCPF0VUuHi0BXBz7eoXGZ6KRl1gataiqTXz4DdSVIXGloKiq + +pod "htpasswd" deleted +``` + +Replace `htpasswd` in `registry/configmap.yaml` with your own `htpasswd` string that generated above. + +```yaml +... + htpasswd: |- + reguser:$2y$05$VLMvcWCPF0VUuHi0BXBz7eoXGZ6KRl1gataiqTXz4DdSVIXGloKiq 👈👈👈 +``` + +Prepare directories for Persistent Volumes defined in `registry/pv.yaml`. + +```bash +sudo mkdir -p /data/registry +``` + +### Deploy Private Container Registry + +Deploy Private Container Registry. + +```bash +kubectl apply -k registry +``` + +Required resources has been deployed in `registry` namespace. + +```bash +$ kubectl get all -n registry +NAME READY STATUS RESTARTS AGE +pod/registry-5b4f874b77-9gb64 1/1 Running 0 27s + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/registry-service ClusterIP 10.43.50.156 5000/TCP 28s + +NAME READY UP-TO-DATE AVAILABLE AGE +deployment.apps/registry 1/1 1 1 27s + +NAME DESIRED CURRENT READY AGE +replicaset.apps/registry-5b4f874b77 1 1 1 27s +``` + +Now your container registry can be used through `registry.example.com` or the hostname you specified. + +## Quick Testing + +### Testing with Docker + +Add your registry as an insecure registry and restart Docker daemon. + +```bash +sudo tee /etc/docker/daemon.json < + ------ + \ + \ + \ + ## . + ## ## ## == + ## ## ## ## === + /""""""""""""""""___/ === + ~~~ {~~ ~~~~ ~~~ ~~~~ ~~ ~ / ===- ~~~ + \______ o __/ + \ \ __/ + \____\______/ +``` + +### Digging into the Registry + +There is an useful CLI tool called [**reg**](https://github.com/genuinetools/reg) to dig into the container registry. + +```bash +# Install reg +sudo curl -fSL https://github.com/genuinetools/reg/releases/download/v0.16.1/reg-linux-amd64 -o /usr/local/bin/reg +sudo chmod +x /usr/local/bin/reg + +# List repositories and tags in the container registry +reg ls -k registry.example.com +reg tags -k registry.example.com/reguser/whalesay + +# Delete tags on the registry +reg rm -k registry.example.com/reguser/whalesay:latest +``` + +## (Optional) Use as Private Container Registry for K3s + +Optionally, this registry can also be registered as a private container registry for K3s. + +### Procedure + +To achieve this, create a `registries.yaml` and restart K3s. + +```bash +sudo tee /etc/rancher/k3s/registries.yaml < + ------ + \ + \ + \ + ## . + ## ## ## == + ## ## ## ## === + /""""""""""""""""___/ === + ~~~ {~~ ~~~~ ~~~ ~~~~ ~~ ~ / ===- ~~~ + \______ o __/ + \ \ __/ + \____\______/ +pod "whalesay" deleted +``` diff --git a/registry/configmap.yaml b/registry/configmap.yaml new file mode 100644 index 0000000..fede2a3 --- /dev/null +++ b/registry/configmap.yaml @@ -0,0 +1,35 @@ +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: registry-configmap + labels: + app: registry +data: + config.yml: |- + version: 0.1 + log: + fields: + service: registry + storage: + cache: + blobdescriptor: inmemory + filesystem: + rootdirectory: /var/lib/registry + delete: + enabled: true + http: + addr: :5000 + headers: + X-Content-Type-Options: [nosniff] + auth: + htpasswd: + realm: basic-realm + path: /etc/docker/registry/htpasswd + health: + storagedriver: + enabled: true + interval: 10s + threshold: 3 + htpasswd: |- + reguser:$2y$05$VLMvcWCPF0VUuHi0BXBz7eoXGZ6KRl1gataiqTXz4DdSVIXGloKiq diff --git a/registry/deployment.yaml b/registry/deployment.yaml new file mode 100644 index 0000000..02f380e --- /dev/null +++ b/registry/deployment.yaml @@ -0,0 +1,36 @@ +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: registry + labels: + app: registry +spec: + replicas: 1 + selector: + matchLabels: + app: registry + template: + metadata: + labels: + app: registry + spec: + containers: + - name: registry + image: registry:2.7 + ports: + - containerPort: 5000 + protocol: TCP + volumeMounts: + - name: registry-volume + mountPath: /var/lib/registry + - name: registry-config + mountPath: /etc/docker/registry + volumes: + - name: registry-volume + persistentVolumeClaim: + claimName: registry-claim + - name: registry-config + configMap: + defaultMode: 420 + name: registry-configmap diff --git a/registry/ingress.yaml b/registry/ingress.yaml new file mode 100644 index 0000000..42e3e93 --- /dev/null +++ b/registry/ingress.yaml @@ -0,0 +1,21 @@ +--- +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + name: registry-ingress +spec: + tls: + - hosts: + - registry.example.com + secretName: registry-secret-tls + rules: + - host: registry.example.com + http: + paths: + - path: / + pathType: ImplementationSpecific + backend: + service: + name: registry-service + port: + number: 5000 diff --git a/registry/kustomization.yaml b/registry/kustomization.yaml new file mode 100644 index 0000000..f5507c9 --- /dev/null +++ b/registry/kustomization.yaml @@ -0,0 +1,23 @@ +--- +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +namespace: registry + +generatorOptions: + disableNameSuffixHash: true + +secretGenerator: + - name: registry-secret-tls + type: kubernetes.io/tls + files: + - tls.crt + - tls.key + +resources: + - namespace.yaml + - pv.yaml + - pvc.yaml + - ingress.yaml + - service.yaml + - configmap.yaml + - deployment.yaml diff --git a/registry/namespace.yaml b/registry/namespace.yaml new file mode 100644 index 0000000..3cf8b7b --- /dev/null +++ b/registry/namespace.yaml @@ -0,0 +1,5 @@ +--- +apiVersion: v1 +kind: Namespace +metadata: + name: registry diff --git a/registry/pv.yaml b/registry/pv.yaml new file mode 100644 index 0000000..bab51bf --- /dev/null +++ b/registry/pv.yaml @@ -0,0 +1,14 @@ +--- +apiVersion: v1 +kind: PersistentVolume +metadata: + name: registry-volume +spec: + accessModes: + - ReadWriteOnce + persistentVolumeReclaimPolicy: Retain + capacity: + storage: 5Gi + storageClassName: registry-volume + hostPath: + path: /data/registry diff --git a/registry/pvc.yaml b/registry/pvc.yaml new file mode 100644 index 0000000..f3156a5 --- /dev/null +++ b/registry/pvc.yaml @@ -0,0 +1,13 @@ +--- +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: registry-claim +spec: + accessModes: + - ReadWriteOnce + volumeMode: Filesystem + resources: + requests: + storage: 5Gi + storageClassName: registry-volume diff --git a/registry/service.yaml b/registry/service.yaml new file mode 100644 index 0000000..4f2d865 --- /dev/null +++ b/registry/service.yaml @@ -0,0 +1,11 @@ +--- +apiVersion: v1 +kind: Service +metadata: + name: registry-service +spec: + ports: + - protocol: TCP + port: 5000 + selector: + app: registry