expand/awx-on-k3s
1
0
Fork 0
mirror of https://github.com/Expand-sys/awx-on-k3s synced 2025-12-15 21:42:15 +11:00
Code Issues Projects Releases Packages Wiki Activity Actions

feat: bump awx operator version to 0.26.0 and modify pv and hostpath for psql 13

Browse source
This commit is contained in:
kurokobo 2022-08-11 11:13:06 +09:00
parent d83b2d5dbe
commit 307c3ac877
18 changed files with 133 additions and 77 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

41
README.md
View file

@ -8,6 +8,8 @@ An example implementation of AWX on single node K3s using AWX Operator, with eas
- Fixed (configurable) passwords for AWX and PostgreSQL
- Fixed (configurable) versions of AWX and PostgreSQL
**If you want to view the guide for the specific version of AWX Operator, switch the page to the desired tag instead of the `main` branch.**
<!-- omit in toc -->
## Table of Contents
@ -27,17 +29,17 @@ An example implementation of AWX on single node K3s using AWX Operator, with eas
- Tested on:
- CentOS Stream 8 (Minimal)
- K3s v1.23.8+k3s2
- K3s v1.24.3+k3s1
- Products that will be deployed:
- AWX Operator 0.25.0
- AWX Operator 0.26.0
- AWX 21.3.0
- PostgreSQL 12
- PostgreSQL 13
## References
- [K3s - Lightweight Kubernetes](https://rancher.com/docs/k3s/latest/en/)
- [INSTALL.md on ansible/awx](https://github.com/ansible/awx/blob/21.3.0/INSTALL.md) @21.3.0
- [README.md on ansible/awx-operator](https://github.com/ansible/awx-operator/blob/0.25.0/README.md) @0.25.0
- [README.md on ansible/awx-operator](https://github.com/ansible/awx-operator/blob/0.26.0/README.md) @0.26.0
## Requirements
@ -55,10 +57,15 @@ An example implementation of AWX on single node K3s using AWX Operator, with eas
### Prepare CentOS Stream 8 host
Disable Firewalld. This is [recommended by K3s](https://rancher.com/docs/k3s/latest/en/advanced/#additional-preparation-for-red-hat-centos-enterprise-linux).
Disable Firewalld and nm-cloud-setup if enabled. This is [recommended by K3s](https://rancher.com/docs/k3s/latest/en/advanced/#additional-preparation-for-red-hat-centos-enterprise-linux).
```bash
# Disable Firewalld
sudo systemctl disable firewalld --now
# Disable nm-cloud-setup if exists and enabled
systemctl disable nm-cloud-setup.service nm-cloud-setup.timer
reboot
```
Install required packages to deploy AWX Operator and AWX.
@ -83,7 +90,7 @@ Install specified version of AWX Operator. Note that this procedure is applicabl
cd ~
git clone https://github.com/ansible/awx-operator.git
cd awx-operator
git checkout 0.25.0
git checkout 0.26.0
```
Export the name of the namespace where you want to deploy AWX Operator as the environment variable `NAMESPACE` and run `make deploy`. The default namespace is `awx`.
@ -114,10 +121,13 @@ replicaset.apps/awx-operator-controller-manager-68d787cfbd 1 1
Clone this repository and change directory.
If you want to use files suitable for the specific version of AWX Operator, [refer tags in this repository](https://github.com/kurokobo/awx-on-k3s/tags) and specify desired tag in `git checkout`.
```bash
cd ~
git clone https://github.com/kurokobo/awx-on-k3s.git
cd awx-on-k3s
git checkout 0.26.0
```
Generate a Self-Signed certificate. Note that IP address can't be specified. If you want to use a certificate from public ACME CA such as Let's Encrypt or ZeroSSL instead of Self-Signed certificate, follow the guide on [📁 **Use SSL Certificate from Public ACME CA**](acme) first and come back to this step when done.
@ -146,7 +156,7 @@ Modify two `password`s in `base/kustomization.yaml`. Note that the `password` un
- name: awx-postgres-configuration
type: Opaque
literals:
- host=awx-postgres
- host=awx-postgres-13
- port=5432
- database=awx
- username=awx
@ -163,9 +173,9 @@ Modify two `password`s in `base/kustomization.yaml`. Note that the `password` un
Prepare directories for Persistent Volumes defined in `base/pv.yaml`. These directories will be used to store your databases and project files. Note that the size of the PVs and PVCs are specified in some of the files in this repository, but since their backends are `hostPath`, its value is just like a label and there is no actual capacity limitation.
```bash
sudo mkdir -p /data/postgres
sudo mkdir -p /data/postgres-13
sudo mkdir -p /data/projects
sudo chmod 755 /data/postgres
sudo chmod 755 /data/postgres-13
sudo chown 1000:0 /data/projects
```
@ -190,7 +200,7 @@ $ kubectl -n awx logs -f deployments/awx-operator-controller-manager -c awx-mana
...
----- Ansible Task Status Event StdOut (awx.ansible.com/v1beta1, Kind=AWX, awx/awx) -----
PLAY RECAP *********************************************************************
localhost : ok=67 changed=0 unreachable=0 failed=0 skipped=44 rescued=0 ignored=0
localhost : ok=71 changed=0 unreachable=0 failed=0 skipped=48 rescued=0 ignored=0
```
Required objects has been deployed next to AWX Operator in `awx` namespace.
@ -202,12 +212,12 @@ awx.awx.ansible.com/awx 4m17s
NAME READY STATUS RESTARTS AGE
pod/awx-operator-controller-manager-68d787cfbd-j6k7z 2/2 Running 0 7m43s
pod/awx-postgres-0 1/1 Running 0 4m6s
pod/awx-postgres-13-0 1/1 Running 0 4m6s
pod/awx-84d5c45999-h7xm4 4/4 Running 0 3m59s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/awx-operator-controller-manager-metrics-service ClusterIP 10.43.134.67 <none> 8443/TCP 7m43s
service/awx-postgres ClusterIP None <none> 5432/TCP 4m6s
service/awx-postgres-13 ClusterIP None <none> 5432/TCP 4m6s
service/awx-service ClusterIP 10.43.232.137 <none> 80/TCP 4m
NAME READY UP-TO-DATE AVAILABLE AGE
@ -218,21 +228,18 @@ NAME DESIRED CURRENT
replicaset.apps/awx-operator-controller-manager-68d787cfbd 1 1 1 7m43s
replicaset.apps/awx-84d5c45999 1 1 1 3m59s
NAME READY AGE
statefulset.apps/awx-postgres 1/1 4m6s
NAME READY AGE
statefulset.apps/awx-postgres-13 1/1 4m6s
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress.networking.k8s.io/awx-ingress <none> awx.example.com 192.168.0.100 80, 443 4m
NAME TYPE DATA AGE
secret/default-token-6tp55 kubernetes.io/service-account-token 3 7m43s
secret/awx-operator-controller-manager-token-sz6wq kubernetes.io/service-account-token 3 7m43s
secret/awx-admin-password Opaque 1 4m17s
secret/awx-postgres-configuration Opaque 6 4m17s
secret/awx-secret-tls kubernetes.io/tls 2 4m17s
secret/redhat-operators-pull-secret kubernetes.io/dockerconfigjson 1 4m17s
secret/awx-app-credentials Opaque 3 4m2s
secret/awx-token-jfndh kubernetes.io/service-account-token 3 4m2s
secret/awx-secret-key Opaque 1 4m13s
secret/awx-broadcast-websocket Opaque 1 4m9s
```

2
acme/README.md
View file

@ -42,7 +42,7 @@ This guide does not provide any information how to configure Azure, other DNS se
Deploy cert-manager first.
```bash
kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.8.2/cert-manager.yaml
kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.9.1/cert-manager.yaml
```
Ensure the pods in `cert-manager` namespace are running.

7
backup/ansible/README.md
View file

@ -32,7 +32,7 @@ An example simple playbook for Ansible is also provided in this repository. This
| - | - | - |
| `awxbackup_namespace` | The name of the NameSpace where the `AWXBackup` resource will be created. | `awx` |
| `awxbackup_name` | The name of the `AWXBackup` resource. Dynamically generated using execution time by default. | `awxbackup-{{ lookup('pipe', 'date +%Y-%m-%d-%H-%M-%S') }}` |
| `awxbackup_spec` | The `spec` of the `AWXBackup` resource. Refer [official documentation](https://github.com/ansible/awx-operator/tree/0.25.0/roles/backup) for acceptable fields. | `deployment_name: awx`<br>`backup_pvc: awx-backup-claim`<br>`clean_backup_on_delete: true` |
| `awxbackup_spec` | The `spec` of the `AWXBackup` resource. Refer [official documentation](https://github.com/ansible/awx-operator/tree/0.26.0/roles/backup) for acceptable fields. | `deployment_name: awx`<br>`backup_pvc: awx-backup-claim`<br>`clean_backup_on_delete: true` |
| `awxbackup_timeout` | Time to wait for backup to complete, in seconds. If exceeded, the playbook will fail. | `600` |
| `awxbackup_keep_days` | Number of days to keep `AWXBackup` resources. `AWXBackup` resources older than this value will be deleted by this playbook. Set `0` to keep forever. | `30` |
@ -58,6 +58,11 @@ rolebinding.rbac.authorization.k8s.io/awx-backup created
Obtain the API Token which required to authenticate the Kubernetes API. This token will be used later.
```bash
# Kubernetes 1.24 or later
$ kubectl -n awx create token awx-backup --duration=87600h
eyJhbGciOiJSUzI...hcGsPI5MzmaMHQvw
# Kubernetes 1.23 or earlier
$ SECRET=$(kubectl -n ${NAMESPACE} get sa awx-backup -o jsonpath='{.secrets[0].name}')
$ kubectl -n ${NAMESPACE} get secret ${SECRET} -o jsonpath='{.data.token}' | base64 -d
eyJhbGciOiJSUzI...hcGsPI5MzmaMHQvw

6
base/awx.yaml
View file

@ -5,8 +5,8 @@ metadata:
name: awx
spec:
# These parameters are designed for use with:
# - AWX Operator: 0.25.0
# https://github.com/ansible/awx-operator/blob/0.25.0/README.md
# - AWX Operator: 0.26.0
# https://github.com/ansible/awx-operator/blob/0.26.0/README.md
# - AWX: 21.3.0
# https://github.com/ansible/awx/blob/21.3.0/INSTALL.md
@ -27,6 +27,8 @@ spec:
projects_persistence: true
projects_existing_claim: awx-projects-claim
postgres_init_container_resource_requirements: {}
postgres_resource_requirements: {}
web_resource_requirements: {}
task_resource_requirements: {}
ee_resource_requirements: {}

10
base/kustomization.yaml
View file

@ -16,7 +16,7 @@ secretGenerator:
- name: awx-postgres-configuration
type: Opaque
literals:
- host=awx-postgres
- host=awx-postgres-13
- port=5432
- database=awx
- username=awx
@ -28,6 +28,14 @@ secretGenerator:
literals:
- password=Ansible123!
# If you want to specify SECRET_KEY for your AWX manually, uncomment following lines and change the value.
# Refer AAC documentation for detail about SECRET_KEY.
# https://docs.ansible.com/automation-controller/latest/html/administration/secret_handling.html
#- name: awx-secret-key
# type: Opaque
# literals:
# - secret_key=MySuperSecureSecretKey123!
# Temporary workaround to suppress errors from AWX Operator 0.21.0.
# Refer: https://github.com/kurokobo/awx-on-k3s/issues/74
- name: redhat-operators-pull-secret

4
base/pv.yaml
View file

@ -2,7 +2,7 @@
apiVersion: v1
kind: PersistentVolume
metadata:
name: awx-postgres-volume
name: awx-postgres-13-volume
spec:
accessModes:
- ReadWriteOnce
@ -11,7 +11,7 @@ spec:
storage: 8Gi
storageClassName: awx-postgres-volume
hostPath:
path: /data/postgres
path: /data/postgres-13
---
apiVersion: v1

2
galaxy/README.md
View file

@ -298,7 +298,7 @@ Modify two `password`s in `pulp/kustomization.yaml`.
Prepare directories for Persistent Volumes defined in `pulp/pv.yaml`.
```bash
sudo mkdir -p /data/galaxy/postgres
sudo mkdir -p /data/galaxy/postgres-13
sudo mkdir -p /data/galaxy/redis
sudo mkdir -p /data/galaxy/pulp
```

2
galaxy/pulp/pv.yaml
View file

@ -11,7 +11,7 @@ spec:
storage: 8Gi
storageClassName: galaxy-postgres-volume
hostPath:
path: /data/galaxy/postgres
path: /data/galaxy/postgres-13
---
apiVersion: v1

2
git/deployment.yaml
View file

@ -17,7 +17,7 @@ spec:
spec:
containers:
- name: git
image: gitea/gitea:1.16
image: gitea/gitea:1.17
ports:
- name: gitea-http
containerPort: 3000

14
restore/README.md
View file

@ -27,19 +27,19 @@ If your AWX instance is running, it is recommended that it be deleted along with
```bash
# Delete AWX resource, PVC, and PV
kubectl -n awx delete awx awx
kubectl -n awx delete pvc postgres-awx-postgres-0
kubectl delete pv awx-postgres-volume
kubectl -n awx delete pvc postgres-13-awx-postgres-13-0
kubectl delete pv awx-postgres-13-volume
# Delete any data in the PV
sudo rm -rf /data/postgres
sudo rm -rf /data/postgres-13
```
Then prepare directories for your PVs. `/data/projects` is required if you are restoring the entire AWX to a new environment.
```bash
sudo mkdir -p /data/postgres
sudo mkdir -p /data/postgres-13
sudo mkdir -p /data/projects
sudo chmod 755 /data/postgres
sudo chmod 755 /data/postgres-13
sudo chown 1000:0 /data/projects
```
@ -67,7 +67,6 @@ If you want to restore from AWXBackup object, specify its name in `restore/awxre
```yaml
...
# Parameters to restore from AWXBackup object
backup_pvc_namespace: awx
backup_name: awxbackup-2021-06-06 👈👈👈
...
```
@ -77,7 +76,6 @@ If the AWXBackup object no longer exists, place the backup files and specify the
```yaml
...
# Parameters to restore from existing files on PVC (without AWXBackup object)
backup_pvc_namespace: awx
backup_pvc: awx-backup-claim 👈👈👈
backup_dir: /backups/tower-openshift-backup-2021-06-06-10:51:49 👈👈👈
...
@ -102,7 +100,7 @@ $ kubectl -n awx logs -f deployments/awx-operator-controller-manager -c awx-mana
...
----- Ansible Task Status Event StdOut (awx.ansible.com/v1beta1, Kind=AWX, awx/awx) -----
PLAY RECAP *********************************************************************
localhost : ok=69 changed=0 unreachable=0 failed=0 skipped=42 rescued=0 ignored=0
localhost : ok=73 changed=0 unreachable=0 failed=0 skipped=46 rescued=0 ignored=0
```
This will create AWXRestore object in the namespace, and now your AWX is restored.

2
restore/awxrestore.yaml
View file

@ -8,11 +8,9 @@ spec:
deployment_name: awx
# Parameters to restore from AWXBackup object
#backup_pvc_namespace: awx
#backup_name: awxbackup-2021-06-06
# Parameters to restore from existing files on PVC (without AWXBackup object)
#backup_pvc_namespace: awx
#backup_pvc: awx-backup-claim
#backup_dir: /backups/tower-openshift-backup-2021-06-06-10:51:49

4
restore/pv.yaml
View file

@ -2,7 +2,7 @@
apiVersion: v1
kind: PersistentVolume
metadata:
name: awx-postgres-volume
name: awx-postgres-13-volume
spec:
accessModes:
- ReadWriteOnce
@ -11,7 +11,7 @@ spec:
storage: 8Gi
storageClassName: awx-postgres-volume
hostPath:
path: /data/postgres
path: /data/postgres-13
---
apiVersion: v1

4
tips/dockerhub-rate-limit.md
View file

@ -4,13 +4,13 @@
If your Pod for PostgreSQL is in `ErrImagePull` and its `Events` shows following events, this is due to [the Rate Limit on Docker Hub](https://docs.docker.com/docker-hub/download-rate-limit/).
```bash
$ kubectl -n awx describe pod awx-postgres-0
$ kubectl -n awx describe pod awx-postgres-13-0
...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
...
Warning Failed 2s kubelet Failed to pull image "postgres:12": rpc error: code = Unknown desc = failed to pull and unpack image "docker.io/library/postgres:12": failed to copy: httpReadSeeker: failed open: unexpected status code https://registry-1.docker.io/v2/library/postgres/manifests/sha256:...: 429 Too Many Requests - Server message: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit
Warning Failed 2s kubelet Failed to pull image "postgres:13": rpc error: code = Unknown desc = failed to pull and unpack image "docker.io/library/postgres:13": failed to copy: httpReadSeeker: failed open: unexpected status code https://registry-1.docker.io/v2/library/postgres/manifests/sha256:...: 429 Too Many Requests - Server message: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit
...
```

10
tips/external-db.md
View file

@ -25,7 +25,7 @@ version: "3"
services:
postgres:
image: postgres:12
image: postgres:13
ports:
- 5432:5432
restart: always
@ -87,14 +87,14 @@ Note that the `type=unmanaged` is the important configuration to use external da
### Modify `base/pv.yaml`
Comment out following unnecessary lines which related to `awx-postgres-volume` in `base/pv.yaml`.
Comment out following unnecessary lines which related to `awx-postgres-13-volume` in `base/pv.yaml`.
```yaml
# --- 👈👈👈
# apiVersion: v1 👈👈👈
# kind: PersistentVolume 👈👈👈
# metadata: 👈👈👈
# name: awx-postgres-volume 👈👈👈
# name: awx-postgres-13-volume 👈👈👈
# spec: 👈👈👈
# accessModes: 👈👈👈
# - ReadWriteOnce 👈👈👈
@ -103,7 +103,7 @@ Comment out following unnecessary lines which related to `awx-postgres-volume` i
# storage: 8Gi 👈👈👈
# storageClassName: awx-postgres-volume 👈👈👈
# hostPath: 👈👈👈
# path: /data/postgres 👈👈👈
# path: /data/postgres-13 👈👈👈
---
apiVersion: v1
@ -115,7 +115,7 @@ metadata:
### Prepare directories
You do not need to create the `/data/postgres` directory that the main guide instructs you to create.
You do not need to create the `/data/postgres-13` directory that the main guide instructs you to create.
## The next steps

2
tips/https-redirection.md
View file

@ -74,7 +74,7 @@ namespace/awx unchanged
secret/awx-admin-password unchanged
secret/awx-postgres-configuration unchanged
secret/awx-secret-tls configured
persistentvolume/awx-postgres-volume unchanged
persistentvolume/awx-postgres-13-volume unchanged
persistentvolume/awx-projects-volume unchanged
persistentvolumeclaim/awx-projects-claim unchanged
awx.awx.ansible.com/awx configured 👈👈👈

43
tips/troubleshooting.md
View file

@ -44,7 +44,7 @@ If the Pods are working properly, its `STATUS` are `Running`. If your Pods are n
$ kubectl -n awx get pod
NAME READY STATUS RESTARTS AGE
awx-operator-controller-manager-68d787cfbd-j6k7z 2/2 Running 0 7m43s
awx-postgres-0 1/1 Running 0 4m6s
awx-postgres-13-0 1/1 Running 0 4m6s
awx-84d5c45999-h7xm4 0/4 Pending 0 3m59s
```
@ -101,7 +101,7 @@ For AWX Operator and AWX, specifically, the following commands are helpful.
- `kubectl -n awx logs -f deployment/awx -c awx-ee`
- `kubectl -n awx logs -f deployment/awx -c redis`
- Logs of PostgreSQL
- `kubectl -n awx logs -f statefulset/awx-postgres`
- `kubectl -n awx logs -f statefulset/awx-postgres-13`
### Reveal "censored" output in the AWX Operator's log
@ -133,15 +133,15 @@ spec:
If your Pod for PostgreSQL is in `ErrImagePull` and its `Events` shows following events, this is due to [the rate limit on Docker Hub](https://docs.docker.com/docker-hub/download-rate-limit/).
```bash
$ kubectl -n awx describe pod awx-postgres-0
$ kubectl -n awx describe pod awx-postgres-13-0
...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Pulling 9s kubelet Pulling image "postgres:12"
Warning Failed 2s kubelet Failed to pull image "postgres:12": rpc error: code = Unknown desc = failed to pull and unpack image "docker.io/library/postgres:12": failed to copy: httpReadSeeker: failed open: unexpected status code https://registry-1.docker.io/v2/library/postgres/manifests/sha256:505d023f030cdea84a42d580c2a4a0e17bbb3e91c30b2aea9c02f2dfb10325ba: 429 Too Many Requests - Server message: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit
Normal Pulling 9s kubelet Pulling image "postgres:13"
Warning Failed 2s kubelet Failed to pull image "postgres:13": rpc error: code = Unknown desc = failed to pull and unpack image "docker.io/library/postgres:13": failed to copy: httpReadSeeker: failed open: unexpected status code https://registry-1.docker.io/v2/library/postgres/manifests/sha256:...: 429 Too Many Requests - Server message: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit
Warning Failed 2s kubelet Error: ErrImagePull
Normal BackOff 1s kubelet Back-off pulling image "postgres:12"
Normal BackOff 1s kubelet Back-off pulling image "postgres:13"
Warning Failed 1s kubelet Error: ImagePullBackOff
```
@ -179,7 +179,7 @@ Typical solutions are one of the following:
ee_resource_requirements: {} 👈👈👈
```
- You can specify more specific value for each containers. Refer [official documentation](https://github.com/ansible/awx-operator/blob/0.25.0/README.md#containers-resource-requirements) for details.
- You can specify more specific value for each containers. Refer [official documentation](https://github.com/ansible/awx-operator/blob/0.26.0/README.md#containers-resource-requirements) for details.
- In this way you can run AWX with fewer resources, but you may encounter performance issues.
### The Pod is `Pending` with "1 pod has unbound immediate PersistentVolumeClaims." event
@ -199,9 +199,9 @@ Check the `STATUS` of your PVs and ensure your PVs doesn't have `Available` or `
```bash
$ kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
awx-projects-volume 2Gi RWO Retain Released awx/awx-projects-claim awx-projects-volume 17h
awx-postgres-volume 8Gi RWO Retain Released awx/postgres-awx-postgres-0 awx-postgres-volume 17h
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
awx-projects-volume 2Gi RWO Retain Released awx/awx-projects-claim awx-projects-volume 17h
awx-postgres-13-volume 8Gi RWO Retain Released awx/postgres-13-awx-postgres-13-0 awx-postgres-volume 17h
```
Probably this is the second (or more) time to deploy AWX for you. These PVs which have `Released` state are tied to your old (and probably no longer exists now) PVCs you created in the past.
@ -259,8 +259,11 @@ This problem occurs when the AWX pod and the PostgreSQL pod cannot communicate p
To solve this, check or try the following:
- Ensure your PostgreSQL (typically the Pod named `awx-postgres-0`) is in `Running` state.
- Ensure your PostgreSQL (typically the Pod named `awx-postgres-0` or `awx-postgres-13-0`) is in `Running` state.
- Ensure `host` under `awx-postgres-configuration` in `base/kustomizaton.yaml` has correct value.
- Specify `awx-postgres` for AWX Operator 0.25.0 or earlier, `awx-postgres-13` for `0.26.0`.
- Ensure your `firewalld`, `ufw` or any kind of firewall has been disabled on your K3s host.
- Ensure your `nm-cloud-setup` service on your K3s host is disabled if exists.
- Ensure your `awx-postgres-configuration` has correct values, especially if you're using external PostgreSQL.
- Uninstall K3s and install it again.
@ -272,33 +275,33 @@ In this situation, your Pod for PostgreSQL is in `CrashLoopBackOff` state and it
$ kubectl -n awx get pod
NAME READY STATUS RESTARTS AGE
awx-operator-controller-manager-68d787cfbd-j6k7z 2/2 Running 0 7m43s
awx-postgres-0 1/1 CrashLoopBackOff 3 4m6s
awx-postgres-13-0 1/1 CrashLoopBackOff 3 4m6s
awx-84d5c45999-h7xm4 4/4 Running 0 3m59s
$ kubectl -n awx logs statefulset/awx-postgres
mkdir: cannot create directory '/var/lib/postgresql/data': Permission denied
```
You should check the permissions and the owner of directories where used as PV on your K3s host. If you followed my guide, it would be `/data/postgres`. There is additional `data` directory created by K3s under `/data/postgres`.
You should check the permissions and the owner of directories where used as PV on your K3s host. If you followed my guide, it would be `/data/postgres-13`. There is additional `data` directory created by K3s under `/data/postgres-13`.
```bash
$ ls -ld /data/postgres /data/postgres/data
drwxr-xr-x. 2 root root 18 Aug 20 10:09 /data/postgres
drwxr-xr-x. 3 root root 20 Aug 20 10:09 /data/postgres/data
$ ls -ld /data/postgres-13 /data/postgres-13/data
drwxr-xr-x. 2 root root 18 Aug 20 10:09 /data/postgres-13
drwxr-xr-x. 3 root root 20 Aug 20 10:09 /data/postgres-13/data
```
In my environment, `755` and `root:root` (`0:0`) works correctly. So you can try:
```bash
sudo chmod 755 /data/postgres /data/postgres/data
sudo chown 0:0 /data/postgres /data/postgres/data
sudo chmod 755 /data/postgres-13 /data/postgres-13/data
sudo chown 0:0 /data/postgres-13 /data/postgres-13/data
```
Or, you can also try `999:0` as owner/group for the directory.
```bash
sudo chmod 755 /data/postgres /data/postgres/data
sudo chown 999:0 /data/postgres /data/postgres/data
sudo chmod 755 /data/postgres-13 /data/postgres-13/data
sudo chown 999:0 /data/postgres-13 /data/postgres-13/data
```
`999` is [the UID of the `postgres` user which used in the container](https://github.com/docker-library/postgres/blob/master/12/bullseye/Dockerfile#L23).

54
tips/upgrade-operator.md
View file

@ -29,7 +29,34 @@ Refer [📝README: Backing up using AWX Operator](../README.md#backing-up-using-
## 📝 Upgrade from `0.14.0` or later (e.g. from `0.14.0` to `0.15.0`)
If you are using AWX Operator `0.14.0` or later and want to upgrade to newer version, simply, deploy the new version of AWX Operator to the same namespace where the old AWX Operator is running.
If you are using AWX Operator `0.14.0` or later and want to upgrade to newer version, basically upgrade is done by deploying the new version of AWX Operator to the same namespace where the old AWX Operator is running.
Note that only when upgrading **from `0.25.0` or earlier to `0.26.0` or later**, since the bundled PostgreSQL version will be changed to 13, so the following additional tasks are required.
```bash
# Required only when upgrading from 0.25.0 or earlier to 0.26.0 or later
sudo mkdir -p /data/postgres-13
sudo chmod 755 /data/postgres-13
cat <<EOF > pv-postgres-13.yaml
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: awx-postgres-13-volume
spec:
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
capacity:
storage: 8Gi
storageClassName: awx-postgres-volume
hostPath:
path: /data/postgres-13
EOF
kubectl apply -f pv-postgres-13.yaml
```
To upgrade your AWX Operator, perform following steps.
```bash
# Prepare required files
@ -61,6 +88,14 @@ PLAY RECAP *********************************************************************
localhost : ok=56 changed=0 unreachable=0 failed=0 skipped=35 rescued=0 ignored=0
```
If your AWX Operator has upgraded from `0.25.0` or earlier to `0.26.0` or later, old PV for PostgreSQL 12 can be removed since new AWX is running with new PV for PostgreSQL 13.
```bash
# Recommended only when upgraded from 0.25.0 or earlier to 0.26.0 or later
kubectl delete pv awx-postgres-volume
sudo rm -rf /data/postgres
```
## 📝 Upgrade from `0.13.0` (e.g. from `0.13.0` to `0.14.0`)
If you are using AWX Operator `0.13.0` and want to upgrade to newer version, you should consider the big changes in AWX Operator in `0.14.0`. [As described in the documentation](https://github.com/ansible/awx-operator/blob/0.14.0/README.md#v0140), in `0.14.0`, AWX Operator changed from cluster scope to namespace scope. Also, the Operator SDK `1.x` is used.
@ -179,17 +214,12 @@ During the AWX upgrade, a rollout of the Deployment resource will be performed a
For this reason, if we do not have enough free resources on our K3s node, we can manually delete the old AWX instance beforehand in order to free up resources. Note that the rollout history will be lost with this step.
```bash
kubectl -n awx delete deployment awx
```
$ kubectl -n awx delete deployment awx
deployment.apps "awx" deleted
Ensure that it is not the `awx` resource that should be deleted, but the `deployment` resource. If we accidentally delete the `awx` resource or any Secrets, we will not be able to upgrade successfully.
After a few minutes of waiting, our AWX Operator will successfully launch the new Deployment and the Pod for AWX.
```bash
$ kubectl -n awx get all
NAME READY STATUS RESTARTS AGE
pod/awx-postgres-0 1/1 Running 0 8m57s
NAME READY STATUS RESTARTS AGE
pod/awx-postgres-13-0 1/1 Running 0 8m57s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/awx-postgres ClusterIP None <none> 5432/TCP 8m57s
@ -198,3 +228,7 @@ service/awx-service ClusterIP 10.43.248.150 <none> 80/TCP 8m51
NAME READY AGE
statefulset.apps/awx-postgres 1/1 8m58s
```
Ensure that it is not the `awx` resource that should be deleted, but the `deployment` resource. If we accidentally delete the `awx` resource or any Secrets, we will not be able to upgrade successfully.
After a few minutes of waiting, our AWX Operator will successfully launch the new Deployment and the Pod for AWX.

1
tips/version-mapping.md
View file

@ -10,6 +10,7 @@ The table below maps the AWX Operator versions and bundled AWX versions.
| AWX Operator | AWX |
| - | - |
| 0.26.0 | 21.4.0 |
| 0.25.0 | 21.3.0 |
| 0.24.0 | 21.3.0 |
| 0.23.0 | 21.2.0 |