mirror of
https://github.com/Expand-sys/awx-on-k3s
synced 2025-12-16 05:52:15 +11:00
feat: bump postgresql version to 15
This commit is contained in:
parent
50a9fc62a0
commit
2726e781d5
10 changed files with 75 additions and 30 deletions
18
README.md
18
README.md
|
|
@ -33,7 +33,7 @@ An example implementation of AWX on single node K3s using AWX Operator, with eas
|
||||||
- Products that will be deployed:
|
- Products that will be deployed:
|
||||||
- AWX Operator 2.12.2
|
- AWX Operator 2.12.2
|
||||||
- AWX 23.9.0
|
- AWX 23.9.0
|
||||||
- PostgreSQL 13
|
- PostgreSQL 15
|
||||||
|
|
||||||
## References
|
## References
|
||||||
|
|
||||||
|
|
@ -85,8 +85,8 @@ curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=v1.28.6+k3s2 sh -s - --write-
|
||||||
|
|
||||||
### Install AWX Operator
|
### Install AWX Operator
|
||||||
|
|
||||||
> [!WARNING]
|
> [!NOTE]
|
||||||
> If you are planning that creating backup of your AWX instance using AWX Operator by referring to [the backup guide](backup), AWX Operator 2.12.2 is not recommended due to [a known issue for backup](https://github.com/ansible/awx-operator/issues/1734). Use an older version of AWX Operator like [2.12.1](https://github.com/kurokobo/awx-on-k3s/tree/2.12.1) instead.
|
> From AWX Operator 2.13.0, Default PostgreSQL version is bumped from 13 to 15. If you have a plan to upgrade existing AWX Operator and AWX, refer to [📝Tips: Upgrade AWX Operator and AWX](tips/upgrade-operator.md) to perform additional tasks to database migration.
|
||||||
|
|
||||||
Clone this repository and change directory.
|
Clone this repository and change directory.
|
||||||
|
|
||||||
|
|
@ -151,7 +151,7 @@ Modify the two `password` entries in `base/kustomization.yaml`. Note that the `p
|
||||||
- name: awx-postgres-configuration
|
- name: awx-postgres-configuration
|
||||||
type: Opaque
|
type: Opaque
|
||||||
literals:
|
literals:
|
||||||
- host=awx-postgres-13
|
- host=awx-postgres-15
|
||||||
- port=5432
|
- port=5432
|
||||||
- database=awx
|
- database=awx
|
||||||
- username=awx
|
- username=awx
|
||||||
|
|
@ -168,9 +168,9 @@ Modify the two `password` entries in `base/kustomization.yaml`. Note that the `p
|
||||||
Prepare directories for Persistent Volumes defined in `base/pv.yaml`. These directories will be used to store your databases and project files. Note that the size of the PVs and PVCs are specified in some of the files in this repository, but since their backends are `hostPath`, its value is just like a label and there is no actual capacity limitation.
|
Prepare directories for Persistent Volumes defined in `base/pv.yaml`. These directories will be used to store your databases and project files. Note that the size of the PVs and PVCs are specified in some of the files in this repository, but since their backends are `hostPath`, its value is just like a label and there is no actual capacity limitation.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
sudo mkdir -p /data/postgres-13
|
sudo mkdir -p /data/postgres-15
|
||||||
sudo mkdir -p /data/projects
|
sudo mkdir -p /data/projects
|
||||||
sudo chmod 755 /data/postgres-13
|
sudo chmod 755 /data/postgres-15
|
||||||
sudo chown 1000:0 /data/projects
|
sudo chown 1000:0 /data/projects
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
@ -207,13 +207,13 @@ awx.awx.ansible.com/awx 6m15s
|
||||||
|
|
||||||
NAME READY STATUS RESTARTS AGE
|
NAME READY STATUS RESTARTS AGE
|
||||||
pod/awx-operator-controller-manager-57867569c4-ggl29 2/2 Running 0 6m50s
|
pod/awx-operator-controller-manager-57867569c4-ggl29 2/2 Running 0 6m50s
|
||||||
pod/awx-postgres-13-0 1/1 Running 0 5m56s
|
pod/awx-postgres-15-0 1/1 Running 0 5m56s
|
||||||
pod/awx-task-5d8cd9b6b9-8ptjt 4/4 Running 0 5m25s
|
pod/awx-task-5d8cd9b6b9-8ptjt 4/4 Running 0 5m25s
|
||||||
pod/awx-web-66f89bc9cf-6zck5 3/3 Running 0 4m39s
|
pod/awx-web-66f89bc9cf-6zck5 3/3 Running 0 4m39s
|
||||||
|
|
||||||
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
|
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
|
||||||
service/awx-operator-controller-manager-metrics-service ClusterIP 10.43.18.30 <none> 8443/TCP 7m
|
service/awx-operator-controller-manager-metrics-service ClusterIP 10.43.18.30 <none> 8443/TCP 7m
|
||||||
service/awx-postgres-13 ClusterIP None <none> 5432/TCP 5m55s
|
service/awx-postgres-15 ClusterIP None <none> 5432/TCP 5m55s
|
||||||
service/awx-service ClusterIP 10.43.237.218 <none> 80/TCP 5m28s
|
service/awx-service ClusterIP 10.43.237.218 <none> 80/TCP 5m28s
|
||||||
|
|
||||||
NAME READY UP-TO-DATE AVAILABLE AGE
|
NAME READY UP-TO-DATE AVAILABLE AGE
|
||||||
|
|
@ -227,7 +227,7 @@ replicaset.apps/awx-task-5d8cd9b6b9 1 1
|
||||||
replicaset.apps/awx-web-66f89bc9cf 1 1 1 4m39s
|
replicaset.apps/awx-web-66f89bc9cf 1 1 1 4m39s
|
||||||
|
|
||||||
NAME READY AGE
|
NAME READY AGE
|
||||||
statefulset.apps/awx-postgres-13 1/1 5m56s
|
statefulset.apps/awx-postgres-15 1/1 5m56s
|
||||||
|
|
||||||
NAME CLASS HOSTS ADDRESS PORTS AGE
|
NAME CLASS HOSTS ADDRESS PORTS AGE
|
||||||
ingress.networking.k8s.io/awx-ingress traefik awx.example.com 192.168.0.219 80, 443 5m27s
|
ingress.networking.k8s.io/awx-ingress traefik awx.example.com 192.168.0.219 80, 443 5m27s
|
||||||
|
|
|
||||||
|
|
@ -16,7 +16,7 @@ secretGenerator:
|
||||||
- name: awx-postgres-configuration
|
- name: awx-postgres-configuration
|
||||||
type: Opaque
|
type: Opaque
|
||||||
literals:
|
literals:
|
||||||
- host=awx-postgres-13
|
- host=awx-postgres-15
|
||||||
- port=5432
|
- port=5432
|
||||||
- database=awx
|
- database=awx
|
||||||
- username=awx
|
- username=awx
|
||||||
|
|
|
||||||
|
|
@ -2,7 +2,7 @@
|
||||||
apiVersion: v1
|
apiVersion: v1
|
||||||
kind: PersistentVolume
|
kind: PersistentVolume
|
||||||
metadata:
|
metadata:
|
||||||
name: awx-postgres-13-volume
|
name: awx-postgres-15-volume
|
||||||
spec:
|
spec:
|
||||||
accessModes:
|
accessModes:
|
||||||
- ReadWriteOnce
|
- ReadWriteOnce
|
||||||
|
|
@ -11,7 +11,7 @@ spec:
|
||||||
storage: 8Gi
|
storage: 8Gi
|
||||||
storageClassName: awx-postgres-volume
|
storageClassName: awx-postgres-volume
|
||||||
hostPath:
|
hostPath:
|
||||||
path: /data/postgres-13
|
path: /data/postgres-15
|
||||||
|
|
||||||
---
|
---
|
||||||
apiVersion: v1
|
apiVersion: v1
|
||||||
|
|
|
||||||
|
|
@ -29,19 +29,19 @@ If your AWX instance is running, it is recommended that it be deleted along with
|
||||||
```bash
|
```bash
|
||||||
# Delete AWX resource, PVC, and PV
|
# Delete AWX resource, PVC, and PV
|
||||||
kubectl -n awx delete awx awx
|
kubectl -n awx delete awx awx
|
||||||
kubectl -n awx delete pvc postgres-13-awx-postgres-13-0
|
kubectl -n awx delete pvc postgres-15-awx-postgres-15-0
|
||||||
kubectl delete pv awx-postgres-13-volume
|
kubectl delete pv awx-postgres-15-volume
|
||||||
|
|
||||||
# Delete any data in the PV
|
# Delete any data in the PV
|
||||||
sudo rm -rf /data/postgres-13
|
sudo rm -rf /data/postgres-15
|
||||||
```
|
```
|
||||||
|
|
||||||
Then prepare directories for your PVs. `/data/projects` is required if you are restoring the entire AWX to a new environment.
|
Then prepare directories for your PVs. `/data/projects` is required if you are restoring the entire AWX to a new environment.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
sudo mkdir -p /data/postgres-13
|
sudo mkdir -p /data/postgres-15
|
||||||
sudo mkdir -p /data/projects
|
sudo mkdir -p /data/projects
|
||||||
sudo chmod 755 /data/postgres-13
|
sudo chmod 755 /data/postgres-15
|
||||||
sudo chown 1000:0 /data/projects
|
sudo chown 1000:0 /data/projects
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -2,7 +2,7 @@
|
||||||
apiVersion: v1
|
apiVersion: v1
|
||||||
kind: PersistentVolume
|
kind: PersistentVolume
|
||||||
metadata:
|
metadata:
|
||||||
name: awx-postgres-13-volume
|
name: awx-postgres-15-volume
|
||||||
spec:
|
spec:
|
||||||
accessModes:
|
accessModes:
|
||||||
- ReadWriteOnce
|
- ReadWriteOnce
|
||||||
|
|
@ -11,7 +11,7 @@ spec:
|
||||||
storage: 8Gi
|
storage: 8Gi
|
||||||
storageClassName: awx-postgres-volume
|
storageClassName: awx-postgres-volume
|
||||||
hostPath:
|
hostPath:
|
||||||
path: /data/postgres-13
|
path: /data/postgres-15
|
||||||
|
|
||||||
---
|
---
|
||||||
apiVersion: v1
|
apiVersion: v1
|
||||||
|
|
|
||||||
|
|
@ -245,7 +245,7 @@ $ kubectl delete -k base
|
||||||
secret "awx-admin-password" deleted
|
secret "awx-admin-password" deleted
|
||||||
secret "awx-postgres-configuration" deleted
|
secret "awx-postgres-configuration" deleted
|
||||||
secret "awx-secret-tls" deleted
|
secret "awx-secret-tls" deleted
|
||||||
persistentvolume "awx-postgres-13-volume" deleted
|
persistentvolume "awx-postgres-15-volume" deleted
|
||||||
persistentvolume "awx-projects-volume" deleted
|
persistentvolume "awx-projects-volume" deleted
|
||||||
persistentvolumeclaim "awx-projects-claim" deleted
|
persistentvolumeclaim "awx-projects-claim" deleted
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -87,7 +87,7 @@ namespace/awx unchanged
|
||||||
secret/awx-admin-password unchanged
|
secret/awx-admin-password unchanged
|
||||||
secret/awx-postgres-configuration unchanged
|
secret/awx-postgres-configuration unchanged
|
||||||
secret/awx-secret-tls configured
|
secret/awx-secret-tls configured
|
||||||
persistentvolume/awx-postgres-13-volume unchanged
|
persistentvolume/awx-postgres-15-volume unchanged
|
||||||
persistentvolume/awx-projects-volume unchanged
|
persistentvolume/awx-projects-volume unchanged
|
||||||
persistentvolumeclaim/awx-projects-claim unchanged
|
persistentvolumeclaim/awx-projects-claim unchanged
|
||||||
awx.awx.ansible.com/awx configured 👈👈👈
|
awx.awx.ansible.com/awx configured 👈👈👈
|
||||||
|
|
|
||||||
|
|
@ -44,7 +44,7 @@ If the Pods are working properly, its `STATUS` are `Running`. If your Pods are n
|
||||||
$ kubectl -n awx get pod
|
$ kubectl -n awx get pod
|
||||||
NAME READY STATUS RESTARTS AGE
|
NAME READY STATUS RESTARTS AGE
|
||||||
awx-operator-controller-manager-57867569c4-ggl29 2/2 Running 0 8m20s
|
awx-operator-controller-manager-57867569c4-ggl29 2/2 Running 0 8m20s
|
||||||
awx-postgres-13-0 1/1 Running 0 7m26s
|
awx-postgres-15-0 1/1 Running 0 7m26s
|
||||||
awx-task-5d8cd9b6b9-8ptjt 0/4 Pending 0 6m55s
|
awx-task-5d8cd9b6b9-8ptjt 0/4 Pending 0 6m55s
|
||||||
awx-web-66f89bc9cf-6zck5 0/3 Pending 0 6m9s
|
awx-web-66f89bc9cf-6zck5 0/3 Pending 0 6m9s
|
||||||
```
|
```
|
||||||
|
|
@ -108,7 +108,7 @@ For AWX Operator and AWX, specifically, the following commands are helpful.
|
||||||
- `kubectl -n awx logs -f deployment/awx-task -c awx-rsyslog`
|
- `kubectl -n awx logs -f deployment/awx-task -c awx-rsyslog`
|
||||||
- `kubectl -n awx logs -f deployment/awx-task -c redis`
|
- `kubectl -n awx logs -f deployment/awx-task -c redis`
|
||||||
- Logs of PostgreSQL
|
- Logs of PostgreSQL
|
||||||
- `kubectl -n awx logs -f statefulset/awx-postgres-13`
|
- `kubectl -n awx logs -f statefulset/awx-postgres-15`
|
||||||
|
|
||||||
### Reveal "censored" output in the AWX Operator's log
|
### Reveal "censored" output in the AWX Operator's log
|
||||||
|
|
||||||
|
|
@ -208,7 +208,7 @@ Check the `STATUS` of your PVs and ensure your PVs doesn't have `Available` or `
|
||||||
$ kubectl get pv
|
$ kubectl get pv
|
||||||
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
|
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
|
||||||
awx-projects-volume 2Gi RWO Retain Released awx/awx-projects-claim awx-projects-volume 17h
|
awx-projects-volume 2Gi RWO Retain Released awx/awx-projects-claim awx-projects-volume 17h
|
||||||
awx-postgres-13-volume 8Gi RWO Retain Released awx/postgres-13-awx-postgres-13-0 awx-postgres-volume 17h
|
awx-postgres-15-volume 8Gi RWO Retain Released awx/postgres-15-awx-postgres-15-0 awx-postgres-volume 17h
|
||||||
```
|
```
|
||||||
|
|
||||||
Probably this is the second (or more) time to deploy AWX for you. These PVs which have `Released` state are tied to your old (and probably no longer exists now) PVCs you created in the past.
|
Probably this is the second (or more) time to deploy AWX for you. These PVs which have `Released` state are tied to your old (and probably no longer exists now) PVCs you created in the past.
|
||||||
|
|
@ -266,9 +266,9 @@ This problem occurs when the AWX pod and the PostgreSQL pod cannot communicate p
|
||||||
|
|
||||||
To solve this, check or try the following:
|
To solve this, check or try the following:
|
||||||
|
|
||||||
- Ensure your PostgreSQL (typically the Pod named `awx-postgres-0` or `awx-postgres-13-0`) is in `Running` state.
|
- Ensure your PostgreSQL (typically the Pod named `awx-postgres-0`, `awx-postgres-13-0`, or `awx-postgres-15-0`) is in `Running` state.
|
||||||
- Ensure `host` under `awx-postgres-configuration` in `base/kustomization.yaml` has correct value.
|
- Ensure `host` under `awx-postgres-configuration` in `base/kustomization.yaml` has correct value.
|
||||||
- Specify `awx-postgres` for AWX Operator 0.25.0 or earlier, `awx-postgres-13` for `0.26.0` or later.
|
- Specify `awx-postgres` for AWX Operator 0.25.0 or earlier, `awx-postgres-13` for `0.26.0` to `2.12.2`, `awx-postgres-15` for newer versions.
|
||||||
- Ensure your `firewalld`, `ufw` or any kind of firewall has been disabled on your K3s host.
|
- Ensure your `firewalld`, `ufw` or any kind of firewall has been disabled on your K3s host.
|
||||||
- Ensure your `nm-cloud-setup` service on your K3s host is disabled if exists.
|
- Ensure your `nm-cloud-setup` service on your K3s host is disabled if exists.
|
||||||
- Ensure your `awx-postgres-configuration` has correct values, especially if you're using external PostgreSQL.
|
- Ensure your `awx-postgres-configuration` has correct values, especially if you're using external PostgreSQL.
|
||||||
|
|
@ -290,7 +290,9 @@ $ kubectl -n awx logs statefulset/awx-postgres
|
||||||
mkdir: cannot create directory '/var/lib/postgresql/data': Permission denied
|
mkdir: cannot create directory '/var/lib/postgresql/data': Permission denied
|
||||||
```
|
```
|
||||||
|
|
||||||
You should check the permissions and the owner of directories where used as PV on your K3s host. If you followed my guide, it would be `/data/postgres-13`. There is additional `data` directory created by K3s under `/data/postgres-13`.
|
You should check the permissions and the owner of directories where used as PV on your K3s host.
|
||||||
|
|
||||||
|
For the PostgreSQL that deployed by **AWX Operator 2.12.2 or earlier**, if you followed my guide, it would be `/data/postgres-13`. There is additional `data` directory created by K3s under `/data/postgres-13`.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
$ ls -ld /data/postgres-13 /data/postgres-13/data
|
$ ls -ld /data/postgres-13 /data/postgres-13/data
|
||||||
|
|
@ -305,14 +307,31 @@ sudo chmod 755 /data/postgres-13 /data/postgres-13/data
|
||||||
sudo chown 0:0 /data/postgres-13 /data/postgres-13/data
|
sudo chown 0:0 /data/postgres-13 /data/postgres-13/data
|
||||||
```
|
```
|
||||||
|
|
||||||
Or, you can also try `999:0` as owner/group for the directory.
|
Or, you can also try `` as owner/group for the directory.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
sudo chmod 755 /data/postgres-13 /data/postgres-13/data
|
sudo chmod 755 /data/postgres-13 /data/postgres-13/data
|
||||||
sudo chown 999:0 /data/postgres-13 /data/postgres-13/data
|
sudo chown 999:0 /data/postgres-13 /data/postgres-13/data
|
||||||
```
|
```
|
||||||
|
|
||||||
`999` is [the UID of the `postgres` user which used in the container](https://github.com/docker-library/postgres/blob/master/12/bullseye/Dockerfile#L23).
|
`999` is [the UID of the `postgres` user which used in the container](https://github.com/docker-library/postgres/blob/master/13/bullseye/Dockerfile#L13).
|
||||||
|
|
||||||
|
For the PostgreSQL that deployed by **AWX Operator 2.13.0 or later**, if you followed my guide, it would be `/data/postgres-15`. There is additional `data` directory created by K3s under `/data/postgres-15`.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
$ ls -ld /data/postgres-15 /data/postgres-15/data
|
||||||
|
drwxr-xr-x. 2 root root 18 Aug 20 10:09 /data/postgres-15
|
||||||
|
drwxr-xr-x. 3 root root 20 Aug 20 10:09 /data/postgres-15/data
|
||||||
|
```
|
||||||
|
|
||||||
|
In my environment, `755` and `26:0` works correctly. So you can try:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo chmod 755 /data/postgres-15 /data/postgres-15/data
|
||||||
|
sudo chown 26:0 /data/postgres-15 /data/postgres-15/data
|
||||||
|
```
|
||||||
|
|
||||||
|
`26` is [the UID of the user which used in the container](https://github.com/sclorg/postgresql-container/blob/master/15/Dockerfile.c9s#L86).
|
||||||
|
|
||||||
## Troubles during Daily Use
|
## Troubles during Daily Use
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -28,7 +28,7 @@ awx.awx.ansible.com/awx created
|
||||||
can be deleted with the following command with same manifest files. Note that PVC for PostgreSQL should be removed manually since this PVC was created by not `kubectl apply -k` but AWX Operator.
|
can be deleted with the following command with same manifest files. Note that PVC for PostgreSQL should be removed manually since this PVC was created by not `kubectl apply -k` but AWX Operator.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
$ kubectl -n awx delete pvc postgres-13-awx-postgres-13-0 --wait=false
|
$ kubectl -n awx delete pvc postgres-15-awx-postgres-15-0 --wait=false
|
||||||
$ kubectl delete -k base
|
$ kubectl delete -k base
|
||||||
secret "awx-admin-password" deleted
|
secret "awx-admin-password" deleted
|
||||||
secret "awx-postgres-configuration" deleted
|
secret "awx-postgres-configuration" deleted
|
||||||
|
|
|
||||||
|
|
@ -33,6 +33,32 @@ Refer [📝README: Backing up using AWX Operator](../README.md#backing-up-using-
|
||||||
|
|
||||||
If you are using AWX Operator `0.14.0` or later and want to upgrade to newer version, basically upgrade is done by deploying the new version of AWX Operator to the same namespace where the old AWX Operator is running.
|
If you are using AWX Operator `0.14.0` or later and want to upgrade to newer version, basically upgrade is done by deploying the new version of AWX Operator to the same namespace where the old AWX Operator is running.
|
||||||
|
|
||||||
|
> [!WARNING]
|
||||||
|
> If you are planning to upgrade AWX Operator **from `2.12.2` or earlier to `2.13.0` or later**, note that since the bundled PostgreSQL version will be changed to 15, so the following additional tasks are required.
|
||||||
|
>
|
||||||
|
> ```bash
|
||||||
|
> # Required only when upgrading from 2.12.2 or earlier to 2.13.0 or later
|
||||||
|
> sudo mkdir -p /data/postgres-15
|
||||||
|
> sudo chmod 755 /data/postgres-15
|
||||||
|
> cat <<EOF > pv-postgres-15.yaml
|
||||||
|
> ---
|
||||||
|
> apiVersion: v1
|
||||||
|
> kind: PersistentVolume
|
||||||
|
> metadata:
|
||||||
|
> name: awx-postgres-15-volume
|
||||||
|
> spec:
|
||||||
|
> accessModes:
|
||||||
|
> - ReadWriteOnce
|
||||||
|
> persistentVolumeReclaimPolicy: Retain
|
||||||
|
> capacity:
|
||||||
|
> storage: 8Gi
|
||||||
|
> storageClassName: awx-postgres-volume
|
||||||
|
> hostPath:
|
||||||
|
> path: /data/postgres-15
|
||||||
|
> EOF
|
||||||
|
> kubectl apply -f pv-postgres-15.yaml
|
||||||
|
> ```
|
||||||
|
|
||||||
> [!WARNING]
|
> [!WARNING]
|
||||||
> If you are planning to upgrade AWX Operator **from `2.0.0` to `2.0.1` or later**, note that [the `extra_volumes` and `extra_volumes` in `base/awx.yaml` for `2.0.0` as a workaround for specific issue](https://github.com/kurokobo/awx-on-k3s/blob/2.0.0/base/awx.yaml#L42-L51) causes failure of upgrading.
|
> If you are planning to upgrade AWX Operator **from `2.0.0` to `2.0.1` or later**, note that [the `extra_volumes` and `extra_volumes` in `base/awx.yaml` for `2.0.0` as a workaround for specific issue](https://github.com/kurokobo/awx-on-k3s/blob/2.0.0/base/awx.yaml#L42-L51) causes failure of upgrading.
|
||||||
>
|
>
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue