expand/awx-on-k3s
1
0
Fork 0
mirror of https://github.com/Expand-sys/awx-on-k3s synced 2025-12-17 06:22:14 +11:00
Code Issues Projects Releases Packages Wiki Activity Actions
awx-on-k3s/rulebooks/README.md
kurokobo 12bd4517ad docs: fix command and typos
2023-08-10 21:31:42 +09:00

436 lines
19 KiB
Markdown
Raw Blame History

<!-- omit in toc -->
# [Experimental] Integrate AWX with EDA Controller
The guide to deploy and use Event Driven Ansible Controller (EDA Controller) with AWX on K3s.
In this guide, [EDA Controller Operator](https://github.com/ansible/eda-server-operator) is used to deploy EDA Controller.
**Note that [EDA Controller Operator](https://github.com/ansible/eda-server-operator) is not a fully supported installation method for EDA Controller since it's not listed in [the deployment guide](https://github.com/ansible/eda-server/blob/main/docs/deployment.md).**
- [Ansible Blog | Ansible.com | Event-Driven Ansible](https://www.ansible.com/blog)
- [Welcome to Ansible Rulebook documentation — Ansible Rulebook Documentation](https://ansible.readthedocs.io/projects/rulebook/en/latest/)
- [ansible/eda-server](https://github.com/ansible/eda-server)
- [ansible/eda-server-operator](https://github.com/ansible/eda-server-operator)
<!-- omit in toc -->
## Table of Contents
- [Prerequisites](#prerequisites)
- [Deployment Instruction](#deployment-instruction)
- [Install EDA Controller Operator](#install-eda-controller-operator)
- [Prepare required files to deploy EDA Controller](#prepare-required-files-to-deploy-eda-controller)
- [Deploy EDA Controller](#deploy-eda-controller)
- [Demo: Use EDA Controller](#demo-use-eda-controller)
- [Configure EDA Controller](#configure-eda-controller)
- [Issue new token for AWX and add it on EDA Controller](#issue-new-token-for-awx-and-add-it-on-eda-controller)
- [Add Decision Environment on EDA Controller](#add-decision-environment-on-eda-controller)
- [Add Project on EDA Controller](#add-project-on-eda-controller)
- [Activate Rulebook](#activate-rulebook)
- [Deploy Ingress resource for the webhook](#deploy-ingress-resource-for-the-webhook)
- [Trigger Rule using Webhook](#trigger-rule-using-webhook)
- [Appendix: Use MQTT as a source](#appendix-use-mqtt-as-a-source)
## Prerequisites
EDA Controller is designed to use with AWX, so we have to have working AWX instance. Refer to [the main guide on this repository](../README.md) to deploy AWX on K3s.
## Deployment Instruction
### Install EDA Controller Operator
Clone this repository and change directory.
```bash
cd ~
git clone https://github.com/kurokobo/awx-on-k3s.git
cd awx-on-k3s
```
Then invoke `kubectl apply -k rulebooks/operator` to deploy EDA Controller Operator.
```bash
kubectl apply -k rulebooks/operator
```
The EDA Controller Operator will be deployed to the namespace `eda`.
```bash
$ kubectl -n eda get all
NAME READY STATUS RESTARTS AGE
pod/eda-server-operator-controller-manager-7bf7578d44-7r87w 2/2 Running 0 12s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/eda-server-operator-controller-manager-metrics-service ClusterIP 10.43.3.124 <none> 8443/TCP 12s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/eda-server-operator-controller-manager 1/1 1 1 12s
NAME DESIRED CURRENT READY AGE
replicaset.apps/eda-server-operator-controller-manager-7bf7578d44 1 1 1 12s
```
### Prepare required files to deploy EDA Controller
Generate a Self-Signed certificate for the Web UI and API for EDA Controller. Note that IP address can't be specified.
```bash
EDA_HOST="eda.example.com"
openssl req -x509 -nodes -days 3650 -newkey rsa:2048 -out ./rulebooks/controller/tls.crt -keyout ./rulebooks/controller/tls.key -subj "/CN=${EDA_HOST}/O=${EDA_HOST}" -addext "subjectAltName = DNS:${EDA_HOST}"
```
Modify `hostname` and `automation_server_url` in `rulebooks/controller/eda.yaml`. Note `hostname` is the hostname for your EDA Controller instance, and `automation_server_url` is the URL for your AWX instance that accessible from EDA Controller.
```yaml
...
spec:
...
ingress_type: ingress
ingress_tls_secret: eda-secret-tls
hostname: eda.example.com 👈👈👈
automation_server_url: https://awx.example.com/ 👈👈👈
automation_server_ssl_verify: no
...
```
Modify two `password`s in `rulebooks/controller/kustomization.yaml`.
```yaml
...
- name: eda-database-configuration
type: Opaque
literals:
- host=eda-postgres-13
- port=5432
- database=eda
- username=eda
- password=Ansible123! 👈👈👈
- type=managed
- name: eda-admin-password
type: Opaque
literals:
- password=Ansible123! 👈👈👈
...
```
<!--
Prepare directories for Persistent Volumes defined in `base/pv.yaml`. This directory will be used to store your database.
```bash
sudo mkdir -p /data/eda/postgres-13/data
sudo chmod 755 /data/eda/postgres-13/data
sudo chown 26:0 /data/eda/postgres-13/data
```
-->
### Deploy EDA Controller
Deploy EDA Controller, this takes few minutes to complete.
```bash
kubectl apply -k rulebooks/controller
```
To monitor the progress of the deployment, check the logs of `deployment/eda-server-operator-controller-manager`:
```bash
kubectl -n eda logs -f deployment/eda-server-operator-controller-manager
```
When the deployment completes successfully, the logs end with:
```txt
$ kubectl -n eda logs -f deployment/eda-server-operator-controller-manager
...
----- Ansible Task Status Event StdOut (eda.ansible.com/v1alpha1, Kind=EDA, eda/eda) -----
PLAY RECAP *********************************************************************
localhost : ok=54 changed=0 unreachable=0 failed=0 skipped=16 rescued=0 ignored=0
```
Required objects has been deployed next to AWX Operator in `awx` namespace.
```bash
$ kubectl -n eda get eda,all,ingress,configmap,secret
NAME AGE
eda.eda.ansible.com/eda 3m50s
NAME READY STATUS RESTARTS AGE
pod/eda-server-operator-controller-manager-7bf7578d44-2wm69 2/2 Running 0 6m29s
pod/eda-redis-7d78cdf7d5-z87kk 1/1 Running 0 3m34s
pod/eda-postgres-13-0 1/1 Running 0 3m25s
pod/eda-ui-647b989ccb-stqkp 1/1 Running 0 2m36s
pod/eda-worker-fd594c44-96d9p 1/1 Running 0 2m32s
pod/eda-api-5c467d6c48-88m8z 2/2 Running 0 2m39s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/eda-server-operator-controller-manager-metrics-service ClusterIP 10.43.133.61 <none> 8443/TCP 6m29s
service/eda-redis-svc ClusterIP 10.43.144.67 <none> 6379/TCP 3m36s
service/eda-postgres-13 ClusterIP None <none> 5432/TCP 3m27s
service/eda-api ClusterIP 10.43.89.128 <none> 8000/TCP 2m41s
service/eda-daphne ClusterIP 10.43.12.68 <none> 8001/TCP 2m41s
service/eda-ui ClusterIP 10.43.136.60 <none> 80/TCP 2m38s
service/eda-worker ClusterIP 10.43.201.230 <none> 8080/TCP 2m33s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/eda-server-operator-controller-manager 1/1 1 1 6m29s
deployment.apps/eda-redis 1/1 1 1 3m34s
deployment.apps/eda-ui 1/1 1 1 2m36s
deployment.apps/eda-worker 1/1 1 1 2m32s
deployment.apps/eda-api 1/1 1 1 2m39s
NAME DESIRED CURRENT READY AGE
replicaset.apps/eda-server-operator-controller-manager-7bf7578d44 1 1 1 6m29s
replicaset.apps/eda-redis-7d78cdf7d5 1 1 1 3m34s
replicaset.apps/eda-ui-647b989ccb 1 1 1 2m36s
replicaset.apps/eda-worker-fd594c44 1 1 1 2m32s
replicaset.apps/eda-api-5c467d6c48 1 1 1 2m39s
NAME READY AGE
statefulset.apps/eda-postgres-13 1/1 3m25s
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress.networking.k8s.io/eda-ingress traefik eda.example.com 192.168.0.219 80, 443 2m35s
NAME DATA AGE
configmap/kube-root-ca.crt 1 6m29s
configmap/eda-eda-configmap 2 2m43s
configmap/eda-server-operator 0 6m28s
NAME TYPE DATA AGE
secret/redhat-operators-pull-secret Opaque 1 6m29s
secret/eda-admin-password Opaque 1 3m50s
secret/eda-database-configuration Opaque 6 3m50s
secret/eda-secret-tls kubernetes.io/tls 2 3m50s
secret/eda-db-fields-encryption-secret Opaque 1 2m51s
```
Now your EDA Controller is available at `https://eda.example.com/` or the hostname you specified.
## Demo: Use EDA Controller
Here is a demo of configuring a webhook on the EDA Controller side, and triggering a Job Template on AWX by posting payload that contains a specific `message` to the webhook.
In this demo, following example Rulebook is used. Review the Rulebook.
- **Webhook as a source**: [demo_webhook.yaml](demo_webhook.yaml)
- The webhook that listening on `0.0.0.0:5000` is defined as a source of the Ruleset.
- This Ruleset has a rule that if the payload contains `message` field with the body `Hello EDA`, trigger `Demo Job Template` in `Default` organization on AWX.
In addition to the webhook demo, a quick demo to use MQTT as a source is also provided.
- **MQTT as a source**: [demo_mqtt.yaml](demo_mqtt.yaml)
- As a source of the Ruleset, subscribing MQTT topic on the MQTT broker is defined. Actual connection information for MQTT can be defined by Rulebook Variables.
- This Ruleset has a rule that if the received data contains `message` field with the body `Hello EDA`, trigger `Demo Job Template` in `Default` organization on AWX.
### Configure EDA Controller
In order to the webhook to be ready to receive messages, the following tasks need to be done.
- Issue new token for AWX and add it on EDA Controller
- Add Decision Environment on EDA Controller
- Add Project on EDA Controller
- Activate Rulebook
- Deploy Ingress resource for the webhook
#### Issue new token for AWX and add it on EDA Controller
EDA Controller uses a token to access AWX. This token has to be issued by AWX and registered on EDA Controller.
To issue new token by AWX, in the Web UI for AWX, open `User Details` page (accessible by user icon at the upper right corner), follow to the `Tokens` tab, and then click `Add` button. Specify `Write` as `Scope` and click `Save`, then keep the issued token in the safe place.
Alternatively we can issue new token by CLI as follows:
```bash
$ kubectl -n awx exec deployment/awx-task -- awx-manage create_oauth2_token --user=admin
4sIZrWXi**************8xChmahb
```
To register the token on EDA Controller, in the Web UI for EDA Controller, open `User details` page (accessible by user icon at the upper right corner), follow to the `Controller Tokens` tab, and then click `Create controller token` button.
Fill the form as follows, then click `Create controller token` button on the bottom of the page:
| Key | Value |
| - | - |
| Name | `awx.example.com` |
| Token | `<YOUR_TOKEN>` |
#### Add Decision Environment on EDA Controller
Decision Environment (DE) is an environment for running Ansible Rulebook (`ansible-rulebook`) by the EDA Controller, like Execution Environment (EE) for running Ansible Runner (`ansible-runner`) by the AWX.
There is no default DE on EDA Controller, so we have to register new one.
Open `Decision Environments` under `Resources` on Web UI for EDA Controller, then click `Create decision environment` button.
Fill the form as follows, then click `Create decision environment` button on the bottom of the page:
| Key | Value |
| - | - |
| Name | `Minimal DE` |
| Image | `quay.io/ansible/ansible-rulebook:latest` |
#### Add Project on EDA Controller
To run Ansible Rulebook by EDA Controller, the repository on SCM that contains Rulebooks have to be registered as Project on EDA Controller.
This repository contains some example Rulebooks under [rulebooks](./) directory, so we can register this repository as Project.
Open `Projects` under `Resources` on Web UI for EDA Controller, then click `Create project` button.
Fill the form as follows, then click `Create project` button on the bottom of the page:
| Key | Value |
| - | - |
| Name | `Demo Project` |
| SCM URL | `https://github.com/kurokobo/awx-on-k3s.git` |
Refresh the page and wait for the `Status` for the project to be `Completed`.
#### Activate Rulebook
To run Ansible Rulebook by EDA Controller, activate the Rulebook.
Open `Rulebook Activations` under `Views` on Web UI for EDA Controller, then click `Create rulebook activation` button.
Fill the form as follows, then click `Create rulebook activation` button on the bottom of the page:
| Key | Value |
| - | - |
| Name | `Trigger Demo Job Template by Webhook` |
| Project | `Demo Project` |
| Rulebook | `demo_webhook.yaml` |
| Decision environment | `Minimal DE` |
Refresh the page and wait for the `Activation status` for the Rulebook to be `Running`.
Ensure you have Activation ID which can be found at the end of the URL, e.g. `http://eda.example.com/eda/rulebook-activations/details/1`.
The new Job is created on `eda` namespace.
```bash
$ ACTIVATION_ID=1
$ kubectl -n eda get job -l activation-id=${ACTIVATION_ID}
NAME COMPLETIONS DURATION AGE
activation-job-1 0/1 8m45s 11m
```
By this Job, new Pod that `ansible-rulebook` running on is also created.
```bash
$ JOB_NAME=activation-job-${ACTIVATION_ID}
$ kubectl -n eda get pod -l job-name=${JOB_NAME}
NAME READY STATUS RESTARTS AGE
activation-job-1-h9kjt 1/1 Running 0 11m
```
The new Service is also created by EDA Controller. This service provides the endpoint for the webhook.
```bash
$ kubectl -n eda get service -l job-name=${JOB_NAME}
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
activation-job-1-5000 ClusterIP 10.43.221.234 <none> 5000/TCP 11m
```
#### Deploy Ingress resource for the webhook
To make the webhook externally accessible, we have to expose the Service that created by EDA Controller.
To achieve this, in this example, we create new Ingress.
Modify `hosts`, `host`, and `name` under `service` in `rulebooks/webhook/ingress.yaml`. Here, the same hostname as the EDA Controller are specified so that the endpoint for webhook can be accessed under the same URL as the EDA Controller. Note that the `name` of the `service` has to be the name of the Service that created by EDA Controller, as reviewed above.
```yaml
...
spec:
tls:
- hosts:
- eda.example.com 👈👈👈
secretName: eda-secret-tls
rules:
- host: eda.example.com 👈👈👈
http:
paths:
- path: /webhooks/demo
pathType: ImplementationSpecific
backend:
service:
name: activation-job-1-5000 👈👈👈
port:
number: 5000
```
Modify `replicas` for `worker` in the same file. The number of rulebooks that can be activated simultaneously is equal to the number of this value.
```yaml
...
worker:
replicas: 2 👈👈👈
resource_requirements:
requests: {}
...
```
By applying this file, your webhook can be accessed on the URL `https://eda.example.com/webhooks/demo`.
```bash
$ kubectl apply -f rulebooks/webhook/ingress.yaml
...
$ kubectl -n eda get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
eda-ingress traefik eda.example.com 192.168.0.219 80, 443 4h45m
eda-ingress-webhook traefik eda.example.com 192.168.0.219 80, 443 1s 👈👈👈
```
### Trigger Rule using Webhook
In [the Rulebook (`demo_webhook.yaml`) that used in this demo](demo_webhook.yaml), the Job Template (`Demo Job Template`) is invoked on AWX on the condition that `message` in the payload that posted to the webhook is `Hello EDA`.
Post that payload to the webhook, and review the Job Template is triggered.
```bash
$ curl -k \
-H "Content-Type: application/json" \
-d '{"message": "Hello EDA"}' \
https://eda.example.com/webhooks/demo
```
Review `Rule Audit` page under `Views` on the Web UI for EDA Controller, and `Jobs` page under `Views` on the Web UI for AWX.
### Appendix: Use MQTT as a source
For a more authentic example, here is an example of using MQTT as a source.
To use MQTT, a MQTT broker is required. If you don't have one, you can use `kubectl apply -k rulebooks/mqtt/broker` to deploy a minimal MQTT broker on K3s that listens on `31883` or use a public broker such as [test.mosquitto.org](https://test.mosquitto.org/). Also, this demonstration is prepared assuming that neither authentication nor encryption is used.
Define the Decision Environment with the following information, just as you configured for the webhook.
| Key | Value |
| - | - |
| Name | `Minimal DE with MQTT` |
| Image | `docker.io/kurokobo/ansible-rulebook:v1.0.1-mqtt` |
Note that the image specified above is based on `quay.io/ansible/ansible-rulebook:v1.0.1` and includes [PR #113](https://github.com/ansible/event-driven-ansible/pull/113) with small patch to make `ansible.eda.mqtt` workable. The Dockerfile for this image is available under [mqtt/de directory](./mqtt/de).
Then define Rulebook Activation as follows. Note that you should modify actual values for `Variables` to suit your environment:
| Key | Value |
| - | - |
| Name | `Trigger Demo Job Template by MQTT` |
| Project | `Demo Project` |
| Rulebook | `demo_mqtt.yaml` |
| Decision environment | `Minimal DE with MQTT` |
| Variables | `mqtt_host: mqtt.example.com`<br>`mqtt_port: 31883`<br>`mqtt_topic: demo` |
Activate the Rulebook, and publish specific message that matches the condition in the Rule to the topic you've defined.
```bash
docker run -it --rm efrecon/mqtt-client pub \
-h mqtt.example.com \
-p 31883 \
-t demo \
-m '{"message": "Hello EDA"}'
```
Ensure your Job Template on AWX has been triggered.
Reference in a new issue View git blame Copy permalink