diff --git a/.gitignore b/.gitignore index be870b4..136676c 100644 --- a/.gitignore +++ b/.gitignore @@ -1,2 +1,3 @@ +debug *.crt *.key diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..bc6713d --- /dev/null +++ b/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2021 kurokobo + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/README.md b/README.md index c3092e0..5212bf6 100644 --- a/README.md +++ b/README.md @@ -26,7 +26,13 @@ An example implementation of AWX on single node K3s using AWX Operator, with eas - [Restoring using AWX Operator](#restoring-using-awx-operator) - [Prepare for Restore](#prepare-for-restore) - [Invoke Manual Restore](#invoke-manual-restore) +- [Deploy Private Git Repository](#deploy-private-git-repository) - [Deploy Private Container Registry](#deploy-private-container-registry) +- [Use Ansible Builder](#use-ansible-builder) +- [Use Ansible Runner](#use-ansible-runner) +- [Additional Configuration for AWX](#additional-configuration-for-awx) + - [Configure AWX to use Git Repository with Self-Signed Certificate](#configure-awx-to-use-git-repository-with-self-signed-certificate) + - [Expose your /etc/hosts to K3s](#expose-your-etchosts-to-k3s) ## Environment @@ -327,10 +333,82 @@ Then restore the Secret for TLS manually (or create newly using original certifi kubectl apply -f awx-secret-tls.yaml ``` +## Deploy Private Git Repository + +To use AWX with SCM, this repository includes the manifests to deploy [Gitea](https://gitea.io/en-us/). + +See [📝`git/README.md`](git) for instructions. + ## Deploy Private Container Registry To use Execution Environments in AWX (AWX-EE), we have to push the container image built with `ansible-builder` to the container registry. If we don't want to push our container images to Docker Hub or other cloud services, we can deploy a private container registry on K3s. -See [📝 `registry/README.md`](registry/README.md) for instructions. +See [📝`registry/README.md`](registry) for instructions. + +## Use Ansible Builder + +See [📝`builder/README.md`](builder) for instructions. + +## Use Ansible Runner + +See [📝`runner/README.md`](runner) for instructions. + +## Additional Configuration for AWX + +### Configure AWX to use Git Repository with Self-Signed Certificate + +1. Add Credentials for SCM +2. Allow Self-Signed Certificate such as this Gitea + - Open `Settings` > `Jobs settings` in AWX + - Press `Edit` and scroll down to `Extra Environment Variables`, then add `"GIT_SSL_NO_VERIFY": "True"` in `{}` + - Press `Save` + +### Expose your /etc/hosts to K3s + +If we don't have a DNS server and are using `/etc/hosts`, we will need to do some additional tasks to get the Pods on K3s to resolve names according to `/etc/hosts`. + +This is necessary for AWX to resolve the hostname for your Private Git Repository or pull images from the Container Registry. + +One easy way to do this is to use `dnsmasq`. + +1. Add entries to `/etc/hosts` on your K3s host. Note that the IP addresses have to be replaced with your K3s host's one. + + ```bash + sudo echo "192.168.0.100 awx.example.com" >> /etc/hosts + sudo echo "192.168.0.100 registry.example.com" >> /etc/hosts + sudo echo "192.168.0.100 git.example.com" >> /etc/hosts + ``` + +2. Install and start `dnsmasq` with default configuration. + + ```bash + sudo dnf install dnsmasq + sudo systemctl enable dnsmasq --now + ``` + +3. Create new `resolv.conf` to use K3s. Note that the IP addresses have to be replaced with your K3s host's one. + + ```bash + sudo echo "nameserver 192.168.0.100" > /etc/rancher/k3s/resolv.conf + ``` + +4. Add `--resolv-conf /etc/rancher/k3s/resolv.conf` as an argument for `k3s server` command. + + ```bash + $ cat /etc/systemd/system/k3s.service + ... + ExecStart=/usr/local/bin/k3s \ + server \ + '--write-kubeconfig-mode' \ + '644' \ + '--resolv-conf' \ 👈👈👈 + '/etc/rancher/k3s/resolv.conf' \ 👈👈👈 + ``` + +5. Restart K3s. + + ```bash + sudo systemctl restart k3s + ``` diff --git a/builder/.gitignore b/builder/.gitignore new file mode 100644 index 0000000..dfd772f --- /dev/null +++ b/builder/.gitignore @@ -0,0 +1 @@ +context diff --git a/builder/README.md b/builder/README.md new file mode 100644 index 0000000..b613f3d --- /dev/null +++ b/builder/README.md @@ -0,0 +1,96 @@ +# Examples of Ansible Builder + +- [ansible/ansible-builder](https://github.com/ansible/ansible-builder) +- [Introduction — ansible-builder documentation](https://ansible-builder.readthedocs.io/en/latest/index.html) + +## Environment in This Example + +- CentOS 8.2 +- Python 3.9 +- Docker 20.10.7 + +## Install + +```bash +python3 -m pip install ansible-builder +``` + +## Prepare Required Files + +`execution-environment.yml` is required file to build Execution Environment. + +The base image can be chosen from the tags from [http://quay.io/ansible/ansible-runner](https://quay.io/repository/ansible/ansible-runner?tab=tags). + +## Build Execution Environment + +`ansible-builder build` command builds Execution Environment as a container image according to the definition in `execution-environment.yml`. + +```bash +ansible-builder build --tag registry.example.com/ansible/ee:2.10-custom --container-runtime docker --verbosity 3 +``` + +```bash +$ ansible-builder build --tag registry.example.com/ansible/ee:2.10-custom --container-runtime docker --verbosity 3 +Ansible Builder is building your execution environment image, "registry.example.com/ansible/ee:2.10-custom". +File context/_build/requirements.yml will be created. +File context/_build/requirements.txt will be created. +File context/_build/bindep.txt will be created. +File context/_build/ansible.cfg will be created. +Rewriting Containerfile to capture collection requirements +Running command: + docker build -f context/Dockerfile -t registry.example.com/ansible/ee:2.10-custom context +Sending build context to Docker daemon 7.68kB +Step 1/25 : ARG EE_BASE_IMAGE=quay.io/ansible/ansible-runner:stable-2.10-devel +Step 2/25 : ARG EE_BUILDER_IMAGE=quay.io/ansible/ansible-builder:latest +Step 3/25 : FROM $EE_BASE_IMAGE as galaxy +... +Removing intermediate container a083001a665a + ---> 050cf7076379 +Successfully built 050cf7076379 +Successfully tagged registry.example.com/ansible/ee:2.10-custom + +Complete! The build context can be found at: /home/********/awx-on-k3s/builder/context +``` + +```bash +$ docker image ls +REPOSITORY TAG IMAGE ID CREATED SIZE +registry.example.com/ansible/ee 2.10-custom 6fb343319a80 2 minutes ago 871MB +``` + +Now you can push this image to your [private container registry](../registry/README.md) to use as Execution Environment on AWX. + +```bash +$ docker push registry.example.com/ansible/ee:2.10-custom +The push refers to repository [registry.example.com/ansible/ee] +... +2.10-custom: digest: sha256:0138445c58253c733f2e255b618469d9f61337901c13e3be6412984fd835ad55 size: 3880 +``` + +The `Dockerfile` generated for the build will be saved under the `context` directory. + +```bash +$ cat context/Dockerfile +ARG EE_BASE_IMAGE=quay.io/ansible/ansible-runner:stable-2.10-devel +ARG EE_BUILDER_IMAGE=quay.io/ansible/ansible-builder:latest + +FROM $EE_BASE_IMAGE as galaxy +ARG ANSIBLE_GALAXY_CLI_COLLECTION_OPTS= +USER root +... +``` + +## Create Context Files + +`ansible-builder create` command only creates context files like `Dockerfile`, it does not invoke building the image. + +```bash +$ ansible-builder create --verbosity 3 +Ansible Builder is generating your execution environment build context. +File context/_build/requirements.yml will be created. +File context/_build/requirements.txt will be created. +File context/_build/bindep.txt will be created. +File context/_build/ansible.cfg will be created. +Rewriting Containerfile to capture collection requirements +Complete! The build context can be found at: /home/********/awx-on-k3s/builder/context +``` diff --git a/builder/ansible.cfg b/builder/ansible.cfg new file mode 100644 index 0000000..14c8065 --- /dev/null +++ b/builder/ansible.cfg @@ -0,0 +1,2 @@ +[defaults] +host_key_checking = False diff --git a/builder/bindep.txt b/builder/bindep.txt new file mode 100644 index 0000000..64eb27d --- /dev/null +++ b/builder/bindep.txt @@ -0,0 +1 @@ +bind-utils diff --git a/builder/execution-environment.yml b/builder/execution-environment.yml new file mode 100644 index 0000000..c7d5d4c --- /dev/null +++ b/builder/execution-environment.yml @@ -0,0 +1,20 @@ +--- +version: 1 + +build_arg_defaults: + EE_BASE_IMAGE: quay.io/ansible/ansible-runner:stable-2.10-devel + +ansible_config: ansible.cfg + +dependencies: + galaxy: requirements.yml + python: requirements.txt + system: bindep.txt + +additional_build_steps: + prepend: + - RUN whoami + - RUN cat /etc/os-release + append: + - RUN echo This is a post-install command! + - RUN ls -la /etc diff --git a/builder/requirements.txt b/builder/requirements.txt new file mode 100644 index 0000000..1fdf96c --- /dev/null +++ b/builder/requirements.txt @@ -0,0 +1 @@ +example-pypi-package diff --git a/builder/requirements.yml b/builder/requirements.yml new file mode 100644 index 0000000..63870da --- /dev/null +++ b/builder/requirements.yml @@ -0,0 +1,5 @@ +--- +collections: + - name: community.general + version: 3.3.2 + source: https://galaxy.ansible.com diff --git a/git/README.md b/git/README.md new file mode 100644 index 0000000..ec8ebfb --- /dev/null +++ b/git/README.md @@ -0,0 +1,76 @@ + +# Deploy Private Git Repository using Gitea + +Deploying your private Git repository using [Gitea](https://gitea.io/en-us/) to use AWX with the playbooks on SCM. + +Note that this sample manifest does not include any databases, so the SQLite3 has to be selected as `Database Type` for Gitea. + + +## Table of Contents + +- [Procedure](#procedure) + - [Prepare required files](#prepare-required-files) + - [Deploy Private Git Repository](#deploy-private-git-repository) + +## Procedure + +### Prepare required files + +Generate a Self-Signed Certificate. Note that IP address can't be specified. + +```bash +GIT_HOST="git.example.com" +openssl req -x509 -nodes -days 3650 -newkey rsa:2048 -out ./git/tls.crt -keyout ./git/tls.key -subj "/CN=${GIT_HOST}/O=${GIT_HOST}" -addext "subjectAltName = DNS:${GIT_HOST}" +``` + +Modify `hosts` and `host` in `git/ingress.yaml`. + +```yaml +... + - hosts: + - git.example.com 👈👈👈 + secretName: git-secret-tls + rules: + - host: git.example.com 👈👈👈 +... +``` + +Prepare directories for Persistent Volumes defined in `git/pv.yaml`. + +```bash +sudo mkdir -p /data/git +``` + +### Deploy Private Git Repository + +Deploy Private Git Repository. + +```bash +kubectl apply -k git +``` + +Required resources has been deployed in `git` namespace. + +```bash +$ kubectl get all -n git +NAME READY STATUS RESTARTS AGE +pod/git-576868dc5b-z7z55 1/1 Running 0 31s + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/git-service ClusterIP 10.43.192.81 3000/TCP,22/TCP 31s + +NAME READY UP-TO-DATE AVAILABLE AGE +deployment.apps/git 1/1 1 1 31s + +NAME DESIRED CURRENT READY AGE +replicaset.apps/git-576868dc5b 1 1 1 31s +``` + +Now your Git repository is accesible through `https://git.example.com/` or the hostname you specified. Visit the URL and follow the installation wizard. + +Note that this sample manifest does not include any databases, so the SQLite3 has to be selected as `Database Type` for Gitea. + +| Configration | Recommemded Value | +| -------------- | -------------------------------------------------------- | +| Database Type | `SQLite3` | +| Gitea Base URL | `https://git.example.com/` or the hostname you specified | diff --git a/git/deployment.yaml b/git/deployment.yaml new file mode 100644 index 0000000..12e2d6e --- /dev/null +++ b/git/deployment.yaml @@ -0,0 +1,32 @@ +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: git + labels: + app: git +spec: + replicas: 1 + selector: + matchLabels: + app: git + template: + metadata: + labels: + app: git + spec: + containers: + - name: git + image: gitea/gitea:1.14 + ports: + - name: gitea-http + containerPort: 3000 + - name: gitea-ssh + containerPort: 22 + volumeMounts: + - name: git-volume + mountPath: /data + volumes: + - name: git-volume + persistentVolumeClaim: + claimName: git-claim diff --git a/git/ingress.yaml b/git/ingress.yaml new file mode 100644 index 0000000..611f8b2 --- /dev/null +++ b/git/ingress.yaml @@ -0,0 +1,21 @@ +--- +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + name: git-ingress +spec: + tls: + - hosts: + - git.example.com + secretName: git-secret-tls + rules: + - host: git.example.com + http: + paths: + - path: / + pathType: ImplementationSpecific + backend: + service: + name: git-service + port: + number: 3000 diff --git a/git/kustomization.yaml b/git/kustomization.yaml new file mode 100644 index 0000000..aa0f3ed --- /dev/null +++ b/git/kustomization.yaml @@ -0,0 +1,22 @@ +--- +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +namespace: git + +generatorOptions: + disableNameSuffixHash: true + +secretGenerator: + - name: git-secret-tls + type: kubernetes.io/tls + files: + - tls.crt + - tls.key + +resources: + - namespace.yaml + - pv.yaml + - pvc.yaml + - ingress.yaml + - service.yaml + - deployment.yaml diff --git a/git/namespace.yaml b/git/namespace.yaml new file mode 100644 index 0000000..9c82b95 --- /dev/null +++ b/git/namespace.yaml @@ -0,0 +1,5 @@ +--- +apiVersion: v1 +kind: Namespace +metadata: + name: git diff --git a/git/pv.yaml b/git/pv.yaml new file mode 100644 index 0000000..bb5151b --- /dev/null +++ b/git/pv.yaml @@ -0,0 +1,14 @@ +--- +apiVersion: v1 +kind: PersistentVolume +metadata: + name: git-volume +spec: + accessModes: + - ReadWriteOnce + persistentVolumeReclaimPolicy: Retain + capacity: + storage: 5Gi + storageClassName: git-volume + hostPath: + path: /data/git diff --git a/git/pvc.yaml b/git/pvc.yaml new file mode 100644 index 0000000..2422e69 --- /dev/null +++ b/git/pvc.yaml @@ -0,0 +1,13 @@ +--- +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: git-claim +spec: + accessModes: + - ReadWriteOnce + volumeMode: Filesystem + resources: + requests: + storage: 2Gi + storageClassName: git-volume diff --git a/git/service.yaml b/git/service.yaml new file mode 100644 index 0000000..19e4aa0 --- /dev/null +++ b/git/service.yaml @@ -0,0 +1,15 @@ +--- +apiVersion: v1 +kind: Service +metadata: + name: git-service +spec: + ports: + - name: gitea-http + protocol: TCP + port: 3000 + - name: gitea-ssh + protocol: TCP + port: 22 + selector: + app: git diff --git a/registry/README.md b/registry/README.md index d66b0a3..389d949 100644 --- a/registry/README.md +++ b/registry/README.md @@ -12,7 +12,7 @@ Deploying your private container registry on your K3s to use with AWX. - [Quick Testing](#quick-testing) - [Testing with Docker](#testing-with-docker) - [Digging into the Registry](#digging-into-the-registry) -- [(Optional) Use as Private Container Registry for K3s](#optional-use-as-private-container-registry-for-k3s) +- [Use as Private Container Registry for K3s](#use-as-private-container-registry-for-k3s) - [Procedure](#procedure-1) - [Testing](#testing) @@ -174,9 +174,9 @@ reg tags -k registry.example.com/reguser/whalesay reg rm -k registry.example.com/reguser/whalesay:latest ``` -## (Optional) Use as Private Container Registry for K3s +## Use as Private Container Registry for K3s -Optionally, this registry can also be registered as a private container registry for K3s. +This registry can also be registered as a private container registry for K3s. This is required procedure to use this registry to store images for AWX Execution Environment. ### Procedure diff --git a/runner/.gitignore b/runner/.gitignore new file mode 100644 index 0000000..de153db --- /dev/null +++ b/runner/.gitignore @@ -0,0 +1 @@ +artifacts diff --git a/runner/README.md b/runner/README.md new file mode 100644 index 0000000..7084f3c --- /dev/null +++ b/runner/README.md @@ -0,0 +1,39 @@ +# Examples of Ansible Runner + +- [ansible/ansible-runner](https://github.com/ansible/ansible-runner) +- [Ansible Runner — ansible-runner 2.0.1 documentation](https://ansible-runner.readthedocs.io/en/stable/index.html) + +## Environment in This Example + +- CentOS 8.2 +- Python 3.9 +- Docker 20.10.7 + +## Install + +```bash +python3 -m pip install ansible-runner +``` + +## Prepare Required Files + +Refer [`projects/demo.yml`](projects/demo.yml) and [`env/settings`](env/settings). + +Your own Execution Environment can be used using `container_image` in [`env/settings`](env/settings). + +## Invoke Ansible Runner + +The sample playbook will help you figure out the differences between Execution Environments, such as Ansible version, `pip list`, etc. + +```bash +ansible-runner run . -p demo.yml +``` + +The settings can also be given directly as arguments like `--container-image` without using `env/settings`. + +## Tips + +- If `process_isolation` is set to `false` as default, `ansible-runner` will just work as a wrapper for the local `ansible-playbook` command. +- If `process_isolation` is set to `true`, `connection: local` points to the container itself, so the playbook cannot affect `localhost` which means the container host. To avoid this, you need to SSH to local host without using `connection: local`. [Related issues is here](https://github.com/ansible/ansible-runner/issues/752). +- `container_image` defaults to `quay.io/ansible/ansible-runner:devel`. You can check available tags for the public Execution Environment at [`quay.io/ansible/ansible-runner`](https://quay.io/repository/ansible/ansible-runner?tab=tags) +- The `process_isolation_show_paths` described in the documentation does not work with Docker and Podman. Instead, you can use `container_volume_mounts` as described in [`env/settings`](env/settings), but this is not documented. diff --git a/runner/env/settings b/runner/env/settings new file mode 100644 index 0000000..bbeea7b --- /dev/null +++ b/runner/env/settings @@ -0,0 +1,9 @@ +--- +process_isolation: true +process_isolation_executable: docker +container_image: registry.example.com/ansible/ee:2.10-custom + +# Specify additional bind-mount points to an isolated Docker or Podman container. +# Note that this is UNDOCUMENTED option. +#container_volume_mounts: +# - /etc/sample:/etc/sample diff --git a/runner/project/demo.yml b/runner/project/demo.yml new file mode 100644 index 0000000..0c15149 --- /dev/null +++ b/runner/project/demo.yml @@ -0,0 +1,38 @@ +--- +- hosts: localhost + connection: local + + tasks: + - name: Ensure that the host is reachable + ansible.builtin.ping: + + - name: Print variables for debugging + ansible.builtin.debug: + var: data + vars: + data: + ansible_playbook_python: "{{ ansible_playbook_python }}" + ansible_python_version: "{{ ansible_python_version }}" + ansible_python.executable: "{{ ansible_python.executable }}" + ansible_version.full: "{{ ansible_version.full }}" + + - name: Invoke debug commands + ansible.builtin.command: "{{ item }}" + changed_when: false + failed_when: false + loop: + - hostname + - whoami + - pwd + - python3 -m pip list + - ansible-galaxy collection list -p . + register: command_results + - ansible.builtin.debug: + msg: "{{ item.stdout_lines }}" + loop: "{{ command_results.results }}" + loop_control: + label: "{{ item.cmd }}" + + # - name: Simply pause for a while + # ansible.builtin.pause: + # minutes: 10