fix: improve the guide for builder and add support to use ee without registry

README.md

@@ -256,6 +256,8 @@ Refer [📁 **Back up AWX using AWX Operator**](backup) and [📁 **Restore AWX
   - This guide includes not only the way to make backup manually, but also an example simple playbook for Ansible, which can be use with scheduling feature on AWX.
 - [📁 **Restore AWX using AWX Operator**](restore)
   - The guide to restore your AWX using AWX Operator.
+- [📁 **Build and Use your own Execution Environment**](builder)
+  - The guide to use Ansible Builder to build our own Execution Environment.
 - [📁 **Deploy Private Git Repository on Kubernetes**](git)
   - The guide to use AWX with SCM. This repository includes the manifests to deploy [Gitea](https://gitea.io/en-us/).
 - [📁 **Deploy Private Container Registry on Kubernetes**](registry)
@@ -267,8 +269,6 @@ Refer [📁 **Back up AWX using AWX Operator**](backup) and [📁 **Restore AWX
   - **All information on the guide is for development, testing and study purposes only.**
 - [📁 **Use SSL Certificate from Public ACME CA**](acme)
   - The guide to use a certificate from public ACME CA such as Let's Encrypt or ZeroSSL instead of Self-Signed certificate.
-- [📁 **Use Ansible Builder**](builder)
-  - The guide to use Ansible Builder to build our own Execution Environment.
 - [📁 **Use Ansible Runner**](runner)
   - The guide to use Ansible Runner to run playbook using Execution Environment.
 - [📁 **Use Customized Pod Specification for your Execution Environment**](containergroup)

builder/README.md

@@ -1,38 +1,103 @@
-# Examples of Ansible Builder
+<!-- omit in toc -->
+# Build and use your own Execution Environment
 
-This repository includes ready-to-use files as an example to use Ansible Builder.
+This page mainly shows you how to build and use your own custom Execution Environment.
+
+<!-- omit in toc -->
+## Table of Contents
+
+- [Introduction](#introduction)
+- [Build your own custom EE](#build-your-own-custom-ee)
+  - [Environment in This Example](#environment-in-this-example)
+  - [Install Ansible Builder](#install-ansible-builder)
+  - [Prepare Required Files](#prepare-required-files)
+  - [Build EE](#build-ee)
+- [Use EE](#use-ee)
+  - [Use EE in AWX](#use-ee-in-awx)
+    - [Use EE in AWX with container registry](#use-ee-in-awx-with-container-registry)
+    - [Use EE in AWX without container registry](#use-ee-in-awx-without-container-registry)
+  - [Use EE in Ansible Runner](#use-ee-in-ansible-runner)
+- [Tips](#tips)
+  - [Get the `Dockerfile` of your custom EE](#get-the-dockerfile-of-your-custom-ee)
+  - [Use Ansible collections without custom EE](#use-ansible-collections-without-custom-ee)
+
+## Introduction
+
+When the Job Template launched on AWX, the playbook runs on the Execution Environment (EE), which is a containerized environment completely isolated from your K3s host.
+
+- [What's new in Ansible Automation Platform 2: automation execution environments](https://www.ansible.com/blog/whats-new-in-ansible-automation-platform-2-automation-execution-environments)
+
+The default EE, [quay.io/ansible/awx-ee:latest](https://quay.io/repository/ansible/awx-ee), has some typical collections, Pip modules, and RPM packages by default, but sometimes your playbooks require additional (non-default) collections, modules or packages.
+
+In such cases, you need to add Ansible collections, Python packages, and RPM packages to EE. There are two ways you can choose to achieve this.
+
+1. **Build and use your own Execution Environment**
+   - This method is used when additional Pip or RPM packages are required in addition to the Ansible collection.
+   - Follow whole of this page to build and use custom EE.
+2. **Place `collections/requirements.yml` in your project**
+   - This is useful if you simply need additional Ansible collections only, and no additional Pip or RPM packages are required. However, this method cannot be used if your project type on AWX is Manual.
+   - See [_Use Ansible collections without custom EE_](#use-ansible-collections-without-custom-ee) at the bottom of this page for instructions.
+
+## Build your own custom EE
+
+To build custom EE, there is a tool called Ansible Builder. You can build your own custom EE with any Ansible collections, Python packages, and RPM packages added.
 
 - [ansible/ansible-builder](https://github.com/ansible/ansible-builder)
 - [Introduction — ansible-builder documentation](https://ansible-builder.readthedocs.io/en/latest/)
 
-## Environment in This Example
+This repository includes ready-to-use files as an example to use Ansible Builder. You can clone my repository to start with my ready-to-use example files.
+
+```bash
+cd ~
+git clone https://github.com/kurokobo/awx-on-k3s.git
+cd awx-on-k3s/builder
+```
+
+### Environment in This Example
 
 - CentOS Stream 8 (Minimal)
 - Python 3.9
 - Docker 20.10.17
 - Ansible Builder 1.1.0
 
-## Install
+### Install Ansible Builder
+
+You can install Ansible Builder by using Pip.
 
 ```bash
 python3 -m pip install ansible-builder
 ```
 
-## Prepare Required Files
+### Prepare Required Files
 
-`execution-environment.yml` is required file to build Execution Environment.
+At least, the file `execution-environment.yml` is required to build EE.
 
-The base image can be chosen from the tags from [http://quay.io/ansible/ansible-runner](https://quay.io/repository/ansible/ansible-runner?tab=tags).
+This repository contains [`execution-environment.yml` as a ready-to-use example](execution-environment.yml). This file is made to achieve following requirements.
 
-## Build Execution Environment
+- Use `quay.io/ansible/ansible-runner:stable-2.12-latest` as the base image
+- Add Ansible collections that listed in [`requirements.yml`](requirements.yml)
+- Add Python packages that listed in [`requirements.txt`](requirements.txt)
+- Add RPM Packages that listed in [`bindep.txt`](bindep.txt)
+- Run some commands before build steps and after build steps
 
-`ansible-builder build` command builds Execution Environment as a container image according to the definition in `execution-environment.yml`.
+You can review modify [`execution-environment.yml`](execution-environment.yml) and any YAML or TEXT file referenced from this file to suit your requirements.
+
+Note the base image can be chosen from the tags from [quay.io/ansible/ansible-runner](https://quay.io/repository/ansible/ansible-runner?tab=tags).
+
+The syntax of `requirements.yml` is [the same as for Ansible Galaxy](https://docs.ansible.com/ansible/latest/galaxy/user_guide.html#install-multiple-collections-with-a-requirements-file). The syntax of `requirements.txt` is [the same as for Pip](https://pip.pypa.io/en/stable/reference/requirements-file-format/), and `bindep.txt` is [for Bindep](https://docs.opendev.org/opendev/bindep/latest/readme.html).
+
+Other customization is possible besides this. Refer to [the official Ansible Builder documentation](https://ansible-builder.readthedocs.io/en/latest/) for details.
+
+### Build EE
+
+Once your files are ready, run `ansible-builder build` command to build EE as a container image according to the definition in `execution-environment.yml`. Specify a tag (`--tag`) to suit your requiremnts.
 
 ```bash
-cd builder
 ansible-builder build --tag registry.example.com/ansible/ee:2.12-custom --container-runtime docker --verbosity 3
 ```
 
+Below is an example output of this command.
+
 ```bash
 $ ansible-builder build --tag registry.example.com/ansible/ee:2.12-custom --container-runtime docker --verbosity 3
 Ansible Builder is building your execution environment image, "registry.example.com/ansible/ee:2.12-custom".
@@ -56,13 +121,25 @@ Successfully tagged registry.example.com/ansible/ee:2.12-custom
 Complete! The build context can be found at: /home/********/awx-on-k3s/builder/context
 ```
 
+Once the command is complete, your custom EE image is built and stored on Docker or Podman.
+
 ```bash
 $ docker image ls
 REPOSITORY                        TAG                  IMAGE ID       CREATED         SIZE
 registry.example.com/ansible/ee   2.12-custom          f6c3375db22e   4 minutes ago   748MB
 ```
 
-Now you can push this image to the container registry to use as Execution Environment on AWX. If you want to deploy your own private container registry, refer [additional guide on this repository](../registry).
+## Use EE
+
+You can use your EE in your AWX or Ansible Runner.
+
+### Use EE in AWX
+
+To use your EE in AWX, in typical use cases, your EE should be stored on some container registry and should be pulled by AWX, but technically you can use your EE with your AWX without any container registry.
+
+#### Use EE in AWX with container registry
+
+Simply you can push your EE image to some container registry. Any registry can be acceptable. If you want to deploy your own private container registry, refer [additional guide on this repository](../registry).
 
 ```bash
 $ docker push registry.example.com/ansible/ee:2.12-custom
@@ -71,7 +148,49 @@ The push refers to repository [registry.example.com/ansible/ee]
 2.12-custom: digest: sha256:043a2bd19f4fcc5bd189f0ef0e8fb4e3b436c90e984f23f7dcf0e6b3da4443e0 size: 4515
 ```
 
-The `Dockerfile` generated for the build will be saved under the `context` directory.
+Then you can specify `registry.example.com/ansible/ee:2.12-custom` as your own custom EE in AWX. Specify registry credentials if your container registry requires authentication.
+
+#### Use EE in AWX without container registry
+
+The EEs in AWX are just Pods on Kubernetes. When executing a Job Template in AWX, AWX instructs Kubernetes to create a Pod using specified EE image. Of course Kubernetes requires the specified container image when creating the Pod. So while creating the Pod, Kubernetes usually pulls specified image from the container registry.
+
+But this happens only if the image is not cached on the container runtime for Kubernetes. If Kubernetes has the cache of the specified image, Kubernetes never pulls the image from the container registry (Technically, this is the default behavior and stated as `imagePullPolicy` is `IfNotPresent`).
+
+This means that if your Kubernetes has all the EE images you need in its cache in advance, you can use any EE from AWX without any container registry. To achieve this, you should export your EE image from Docker or Podman, and import it to containerd that the container runtime for K3s.
+
+```bash
+# Save your EE image as Tar file
+docker save registry.example.com/ansible/ee:2.12-custom -o custom-ee.tar
+
+# Import the Tar file to containerd
+sudo /usr/local/bin/k3s ctr images import --compress-blobs --base-name registry.example.com/ansible/ee:2.12-custom custom-ee.tar
+```
+
+Ensure your imported image is listed.
+
+```bash
+$ sudo /usr/local/bin/k3s crictl images
+IMAGE                             TAG           IMAGE ID            SIZE
+...
+registry.example.com/ansible/ee   2.12-custom   cbd5e7519054c       515MB
+...
+```
+
+Now you can specify `registry.example.com/ansible/ee:2.12-custom` as your own custom EE in AWX without any container registry and any credentials.
+
+You can change the policy of pulling the image in `Edit` page of your EE. The default `Only pull the image if not present before running` is ok, but to be safe you should specify `Never pull container before running`.
+
+### Use EE in Ansible Runner
+
+Refer [the guide for Ansible Runner on this repository](../runner).
+
+## Tips
+
+Some additional information around Ansible Builder and EE.
+
+### Get the `Dockerfile` of your custom EE
+
+The `Dockerfile` is generated and stored under the `context` directory once your `ansible-builder build` command is completed.
 
 ```bash
 $ cat context/Dockerfile
@@ -84,9 +203,7 @@ USER root
 ...
 ```
 
-## Create Context Files
+If you want to create `Dockerfile` without building image, you can use `ansible-builder create` command.
-
-`ansible-builder create` command only creates context files like `Dockerfile`, it does not invoke building the image.
 
 ```bash
 $ ansible-builder create --verbosity 3
@@ -98,3 +215,14 @@ File context/_build/ansible.cfg will be created.
 Rewriting Containerfile to capture collection requirements
 Complete! The build context can be found at: /home/********/awx-on-k3s/builder/context
 ```
+
+### Use Ansible collections without custom EE
+
+If you simply need additional Ansible collections only, and no additional Pip or RPM packages are required, this method is the simplest way. However, this method can't be applied for Manual type projects. If you want to add your project as Manual type, you should build custom EE.
+
+The procedure is quite simple, create and place `collections/requirements.yml` on your project root including the list of the collections which you want to use.
+
+The format of `collections/requirements.yml` is the same as [the `requirements.yml` for ansible-galaxy](
+https://docs.ansible.com/ansible/latest/galaxy/user_guide.html#install-multiple-collections-with-a-requirements-file).
+
+If `collections/requirements.yml` is present in your project, AWX will install the collections accordingly while updating project.

tips/troubleshooting.md

@@ -391,14 +391,4 @@ fatal: [localhost]: FAILED! => {"changed": false, "msg": "Failed to import the r
 ...
 ```
 
-When the Job Template launched at AWX, the playbook runs on the Execution Environment, which is a containerized environment completely isolated from the K3s host. The default Execution Environment has few typical collections, Pip modules, and RPM packages by default, but if your playbooks require additional (non-default) modules or packages, there are two ways to achieve this.
+To solve this, refer [the guide about Execution Environment on this repository](../builder).
-
-- **Place `collections/requirements.yml` in your project.**
-  - Note that this way is applicable for **adding Collections only**. If you want to add not only Collections but also Pip modules or RPM Packages, follow the next method to build your own Execution Environment.
-  - You can create and place your own `collections/requirements.yml` including collections which you want to use. The format is the same as [the `requirements.yml` for ansible-galaxy](
-https://docs.ansible.com/ansible/latest/galaxy/user_guide.html#install-multiple-collections-with-a-requirements-file).
-  - If `collections/requirements.yml` is present in your project, AWX will install the collections accordingly.
-- **Build your own Execution Environment.**
-  - You can build your own Execution Environment. This method can be used for adding Collections, Pip modules and RPM packages.
-  - [There is a guide to use Ansible Builder on this repository](https://github.com/kurokobo/awx-on-k3s/tree/main/builder) to build Execution Environment. You can customize `requirements.yml` for Collections, `requirements.txt` for Pip modules, and `bindep.txt` for RPM packages.
-  - This method also requires Docker or Podman, and container registry. If you want to deploy your own private container registry on your K3s, refer [the guide on this repository](../registry).