AWX を (k8s では無く) Docker で起動する

以前に以下のメモを書きました。

• AmazonLinux2 + Nginx + Python3.x + Ansible AWX 環境を構築する
• Ubuntu 22.04.4LTS へ k3s で AWX Operator を起動する

以前 AWX は Docker で起動することが出来たのですが、最新の AWX は k8s で起動する前提に仕様が変わっているようです。 しかし、開発環境の構築に限っては Docker Compose for Development の手順に従うことで Docker Compose から起動することが出来るようです。 今回はこの手順をメモしておきます。

検証環境

対象	バージョン
Ubuntu	22.04.4LTS

AWX を Docker Compose でビルドする

ビルド時は Ansible が利用される為、予めインストールしておきます。

python3 -m pip install ansible

GitHub から AWX のソースコードを Clone します。 Docker イメージを作成する際は Git Repository の最新ファイルだけあれば良いので --depth=1 を指定し、「不要なファイルは Clone しないことでストレージ容量の節約と Clone にかかる処理時間の短縮」を行います。

git clone --depth=1 https://github.com/ansible/awx
cd awx

公式の手順では、ビルドは Makefile を利用します。 Starting the Development Environment では make docker-compose-build を利用するように記載されています。 docker-compose-build の定義部分は以下の通りです。

## Build awx_devel image for docker compose development environment
docker-compose-build: Dockerfile.dev
    DOCKER_BUILDKIT=1 docker build \
        -f Dockerfile.dev \
        -t $(DEVEL_IMAGE_NAME) \
        --build-arg BUILDKIT_INLINE_CACHE=1 \
        --cache-from=$(DEV_DOCKER_TAG_BASE)/awx_devel:$(COMPOSE_TAG) .

docker-compose-build から参照されている Dockerfile.dev の PHONY は以下のように定義されています。

.PHONY: Dockerfile.dev
## Generate Dockerfile.dev for awx_devel image
Dockerfile.dev: tools/ansible/roles/dockerfile/templates/Dockerfile.j2
    ansible-playbook tools/ansible/dockerfile.yml \
        -e dockerfile_name=Dockerfile.dev \
        -e build_dev=True \
        -e receptor_image=$(RECEPTOR_IMAGE)

tools/ansible/roles/dockerfile/templates/Dockerfile.j2 の内容はこのメモ末尾に参考として転記しておきます。 ビルドします。

make docker-compose-build

docker images を実行してコンテナイメージが作成されたことを確認します。

# docker images
REPOSITORY                  TAG       IMAGE ID       CREATED          SIZE
ghcr.io/ansible/awx_devel   devel     99af155251cf   40 minutes ago   1.96GB

AWX の起動

AWX の起動手順は Run AWX 記載されています。 make docker-compose を実行するように記載されていますが、下記とも記載されています。

For running docker-compose detached mode, start the containers using the following command: $ make docker-compose COMPOSE_UP_OPTS=-d

Makefile 中、該当箇所の定義は以下のようになっています。

docker-compose: awx/projects docker-compose-sources
    ansible-galaxy install --ignore-certs -r tools/docker-compose/ansible/requirements.yml;
    ansible-playbook -i tools/docker-compose/inventory tools/docker-compose/ansible/initialize_containers.yml \
        -e enable_vault=$(VAULT) \
        -e vault_tls=$(VAULT_TLS) \
        -e enable_ldap=$(LDAP); \
    $(MAKE) docker-compose-up

docker-compose-up:
    $(DOCKER_COMPOSE) -f tools/docker-compose/_sources/docker-compose.yml $(COMPOSE_OPTS) up $(COMPOSE_UP_OPTS) --remove-orphans

今回は detached mode にしてバックグラウンド実行したいので make docker-compose COMPOSE_UP_OPTS=-d を実行します。

make docker-compose COMPOSE_UP_OPTS=-d

コンテナの状態を確認する

コンテナの状態を確認するには docker compose -f tools/docker-compose/_sources/docker-compose.yml ps -a を実行します。

docker compose -f tools/docker-compose/_sources/docker-compose.yml ps -a

実際の実行例は以下の通りです。

# docker compose -f tools/docker-compose/_sources/docker-compose.yml ps -a
WARN[0000] /root/awx/tools/docker-compose/_sources/docker-compose.yml: `version` is obsolete
NAME               IMAGE                              COMMAND                  SERVICE    CREATED         STATUS         PORTS
tools_awx_1        ghcr.io/ansible/awx_devel:devel    "/entrypoint.sh laun…"   awx_1      8 minutes ago   Up 8 minutes   0.0.0.0:2222->2222/tcp, :::2222->2222/tcp, 0.0.0.0:6899->6899/tcp, :::6899->6899/tcp, 0.0.0.0:7899-7999->7899-7999/tcp, :::7899-7999->7899-7999/tcp, 0.0.0.0:8013->8013/tcp, :::8013->8013/tcp, 0.0.0.0:8043->8043/tcp, :::8043->8043/tcp, 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp, 22/tcp, 0.0.0.0:9888->9888/tcp, :::9888->9888/tcp, 0.0.0.0:3000->3001/tcp, :::3000->3001/tcp
tools_postgres_1   quay.io/sclorg/postgresql-15-c9s   "container-entrypoin…"   postgres   8 minutes ago   Up 8 minutes   0.0.0.0:5441->5432/tcp, :::5441->5432/tcp
tools_redis_1      redis:latest                       "redis-server /usr/l…"   redis_1    8 minutes ago   Up 8 minutes   6379/tcp

コンテナのログを確認するには docker compose -f tools/docker-compose/_sources/docker-compose.yml logs を実行します。

docker compose -f tools/docker-compose/_sources/docker-compose.yml logs

AWX へログインする

AWX のログが以下のようになっていれば (auth.0001_initial... OK になっていれば) 必要最小限の初期化処理は完了した、と判断出来るようです。

# docker compose -f tools/docker-compose/_sources/docker-compose.yml logs
    ・
    ・
    ・
tools_awx_1    | Running migrations:
tools_awx_1    |   Applying contenttypes.0001_initial... OK
tools_awx_1    |   Applying contenttypes.0002_remove_content_type_name... OK
tools_awx_1    |   Applying auth.0001_initial... OK

この状態になったら以下のコマンドを実行し、初期化します。

docker exec tools_awx_1 make clean-ui ui-devel

初期状態ではユーザが存在しない為、管理用ユーザを作成します。

docker exec -ti tools_awx_1 awx-manage createsuperuser

Web ブラウザから https://ADDRESS:8043 へアクセスするとログイン画面が表示されるはずです。

参考

tools/ansible/roles/dockerfile/templates/Dockerfile.j2

### This file is generated from
### tools/ansible/roles/dockerfile/templates/Dockerfile.j2
###
### DO NOT EDIT
###

# Build container
FROM quay.io/centos/centos:stream9 as builder

ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8
ENV AWX_LOGGING_MODE stdout


USER root

# Import the gpg key for DNF to work
RUN rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial

# Install build dependencies
RUN dnf -y update && dnf install -y 'dnf-command(config-manager)' && \
    dnf config-manager --set-enabled crb && \
    dnf -y install \
    iputils \
    gcc \
    gcc-c++ \
    git-core \
    gettext \
    glibc-langpack-en \
    libffi-devel \
    libtool-ltdl-devel \
    make \
{% if not headless|bool %}
    nodejs \
{% endif %}
    nss \
    openldap-devel \
    patch \
    postgresql \
    postgresql-devel \
    python3.11 \
    "python3.11-devel" \
    "python3.11-pip" \
    "python3.11-setuptools" \
    "python3.11-packaging" \
    "python3.11-psycopg2" \
    swig \
    unzip \
    xmlsec1-devel \
    xmlsec1-openssl-devel

RUN pip3.11 install -vv build

{% if image_architecture == 'ppc64le'  %}
    RUN dnf -y update && dnf install -y wget && \
    wget https://static.rust-lang.org/dist/rust-1.41.0-powerpc64le-unknown-linux-gnu.tar.gz && \
    tar -zxvf rust-1.41.0-powerpc64le-unknown-linux-gnu.tar.gz && \
    cd rust-1.41.0-powerpc64le-unknown-linux-gnu && \
    sh install.sh ;
{% endif %}

# Install & build requirements
ADD Makefile /tmp/Makefile
RUN mkdir /tmp/requirements
ADD requirements/requirements.txt \
    requirements/requirements_tower_uninstall.txt \
    requirements/requirements_git.txt \
    /tmp/requirements/

RUN cd /tmp && make requirements_awx

ARG VERSION
ARG SETUPTOOLS_SCM_PRETEND_VERSION
ARG HEADLESS

{% if (build_dev|bool) or (kube_dev|bool) %}
ADD requirements/requirements_dev.txt /tmp/requirements
RUN cd /tmp && make requirements_awx_dev
{% else %}
# Use the distro provided npm to bootstrap our required version of node

{% if not headless|bool %}
RUN npm install -g n && n 16.13.1
{% endif %}

# Copy source into builder, build sdist, install it into awx venv
COPY . /tmp/src/
WORKDIR /tmp/src/
RUN make sdist && /var/lib/awx/venv/awx/bin/pip install dist/awx.tar.gz

{% if not headless|bool %}
RUN DJANGO_SETTINGS_MODULE=awx.settings.defaults SKIP_SECRET_KEY_CHECK=yes SKIP_PG_VERSION_CHECK=yes /var/lib/awx/venv/awx/bin/awx-manage collectstatic --noinput --clear
{% endif %}

{% endif %}

# Final container(s)
FROM quay.io/centos/centos:stream9

ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8
ENV AWX_LOGGING_MODE stdout

USER root

# Import the gpg key for DNF to work
RUN rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial

ADD https://copr.fedorainfracloud.org/coprs/ansible/Rsyslog/repo/epel-9/ansible-Rsyslog-epel-9.repo /etc/yum.repos.d/ansible-Rsyslog-epel-9.repo

# Install runtime requirements
RUN dnf -y update && dnf install -y 'dnf-command(config-manager)' && \
    dnf config-manager --set-enabled crb && \
    dnf -y install acl \
    git-core \
    git-lfs \
    glibc-langpack-en \
    krb5-workstation \
    nginx \
    "openldap >= 2.6.2-3" \
    postgresql \
    python3.11 \
    "python3.11-devel" \
    "python3.11-pip*" \
    "python3.11-setuptools" \
    "python3.11-packaging" \
    "python3.11-psycopg2" \
    rsync \
    rsyslog-8.2102.0-106.el9 \
    subversion \
    sudo \
    vim-minimal \
    which \
    unzip \
    xmlsec1-openssl && \
    dnf -y clean all

RUN pip3.11 install -vv virtualenv supervisor dumb-init build

RUN rm -rf /root/.cache && rm -rf /tmp/*

{% if (build_dev|bool) or (kube_dev|bool) %}
# Install development/test requirements
RUN dnf -y install \
    crun \
    gdb \
    gtk3 \
    gettext \
    hostname \
    procps \
    alsa-lib \
    libX11-xcb \
    libXScrnSaver \
    iproute \
    strace \
    vim \
    nmap-ncat \
    libpq-devel \
    nodejs \
    nss \
    make \
    patch \
    socat \
    tmux \
    wget \
    diffutils \
    unzip && \
    npm install -g n && n 16.13.1 && npm install -g npm@8.5.0 && dnf remove -y nodejs

RUN pip3.11 install -vv git+https://github.com/coderanger/supervisor-stdout.git@973ba19967cdaf46d9c1634d1675fc65b9574f6e
RUN pip3.11 install -vv black setuptools-scm build

# This package randomly fails to download.
# It is nice to have in the dev env, but not necessary.
# Add it back to the list above if the repo ever straighten up.
RUN dnf --enablerepo=baseos-debug -y install python3-debuginfo || :

RUN dnf install -y epel-next-release && dnf install -y inotify-tools && dnf remove -y epel-next-release
{% endif %}

# Copy app from builder
COPY --from=builder /var/lib/awx /var/lib/awx

RUN ln -s /var/lib/awx/venv/awx/bin/awx-manage /usr/bin/awx-manage

{%if build_dev|bool %}
COPY --from={{ receptor_image }} /usr/bin/receptor /usr/bin/receptor

RUN openssl req -nodes -newkey rsa:2048 -keyout /etc/nginx/nginx.key -out /etc/nginx/nginx.csr \
        -subj "/C=US/ST=North Carolina/L=Durham/O=Ansible/OU=AWX Development/CN=awx.localhost" && \
    openssl x509 -req -days 365 -in /etc/nginx/nginx.csr -signkey /etc/nginx/nginx.key -out /etc/nginx/nginx.crt && \
    chmod 640 /etc/nginx/nginx.{csr,key,crt}
{% endif %}

{% if build_dev|bool %}
RUN dnf install -y podman && rpm --restore shadow-utils 2>/dev/null

# chmod containers.conf and adjust storage.conf to enable Fuse storage.
RUN sed -i -e 's|^#mount_program|mount_program|g' -e '/additionalimage.*/a "/var/lib/shared",' -e 's|^mountopt[[:space:]]*=.*$|mountopt = "nodev,fsync=0"|g' /etc/containers/storage.conf

ENV _CONTAINERS_USERNS_CONFIGURED=""

# Ensure we must use fully qualified image names
# This prevents podman prompt that hangs when trying to pull unqualified images
RUN mkdir -p /etc/containers/registries.conf.d/ && echo "unqualified-search-registries = []" >> /etc/containers/registries.conf.d/force-fully-qualified-images.conf && chmod 644 /etc/containers/registries.conf.d/force-fully-qualified-images.conf
{% endif %}

ADD tools/ansible/roles/dockerfile/files/rsyslog.conf /var/lib/awx/rsyslog/rsyslog.conf
ADD tools/ansible/roles/dockerfile/files/wait-for-migrations /usr/local/bin/wait-for-migrations
ADD tools/ansible/roles/dockerfile/files/stop-supervisor /usr/local/bin/stop-supervisor

ADD tools/ansible/roles/dockerfile/files/uwsgi.ini /etc/tower/uwsgi.ini

## File mappings
{% if build_dev|bool %}
ADD tools/docker-compose/launch_awx.sh /usr/bin/launch_awx.sh
ADD tools/docker-compose/start_tests.sh /start_tests.sh
ADD tools/docker-compose/bootstrap_development.sh /usr/bin/bootstrap_development.sh
ADD tools/docker-compose/entrypoint.sh /entrypoint.sh
ADD tools/scripts/config-watcher /usr/bin/config-watcher
ADD tools/docker-compose/containers.conf /etc/containers/containers.conf
ADD tools/docker-compose/podman-containers.conf /var/lib/awx/.config/containers/containers.conf
{% elif kube_dev|bool %}
RUN ln -sf /awx_devel/tools/ansible/roles/dockerfile/files/launch_awx_web.sh /usr/bin/launch_awx_web.sh
RUN ln -sf /awx_devel/tools/ansible/roles/dockerfile/files/launch_awx_task.sh /usr/bin/launch_awx_task.sh
RUN ln -sf /awx_devel/tools/ansible/roles/dockerfile/files/launch_awx_rsyslog.sh /usr/bin/launch_awx_rsyslog.sh
RUN ln -sf /awx_devel/{{ template_dest }}/supervisor_web.conf /etc/supervisord_web.conf
RUN ln -sf /awx_devel/{{ template_dest }}/supervisor_task.conf /etc/supervisord_task.conf
RUN ln -sf /awx_devel/{{ template_dest }}/supervisor_rsyslog.conf /etc/supervisord_rsyslog.conf
{% else %}
ADD tools/ansible/roles/dockerfile/files/launch_awx_web.sh /usr/bin/launch_awx_web.sh
ADD tools/ansible/roles/dockerfile/files/launch_awx_task.sh /usr/bin/launch_awx_task.sh
ADD tools/ansible/roles/dockerfile/files/launch_awx_rsyslog.sh /usr/bin/launch_awx_rsyslog.sh
ADD tools/scripts/rsyslog-4xx-recovery /usr/bin/rsyslog-4xx-recovery
ADD {{ template_dest }}/supervisor_web.conf /etc/supervisord_web.conf
ADD {{ template_dest }}/supervisor_task.conf /etc/supervisord_task.conf
ADD {{ template_dest }}/supervisor_rsyslog.conf /etc/supervisord_rsyslog.conf
ADD tools/scripts/awx-python /usr/bin/awx-python
{% endif %}

{% if (build_dev|bool) or (kube_dev|bool) %}
RUN echo /awx_devel > /var/lib/awx/venv/awx/lib/python3.11/site-packages/awx.egg-link
RUN echo /awx_devel > /var/lib/awx/venv/awx/lib/python3.11/site-packages/awx.pth
RUN ln -sf /awx_devel/tools/docker-compose/awx-manage /usr/local/bin/awx-manage
RUN ln -sf /awx_devel/tools/scripts/awx-python /usr/bin/awx-python
RUN ln -sf /awx_devel/tools/scripts/rsyslog-4xx-recovery /usr/bin/rsyslog-4xx-recovery
{% endif %}

# Pre-create things we need to access
RUN for dir in \
      /var/lib/awx \
      /var/lib/awx/rsyslog \
      /var/lib/awx/rsyslog/conf.d \
      /var/lib/awx/.local/share/containers/storage \
      /var/run/awx-rsyslog \
      /var/log/nginx \
      /var/lib/pgsql \
      /var/run/supervisor \
      /var/run/awx-receptor \
      /var/lib/nginx ; \
    do mkdir -m 0775 -p $dir ; chmod g+rwx $dir ; chgrp root $dir ; done && \
    for file in \
      /etc/subuid \
      /etc/subgid \
      /etc/group \
      /etc/passwd \
      /var/lib/awx/rsyslog/rsyslog.conf ; \
    do touch $file ; chmod g+rw $file ; chgrp root $file ; done

{% if (build_dev|bool) or (kube_dev|bool) %}
RUN for dir in \
      /etc/containers \
      /var/lib/awx/.config/containers \
      /var/lib/awx/.config/cni \
      /var/lib/awx/.local \
      /var/lib/awx/venv \
      /var/lib/awx/venv/awx/bin \
      /var/lib/awx/venv/awx/lib/python3.11 \
      /var/lib/awx/venv/awx/lib/python3.11/site-packages \
      /var/lib/awx/projects \
      /var/lib/awx/rsyslog \
      /var/run/awx-rsyslog \
      /.ansible \
      /var/lib/shared/overlay-images \
      /var/lib/shared/overlay-layers \
      /var/lib/shared/vfs-images \
      /var/lib/shared/vfs-layers \
      /var/lib/awx/vendor ; \
    do mkdir -m 0775 -p $dir ; chmod g+rwx $dir ; chgrp root $dir ; done && \
    for file in \
      /etc/containers/containers.conf \
      /var/lib/awx/.config/containers/containers.conf \
      /var/lib/shared/overlay-images/images.lock \
      /var/lib/shared/overlay-layers/layers.lock \
      /var/lib/shared/vfs-images/images.lock \
      /var/lib/shared/vfs-layers/layers.lock \
      /var/run/nginx.pid; \
    do touch $file ; chmod g+rw $file ; done && \
    echo "\setenv PAGER 'less -SXF'" > /var/lib/awx/.psqlrc
{% endif %}

{% if not build_dev|bool %}
RUN ln -sf /dev/stdout /var/log/nginx/access.log && \
    ln -sf /dev/stderr /var/log/nginx/error.log
{% endif %}

ENV HOME="/var/lib/awx"

{% if build_dev|bool %}
ENV PATH="/var/lib/awx/venv/awx/bin/:${PATH}"

EXPOSE 8043 8013 8080 22

ENTRYPOINT ["/entrypoint.sh"]
CMD ["/bin/bash"]
{% else %}
USER 1000
EXPOSE 8052

ENTRYPOINT ["dumb-init", "--"]
VOLUME /var/lib/nginx
VOLUME /var/lib/awx/.local/share/containers
{% endif %}

tools/docker-compose/ansible/requirements.yml

---
collections:
  - awx.awx
  - flowerysong.hvault
  - community.docker