This article currently covers automated installation of AWX on an Ubuntu-18.04 LXD guest via cloud init.
The AWX install process process is in the AWX Project repository. A good walk through is also available on youtube. These instructions favor CentOS7 and use ansible to create and install microservices into Docker containers directly or via Openshift or Kubernetes container management platforms.
The instructions favor RedHat Enterprise Linux 7 or the various derived flavors like CentOS7 or Oracle Secure linux that can use the EPEL RPM/yum package repositories.
Installation to apt package managed distributions like Ubuntu is supported under the Tower product and works fine, but documentation favours RHEL7 and it probably receives slightly more testing.
‘’’The nodejs version’’’ called out in the INSTALL.md doc is behind what is released under Ubutu-18.04, but the newer version seems to work so far. The python-docker-py package is also not released under Ubuntu 18.04 and is more confusing in the INSTALL.md doc. Python-docker-py is an old python interface to docker that has been replaced by the newer python-docker module. Build works just fine if you install one of either python-docker or python-docker-py which can still be found in pip but it breaks if both are installed. The situation is more confusing since docker-compose pulls in python-docker as a dependency.
The AWX Installation Video walks though the process and is 20 minutes well spent if it’s your first time through.
As it turns out, the entire install works fine in an LXD ubuntu-18.04 guest on ubuntu-18.04 host. The install requires a privileged user. After creating the lxd guest, you can switch it back to unpriviliged with security.nesting=true and things run fine. I believe the extra privileges are required for docker container building due to use of mknod in initial setup, though I stopped short of validating that to 100%.
It is noteworthy that the ubuntu lxd image repo for has a history of being ahead of images: linuxcontainers.org repo in terms of cloud-init support though both appear to have at least some cloud-init support at this point.
Along the way, a manual CentOS7 lxd guest install was validated manaually as well and may have the advantage of being closer to the target developed to since Tower, Ansible, and AWX are RedHat opensource projects, but the ubuntu guest has yet to show flaws.
Note that the user.user-data string is itself a yaml document. When developing this, it’s useful to use a separate file starting at #cloud-config so you can lint the internal data.
You can also use the yaml2json.py script below to visualize your embedded data structure.
---
config:
linux.kernel_modules: ip_tables
security.nesting: "true"
security.privileged: "true"
user.user-data: |
#cloud-config
output:
all: '| tee -a /var/log/cloud-init-output.log'
package_update: true
package_upgrade: true
packages:
- docker.io
- docker-compose
- build-essential
- cloud-utils
- vim
- ansible
- nodejs
- npm
- git
runcmd:
- set -xe
- mkdir -p /awx/src
- mkdir /awx/ca-trust
- cat > '/awx/ca-trust/cacert.pem' << 'EOF'
- |-
-----BEGIN CERTIFICATE-----
YOUR INTERNAL CA PEM HERE
-----END CERTIFICATE-----
EOF
- chmod 444 '/awx/ca-trust/cacert.pem'
- chmod 555 '/awx/ca-trust'
- git clone 'https://github.com/ansible/awx.git' '/awx/src/awx'
- git clone 'https://github.com/ansible/awx-logos.git' '/awx/src/awx-logos'
- systemctl start docker
- systemctl enable docker
- sed -i 's/postgres_data_dir/# postgres_data_dir/' '/awx/src/awx/installer/inventory'
- cat >> '/awx/src/awx/installer/inventory' << 'EOF'
- |-
postgres_data_dir=/awx/db
awx_official=true
project_data_dir=/awx/projects
ca_trust_dir=/awx/ca-trust
use_docker_compose=false
EOF
- |-
env HOME=/root USER=root ansible-playbook -vi /awx/src/awx/installer/inventory \
/awx/src/awx/installer/install.yml
users:
- groups:
- adm
- audio
- cdrom
- dialout
- dip
- floppy
- netdev
- plugdev
- sudo
- video
- docker
lock_passwd: true
name: ubuntu
shell: /bin/bash
ssh-authorized-keys:
- ssh-rsa PUT YOUR SSH KEYS HERE user@asdf
- ssh-rsa PUT YOUR SSH KEYS HERE user@jkl;
sudo:
- ALL=(ALL) NOPASSWD:ALL
description: Installs awx in lxd
devices: {}
name: awx
used_by: []
Note that the image copy and profile load is persistent on the LXD host only required the first time.
user@mendota$ lxc image copy ubuntu:18.04/amd64 local: --copy-aliases
user@mendota$ lxc profile create awx
user@mendota$ lxc profile edit awx <awx.yml
user@mendota$ lxc launch 18.04/amd64 awx01 -p default -p awx -v ;\
sleep 10 ;\
lxc exec monona:awx01 tail -100f /var/log/cloud-init-output.log
This should scroll past a bunch of data with a final success message like the following.
PLAY RECAP *********************************************************************
localhost : ok=12 changed=5 unreachable=0 failed=0
Cloud-init v. 18.2 finished at Fri, 15 Jun 2018 15:19:45 +0000. Datasource DataSourceNoCloud [seed=/var/lib/cloud/seed/nocloud-net][dsmode=net]. Up 355.00 seconds
At this point you should be able to visit the LXD guest via http://awx01 and see an upgrading spinner. The first run calls a playbook to update the install after which you will get a login screen with default credentials of admin and password.
For extra security you can switch the guest back to unprivileged mode.
user@mendota$ lxc exec awx01 -- shutdown -h now
user@mendota$ lxc config set awx security.privileged false
user@mendota$ lxc start awx01
Since these are containers, you can access the nested namespace pids from the host as well.
user@monona ~ 0 $ lxc list local:awx01 --columns=ns4p
+-------+---------+----------------------+------+
| NAME | STATE | IPV4 | PID |
+-------+---------+----------------------+------+
| awx01 | RUNNING | 172.17.0.1 (docker0) | 9442 |
| | | 10.0.2.208 (eth0) | |
+-------+---------+----------------------+------+
user@monona ~ 0 $ sudo pstree -pTu 9442
systemd(9442,1000000)─┬─accounts-daemon(12267)
├─agetty(12760)
├─atd(12276,1000001)
├─cron(12272)
├─dbus-daemon(12385,1000103)
├─dockerd(12258)─┬─docker-containe(12518)─┬─docker-containe(13248)───memcached(13717,1001000)
│ │ ├─docker-containe(13268)───sh(13836)─┬─epmd(15991,1000100)
│ │ │ └─launch.sh(14371)───rabbitmq-server(14401,1000100)───beam.smp(+
│ │ ├─docker-containe(13422)───postgres(13909,1000999)─┬─postgres(15413)
│ │ │ ├─postgres(15414)
│ │ │ ├─postgres(15415)
│ │ │ ├─postgres(15416)
│ │ │ ├─postgres(15417)
│ │ │ ├─postgres(19731)
│ │ │ └─postgres(19796)
│ │ ├─docker-containe(14801)───tini(15212)───bash(15495)───supervisord(18009)─┬─daphne(18356)
│ │ │ ├─nginx(18357)───nginx(183+
│ │ │ ├─python(18355)
│ │ │ └─uwsgi(18359)─┬─uwsgi(183+
│ │ │ ├─uwsgi(183+
│ │ │ ├─uwsgi(183+
│ │ │ ├─uwsgi(183+
│ │ │ └─uwsgi(184+
│ │ └─docker-containe(15540)───tini(15695)───bash(15906)───supervisord(19625)─┬─celery(19635)─┬─celery(1+
│ │ │ ├─celery(1+
│ │ │ ├─celery(1+
│ │ │ └─celery(1+
│ │ ├─python(19634)
│ │ ├─python(19636)
│ │ └─python(19637)─┬─python(1+
│ │ ├─python(1+
│ │ ├─python(1+
│ │ └─python(1+
│ └─docker-proxy(14675)
├─networkd-dispat(12280)
├─polkitd(12643)
├─rsyslogd(12545,1000102)
├─snapd(12259)
├─sshd(12637)
├─systemd-journal(9981)
├─systemd-logind(12543)
├─systemd-network(11145,1000100)
├─systemd-resolve(11451,1000101)
└─systemd-udevd(10427)
The below script can be used to parse through and view interpreted data structures produces by yaml.
#!/usr/bin/python
import sys, json, yaml
try:
if sys.argv[1] == "-":
y=yaml.load(sys.stdin.read())
else:
with open(sys.argv[1]) as i:
y=yaml.load(i.read())
except:
print "Usage: \n\t%s [file.yml|-] {key} [key} ...\n"
print "Dumps yaml content as beuatified json\n"
print "if argv[1] is -, reads yaml from stdin\n"
print "if optional key is given, key is extracted and printed to stdout\n"
else:
i=2
while len(sys.argv) > i:
y=y.get(sys.argv[i])
i += 1
if isinstance(y,(str,unicode)) is False and isinstance(y,(list,dict)) is True:
print json.dumps(y,indent=2)
else:
print y
user@monona:/tmp$ yaml2json.py
Usage:
%s [file.yml|-] {key} [key} ...
Dumps yaml content as beuatified json
if argv[1] is -, reads yaml from stdin
if optional key is given, key is extracted and printed to stdout
user@monona:/tmp$ yaml2json.py /tmp/awx.yaml config user.user-data | yaml2json.py -
{
"users": [
{
"ssh-authorized-keys": [
"ssh-rsa ... me@booger",
"ssh-rsa ... me@mendota"
],
"shell": "/bin/bash",
"name": "ubuntu",
"groups": [
"adm",
"audio",
"cdrom",
"dialout",
"dip",
"floppy",
"netdev",
"plugdev",
"sudo",
"video",
"docker"
],
"sudo": [
"ALL=(ALL) NOPASSWD:ALL"
],
"lock_passwd": true
}
],
"output": {
"all": "| tee -a /var/log/cloud-init-output.log"
},
"package_update": true,
"packages": [
"docker.io",
"build-essential",
"cloud-utils",
"vim",
"ansible",
"python-pip",
"nodejs",
"npm",
"git"
],
"package_upgrade": true,
"runcmd": [
"set -xe",
"pip install docker-py",
"env",
"mkdir -p /awx/src",
"mkdir /awx/ca-trust",
"cat > '/awx/ca-trust/cacert.pem' << 'EOF'",
"-----BEGIN CERTIFICATE-----\nMIID9TCCAl2gAwIBAgIMWw23QDQvMTzGywMA0GCSqGSIb3DQEBCwUAMBYxFDAS\nBgNVBAMTC2RldmVuZG9yLmlvMB4XDTE4MDUyOTIwMjUzNloXDTIxMDIyMTIwMjUz\nNlowFjEUMBIGA1UEAxMLZGV2ZW5kb3IuaW8wggGiMA0GCSqGSIb3DQEBAQUAA4IB\njwAwggGKAoIBgQDlcLd4os+jXIu8w6mRDI4Tv1XHiFBfAgGCHXiCWgrZZ2NK9hVR\nYsqQ6yh2phJN9TQRU5xCVjiojoivVTfOKbIbYcWsBSam3PSoMYgfejvREgiXT2PP\nW9WyteTSRgDiBYua9PHYmYXMniqhvJ4rP2m9MATuso0yuDs+EXAo0U6wCSLoLnsA\ndpnBiL71xF5kg7/V5sDhgaGnQD21r38oRqJ4UV2BfvdavLNGOtcaCfS1NZpngEHv\nN3FoWkMoA4Vha01K20PY79uWLHlDR3bHAKOs8TKCcH+RvIySM8Vi6/nVSbE6vaz+\nfvNhtNZEVaBRi/Wjt6MUD4JYY8JLi3UiQv1EPNFZi5jaVmO168k+7Au0s93jFZ3n\nr5Je7L7Bb6jQntHECcicTpBpVI7+M/MdlKq88VyUBNWMpVXp2Ieq81Wg2h6MlhBT\nTeH9veHY/B/2apzl08hhqbxb+Or/4LeHEL69fA69AuLfGkGKHLCsUh/71RXjHdfc\n0OE2OEgFEfGz0cMCAwEAAaNDMEEwDwYDVR0TAQH/BAUwAwEB/zAPBgNVHQ8BAf8E\nBQMDBwQAMB0GA1UdDgQWBBTK45tHZnjDCvEipkkIHnnZled98zANBgkqhkiG9w0B\nAQsFAAOCAYEA4GtL+LZ79OjxY0KW05e0sMmdDjMQHVhdfyRe0R6d3A1QeY1gsD6E\nAQ7zOvllE8dcvTeZRKuJ7nK4PECNlBmlD8P/WBK95gHoQh8Ljbr+tWeFAaGTFAAK\nqcldcNM0GHB2TUYPZzpJ/wIv2ry6SUOswffkqqBoWbs//Ho8lPV3YGOAtBcTvJYu\nxmzPioxGPA/9vlC8pcown0KJub8oCVlFq31GbINSe25/ZiLcAp1S/msblzskAbgC\n78FruivGdjN8aWMKY2xzZrrPBYy82hoy4BfHIa8ps61/EGRt4Da73yS8dDDqNz6n\nTEV4woVUxYWUnYmIv0Ghh0IWZAWpAZeuGy6TiNbqPisadscXah+31QV5HbLaits+\nJF2GfiGVkiDJ5qKOH1QYnIbphj89Fr0I4dMa18SpQT1LRk7FXPXxhRJ7vKrLpLua\n9dYm0J/WrpilTBt7o5De1EoElt2oS1nkziFRdQMmcXwVg/Zxk73fRlafudjJoC72\nbJDHwXBjx0KC\n-----END CERTIFICATE-----\nEOF",
"chmod 444 '/awx/ca-trust/cacert.pem'",
"chmod 555 '/awx/ca-trust'",
"git clone 'https://github.com/ansible/awx.git' '/awx/src/awx'",
"git clone 'https://github.com/ansible/awx-logos.git' '/awx/src/awx-logos'",
"systemctl start docker",
"systemctl enable docker",
"sed -i 's/postgres_data_dir/# postgres_data_dir/' '/awx/src/awx/installer/inventory'",
"cat >> '/awx/src/awx/installer/inventory' << 'EOF'",
"postgres_data_dir=/awx/db\nawx_official=true\nproject_data_dir=/awx/projects\nca_trust_dir=/awx/ca-trust\n\nEOF",
"env HOME=/root USER=root ansible-playbook -vvi /awx/src/awx/installer/inventory /awx/src/awx/installer/install.yml"
]
}
Ansible the company started the opensource ‘’ansible’’ software and the paide Ansible Tower product attached to it. Ansible has since been aquired by redhat .
Redhat pushed the Ansible Tower source code in the commons under the AWX Project while retaining Tower Product as a supported and curated version of AWX.
Is the opensource project behind Ansible agentless devops automation software based on python and various remote connectivity options.
The productized version of Ansible AWX sold by RedHat. This is a management product build on top of Ansible Engine that layers on additional functionality for ansible including a rest api and web ui. Ansible Tower is reportedly just a curated and supported version of code developed under the AWX Project
Refers to a productized and supported version of Ansible provided by redhat and developed under the Ansible Project.
The free version of Tower as found in the AWX Project. Reference documentation for usage is found under Ansible Tower with a few exceptions such as AWX install process found in the github repo for the AWX Project
Agentless devops automation software based on python and many remote connectivity options with ssh being the protocol supported, though windows remote access, snmp, web services, cloud-ap, and several other options have matured and extend ansible well beyond managing ssh enabled devices.
Lightweight container platform designed for use in deployment and repeatibility. Original versions of docker share a common ancestor in LXC
Next generation LXC that included a daemon and rest interface. Lightweight containers like Docker with fuller support for something closer to a virtual machines without the hypervisor overhead. The opensource LXD project was founded and is currently lead by Canonical Ltd.
A an opensource container management platform developed by google with versions available under both redhat and ubuntu that appear to bundle or preconfigure some addons to round out a cloud platform.
A RedHat branded project based on kubernetes.
Cloud-init is the defacto multi-distribution package that handles early initialization of a cloud instance.
Todo
Setup a trial instance of Ansible Tower to compare.
Todo
Load up awx with some useful stuff.
Todo
Switching docker_compose=true works fine. It’s a little unclear to me at this time what the difference amounts to.