AWX Server Error Has Occurred

By default AWX installs the database in the /tmp directory. This can be avoided during install by modifying the inventory file.  However, if by chance AWX was installed using the default settings, you will eventually have a corrupt database and a Server Error page will be displayed.  If you are fortunate to have a snapshot of a virtual machine or a current backup of the database, you can get things working again, but this will be temporary.  There is a process I developed and successfully implemented that successfully moves the database and preserves the installation rather painlessly.

Resolution

No guarantees, but this process worked for one instance.

# Change the /root/awx_upgrade/awx/installer/inventory file from the terrible /tmp/pgdocker and /tmp/awxcompose to..
postgres_data_dir=/opt/pgdocker
docker_compose_dir=/opt/awxcompose

# Create a new location for the database and files.
mkdir /opt/pgdocker /opt/awxcompose

# Backup
cd /tmp
tar czvf awxcompose.tgz awxcompose/*
tar czvf pgdocker.tgz pgdocker/*


# Rerun the Ansible Playbook.  This will create a new database.
cd /root/awx_upgrade/awx/installer
ansible-playbook -i inventory install.yml

# Stop the services
sudo docker stop awx_task awx_web awx_memcached awx_rabbitmq awx_postgres

# Copy the old database to the new location.
cd /opt/pgdocker
scp -r /tmp/pgdocker/* .
chown polkitd:ssh_keys pgdata/ -R
chown polkitd:root pgdata

sudo docker stop awx_task awx_web awx_memcached awx_rabbitmq awx_postgres

# Should be permissive anyway, verified that it is
getenforce
sed -i --follow-symlinks 's/\(^SELINUX=\)\(.*\)/\1permissive/g' /etc/selinux/config
setenforce 0

Source(s)

  • https://github.com/ansible/awx/issues/58
  • https://github.com/ansible/awx/issues/1212
  • https://ansible-dellos-docs.readthedocs.io/en/latest/ztd_example.html
  • https://akai-tsuki.hatenablog.com/entry/2018/07/29/163710
  • https://github.com/ansible/awx/blob/devel/INSTALL.md#postgresql-1
  • https://github.com/ansible/awx/issues/1685
  • https://www.howtoforge.com/community/threads/awx-install-stuck.81007/
  • https://techblog.jeppson.org/2018/12/upgrading-awx/