Ubuntu binary ironic inspector missing pymysql

Seen in WIP ironic CI test:

TASK [ironic : Running Ironic Inspector bootstrap container] *******************
task path: /home/zuul/src/opendev.org/openstack/kolla-ansible/ansible/roles/ironic/tasks/bootstrap_service.yml:27
<23.253.160.154> ESTABLISH SSH CONNECTION FOR USER: kolla
<23.253.160.154> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/home/zuul/.ssh/id_rsa_kolla"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="kolla"' -o ConnectTimeout=10 -o ControlPath=/home/zuul/.ansible/cp/25772d2a5f 23.253.160.154 '/bin/sh -c '"'"'echo ~kolla && sleep 0'"'"''
<23.253.160.154> (0, '/home/kolla\n', '')
<23.253.160.154> ESTABLISH SSH CONNECTION FOR USER: kolla
<23.253.160.154> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/home/zuul/.ssh/id_rsa_kolla"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="kolla"' -o ConnectTimeout=10 -o ControlPath=/home/zuul/.ansible/cp/25772d2a5f 23.253.160.154 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /home/kolla/.ansible/tmp/ansible-tmp-1559570225.06-90034726882536 `" && echo ansible-tmp-1559570225.06-90034726882536="` echo /home/kolla/.ansible/tmp/ansible-tmp-1559570225.06-90034726882536 `" ) && sleep 0'"'"''
<23.253.160.154> (0, 'ansible-tmp-1559570225.06-90034726882536=/home/kolla/.ansible/tmp/ansible-tmp-1559570225.06-90034726882536\n', '')
Using module file /home/zuul/src/opendev.org/openstack/kolla-ansible/ansible/library/kolla_docker.py
<23.253.160.154> PUT /home/zuul/.ansible/tmp/ansible-local-13134L07LR2/tmpF4mwj5 TO /home/kolla/.ansible/tmp/ansible-tmp-1559570225.06-90034726882536/AnsiballZ_kolla_docker.py
<23.253.160.154> SSH: EXEC sftp -b - -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/home/zuul/.ssh/id_rsa_kolla"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="kolla"' -o ConnectTimeout=10 -o ControlPath=/home/zuul/.ansible/cp/25772d2a5f '[23.253.160.154]'
<23.253.160.154> (0, 'sftp> put /home/zuul/.ansible/tmp/ansible-local-13134L07LR2/tmpF4mwj5 /home/kolla/.ansible/tmp/ansible-tmp-1559570225.06-90034726882536/AnsiballZ_kolla_docker.py\n', '')
<23.253.160.154> ESTABLISH SSH CONNECTION FOR USER: kolla
<23.253.160.154> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/home/zuul/.ssh/id_rsa_kolla"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="kolla"' -o ConnectTimeout=10 -o ControlPath=/home/zuul/.ansible/cp/25772d2a5f 23.253.160.154 '/bin/sh -c '"'"'chmod u+x /home/kolla/.ansible/tmp/ansible-tmp-1559570225.06-90034726882536/ /home/kolla/.ansible/tmp/ansible-tmp-1559570225.06-90034726882536/AnsiballZ_kolla_docker.py && sleep 0'"'"''
<23.253.160.154> (0, '', '')
<23.253.160.154> ESTABLISH SSH CONNECTION FOR USER: kolla
<23.253.160.154> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/home/zuul/.ssh/id_rsa_kolla"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="kolla"' -o ConnectTimeout=10 -o ControlPath=/home/zuul/.ansible/cp/25772d2a5f -tt 23.253.160.154 '/bin/sh -c '"'"'sudo -H -S -n -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-qmlqxafdugapubjysnotmbhypqewahxy ; /usr/bin/python /home/kolla/.ansible/tmp/ansible-tmp-1559570225.06-90034726882536/AnsiballZ_kolla_docker.py'"'"'"'"'"'"'"'"' && sleep 0'"'"''
<23.253.160.154> (1, 'BECOME-SUCCESS-qmlqxafdugapubjysnotmbhypqewahxy\r\n\r\n{"stdout": "", "changed": true, "failed": true, "stderr": "+ sudo -E kolla_set_configs\\nINFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json\\nINFO:__main__:Validating config file\\nINFO:__main__:Kolla config strategy set to: COPY_ALWAYS\\nINFO:__main__:Copying service configuration files\\nINFO:__main__:Deleting /etc/ironic-inspector/inspector.conf\\nINFO:__main__:Copying /var/lib/kolla/config_files/inspector.conf to /etc/ironic-inspector/inspector.conf\\nINFO:__main__:Setting permission for /etc/ironic-inspector/inspector.conf\\nINFO:__main__:Writing out command to execute\\n++ cat /run_command\\n+ CMD=\'ironic-inspector --config-file /etc/ironic-inspector/inspector.conf\'\\n+ ARGS=\\n+ [[ ! -n \'\' ]]\\n+ . kolla_extend_start\\n++ LOG_PATH=/var/log/kolla/ironic-inspector\\n++ [[ ! -d /var/log/kolla/ironic-inspector ]]\\n++ mkdir -p /var/log/kolla/ironic-inspector\\n+++ stat -c %a /var/log/kolla/ironic-inspector\\n++ [[ 2755 != \\\\7\\\\5\\\\5 ]]\\n++ chmod 755 /var/log/kolla/ironic-inspector\\n++ [[ -n 0 ]]\\n++ ironic-inspector-dbsync --config-file /etc/ironic-inspector/inspector.conf upgrade\\nTraceback (most recent call last):\\n File \\"/usr/bin/ironic-inspector-dbsync\\", line 10, in <module>\\n sys.exit(main())\\n File \\"/usr/lib/python3/dist-packages/ironic_inspector/dbsync.py\\", line 92, in main\\n CONF.command.func(config, CONF.command.name)\\n File \\"/usr/lib/python3/dist-packages/ironic_inspector/dbsync.py\\", line 74, in with_revision\\n do_alembic_command(config, cmd, revision)\\n File \\"/usr/lib/python3/dist-packages/ironic_inspector/dbsync.py\\", line 79, in do_alembic_command\\n getattr(alembic_command, cmd)(config, *args, **kwargs)\\n File \\"/usr/lib/python3/dist-packages/alembic/command.py\\", line 254, in upgrade\\n script.run_env()\\n File \\"/usr/lib/python3/dist-packages/alembic/script/base.py\\", line 427, in run_env\\n util.load_python_file(self.dir, \'env.py\')\\n File \\"/usr/lib/python3/dist-packages/alembic/util/pyfiles.py\\", line 81, in load_python_file\\n module = load_module_py(module_id, path)\\n File \\"/usr/lib/python3/dist-packages/alembic/util/compat.py\\", line 82, in load_module_py\\n spec.loader.exec_module(module)\\n File \\"<frozen importlib._bootstrap_external>\\", line 678, in exec_module\\n File \\"<frozen importlib._bootstrap>\\", line 219, in _call_with_frames_removed\\n File \\"/usr/lib/python3/dist-packages/ironic_inspector/migrations/env.py\\", line 83, in <module>\\n run_migrations_online()\\n File \\"/usr/lib/python3/dist-packages/ironic_inspector/migrations/env.py\\", line 69, in run_migrations_online\\n session = db.get_writer_session()\\n File \\"/usr/lib/python3/dist-packages/ironic_inspector/db.py\\", line 206, in get_writer_session\\n return get_context_manager().writer.get_sessionmaker()()\\n File \\"/usr/lib/python3/dist-packages/oslo_db/sqlalchemy/enginefacade.py\\", line 848, in get_sessionmaker\\n return self._factory.get_writer_maker()\\n File \\"/usr/lib/python3/dist-packages/oslo_db/sqlalchemy/enginefacade.py\\", line 392, in get_writer_maker\\n self._start()\\n File \\"/usr/lib/python3/dist-packages/oslo_db/sqlalchemy/enginefacade.py\\", line 510, in _start\\n engine_args, maker_args)\\n File \\"/usr/lib/python3/dist-packages/oslo_db/sqlalchemy/enginefacade.py\\", line 534, in _setup_for_connection\\n sql_connection=sql_connection, **engine_kwargs)\\n File \\"/usr/lib/python3/dist-packages/debtcollector/renames.py\\", line 43, in decorator\\n return wrapped(*args, **kwargs)\\n File \\"/usr/lib/python3/dist-packages/oslo_db/sqlalchemy/engines.py\\", line 177, in create_engine\\n engine = sqlalchemy.create_engine(url, **engine_args)\\n File \\"/usr/lib/python3/dist-packages/sqlalchemy/engine/__init__.py\\", line 425, in create_engine\\n return strategy.create(*args, **kwargs)\\n File \\"/usr/lib/python3/dist-packages/sqlalchemy/engine/strategies.py\\", line 81, in create\\n dbapi = dialect_cls.dbapi(**dbapi_args)\\n File \\"/usr/lib/python3/dist-packages/sqlalchemy/dialects/mysql/pymysql.py\\", line 62, in dbapi\\n return __import__(\'pymysql\')\\nModuleNotFoundError: No module named \'pymysql\'\\n", "rc": 1, "invocation": {"module_args": {"tty": false, "tls_key": null, "image": "kolla/ubuntu-binary-ironic-inspector:master", "labels": {"BOOTSTRAP": null}, "tls_verify": false, "tls_cacert": null, "auth_password": null, "graceful_timeout": 10, "environment": {"KOLLA_SERVICE_NAME": "bootstrap-ironic-inspector", "KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS", "KOLLA_BOOTSTRAP": null}, "auth_registry": null, "state": "running", "volumes_from": null, "security_opt": [], "privileged": false, "api_version": "auto", "remove_on_exit": true, "restart_retries": 10, "detach": false, "auth_username": null, "dimensions": {}, "name": "bootstrap_ironic_inspector", "tls_cert": null, "cap_add": [], "restart_policy": "never", "auth_email": null, "command": null, "volumes": ["/etc/kolla//ironic-inspector/:/var/lib/kolla/config_files/:ro", "/etc/localtime:/etc/localtime:ro", "kolla_logs:/var/log/kolla", "ironic_inspector_dhcp_hosts:/var/lib/ironic-inspector/dhcp-hostsdir"], "action": "start_container"}}, "msg": "Container exited with non-zero return code 1"}\r\n', 'Shared connection to 23.253.160.154 closed.\r\n')
<23.253.160.154> Failed to connect to the host via ssh: Shared connection to 23.253.160.154 closed.
<23.253.160.154> ESTABLISH SSH CONNECTION FOR USER: kolla
<23.253.160.154> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o 'IdentityFile="/home/zuul/.ssh/id_rsa_kolla"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="kolla"' -o ConnectTimeout=10 -o ControlPath=/home/zuul/.ansible/cp/25772d2a5f 23.253.160.154 '/bin/sh -c '"'"'rm -f -r /home/kolla/.ansible/tmp/ansible-tmp-1559570225.06-90034726882536/ > /dev/null 2>&1 && sleep 0'"'"''
<23.253.160.154> (0, '', '')
fatal: [primary]: FAILED! => {
    "changed": true,
    "invocation": {
        "module_args": {
            "action": "start_container",
            "api_version": "auto",
            "auth_email": null,
            "auth_password": null,
            "auth_registry": null,
            "auth_username": null,
            "cap_add": [],
            "command": null,
            "detach": false,
            "dimensions": {},
            "environment": {
                "KOLLA_BOOTSTRAP": null,
                "KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS",
                "KOLLA_SERVICE_NAME": "bootstrap-ironic-inspector"
            },
            "graceful_timeout": 10,
            "image": "kolla/ubuntu-binary-ironic-inspector:master",
            "labels": {
                "BOOTSTRAP": null
            },
            "name": "bootstrap_ironic_inspector",
            "privileged": false,
            "remove_on_exit": true,
            "restart_policy": "never",
            "restart_retries": 10,
            "security_opt": [],
            "state": "running",
            "tls_cacert": null,
            "tls_cert": null,
            "tls_key": null,
            "tls_verify": false,
            "tty": false,
            "volumes": [
                "/etc/kolla//ironic-inspector/:/var/lib/kolla/config_files/:ro",
                "/etc/localtime:/etc/localtime:ro",
                "kolla_logs:/var/log/kolla",
                "ironic_inspector_dhcp_hosts:/var/lib/ironic-inspector/dhcp-hostsdir"
            ],
            "volumes_from": null
        }
    },
    "msg": "Container exited with non-zero return code 1",
    "rc": 1,
    "stderr": "+ sudo -E kolla_set_configs\nINFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json\nINFO:__main__:Validating config file\nINFO:__main__:Kolla config strategy set to: COPY_ALWAYS\nINFO:__main__:Copying service configuration files\nINFO:__main__:Deleting /etc/ironic-inspector/inspector.conf\nINFO:__main__:Copying /var/lib/kolla/config_files/inspector.conf to /etc/ironic-inspector/inspector.conf\nINFO:__main__:Setting permission for /etc/ironic-inspector/inspector.conf\nINFO:__main__:Writing out command to execute\n++ cat /run_command\n+ CMD='ironic-inspector --config-file /etc/ironic-inspector/inspector.conf'\n+ ARGS=\n+ [[ ! -n '' ]]\n+ . kolla_extend_start\n++ LOG_PATH=/var/log/kolla/ironic-inspector\n++ [[ ! -d /var/log/kolla/ironic-inspector ]]\n++ mkdir -p /var/log/kolla/ironic-inspector\n+++ stat -c %a /var/log/kolla/ironic-inspector\n++ [[ 2755 != \\7\\5\\5 ]]\n++ chmod 755 /var/log/kolla/ironic-inspector\n++ [[ -n 0 ]]\n++ ironic-inspector-dbsync --config-file /etc/ironic-inspector/inspector.conf upgrade\nTraceback (most recent call last):\n File \"/usr/bin/ironic-inspector-dbsync\", line 10, in <module>\n sys.exit(main())\n File \"/usr/lib/python3/dist-packages/ironic_inspector/dbsync.py\", line 92, in main\n CONF.command.func(config, CONF.command.name)\n File \"/usr/lib/python3/dist-packages/ironic_inspector/dbsync.py\", line 74, in with_revision\n do_alembic_command(config, cmd, revision)\n File \"/usr/lib/python3/dist-packages/ironic_inspector/dbsync.py\", line 79, in do_alembic_command\n getattr(alembic_command, cmd)(config, *args, **kwargs)\n File \"/usr/lib/python3/dist-packages/alembic/command.py\", line 254, in upgrade\n script.run_env()\n File \"/usr/lib/python3/dist-packages/alembic/script/base.py\", line 427, in run_env\n util.load_python_file(self.dir, 'env.py')\n File \"/usr/lib/python3/dist-packages/alembic/util/pyfiles.py\", line 81, in load_python_file\n module = load_module_py(module_id, path)\n File \"/usr/lib/python3/dist-packages/alembic/util/compat.py\", line 82, in load_module_py\n spec.loader.exec_module(module)\n File \"<frozen importlib._bootstrap_external>\", line 678, in exec_module\n File \"<frozen importlib._bootstrap>\", line 219, in _call_with_frames_removed\n File \"/usr/lib/python3/dist-packages/ironic_inspector/migrations/env.py\", line 83, in <module>\n run_migrations_online()\n File \"/usr/lib/python3/dist-packages/ironic_inspector/migrations/env.py\", line 69, in run_migrations_online\n session = db.get_writer_session()\n File \"/usr/lib/python3/dist-packages/ironic_inspector/db.py\", line 206, in get_writer_session\n return get_context_manager().writer.get_sessionmaker()()\n File \"/usr/lib/python3/dist-packages/oslo_db/sqlalchemy/enginefacade.py\", line 848, in get_sessionmaker\n return self._factory.get_writer_maker()\n File \"/usr/lib/python3/dist-packages/oslo_db/sqlalchemy/enginefacade.py\", line 392, in get_writer_maker\n self._start()\n File \"/usr/lib/python3/dist-packages/oslo_db/sqlalchemy/enginefacade.py\", line 510, in _start\n engine_args, maker_args)\n File \"/usr/lib/python3/dist-packages/oslo_db/sqlalchemy/enginefacade.py\", line 534, in _setup_for_connection\n sql_connection=sql_connection, **engine_kwargs)\n File \"/usr/lib/python3/dist-packages/debtcollector/renames.py\", line 43, in decorator\n return wrapped(*args, **kwargs)\n File \"/usr/lib/python3/dist-packages/oslo_db/sqlalchemy/engines.py\", line 177, in create_engine\n engine = sqlalchemy.create_engine(url, **engine_args)\n File \"/usr/lib/python3/dist-packages/sqlalchemy/engine/__init__.py\", line 425, in create_engine\n return strategy.create(*args, **kwargs)\n File \"/usr/lib/python3/dist-packages/sqlalchemy/engine/strategies.py\", line 81, in create\n dbapi = dialect_cls.dbapi(**dbapi_args)\n File \"/usr/lib/python3/dist-packages/sqlalchemy/dialects/mysql/pymysql.py\", line 62, in dbapi\n return __import__('pymysql')\nModuleNotFoundError: No module named 'pymysql'\n",
    "stderr_lines": [
        "+ sudo -E kolla_set_configs",
        "INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json",
        "INFO:__main__:Validating config file",
        "INFO:__main__:Kolla config strategy set to: COPY_ALWAYS",
        "INFO:__main__:Copying service configuration files",
        "INFO:__main__:Deleting /etc/ironic-inspector/inspector.conf",
        "INFO:__main__:Copying /var/lib/kolla/config_files/inspector.conf to /etc/ironic-inspector/inspector.conf",
        "INFO:__main__:Setting permission for /etc/ironic-inspector/inspector.conf",
        "INFO:__main__:Writing out command to execute",
        "++ cat /run_command",
        "+ CMD='ironic-inspector --config-file /etc/ironic-inspector/inspector.conf'",
        "+ ARGS=",
        "+ [[ ! -n '' ]]",
        "+ . kolla_extend_start",
        "++ LOG_PATH=/var/log/kolla/ironic-inspector",
        "++ [[ ! -d /var/log/kolla/ironic-inspector ]]",
        "++ mkdir -p /var/log/kolla/ironic-inspector",
        "+++ stat -c %a /var/log/kolla/ironic-inspector",
        "++ [[ 2755 != \\7\\5\\5 ]]",
        "++ chmod 755 /var/log/kolla/ironic-inspector",
        "++ [[ -n 0 ]]",
        "++ ironic-inspector-dbsync --config-file /etc/ironic-inspector/inspector.conf upgrade",
        "Traceback (most recent call last):",
        " File \"/usr/bin/ironic-inspector-dbsync\", line 10, in <module>",
        " sys.exit(main())",
        " File \"/usr/lib/python3/dist-packages/ironic_inspector/dbsync.py\", line 92, in main",
        " CONF.command.func(config, CONF.command.name)",
        " File \"/usr/lib/python3/dist-packages/ironic_inspector/dbsync.py\", line 74, in with_revision",
        " do_alembic_command(config, cmd, revision)",
        " File \"/usr/lib/python3/dist-packages/ironic_inspector/dbsync.py\", line 79, in do_alembic_command",
        " getattr(alembic_command, cmd)(config, *args, **kwargs)",
        " File \"/usr/lib/python3/dist-packages/alembic/command.py\", line 254, in upgrade",
        " script.run_env()",
        " File \"/usr/lib/python3/dist-packages/alembic/script/base.py\", line 427, in run_env",
        " util.load_python_file(self.dir, 'env.py')",
        " File \"/usr/lib/python3/dist-packages/alembic/util/pyfiles.py\", line 81, in load_python_file",
        " module = load_module_py(module_id, path)",
        " File \"/usr/lib/python3/dist-packages/alembic/util/compat.py\", line 82, in load_module_py",
        " spec.loader.exec_module(module)",
        " File \"<frozen importlib._bootstrap_external>\", line 678, in exec_module",
        " File \"<frozen importlib._bootstrap>\", line 219, in _call_with_frames_removed",
        " File \"/usr/lib/python3/dist-packages/ironic_inspector/migrations/env.py\", line 83, in <module>",
        " run_migrations_online()",
        " File \"/usr/lib/python3/dist-packages/ironic_inspector/migrations/env.py\", line 69, in run_migrations_online",
        " session = db.get_writer_session()",
        " File \"/usr/lib/python3/dist-packages/ironic_inspector/db.py\", line 206, in get_writer_session",
        " return get_context_manager().writer.get_sessionmaker()()",
        " File \"/usr/lib/python3/dist-packages/oslo_db/sqlalchemy/enginefacade.py\", line 848, in get_sessionmaker",
        " return self._factory.get_writer_maker()",
        " File \"/usr/lib/python3/dist-packages/oslo_db/sqlalchemy/enginefacade.py\", line 392, in get_writer_maker",
        " self._start()",
        " File \"/usr/lib/python3/dist-packages/oslo_db/sqlalchemy/enginefacade.py\", line 510, in _start",
        " engine_args, maker_args)",
        " File \"/usr/lib/python3/dist-packages/oslo_db/sqlalchemy/enginefacade.py\", line 534, in _setup_for_connection",
        " sql_connection=sql_connection, **engine_kwargs)",
        " File \"/usr/lib/python3/dist-packages/debtcollector/renames.py\", line 43, in decorator",
        " return wrapped(*args, **kwargs)",
        " File \"/usr/lib/python3/dist-packages/oslo_db/sqlalchemy/engines.py\", line 177, in create_engine",
        " engine = sqlalchemy.create_engine(url, **engine_args)",
        " File \"/usr/lib/python3/dist-packages/sqlalchemy/engine/__init__.py\", line 425, in create_engine",
        " return strategy.create(*args, **kwargs)",
        " File \"/usr/lib/python3/dist-packages/sqlalchemy/engine/strategies.py\", line 81, in create",
        " dbapi = dialect_cls.dbapi(**dbapi_args)",
        " File \"/usr/lib/python3/dist-packages/sqlalchemy/dialects/mysql/pymysql.py\", line 62, in dbapi",
        " return __import__('pymysql')",
        "ModuleNotFoundError: No module named 'pymysql'"
    ],
    "stdout": "",
    "stdout_lines": []
}