We've seen random failures in the bifrost container builds around timeouts collecting facts. I believe this is related to https://github.com/ansible/ansible/issues/43884 which we've seen in TripleO and increased our gather_facts timeout to 30 seconds. https://review.openstack.org/#/q/Ia4aeac06d4c0e237180e4ba60063828b0d1c5350
http://logs.openstack.org/98/631098/2/check/kolla-build-centos-source/983159a/logs/build/000_FAILED_bifrost-base.txt.gz
INFO:kolla.common.utils.bifrost-base:PLAYBOOK: install.yaml *********************************************************
INFO:kolla.common.utils.bifrost-base:1 plays in /bifrost/playbooks/install.yaml
INFO:kolla.common.utils.bifrost-base:PLAY [Install Ironic on the target host.] **************************************
INFO:kolla.common.utils.bifrost-base:TASK [Gathering Facts] *********************************************************
INFO:kolla.common.utils.bifrost-base:task path: /bifrost/playbooks/install.yaml:2
INFO:kolla.common.utils.bifrost-base:<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: root
INFO:kolla.common.utils.bifrost-base:<127.0.0.1> EXEC /bin/sh -c 'echo ~root && sleep 0'
INFO:kolla.common.utils.bifrost-base:<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp/ansible-tmp-1547752997.83-171480993999023 `" && echo ansible-tmp-1547752997.83-171480993999023="` echo /root/.ansible/tmp/ansible-tmp-1547752997.83-171480993999023 `" ) && sleep 0'
INFO:kolla.common.utils.bifrost-base:Using module file /var/lib/kolla/venv/lib/python2.7/site-packages/ansible/modules/system/setup.py
INFO:kolla.common.utils.bifrost-base:<127.0.0.1> PUT /root/.ansible/tmp/ansible-local-795G3egY3/tmpArhUyi TO /root/.ansible/tmp/ansible-tmp-1547752997.83-171480993999023/setup.py
INFO:kolla.common.utils.bifrost-base:<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1547752997.83-171480993999023/ /root/.ansible/tmp/ansible-tmp-1547752997.83-171480993999023/setup.py && sleep 0'
INFO:kolla.common.utils.bifrost-base:<127.0.0.1> EXEC /bin/sh -c 'no_proxy='"'"''"'"' https_proxy='"'"''"'"' http_proxy='"'"''"'"' /var/lib/kolla/venv/bin/python /root/.ansible/tmp/ansible-tmp-1547752997.83-171480993999023/setup.py && sleep 0'
INFO:kolla.common.utils.bifrost-base:<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /root/.ansible/tmp/ansible-tmp-1547752997.83-171480993999023/ > /dev/null 2>&1 && sleep 0'
INFO:kolla.common.utils.bifrost-base:The full traceback is:
INFO:kolla.common.utils.bifrost-base:Traceback (most recent call last):
INFO:kolla.common.utils.bifrost-base: File "/tmp/ansible_3XDUCc/ansible_modlib.zip/ansible/module_utils/basic.py", line 2861, in run_command
INFO:kolla.common.utils.bifrost-base: cmd = subprocess.Popen(args, **kwargs)
INFO:kolla.common.utils.bifrost-base: File "/usr/lib64/python2.7/subprocess.py", line 711, in __init__
INFO:kolla.common.utils.bifrost-base: errread, errwrite)
INFO:kolla.common.utils.bifrost-base: File "/usr/lib64/python2.7/subprocess.py", line 1308, in _execute_child
INFO:kolla.common.utils.bifrost-base: data = _eintr_retry_call(os.read, errpipe_read, 1048576)
INFO:kolla.common.utils.bifrost-base: File "/usr/lib64/python2.7/subprocess.py", line 478, in _eintr_retry_call
INFO:kolla.common.utils.bifrost-base: return func(*args)
INFO:kolla.common.utils.bifrost-base: File "/tmp/ansible_3XDUCc/ansible_modlib.zip/ansible/module_utils/facts/timeout.py", line 37, in _handle_timeout
INFO:kolla.common.utils.bifrost-base: raise TimeoutError(msg)
INFO:kolla.common.utils.bifrost-base:TimeoutError: Timer expired after 10 seconds
INFO:kolla.common.utils.bifrost-base:fatal: [127.0.0.1]: FAILED! => {
INFO:kolla.common.utils.bifrost-base: "changed": false,
INFO:kolla.common.utils.bifrost-base: "cmd": "/usr/sbin/udevadm info --query property --name /dev/loop0",
INFO:kolla.common.utils.bifrost-base: "invocation": {
INFO:kolla.common.utils.bifrost-base: "module_args": {
INFO:kolla.common.utils.bifrost-base: "fact_path": "/etc/ansible/facts.d",
INFO:kolla.common.utils.bifrost-base: "filter": "*",
INFO:kolla.common.utils.bifrost-base: "gather_subset": [
INFO:kolla.common.utils.bifrost-base: "all"
INFO:kolla.common.utils.bifrost-base: ],
INFO:kolla.common.utils.bifrost-base: "gather_timeout": 10
INFO:kolla.common.utils.bifrost-base: }
INFO:kolla.common.utils.bifrost-base: },
INFO:kolla.common.utils.bifrost-base: "msg": "Timer expired after 10 seconds",
INFO:kolla.common.utils.bifrost-base: "rc": 257
INFO:kolla.common.utils.bifrost-base:}
INFO:kolla.common.utils.bifrost-base: to retry, use: --limit @/bifrost/playbooks/install.retry
INFO:kolla.common.utils.bifrost-base:PLAY RECAP *********************************************************************
INFO:kolla.common.utils.bifrost-base:127.0.0.1 : ok=0 changed=0 unreachable=0 failed=1
https:/