All OVB job in master periodic pipeline are failing to load ilo driver during openstack overcloud node import instackenv.json

All periodic OVB jobs in master show a log similar to this:

https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/7bcad2a/logs/undercloud/home/zuul/overcloud_prep_images.log.txt.gz#_2019-03-01_01_21_02

2019-03-01 01:21:02 | Exception occured while running the command
2019-03-01 01:21:02 | Traceback (most recent call last):
2019-03-01 01:21:02 | File "/usr/lib/python2.7/site-packages/tripleoclient/command.py", line 29, in run
2019-03-01 01:21:02 | super(Command, self).run(parsed_args)
2019-03-01 01:21:02 | File "/usr/lib/python2.7/site-packages/osc_lib/command/command.py", line 41, in run
2019-03-01 01:21:02 | return super(Command, self).run(parsed_args)
2019-03-01 01:21:02 | File "/usr/lib/python2.7/site-packages/cliff/command.py", line 184, in run
2019-03-01 01:21:02 | return_code = self.take_action(parsed_args) or 0
2019-03-01 01:21:02 | File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_node.py", line 284, in take_action
2019-03-01 01:21:02 | instance_boot_option=parsed_args.instance_boot_option
2019-03-01 01:21:02 | File "/usr/lib/python2.7/site-packages/tripleoclient/workflows/baremetal.py", line 80, in register_or_update
2019-03-01 01:21:02 | 'Exception registering nodes: {}'.format(payload['message']))
2019-03-01 01:21:02 | RegisterOrUpdateError: Exception registering nodes: Resource temporarily unavailable, please retry. (HTTP 503)
2019-03-01 01:21:02 | Exception registering nodes: Resource temporarily unavailable, please retry. (HTTP 503)
2019-03-01 01:21:02 | Resource temporarily unavailable, please retry. (HTTP 503)

Mistral logs at

https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master/d6d092d/logs/undercloud/var/log/containers/mistral/executor.log.txt.gz#_2019-03-01_01_05_46_971

show

2019-03-01 01:05:46.971 8 ERROR tripleo_common.actions.baremetal [req-d5f2acc5-9e93-4f11-94de-a65878c8893c 237656d81d3c45e2853ef02d1cf1ac12 f484b8ae54484d5ebbe6ea7d19f77372 - default default] Error registering nodes with ironic.: ServiceUnavailable: Resource temporarily unavailable, please retry. (HTTP 503)
2019-03-01 01:05:46.971 8 ERROR tripleo_common.actions.baremetal Traceback (most recent call last):
2019-03-01 01:05:46.971 8 ERROR tripleo_common.actions.baremetal File "/usr/lib/python2.7/site-packages/tripleo_common/actions/baremetal.py", line 73, in run
2019-03-01 01:05:46.971 8 ERROR tripleo_common.actions.baremetal ramdisk_name=self.ramdisk_name)
2019-03-01 01:05:46.971 8 ERROR tripleo_common.actions.baremetal File "/usr/lib/python2.7/site-packages/tripleo_common/utils/nodes.py", line 591, in register_all_nodes
2019-03-01 01:05:46.971 8 ERROR tripleo_common.actions.baremetal node = _update_or_register_ironic_node(node, node_map, client=client)
2019-03-01 01:05:46.971 8 ERROR tripleo_common.actions.baremetal File "/usr/lib/python2.7/site-packages/tripleo_common/utils/nodes.py", line 546, in _update_or_register_ironic_node
2019-03-01 01:05:46.971 8 ERROR tripleo_common.actions.baremetal ironic_node = register_ironic_node(node, client)
2019-03-01 01:05:46.971 8 ERROR tripleo_common.actions.baremetal File "/usr/lib/python2.7/site-packages/tripleo_common/utils/nodes.py", line 423, in register_ironic_node
2019-03-01 01:05:46.971 8 ERROR tripleo_common.actions.baremetal ironic_node = client.node.create(**create_map)
2019-03-01 01:05:46.971 8 ERROR tripleo_common.actions.baremetal File "/usr/lib/python2.7/site-packages/ironicclient/common/base.py", line 278, in create
2019-03-01 01:05:46.971 8 ERROR tripleo_common.actions.baremetal resp, body = self.api.json_request('POST', url, body=new)
2019-03-01 01:05:46.971 8 ERROR tripleo_common.actions.baremetal File "/usr/lib/python2.7/site-packages/ironicclient/common/http.py", line 496, in json_request
2019-03-01 01:05:46.971 8 ERROR tripleo_common.actions.baremetal resp, body_iter = self._http_request(url, method, **kwargs)
2019-03-01 01:05:46.971 8 ERROR tripleo_common.actions.baremetal File "/usr/lib/python2.7/site-packages/ironicclient/common/http.py", line 287, in wrapper
2019-03-01 01:05:46.971 8 ERROR tripleo_common.actions.baremetal return func(self, url, method, **kwargs)
2019-03-01 01:05:46.971 8 ERROR tripleo_common.actions.baremetal File "/usr/lib/python2.7/site-packages/ironicclient/common/http.py", line 477, in _http_request
2019-03-01 01:05:46.971 8 ERROR tripleo_common.actions.baremetal error_json.get('debuginfo'), method, url)
2019-03-01 01:05:46.971 8 ERROR tripleo_common.actions.baremetal ServiceUnavailable: Resource temporarily unavailable, please retry. (HTTP 503)
2019-03-01 01:05:46.971 8 ERROR tripleo_common.actions.baremetal

Ironic logs repeatedly show this error

https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset021-master/15e724f/logs/undercloud/var/log/containers/ironic/ironic-conductor.log.txt.gz#_2019-03-01_01_28_03_988

2019-03-01 01:28:03.988 10 ERROR oslo_service.service [req-a798bd9b-57ce-42b5-a0d5-2ef17b35de7a - - - - -] Error starting thread.: DriverLoadError: Driver, hardware type or interface ilo could not be loaded. Reason: 'module' object has no attribute 'noValue'.
2019-03-01 01:28:03.988 10 ERROR oslo_service.service Traceback (most recent call last):
2019-03-01 01:28:03.988 10 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/oslo_service/service.py", line 796, in run_service
2019-03-01 01:28:03.988 10 ERROR oslo_service.service service.start()
2019-03-01 01:28:03.988 10 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/ironic/common/rpc_service.py", line 61, in start
2019-03-01 01:28:03.988 10 ERROR oslo_service.service self.manager.init_host(admin_context)
2019-03-01 01:28:03.988 10 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/ironic/conductor/base_manager.py", line 128, in init_host
2019-03-01 01:28:03.988 10 ERROR oslo_service.service hardware_types = driver_factory.hardware_types()
2019-03-01 01:28:03.988 10 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/ironic/common/driver_factory.py", line 253, in hardware_types
2019-03-01 01:28:03.988 10 ERROR oslo_service.service return _get_all_drivers(HardwareTypesFactory())
2019-03-01 01:28:03.988 10 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/ironic/common/driver_factory.py", line 323, in __init__
2019-03-01 01:28:03.988 10 ERROR oslo_service.service self.__class__._init_extension_manager()
2019-03-01 01:28:03.988 10 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 328, in inner
2019-03-01 01:28:03.988 10 ERROR oslo_service.service return f(*args, **kwargs)
2019-03-01 01:28:03.988 10 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/ironic/common/driver_factory.py", line 393, in _init_extension_manager
2019-03-01 01:28:03.988 10 ERROR oslo_service.service on_missing_entrypoints_callback=missing_callback))
2019-03-01 01:28:03.988 10 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/stevedore/named.py", line 81, in __init__
2019-03-01 01:28:03.988 10 ERROR oslo_service.service verify_requirements)
2019-03-01 01:28:03.988 10 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/stevedore/extension.py", line 203, in _load_plugins
2019-03-01 01:28:03.988 10 ERROR oslo_service.service self._on_load_failure_callback(self, ep, err)
2019-03-01 01:28:03.988 10 ERROR oslo_service.service File "/usr/lib/python2.7/site-packages/ironic/common/driver_factory.py", line 378, in _catch_driver_not_found
2019-03-01 01:28:03.988 10 ERROR oslo_service.service raise exception.DriverLoadError(driver=ep.name, reason=exc)
2019-03-01 01:28:03.988 10 ERROR oslo_service.service DriverLoadError: Driver, hardware type or interface ilo could not be loaded. Reason: 'module' object has no attribute 'noValue'.
2019-03-01 01:28:03.988 10 ERROR oslo_service.service