it was timeout when creating cluster

Having a similar issue in a kolla pike ubuntu/binary deployment:

creating cluster fails:
2017-09-15 09:41:17.990 25 INFO heat.engine.resource [req-2babed61-5ed3-4866-b625-a227846e74ad - default - 55f399ca8a814806b36e5e11c03f9df7 55f399ca8a814806b36e5e11c03f9df7] creating Port "kube_master_eth0" Stack "k8s-atomic-rpksi3fj6zha-kube_masters-iu2i7wwshfyo-1-xcatmwflb6wz" [8354b7e6-392b-4d3a-ac66-24855844e6c2]
2017-09-15 09:41:18.875 26 INFO heat.engine.resource [req-2babed61-5ed3-4866-b625-a227846e74ad - default - 55f399ca8a814806b36e5e11c03f9df7 55f399ca8a814806b36e5e11c03f9df7] creating SoftwareConfig "configure_kubernetes" Stack "k8s-atomic-rpksi3fj6zha-kube_masters-iu2i7wwshfyo-1-xcatmwflb6wz" [8354b7e6-392b-4d3a-ac66-24855844e6c2]
2017-09-15 09:41:19.453 23 ERROR heat.engine.resource [req-819c8ffb-857b-4e63-80d9-9e417a28364c - default - 55f399ca8a814806b36e5e11c03f9df7 55f399ca8a814806b36e5e11c03f9df7] store_attributes rsrc kube_master_eth0 594 DB error type object 'ResourcePropertiesData' has no attribute 'attr_id'
2017-09-15 09:41:19.966 25 INFO heat.engine.check_resource [req-2babed61-5ed3-4866-b625-a227846e74ad - default - 55f399ca8a814806b36e5e11c03f9df7 55f399ca8a814806b36e5e11c03f9df7] Re-trigger resource: (594, True)
2017-09-15 09:41:20.044 26 ERROR heat.engine.resource [req-e5609609-ef5a-4726-af45-dd86ca6e2ad8 - default - 55f399ca8a814806b36e5e11c03f9df7 55f399ca8a814806b36e5e11c03f9df7] store_attributes rsrc kube_master_eth0 594 DB error type object 'ResourcePropertiesData' has no attribute 'attr_id'

Not sure if they are the same, attaching my log here.

Heat stack creation is failing while creating floating ip with exception - DB error type object 'ResourcePropertiesData' has no attribute 'attr_id'.

Rally server log trace,

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/rally/task/runner.py", line 72, in _run_scenario_once
    getattr(scenario_inst, method_name)(**scenario_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/rally/plugins/openstack/scenarios/heat/stacks.py", line 92, in run
    files, environment)
  File "/usr/local/lib/python2.7/dist-packages/rally/task/atomic.py", line 84, in func_atomic_actions
    f = func(self, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/rally/plugins/openstack/scenarios/heat/utils.py", line 143, in _create_stack
    stack_id = self.clients("heat").stacks.create(**kw)["stack"]["id"]
  File "/usr/local/lib/python2.7/dist-packages/heatclient/v1/stacks.py", line 172, in create
    data=kwargs, headers=headers)
  File "/usr/local/lib/python2.7/dist-packages/keystoneauth1/adapter.py", line 223, in post
    return self.request(url, 'POST', **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/heatclient/common/http.py", line 318, in request
    raise exc.from_response(resp)
HTTPInternalServerError: ERROR: Timed out waiting for a reply to message ID d26eba4fc7f246a7a4f57712bd3936f2

h-api log trace,

h-api.log:2017-10-12 14:07:22.303 23229 ERROR heat.common.wsgi [req-a6607dca-371a-4ba0-81ed-f996e03c602e c_rally_58d08eb9_VXEFFhtp c_rally_58d08eb9_qZyHUKPK] Unexpected error occurred serving API: Timed out waiting for a reply to message ID d26eba4fc7f246a7a4f57712bd3936f2: MessagingTimeout: Timed out waiting for a reply to message ID d26eba4fc7f246a7a4f57712bd3936f2

h-api.log:2017-10-12 14:07:38.327 23229 INFO oslo_messaging._drivers.amqpdriver [-] No calling threads waiting for msg_id : d26eba4fc7f246a7a4f57712bd3936f2

h-eng.log trace,

h-eng.log:2017-10-12 14:09:05.831 23533 INFO heat.engine.resource [req-a6607dca-371a-4ba0-81ed-f996e03c602e c_rally_58d08eb9_VXEFFhtp -] creating FloatingIP "floatingip_jump" Stack "s_rally_58d08eb9_9NIlVkSQ" [a1d19fc4-21d7-4408-b3d0-926e4eabe4ef]
h-eng.log:2017-10-12 13:58:08.195 23532 ERROR heat.engine.resource [req-3b8051b2-af38-4941-860e-b25dacd3e691 c_rally_58d08eb9_VXEFFhtp -] store_attributes rsrc floatingip_jump 381 DB error type object 'ResourcePropertiesData' has no attribute 'attr_id'
h-eng.log:2017-10-12 14:11:25.581 23533 ERROR heat.engine.resource [req-6dab48a3-c516-4230-908a-5ac99becf222 c_rally_58d08eb9_VXEFFhtp -] store_attributes rsrc floatingip_jump 511 DB error type object 'ResourcePropertiesData' has no attribute 'attr_id'
h-eng.log:2017-10-12 14:11:29.276 23533 ERROR heat.engine.resource [req-5157bf50-399e-423e-88fc-597e1656c28f c_rally_58d08eb9_VXEFFhtp -] store_attributes rsrc floatingip_jump 498 DB error type object 'ResourcePropertiesData' has no attribute 'attr_id'
h-eng.log:2017-10-12 14:11:47.245 23532 ERROR heat.engine.resource [req-bdba36ca-a963-46fe-a8e3-f8396f9facfb c_rally_58d08eb9_VXEFFhtp -] store_attributes rsrc floatingip_jump 521 DB error type object 'ResourcePropertiesD...

Fix proposed to branch: master
Review: https://review.openstack.org/511752

How is this confirmed in magnum? We don't see any problem's with magnum pike and our CI passes without any problems

This run is from yesterday and it shows just a timeout, nothing broken. On openstack-infra extremely frequently is related to powerless resources (i.e. a m1.large vm running all openstack services plus a magnum cluster).

I changed this bug to incomplete from magnum.

Reviewed: https://review.openstack.org/511752
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=a532535b613777b144b2e159e349783d2a21fd1e
Submitter: Zuul
Branch: master

commit a532535b613777b144b2e159e349783d2a21fd1e
Author: huangtianhua <email address hidden>
Date: Fri Oct 13 15:46:53 2017 +0800

    Correct the filter attribute of db model 'ResourcePropertiesData'

    Make sure to delete the ResourcePropertiesData row
    if the resource 'attr_data_id' has been set already.

    Change-Id: I2df5dfc72d051b6619c769c1260ff24b016bc294
    Closes-Bug: #1708378

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/512907

Reviewed: https://review.openstack.org/512907
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=95ed444fbbc83e26d99ff2fcdfed874a3710cdc7
Submitter: Zuul
Branch: stable/pike

commit 95ed444fbbc83e26d99ff2fcdfed874a3710cdc7
Author: huangtianhua <email address hidden>
Date: Fri Oct 13 15:46:53 2017 +0800

    Correct the filter attribute of db model 'ResourcePropertiesData'

    Make sure to delete the ResourcePropertiesData row
    if the resource 'attr_data_id' has been set already.

    Change-Id: I2df5dfc72d051b6619c769c1260ff24b016bc294
    Closes-Bug: #1708378
    (cherry picked from commit a532535b613777b144b2e159e349783d2a21fd1e)

This issue was fixed in the openstack/heat 10.0.0.0b1 development milestone.

This issue was fixed in the openstack/heat 9.0.1 release.