in tripleo we have a template which is a puppet manifest like [1] (delivered as a SoftwareConfig). Recently this file has grown beyond 65535 chars, resulting in
DEBUG oslo_db.sqlalchemy.session [req-a797cf52-99e4-4a85-8af0-db44167ead0b - -] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/session.py:513\n2015-09-08 21:09:39.788 32609 ERROR oslo_messaging._drivers.impl_rabbit [-] Failed to consume message from queue: \n2015-09-08 21:09:40.778 2133 DEBUG oslo_db.sqlalchemy.session [req-1a71f3cf-993a-45b2-89db-96bf22664aca - -] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/session.py:513\n2015-09-08 22:00:37.468 2133 ERROR oslo_db.sqlalchemy.exc_filters [-] DBAPIError exception wrapped from (_mysql_exceptions.DataError) (1406, \"Data too long for column 'resource_properties' at row 1\") [SQL: u'INSERT INTO event (created_at, updated_at, stack_id, uuid, resource_action, resource_status, resource_name, physical_resource_id, resource_status_reason, resource_type, resource_properties) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)'] [parameters: (datetime.datetime(2015, 9, 9, 2, 0, 37, 466247), None, '8526154b-6ca6-424a-af5a-c91429707932', '17be30fd-e595-4dfa-9f3d-7bf2ada982c5', 'CREATE', 'IN_PROGRESS', 'ControllerPuppetConfigImpl', None, 'state changed', 'OS::Heat::SoftwareConfig', '\\x80\\x02}q\\x01(U\\x06inputsq\\x02NU\\x06configq\\x03X\\x17\\x05\\x01\\x00# Copyright 2015 Red Hat, Inc.\\n# All Rights Reserved.\\n#\\n#
(the rest of the trace is the continuation of [1] in its entirety)
It seems heat is trying to store all of that as the resource_properties of an event like at [3]. Looking a bit closer, this column is type PickleType [4][5]. I am not sure what we can do about this type? Any ideas?
thanks
[1] https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/manifests/overcloud_controller_pacemaker.pp
[2] https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/controller-config-pacemaker.yaml
[3] https://github.com/openstack/heat/blob/e4264917fa2f457c159415e5ea8e29411c5812a7/heat/engine/event.py#L42
[4] https://github.com/openstack/heat/blob/master/heat/db/sqlalchemy/models.py#L242
[5] http://docs.sqlalchemy.org/en/latest/core/type_basics.html?highlight=pickletype#sqlalchemy.types.PickleType
I can't think of a good fix that wouldn't require a DB migration, other than to just retry without including the property values if we encounter this failure.