Ceilometer

stable/havana ceilometer fails for latest alembic drop old deprecated paremeter "type" support

Bug #1300105 reported by Irena Berezovsky
30
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Ceilometer
Invalid
Critical
Unassigned
Havana
Fix Released
Critical
gordon chung
Ceilometer 2013.2.3
devstack
Invalid
Undecided
Unassigned

Bug Description

The log of the failure is:
2014-03-31 06:52:08 + /usr/local/bin/ceilometer-dbsync
2014-03-31 06:52:08 2014-03-31 06:52:08.343 7619 INFO migrate.versioning.api [-] 0 -> 1...
2014-03-31 06:52:08 2014-03-31 06:52:08.609 7619 INFO migrate.versioning.api [-] done
2014-03-31 06:52:08 2014-03-31 06:52:08.609 7619 INFO migrate.versioning.api [-] 1 -> 2...
2014-03-31 06:52:08 2014-03-31 06:52:08.625 7619 INFO migrate.versioning.api [-] done
2014-03-31 06:52:08 2014-03-31 06:52:08.625 7619 INFO migrate.versioning.api [-] 2 -> 3...
2014-03-31 06:52:08 2014-03-31 06:52:08.661 7619 INFO migrate.versioning.api [-] done
2014-03-31 06:52:08 2014-03-31 06:52:08.662 7619 INFO migrate.versioning.api [-] 3 -> 4...
2014-03-31 06:52:08 2014-03-31 06:52:08.678 7619 INFO migrate.versioning.api [-] done
2014-03-31 06:52:08 2014-03-31 06:52:08.679 7619 INFO migrate.versioning.api [-] 4 -> 5...
2014-03-31 06:52:08 2014-03-31 06:52:08.696 7619 INFO migrate.versioning.api [-] done
2014-03-31 06:52:08 2014-03-31 06:52:08.697 7619 INFO migrate.versioning.api [-] 5 -> 6...
2014-03-31 06:52:08 2014-03-31 06:52:08.710 7619 INFO migrate.versioning.api [-] done
2014-03-31 06:52:08 2014-03-31 06:52:08.710 7619 INFO migrate.versioning.api [-] 6 -> 7...
2014-03-31 06:52:08 2014-03-31 06:52:08.783 7619 INFO migrate.versioning.api [-] done
2014-03-31 06:52:08 2014-03-31 06:52:08.783 7619 INFO migrate.versioning.api [-] 7 -> 8...
2014-03-31 06:52:08 2014-03-31 06:52:08.899 7619 INFO migrate.versioning.api [-] done
2014-03-31 06:52:08 2014-03-31 06:52:08.899 7619 INFO migrate.versioning.api [-] 8 -> 9...
2014-03-31 06:52:08 2014-03-31 06:52:08.921 7619 INFO migrate.versioning.api [-] done
2014-03-31 06:52:08 2014-03-31 06:52:08.921 7619 INFO migrate.versioning.api [-] 9 -> 10...
2014-03-31 06:52:08 2014-03-31 06:52:08.938 7619 INFO migrate.versioning.api [-] done
2014-03-31 06:52:08 2014-03-31 06:52:08.938 7619 INFO migrate.versioning.api [-] 10 -> 11...
2014-03-31 06:52:08 2014-03-31 06:52:08.977 7619 INFO migrate.versioning.api [-] done
2014-03-31 06:52:08 2014-03-31 06:52:08.977 7619 INFO migrate.versioning.api [-] 11 -> 12...
2014-03-31 06:52:09 2014-03-31 06:52:09.075 7619 INFO migrate.versioning.api [-] done
2014-03-31 06:52:09 2014-03-31 06:52:09.076 7619 INFO migrate.versioning.api [-] 12 -> 13...
2014-03-31 06:52:09 2014-03-31 06:52:09.092 7619 INFO migrate.versioning.api [-] done
2014-03-31 06:52:09 2014-03-31 06:52:09.093 7619 INFO migrate.versioning.api [-] 13 -> 14...
2014-03-31 06:52:09 2014-03-31 06:52:09.129 7619 INFO migrate.versioning.api [-] done
2014-03-31 06:52:09 2014-03-31 06:52:09.129 7619 INFO migrate.versioning.api [-] 14 -> 15...
2014-03-31 06:52:09 2014-03-31 06:52:09.224 7619 INFO migrate.versioning.api [-] done
2014-03-31 06:52:09 2014-03-31 06:52:09.224 7619 INFO migrate.versioning.api [-] 15 -> 16...
2014-03-31 06:52:09 2014-03-31 06:52:09.282 7619 INFO migrate.versioning.api [-] done
2014-03-31 06:52:09 2014-03-31 06:52:09.282 7619 INFO migrate.versioning.api [-] 16 -> 17...
2014-03-31 06:52:09 2014-03-31 06:52:09.320 7619 INFO migrate.versioning.api [-] done
2014-03-31 06:52:09 2014-03-31 06:52:09.321 7619 INFO migrate.versioning.api [-] 17 -> 18...
2014-03-31 06:52:09 2014-03-31 06:52:09.334 7619 INFO migrate.versioning.api [-] done
2014-03-31 06:52:09 2014-03-31 06:52:09.334 7619 INFO migrate.versioning.api [-] 18 -> 19...
2014-03-31 06:52:09 2014-03-31 06:52:09.348 7619 INFO migrate.versioning.api [-] done
2014-03-31 06:52:09 /usr/local/lib/python2.7/dist-packages/alembic/util.py:326: UserWarning: Argument 'type' is now named 'type_' for function 'drop_constraint'
2014-03-31 06:52:09 (oldname, newname, fn.__name__))
2014-03-31 06:52:09 CRITI [ceilometer] __init__() got an unexpected keyword argument 'match'
2014-03-31 06:52:09 Traceback (most recent call last):
2014-03-31 06:52:09 File "/usr/local/bin/ceilometer-dbsync", line 10, in <module>
2014-03-31 06:52:09 sys.exit(dbsync())
2014-03-31 06:52:09 File "/opt/stack/new/ceilometer/ceilometer/storage/__init__.py", line 139, in dbsync
2014-03-31 06:52:09 get_connection(cfg.CONF).upgrade()
2014-03-31 06:52:09 File "/opt/stack/new/ceilometer/ceilometer/storage/impl_sqlalchemy.py", line 153, in upgrade
2014-03-31 06:52:09 migration.db_sync(session.get_bind())
2014-03-31 06:52:09 File "/opt/stack/new/ceilometer/ceilometer/storage/sqlalchemy/migration.py", line 69, in db_sync
2014-03-31 06:52:09 alembic.command.upgrade(_alembic_config(), "head")
2014-03-31 06:52:09 File "/usr/local/lib/python2.7/dist-packages/alembic/command.py", line 125, in upgrade
2014-03-31 06:52:09 script.run_env()
2014-03-31 06:52:09 File "/usr/local/lib/python2.7/dist-packages/alembic/script.py", line 203, in run_env
2014-03-31 06:52:09 util.load_python_file(self.dir, 'env.py')
2014-03-31 06:52:09 File "/usr/local/lib/python2.7/dist-packages/alembic/util.py", line 212, in load_python_file
2014-03-31 06:52:09 module = load_module_py(module_id, path)
2014-03-31 06:52:09 File "/usr/local/lib/python2.7/dist-packages/alembic/compat.py", line 58, in load_module_py
2014-03-31 06:52:09 mod = imp.load_source(module_id, path, fp)
2014-03-31 06:52:09 File "/opt/stack/new/ceilometer/ceilometer/storage/sqlalchemy/alembic/env.py", line 77, in <module>
2014-03-31 06:52:09 run_migrations_online()
2014-03-31 06:52:09 File "/opt/stack/new/ceilometer/ceilometer/storage/sqlalchemy/alembic/env.py", line 71, in run_migrations_online
2014-03-31 06:52:09 context.run_migrations()
2014-03-31 06:52:09 File "<string>", line 7, in run_migrations
2014-03-31 06:52:09 File "/usr/local/lib/python2.7/dist-packages/alembic/environment.py", line 681, in run_migrations
2014-03-31 06:52:09 self.get_context().run_migrations(**kw)
2014-03-31 06:52:09 File "/usr/local/lib/python2.7/dist-packages/alembic/migration.py", line 225, in run_migrations
2014-03-31 06:52:09 change(**kw)
2014-03-31 06:52:09 File "/opt/stack/new/ceilometer/ceilometer/storage/sqlalchemy/alembic/versions/2c3ccda5a3ad_fix_uniq_name.py", line 65, in upgrade
2014-03-31 06:52:09 change_uniq(TABLE_NAME, UNIQ_NAME, COLUMNS)
2014-03-31 06:52:09 File "/opt/stack/new/ceilometer/ceilometer/storage/sqlalchemy/alembic/versions/2c3ccda5a3ad_fix_uniq_name.py", line 59, in change_uniq
2014-03-31 06:52:09 ['meter_id'], ['id'])
2014-03-31 06:52:09 File "<string>", line 7, in create_foreign_key
2014-03-31 06:52:09 File "/usr/local/lib/python2.7/dist-packages/alembic/operations.py", line 577, in create_foreign_key
2014-03-31 06:52:09 initially=initially, match=match, **dialect_kw)
2014-03-31 06:52:09 File "/usr/local/lib/python2.7/dist-packages/alembic/operations.py", line 102, in _foreign_key_constraint
2014-03-31 06:52:09 **dialect_kw
2014-03-31 06:52:09 TypeError: __init__() got an unexpected keyword argument 'match'
2014-03-31 06:52:09 ++ failed
the link to the failed patch log:
http://logs.openstack.org/18/73918/4/check/check-tempest-dsvm-full/0e635c5/logs/devstacklog.txt.gz

Revision history for this message
Accela Zhao (accelazh) wrote :

Ceilometer's requirements.txt specifies alembic>=0.4.1. However the latest release alembic 0.6.4 doesn't work with it. Install alembic 0.6.3 will fix it. (alembic 0.6.4 comes out just 3 days ago.)

Revision history for this message
gordon chung (chungg) wrote :

this seems like a bug in alembic as it doesn't mention anything regarding dropping support in changelogs...

the alternative is to bump requirements in havana to at least alembic>=0.5.0 as that is when type_ attribute is first introduced.

Revision history for this message
gordon chung (chungg) wrote :

ok, i took a closer look at this. the type_ attribute is not throwing the error (just a warning.)

seems like new alembic is dependent on sqlalchemy>=0.9.2, it passes in **dialect_kwargs to schema:
https://bitbucket.org/zzzeek/alembic/src/e5a066c7caf9256e006121d87e859b8ca51f7769/alembic/operations.py?at=master#cl-93

but it's not in sqlalchemy until 0.9.2 (according to docstring):
https://github.com/zzzeek/sqlalchemy/blob/c464960fddf0c3d2db3acff9f772b057e32bcf67/lib/sqlalchemy/sql/schema.py#L1375

adding requirements as affected project.

Revision history for this message
gordon chung (chungg) wrote :

patch to requirements is here: https://review.openstack.org/#/c/84225/

Revision history for this message
Irena Berezovsky (irenab) wrote :

Same issue was reported by the following bug in devstack:
https://bugs.launchpad.net/devstack/+bug/1300541

no longer affects: devstack
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to ceilometer (stable/havana)

Fix proposed to branch: stable/havana
Review: https://review.openstack.org/84433

gordon chung (chungg)
Changed in ceilometer:
status: New → In Progress
importance: Undecided → Critical
gordon chung (chungg)
Changed in ceilometer:
assignee: nobody → gordon chung (chungg)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to ceilometer (stable/havana)

Reviewed: https://review.openstack.org/84433
Committed: https://git.openstack.org/cgit/openstack/ceilometer/commit/?id=c92c1065abac937e9b44e6ee5defb786ab89050c
Submitter: Jenkins
Branch: stable/havana

commit c92c1065abac937e9b44e6ee5defb786ab89050c
Author: Gordon Chung <email address hidden>
Date: Tue Apr 1 09:36:48 2014 -0400

    latest alembic imcompatible with current sqlalchemy requirements

    ceilometer migrations are failing in stable/havana build because
    latest alembic release utilises code added in sqlalchemy>=0.9.2.
    set upper bound for alembic to avoid issue.

    Change-Id: Icc9e647d0c2ff418ead544fe671f3f2fe3c22da2
    Closes-Bug: #1300105

tags: added: in-stable-havana
Revision history for this message
gordon chung (chungg) wrote :

fix committed to Ceilometer. i don't think we need any changes in devstack so we can probably mark that as invalid.

Changed in ceilometer:
status: In Progress → Fix Committed
Alan Pevec (apevec)
Changed in devstack:
status: New → Invalid
Changed in ceilometer:
status: Fix Committed → Invalid
assignee: gordon chung (chungg) → nobody
tags: removed: in-stable-havana
Revision history for this message
Mike Bayer (zzzeek) wrote :

the "match" keyword is supported in SQLAlchemy as of 0.8.0. Are you sure this stack trace is not running an 0.7 version of SQLAlchemy? that would be the issue here. The note about SQLAlchemy 0.9.2 is being misread here, that applies to the "dialect_kw" argument, not the "match" argument.

Revision history for this message
gordon chung (chungg) wrote :

yeah, i should've expanded that comment. at the time global-requirements were for sqlalchemy 0.7.x.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.