sahara db sync fails with duplicate column name storage_devices_number

Bug #1630550 reported by Roman Sokolkov
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mirantis OpenStack
Fix Released
High
Vitalii Gridnev
8.0.x
Won't Fix
High
Vitalii Gridnev

Bug Description

I have Mitaka Shara and doing db sync against Kilo database.

Getting following failure

root@ic3-ctl02-scc:~# sahara-db-manage upgrade head
INFO [alembic.runtime.migration] Context impl MySQLImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
INFO [alembic.runtime.migration] Running upgrade 022 -> 023, add_use_autoconfig
INFO [alembic.runtime.migration] Running upgrade 023 -> 024, manila_shares
INFO [alembic.runtime.migration] Running upgrade 024 -> 025, Increase internal_ip and management_ip column size to work with IPv6
INFO [alembic.runtime.migration] Running upgrade 025 -> 026, add is_public and is_protected flags
INFO [alembic.runtime.migration] Running upgrade 026 -> 027, Rename oozie_job_id
INFO [alembic.runtime.migration] Running upgrade 027 -> 028, add_storage_devices_number
Traceback (most recent call last):
  File "/usr/bin/sahara-db-manage", line 10, in <module>
    sys.exit(main())
  File "/usr/lib/python2.7/dist-packages/sahara/db/migration/cli.py", line 109, in main
    CONF.command.func(config, CONF.command.name)
  File "/usr/lib/python2.7/dist-packages/sahara/db/migration/cli.py", line 49, in do_upgrade_downgrade
    do_alembic_command(config, cmd, revision, sql=CONF.command.sql)
  File "/usr/lib/python2.7/dist-packages/sahara/db/migration/cli.py", line 30, in do_alembic_command
    getattr(alembic_cmd, cmd)(config, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/alembic/command.py", line 174, in upgrade
    script.run_env()
  File "/usr/lib/python2.7/dist-packages/alembic/script/base.py", line 397, in run_env
    util.load_python_file(self.dir, 'env.py')

  File "/usr/lib/python2.7/dist-packages/alembic/util/pyfiles.py", line 93, in load_python_file
    module = load_module_py(module_id, path)
  File "/usr/lib/python2.7/dist-packages/alembic/util/compat.py", line 79, in load_module_py
    mod = imp.load_source(module_id, path, fp)
  File "/usr/lib/python2.7/dist-packages/sahara/db/migration/alembic_migrations/env.py", line 95, in <module>
    run_migrations_online()
  File "/usr/lib/python2.7/dist-packages/sahara/db/migration/alembic_migrations/env.py", line 88, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/usr/lib/python2.7/dist-packages/alembic/runtime/environment.py", line 797, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/usr/lib/python2.7/dist-packages/alembic/runtime/migration.py", line 312, in run_migrations
    step.migration_fn(**kw)
  File "/usr/lib/python2.7/dist-packages/sahara/db/migration/alembic_migrations/versions/028_storage_devices_number.py", line 35, in upgrade
    nullable=True))
  File "<string>", line 8, in add_column
  File "<string>", line 3, in add_column
  File "/usr/lib/python2.7/dist-packages/alembic/operations/ops.py", line 1535, in add_column
    return operations.invoke(op)
  File "/usr/lib/python2.7/dist-packages/alembic/operations/base.py", line 318, in invoke
    return fn(self, operation)
  File "/usr/lib/python2.7/dist-packages/alembic/operations/toimpl.py", line 123, in add_column
    schema=schema
  File "/usr/lib/python2.7/dist-packages/alembic/ddl/impl.py", line 172, in add_column
    self._exec(base.AddColumn(table_name, column, schema=schema))
  File "/usr/lib/python2.7/dist-packages/alembic/ddl/impl.py", line 118, in _exec
    return conn.execute(construct, *multiparams, **params)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 914, in execute
    return meth(self, multiparams, params)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/sql/ddl.py", line 68, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 968, in _execute_ddl
    compiled
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
    context)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1341, in _handle_dbapi_exception
    exc_info
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 200, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
    context)
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 450, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 219, in execute
    self.errorhandler(self, exc, value)
  File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 38, in defaulterrorhandler
    raise errorvalue
sqlalchemy.exc.OperationalError: (_mysql_exceptions.OperationalError) (1060, "Duplicate column name 'storage_devices_number'") [SQL: u'ALTER TABLE instances ADD COLUMN storage_devices_number INTEGER']

Details:
Sahara Mitaka 1:4.0.1-3~u14.04+mos14

Steps to reproduce:
1) Install MOS7.0 with sahara
2) Create mysqldump of sahara database and upload to MOS9.1
3) execute sahara-db-manage upgrade head

Tags: area-sahara
affects: sahara → mos
Changed in mos:
importance: Undecided → High
assignee: nobody → Fuel Octane (fuel-octane-team)
milestone: none → 9.2
status: New → Confirmed
tags: added: area-sahara
Changed in mos:
assignee: Fuel Octane (fuel-octane-team) → Vitaly Gridnev (vgridnev)
Revision history for this message
Vitalii Gridnev (vgridnev) wrote :
Changed in mos:
status: Confirmed → In Progress
Revision history for this message
Vitalii Gridnev (vgridnev) wrote :
Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix merged to openstack/sahara (9.0/mitaka)

Reviewed: https://review.fuel-infra.org/27203
Submitter: Pkgs Jenkins <email address hidden>
Branch: 9.0/mitaka

Commit: 4f445ab0a713ea752c063cd1fb9e97e00dafd12c
Author: Vitaly Gridnev <email address hidden>
Date: Thu Oct 6 10:29:05 2016

Fix 7.0 -> 8.0/9.x migration

addition of column storage_devices_number
is wrapped by "try-except" block because during
upgrade this column already exists since
it was introduced in 7.0 by custom migration
with number 022. Also all code from migration
022 was replaced to migration 023 to introduce
needed tables during migration since 7.0.

Change-Id: I8c2e5e98118fc41b13aeb00966fd2cd71c77bdc7
Closes-bug: 1630550

Changed in mos:
status: In Progress → Fix Committed
tags: added: on-verification
Revision history for this message
TatyanaGladysheva (tgladysheva) wrote :

Verified on 9.2 snapshot #572.

Actual results:
root@node-1:~# sahara-db-manage upgrade head
INFO [alembic.runtime.migration] Context impl MySQLImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
INFO [alembic.runtime.migration] Running upgrade 022 -> 023, add_use_autoconfig
INFO [alembic.runtime.migration] Running upgrade 023 -> 024, manila_shares
INFO [alembic.runtime.migration] Running upgrade 024 -> 025, Increase internal_ip and management_ip column size to work with IPv6
INFO [alembic.runtime.migration] Running upgrade 025 -> 026, add is_public and is_protected flags
INFO [alembic.runtime.migration] Running upgrade 026 -> 027, Rename oozie_job_id
INFO [alembic.runtime.migration] Running upgrade 027 -> 028, add_storage_devices_number
(_mysql_exceptions.OperationalError) (1060, "Duplicate column name 'storage_devices_number'") [SQL: u'ALTER TABLE instances ADD COLUMN storage_devices_number INTEGER']
INFO [alembic.runtime.migration] Running upgrade 028 -> 029, set is_protected on is_default
INFO [alembic.runtime.migration] Running upgrade 029 -> 030, health-check
root@node-1:~#

tags: removed: on-verification
Changed in mos:
status: Fix Committed → Fix Released
Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Change abandoned on openstack/sahara (openstack-ci/fuel-8.0/liberty)

Change abandoned by Vitaly Gridnev <email address hidden> on branch: openstack-ci/fuel-8.0/liberty
Review: https://review.fuel-infra.org/27204

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix proposed to openstack/sahara (mcp/newton)

Fix proposed to branch: mcp/newton
Change author: Vitaly Gridnev <email address hidden>
Review: https://review.fuel-infra.org/33721

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix proposed to openstack/sahara (11.0/ocata)

Fix proposed to branch: 11.0/ocata
Change author: Vitaly Gridnev <email address hidden>
Review: https://review.fuel-infra.org/34148

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix proposed to openstack/sahara (mcp/ocata)

Fix proposed to branch: mcp/ocata
Change author: Vitaly Gridnev <email address hidden>
Review: https://review.fuel-infra.org/34813

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Change abandoned on openstack/sahara (11.0/ocata)

Change abandoned by Roman Podoliaka <email address hidden> on branch: 11.0/ocata
Review: https://review.fuel-infra.org/34148
Reason: we do not need 11.0/ocata anymore - use mcp/ocata instead

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Change abandoned on openstack/sahara (mcp/ocata)

Change abandoned by Ihor Kalnytskyi <email address hidden> on branch: mcp/ocata
Review: https://review.fuel-infra.org/34813
Reason: This patch is a hack for upgrading custom Sahara distributed within MOS. It's not required for MCP.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.