Hardcode use of utf8 for bgpvpns table prevents kilo to liberty upgrade

Bug #1515987 reported by Nikolas Hermanns
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
networking-bgpvpn
Fix Released
High
Nikolas Hermanns

Bug Description

The table networks is coded in utf8 in liberty. In kilo it is latin1.
The table bgpvpns is hardcoded in utf8.
The table bgpvpn_network_associations has foreign keys to the tables networks and bgpvpns.

For kilo it is not possible to setup the database because the table and the refered table must have the same codec.

Changed in bgpvpn:
milestone: none → liberty
importance: Undecided → High
status: New → Confirmed
Revision history for this message
Thomas Morin (tmmorin-orange) wrote :

The 'bgpvpns' table was utf8 from its "day 1" in the networking-bgpvpn project:

https://github.com/openstack/networking-bgpvpn/blob/a3e7270d29a169c21dec27c9ee264ba9e4def03d/networking_bgpvpn/neutron/db/migration/alembic_migrations/versions/start_networking_bgpvpn.py

We could use latin1, and this would likely fix the Kilo issue.
But I guess it would create an issue in Liberty, since the 'networks' table is utf8 in Liberty.

If the 'networks' table is latin1 in kilo and utf8 in Liberty, I would expect this change to be covered by an alembic migration tool in Neutron. If such a thing exist, then it seems we should mimick this migration strategy and apply it to the bgpvpn_network_association table (migrating it from latin1 to utf8 during the kilo->liberty migration).

Revision history for this message
Mathieu Rohon (mathieu-rohon) wrote :

I didn't mention utf8 in the first script on purpose, probably a copy/paste from elsewhere.
I can't find anywhere in the neutron/liberty code the mention of ut8 tables.

Nikolas, where did you find that neutron enforce the network table to be coded in utf8?

Maybe SQLAlchemy relies on the underlying SQL engine to choose its default encoding?

Changed in bgpvpn:
status: Confirmed → Incomplete
Revision history for this message
Mathieu Rohon (mathieu-rohon) wrote :

it seems the encoding is managed at the creation of the db :

https://github.com/openstack-dev/devstack/blob/master/lib/databases/mysql#L58

Revision history for this message
Thomas Morin (tmmorin-orange) wrote :

I've looked at neutron/db/migration/* and the default charset is not set for any table (the 'utf8' and 'latin1' strings don't appear anywhere). I guess the right thing to do is to not force any charset in the database creation statement. I have actually no idea why we added it initially.

In fact, forcing the mysql_engine to InnoDB does not seem to be done anywhere either. I think we should drop this one as well.

Objections to removing these two mysql_* statements ?

Revision history for this message
Thomas Morin (tmmorin-orange) wrote :

Parallel replies, but the good news is that we end up in the same direction: the default charset should not be forced by our creation/migration script. The thing results from a copy-paste from internal code we had, not written by me or Mathieu, hence our inability to find any explanation.

I'll do the change at once, because there is no particular reason for doing things differently than other projects.

Nikolas, can you confirm kilo/liberty works fine after removing it ?

Changed in bgpvpn:
assignee: nobody → Thomas Morin (tmmorin-orange)
status: Incomplete → Confirmed
Changed in bgpvpn:
assignee: Thomas Morin (tmmorin-orange) → Nikolas Hermanns (nikolas-hermanns)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on networking-bgpvpn (master)

Change abandoned by Thomas Morin (<email address hidden>) on branch: master
Review: https://review.openstack.org/245201
Reason: abandon in favor of I2e732d8474f6769259e25d69b1b5269bebc9c26f

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to networking-bgpvpn (master)

Reviewed: https://review.openstack.org/245207
Committed: https://git.openstack.org/cgit/openstack/networking-bgpvpn/commit/?id=827129094787ec4dc785544488fe0131f16fa2d9
Submitter: Jenkins
Branch: master

commit 827129094787ec4dc785544488fe0131f16fa2d9
Author: Nikolas Hermanns <email address hidden>
Date: Fri Nov 13 13:32:00 2015 +0100

    Remove hardcoded utf8 coding for bgpvpns table

    to align to the kilo code it is necessary to
    let the database decide to use latin1 or utf8
    as codec. Otherwise bgpvpn_network_associations
    table cannot have foreign keys to networks
    and bgpvpns. The reason is that for foreign keys
    the column in the table and the column in the
    foreign table has to have the same codec.

    Change-Id: I2e732d8474f6769259e25d69b1b5269bebc9c26f
    Closes-Bug: 1515987

Changed in bgpvpn:
status: Confirmed → Fix Committed
Changed in bgpvpn:
status: Fix Committed → Fix Released
Changed in bgpvpn:
status: Fix Released → Fix Committed
Changed in bgpvpn:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.