Neutron-api stuck in config-changed hook failed with sqlalchemy error during functional testing

Bug #1785839 reported by Pedro Guimarães
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Neutron API Charm
In Progress
Undecided
Pedro Guimarães

Bug Description

When running charm-designate dev-basic-bionic-rocky functional tests, my neutron-api instance got sometimes stuck on config-changed hook failed. The error seems to be intermittent as I reproduced twice in 3 runs.

Ran same scenario on bionic-queens one time and it worked with no errors.

Environment:
charm-designate -> running functional tests checked from git.openstack.org directly
Juju version -> 2.5-beta1-bionic-amd64

Here is the error log --> https://pastebin.ubuntu.com/p/ns6S9v6j2X/

The error at the end is: sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) Cannot add a NOT NULL column with default value NULL [SQL: u'ALTER TABLE address_scopes ADD COLUMN ip_version INTEGER NOT NULL']

It looks like my neutron-api is using sqlite instead of percona-cluster, as assigned for this test scenario. In fact, my functional test log gets stuck on application deployment phase and fails as neutron-api failed, and did not execute the relationship phase (which happens just after)

Tried to set manually the relationship between percona-cluster and neutron-api and got:
https://pastebin.ubuntu.com/p/G5QKTThKnr/

/etc/neutron/neutron.conf after relationship set:
https://pastebin.ubuntu.com/p/2R65ZsgwbQ/
Still got connection database set to sqlite, although relationship with percona-cluster was set.

/etc/neutron/plugins/ml2/ml2_conf.ini:
https://pastebin.ubuntu.com/p/NF9KfgGqgF/

Re-running neutron-db-manage command by hand leads to:
https://pastebin.ubuntu.com/p/MdbNyNZ8jP/

On Bionic-Queens scenario, neutron.conf was correctly set to connect with mysql database.
The bug is similar to: https://bugs.launchpad.net/neutron/+bug/1697881 but DB-neutron api relation was supposed to set connection info correctly and move away from sqlite.

Revision history for this message
Corey Bryant (corey.bryant) wrote :

Thanks for reporting this bug Pedro. The problem is that the neutron.conf template hasn't been rendered yet, as can be seen by the database connection config:

[database]
connection = sqlite:////var/lib/neutron/neutron.sqlite

We'll need to dig into this some more to understand why neutron-db-manage is being executed prior to neutron.conf being rendered.

Changed in charm-neutron-api:
assignee: nobody → Pedro Guimarães (pguimaraes)
Revision history for this message
Pedro Guimarães (pguimaraes) wrote :

I can reproduce this issue by executing:
juju deploy <NEUTRON-API-CHARM> --series=bionic --config openstack-origin="cloud:bionic-rocky"

Changed in charm-neutron-api:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to charm-neutron-api (master)

Fix proposed to branch: master
Review: https://review.openstack.org/589830

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.openstack.org/589853

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on charm-neutron-api (master)

Change abandoned by Pedro Guimarães (<email address hidden>) on branch: master
Review: https://review.openstack.org/589853

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Change abandoned by Frode Nordahl (<email address hidden>) on branch: master
Review: https://review.openstack.org/589830
Reason: Marking this review as abandoned due to lack of activity in the past 6 months.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.