Trove use postgresql as backend database make a fault.

Bug #1760805 reported by LittleMice
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack DBaaS (Trove)
New
Undecided
Unassigned

Bug Description

When using Postgresql as the back-end database, the database synchronization script error occurs because the script still uses the mysql data type and postgresql does not recognize it. For example, the Boolean field is True or False with PostgreSQL, while MySQL is represented by 1 or 0. The specific error message is as follows:
"/var/log/trove/trove-manage.log" 1200L, 256473C 1,1 Top
2018-04-03 16:50:58.622 808946 ERROR root File "/usr/lib/python2.7/dist-packages/trove/db/sqlalchemy/api.py", line 108, in db_sync
2018-04-03 16:50:58.622 808946 ERROR root migration.db_sync(options, version, repo_path)
2018-04-03 16:50:58.622 808946 ERROR root File "/usr/lib/python2.7/dist-packages/trove/db/sqlalchemy/migration.py", line 106, in db_sync
2018-04-03 16:50:58.622 808946 ERROR root upgrade(options, version=version, repo_path=repo_path)
2018-04-03 16:50:58.622 808946 ERROR root File "/usr/lib/python2.7/dist-packages/trove/db/sqlalchemy/migration.py", line 64, in upgrade
2018-04-03 16:50:58.622 808946 ERROR root return versioning_api.upgrade(sql_connection, repo_path, version)
2018-04-03 16:50:58.622 808946 ERROR root File "/usr/lib/python2.7/dist-packages/migrate/versioning/api.py", line 186, in upgrade
2018-04-03 16:50:58.622 808946 ERROR root return _migrate(url, repository, version, upgrade=True, err=err, **opts)
2018-04-03 16:50:58.622 808946 ERROR root File "<decorator-gen-15>", line 2, in _migrate
2018-04-03 16:50:58.622 808946 ERROR root File "/usr/lib/python2.7/dist-packages/migrate/versioning/util/__init__.py", line 167, in with_engine
2018-04-03 16:50:58.622 808946 ERROR root return f(*a, **kw)
2018-04-03 16:50:58.622 808946 ERROR root File "/usr/lib/python2.7/dist-packages/migrate/versioning/api.py", line 366, in _migrate
2018-04-03 16:50:58.622 808946 ERROR root schema.runchange(ver, change, changeset.step)
2018-04-03 16:50:58.622 808946 ERROR root File "/usr/lib/python2.7/dist-packages/migrate/versioning/schema.py", line 93, in runchange
2018-04-03 16:50:58.622 808946 ERROR root change.run(self.engine, step)
2018-04-03 16:50:58.622 808946 ERROR root File "/usr/lib/python2.7/dist-packages/migrate/versioning/script/py.py", line 148, in run
2018-04-03 16:50:58.622 808946 ERROR root script_func(engine)
2018-04-03 16:50:58.622 808946 ERROR root File "/usr/lib/python2.7/dist-packages/trove/db/sqlalchemy/migrate_repo/versions/040_module_priority.py", line 39, in upgrade
2018-04-03 16:50:58.622 808946 ERROR root modules.create_column(column)
2018-04-03 16:50:58.622 808946 ERROR root File "/usr/lib/python2.7/dist-packages/migrate/changeset/schema.py", line 475, in create_column
2018-04-03 16:50:58.622 808946 ERROR root column.create(table=self, *p, **kw)
2018-04-03 16:50:58.622 808946 ERROR root File "/usr/lib/python2.7/dist-packages/migrate/changeset/schema.py", line 582, in create
2018-04-03 16:50:58.622 808946 ERROR root engine.execute(stmt)
2018-04-03 16:50:58.622 808946 ERROR root File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2064, in execute
2018-04-03 16:50:58.622 808946 ERROR root return connection.execute(statement, *multiparams, **params)
2018-04-03 16:50:58.622 808946 ERROR root File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 945, in execute
2018-04-03 16:50:58.622 808946 ERROR root return meth(self, multiparams, params)
2018-04-03 16:50:58.622 808946 ERROR root File "/usr/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
2018-04-03 16:50:58.622 808946 ERROR root return connection._execute_clauseelement(self, multiparams, params)
2018-04-03 16:50:58.622 808946 ERROR root File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
2018-04-03 16:50:58.622 808946 ERROR root compiled_sql, distilled_params
2018-04-03 16:50:58.622 808946 ERROR root File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
2018-04-03 16:50:58.622 808946 ERROR root context)
2018-04-03 16:50:58.622 808946 ERROR root File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1402, in _handle_dbapi_exception
2018-04-03 16:50:58.622 808946 ERROR root exc_info
2018-04-03 16:50:58.622 808946 ERROR root File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
2018-04-03 16:50:58.622 808946 ERROR root reraise(type(exception), exception, tb=exc_tb, cause=cause)
2018-04-03 16:50:58.622 808946 ERROR root File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
2018-04-03 16:50:58.622 808946 ERROR root context)
2018-04-03 16:50:58.622 808946 ERROR root File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 470, in do_execute
2018-04-03 16:50:58.622 808946 ERROR root cursor.execute(statement, parameters)
2018-04-03 16:50:58.622 808946 ERROR root ProgrammingError: (psycopg2.ProgrammingError) column "priority_apply" is of type boolean but expression is of type integer
2018-04-03 16:50:58.622 808946 ERROR root LINE 1: UPDATE modules SET priority_apply=0
2018-04-03 16:50:58.622 808946 ERROR root ^
2018-04-03 16:50:58.622 808946 ERROR root HINT: You will need to rewrite or cast the expression.
2018-04-03 16:50:58.622 808946 ERROR root [SQL: 'UPDATE modules SET priority_apply=%(priority_apply)s'] [parameters: {'priority_apply': 0}]
2018-04-03 16:50:58.622 808946 ERROR root

And the script source :
from sqlalchemy.schema import Column
from sqlalchemy.schema import MetaData
from sqlalchemy.sql.expression import update

from trove.db.sqlalchemy.migrate_repo.schema import Boolean
from trove.db.sqlalchemy.migrate_repo.schema import Integer
from trove.db.sqlalchemy.migrate_repo.schema import Table
from trove.db.sqlalchemy.migrate_repo.schema import Text

COLUMN_NAME_1 = 'priority_apply'
COLUMN_NAME_2 = 'apply_order'
COLUMN_NAME_3 = 'is_admin'

def upgrade(migrate_engine):
    meta = MetaData()
    meta.bind = migrate_engine
    modules = Table('modules', meta, autoload=True)
    is_nullable = True if migrate_engine.name == "sqlite" else False
    column = Column(COLUMN_NAME_1, Boolean(), nullable=is_nullable, default=0)
    modules.create_column(column)
    column = Column(COLUMN_NAME_2, Integer(), nullable=is_nullable, default=5)
    modules.create_column(column)
    column = Column(COLUMN_NAME_3, Boolean(), nullable=is_nullable, default=0)
    modules.create_column(column)
    modules.c.contents.alter(Text(length=4294967295))
    # mark all non-visible, auto-apply and all-tenant modules as is_admin
    update(table=modules,
           values=dict(is_admin=1),
           whereclause="visible=0 or auto_apply=1 or tenant_id is null"
           ).execute()

However,the /etc/trove/trove.conf file give the option to set for postgresql:

#connection = postgresql://trove:trove@localhost/trove

And now ,i want to know that how to fix this problem .

Tags: trove
LittleMice (whscfan)
summary: - Trove install with
+ Trove use postgresql as backend database make a fault.
LittleMice (whscfan)
affects: openstack-manuals → trove
Revision history for this message
vanzhiganov (vanzhiganov-ya) wrote :
Download full text (6.6 KiB)

I have this bug too:

2018-09-27 10:53:51.868 29993 INFO migrate.versioning.api [-] 39 -> 40...
2018-09-27 10:53:51.917 29993 DEBUG migrate.versioning.util [-] Disposing SQLAlchemy engine Engine(postgresql://trove:***@controller/trove) with_engine /usr/lib/python2.7/dist-packages/migrate/versioning/util/__init__.py:170
2018-09-27 10:53:51.919 29993 CRITICAL root [-] Unhandled error: ProgrammingError: (psycopg2.ProgrammingError) column "priority_apply" is of type boolean but expression is of type integer
LINE 1: UPDATE modules SET priority_apply=0
                                          ^
HINT: You will need to rewrite or cast the expression.
 [SQL: 'UPDATE modules SET priority_apply=%(priority_apply)s'] [parameters: {'priority_apply': 0}]
2018-09-27 10:53:51.919 29993 ERROR root Traceback (most recent call last):
2018-09-27 10:53:51.919 29993 ERROR root File "/usr/bin/trove-manage", line 10, in <module>
2018-09-27 10:53:51.919 29993 ERROR root sys.exit(main())
2018-09-27 10:53:51.919 29993 ERROR root File "/usr/lib/python2.7/dist-packages/trove/cmd/manage.py", line 288, in main
2018-09-27 10:53:51.919 29993 ERROR root Commands().execute()
2018-09-27 10:53:51.919 29993 ERROR root File "/usr/lib/python2.7/dist-packages/trove/cmd/manage.py", line 59, in execute
2018-09-27 10:53:51.919 29993 ERROR root exec_method(**kwargs)
2018-09-27 10:53:51.919 29993 ERROR root File "/usr/lib/python2.7/dist-packages/trove/cmd/manage.py", line 44, in db_sync
2018-09-27 10:53:51.919 29993 ERROR root self.db_api.db_sync(CONF, repo_path=repo_path)
2018-09-27 10:53:51.919 29993 ERROR root File "/usr/lib/python2.7/dist-packages/trove/db/sqlalchemy/api.py", line 108, in db_sync
2018-09-27 10:53:51.919 29993 ERROR root migration.db_sync(options, version, repo_path)
2018-09-27 10:53:51.919 29993 ERROR root File "/usr/lib/python2.7/dist-packages/trove/db/sqlalchemy/migration.py", line 106, in db_sync
2018-09-27 10:53:51.919 29993 ERROR root upgrade(options, version=version, repo_path=repo_path)
2018-09-27 10:53:51.919 29993 ERROR root File "/usr/lib/python2.7/dist-packages/trove/db/sqlalchemy/migration.py", line 64, in upgrade
2018-09-27 10:53:51.919 29993 ERROR root return versioning_api.upgrade(sql_connection, repo_path, version)
2018-09-27 10:53:51.919 29993 ERROR root File "/usr/lib/python2.7/dist-packages/migrate/versioning/api.py", line 186, in upgrade
2018-09-27 10:53:51.919 29993 ERROR root return _migrate(url, repository, version, upgrade=True, err=err, **opts)
2018-09-27 10:53:51.919 29993 ERROR root File "<decorator-gen-15>", line 2, in _migrate
2018-09-27 10:53:51.919 29993 ERROR root File "/usr/lib/python2.7/dist-packages/migrate/versioning/util/__init__.py", line 167, in with_engine
2018-09-27 10:53:51.919 29993 ERROR root return f(*a, **kw)
2018-09-27 10:53:51.919 29993 ERROR root File "/usr/lib/python2.7/dist-packages/migrate/versioning/api.py", line 366, in _migrate
2018-09-27 10:53:51.919 29993 ERROR root schema.runchange(ver, change, changeset.step)
2018-09-27 10:53:51.919 29993 ERROR root File "/usr/lib/python2.7/dist-packages/migrate/versioning/schema.py", line 93, in runchange
201...

Read more...

Revision history for this message
Junior Zilles (juniorz123) wrote :
Download full text (9.1 KiB)

I have this issue a swell.

2020-02-17 14:12:52.368 | [SQL: UPDATE modules SET is_admin=%(is_admin)s WHERE visible=0 or auto_apply=1 or tenant_id is null]
2020-02-17 14:12:52.368 | [parameters: {'is_admin': True}]
2020-02-17 14:12:52.368 | (Background on this error at: http://sqlalche.me/e/f405)
2020-02-17 14:12:52.368 | 2020-02-17 14:12:52.350 6085 ERROR root Traceback (most recent call last):
2020-02-17 14:12:52.368 | 2020-02-17 14:12:52.350 6085 ERROR root File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1246, in _execute_context
2020-02-17 14:12:52.368 | 2020-02-17 14:12:52.350 6085 ERROR root cursor, statement, parameters, context
2020-02-17 14:12:52.368 | 2020-02-17 14:12:52.350 6085 ERROR root File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/default.py", line 588, in do_execute
2020-02-17 14:12:52.368 | 2020-02-17 14:12:52.350 6085 ERROR root cursor.execute(statement, parameters)
2020-02-17 14:12:52.368 | 2020-02-17 14:12:52.350 6085 ERROR root psycopg2.errors.UndefinedFunction: operator does not exist: boolean = integer
2020-02-17 14:12:52.368 | 2020-02-17 14:12:52.350 6085 ERROR root LINE 1: UPDATE modules SET is_admin=true WHERE visible=0 or auto_app...
2020-02-17 14:12:52.368 | 2020-02-17 14:12:52.350 6085 ERROR root ^
2020-02-17 14:12:52.368 | 2020-02-17 14:12:52.350 6085 ERROR root HINT: No operator matches the given name and argument type(s). You might need to add explicit type casts.
2020-02-17 14:12:52.368 | 2020-02-17 14:12:52.350 6085 ERROR root
2020-02-17 14:12:52.368 | 2020-02-17 14:12:52.350 6085 ERROR root
2020-02-17 14:12:52.368 | 2020-02-17 14:12:52.350 6085 ERROR root The above exception was the direct cause of the following exception:
2020-02-17 14:12:52.368 | 2020-02-17 14:12:52.350 6085 ERROR root
2020-02-17 14:12:52.368 | 2020-02-17 14:12:52.350 6085 ERROR root Traceback (most recent call last):
2020-02-17 14:12:52.368 | 2020-02-17 14:12:52.350 6085 ERROR root File "/usr/local/bin/trove-manage", line 10, in <module>
2020-02-17 14:12:52.368 | 2020-02-17 14:12:52.350 6085 ERROR root sys.exit(main())
2020-02-17 14:12:52.368 | 2020-02-17 14:12:52.350 6085 ERROR root File "/opt/stack/trove/trove/cmd/manage.py", line 283, in main
2020-02-17 14:12:52.368 | 2020-02-17 14:12:52.350 6085 ERROR root Commands().execute()
2020-02-17 14:12:52.368 | 2020-02-17 14:12:52.350 6085 ERROR root File "/opt/stack/trove/trove/cmd/manage.py", line 54, in execute
2020-02-17 14:12:52.368 | 2020-02-17 14:12:52.350 6085 ERROR root exec_method(**kwargs)
2020-02-17 14:12:52.368 | 2020-02-17 14:12:52.350 6085 ERROR root File "/opt/stack/trove/trove/cmd/manage.py", line 39, in db_sync
2020-02-17 14:12:52.368 | 2020-02-17 14:12:52.350 6085 ERROR root self.db_api.db_sync(CONF, repo_path=repo_path)
2020-02-17 14:12:52.368 | 2020-02-17 14:12:52.350 6085 ERROR root File "/opt/stack/trove/trove/db/sqlalchemy/api.py", line 108, in db_sync
2020-02-17 14:12:52.368 | 2020-02-17 14:12:52.350 6085 ERROR root migration.db_sync(options, version, repo_path)
2020-02-17 14:12:52.369 | 2020-02-17 14:12:52.350 6085 ERROR root ...

Read more...

Revision history for this message
LittleMice (whscfan) wrote :

This issue is too long ago! And i have forgot hao i fix it .I just remember that i have modified some source code ,and it has been running steadily for two years.

Revision history for this message
Lingxian Kong (kong) wrote :

Trove is using storyboard for feature/bug tracking, please go to https://storyboard.openstack.org/#!/project_group/trove if issue still exists.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.