tripleo

Not found image: docker://trunk.registry.rdoproject.org/tripleomaster/fedora-binary-cinder-api:59bf4b8a4c6b00c7ebb9b18b67853f6e318b2a51_d9bb6f52

Bug #1823136 reported by Quique Llorente
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Critical
Quique Llorente
tripleo stein-rc1

Bug Description

At fedora28 standalone master looks like some containers has not being pushed
http://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-fedora-28-standalone-master/fd87a6a/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz

2019-04-03 15:47:59 | jinja2 rendering roles Standalone
2019-04-03 15:47:59 | rendering j2 template to file: /home/zuul/tripleo-heat-installer-templates/./network/config/2-linux-bonds-vlans/standalone.yaml
2019-04-03 15:48:02 | Exception: Not found image: docker://trunk.registry.rdoproject.org/tripleomaster/fedora-binary-cinder-api:59bf4b8a4c6b00c7ebb9b18b67853f6e318b2a51_d9bb6f52
2019-04-03 15:48:02 | Traceback (most recent call last):
2019-04-03 15:48:02 | File "/usr/lib/python3.6/site-packages/tripleoclient/v1/tripleo_deploy.py", line 1207, in _standalone_deploy
2019-04-03 15:48:02 | parsed_args)
2019-04-03 15:48:02 | File "/usr/lib/python3.6/site-packages/tripleoclient/v1/tripleo_deploy.py", line 795, in _deploy_tripleo_heat_templates
2019-04-03 15:48:02 | self._prepare_container_images(env, roles_data)
2019-04-03 15:48:02 | File "/usr/lib/python3.6/site-packages/tripleoclient/v1/tripleo_deploy.py", line 741, in _prepare_container_images
2019-04-03 15:48:02 | env, roles_data, dry_run=True)
2019-04-03 15:48:02 | File "/usr/lib/python3.6/site-packages/tripleo_common/image/kolla_builder.py", line 188, in container_images_prepare_multi
2019-04-03 15:48:02 | mirrors=mirrors
2019-04-03 15:48:02 | File "/usr/lib/python3.6/site-packages/tripleo_common/image/kolla_builder.py", line 296, in container_images_prepare
2019-04-03 15:48:02 | images, modify_only_with_labels)
2019-04-03 15:48:02 | File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py", line 601, in filter_images_with_labels
2019-04-03 15:48:02 | url, session=session)
2019-04-03 15:48:02 | File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py", line 291, in _image_labels
2019-04-03 15:48:02 | i = cls._inspect(image_url, session)
2019-04-03 15:48:02 | File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py", line 1495, in _inspect
2019-04-03 15:48:02 | image_url, session=session)
2019-04-03 15:48:02 | File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 292, in wrapped_f
2019-04-03 15:48:02 | return self.call(f, *args, **kw)
2019-04-03 15:48:02 | File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 358, in call
2019-04-03 15:48:02 | do = self.iter(retry_state=retry_state)
2019-04-03 15:48:02 | File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 319, in iter
2019-04-03 15:48:02 | return fut.result()
2019-04-03 15:48:02 | File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 425, in result
2019-04-03 15:48:02 | return self.__get_result()
2019-04-03 15:48:02 | File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result
2019-04-03 15:48:02 | raise self._exception
2019-04-03 15:48:02 | File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 361, in call
2019-04-03 15:48:02 | result = fn(*args, **kwargs)
2019-04-03 15:48:02 | File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py", line 405, in _inspect
2019-04-03 15:48:02 | image_url.geturl())
2019-04-03 15:48:02 | tripleo_common.image.exception.ImageNotFoundException: Not found image: docker://trunk.registry.rdoproject.org/tripleomaster/fedora-binary-cinder-api:59bf4b8a4c6b00c7ebb9b18b67853f6e318b2a51_d9bb6f52
2019-04-03 15:48:02 | None
2019-04-03 15:48:03 | Not cleaning workin

Revision history for this message
Quique Llorente (quiquell) wrote :

Is deterministic
http://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-fedora-28-standalone-master/b54b206/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz

Revision history for this message
Quique Llorente (quiquell) wrote :

Looks like containers build is pointing to centos instead of fedora
http://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-fedora-28-master-containers-build-push/07ac1e5/conf/overcloud_containers.yaml.txt.gz
- imagename: docker.io/tripleomaster/centos-binary-cinder-api:current-tripleo
  image_source: kolla
- imagename: docker.io/tripleomaster/centos-binary-cinder-backup:current-tripleo
  image_source: kolla
- imagename: docker.io/tripleomaster/centos-binary-cinder-scheduler:current-tripleo
  image_source: kolla
- imagename: docker.io/tripleomaster/centos-binary-cinder-volume:current-tripleo
  image_source: kolla

Revision history for this message
Quique Llorente (quiquell) wrote :

Fixed with https://review.openstack.org/#/c/639849/

Changed in tripleo:
status: Triaged → Fix Committed
status: Fix Committed → Fix Released
Revision history for this message
Quique Llorente (quiquell) wrote :

Looks like we still reference centos images at fedora containers-build
http://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-fedora-28-master-containers-build-push/37cf2a6/conf/overcloud_containers.yaml.txt.gz

Changed in tripleo:
status: Fix Released → Fix Committed
Revision history for this message
Quique Llorente (quiquell) wrote :

No we don't pass overcloud_containers.yaml anymore
http://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-fedora-28-master-containers-build-push/37cf2a6/logs/build_containers.sh.txt.gz, it uses the jinja2 template

#!/bin/bash -eux
source /home/zuul/workspace/venv/bin/activate
TRIPLEO_COMMON_PATH="/home/zuul/src/git.openstack.org/openstack/tripleo-common"

openstack overcloud container image build \
    --kolla-config-file /home/zuul/workspace/kolla-build.conf \
                --exclude fluentd \
            --exclude opendaylight \
            --exclude skydive-agent \
            --exclude skydive-analyzer \
            --exclude sensu-client \
            --exclude sensu-base \

Revision history for this message
Quique Llorente (quiquell) wrote :

Logs from latest f28 standalone
http://logs.openstack.org/37/622037/3/check/tripleo-ci-fedora-28-standalone/d6e649d/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz#_2019-04-09_03_15_53
tripleo_common.image.exception.ImageNotFoundException: Not found image: docker://docker.io/tripleomaster/fedora-binary-cinder-api:e1bc201c57c5f4522d75048ddf3de8ba8178e423_6e306722

Revision history for this message
Quique Llorente (quiquell) wrote :

Looks like promoter is failing to push to docker.io so we don't have the containers promoted.

Revision history for this message
Quique Llorente (quiquell) wrote :
Download full text (3.3 KiB)

Looks like they where correcty pushed to docker hub
2019-04-08 23:03:34,580 22611 INFO promoter
2019-04-08 23:03:34,580 22611 INFO promoter TASK [Tag and push images to docker hub] ***************************************
2019-04-08 23:03:34,580 22611 INFO promoter changed: [localhost] => (item=[u'e1bc201c57c5f4522d75048ddf3de8ba8178e423_6e306722', u'cron'])
2019-04-08 23:03:34,580 22611 INFO promoter changed: [localhost] => (item=[u'e1bc201c57c5f4522d75048ddf3de8ba8178e423_6e306722', u'etcd'])
2019-04-08 23:03:34,580 22611 INFO promoter changed: [localhost] => (item=[u'e1bc201c57c5f4522d75048ddf3de8ba8178e423_6e306722', u'haproxy'])
2019-04-08 23:03:34,580 22611 INFO promoter changed: [localhost] => (item=[u'e1bc201c57c5f4522d75048ddf3de8ba8178e423_6e306722', u'iscsid'])
2019-04-08 23:03:34,580 22611 INFO promoter changed: [localhost] => (item=[u'e1bc201c57c5f4522d75048ddf3de8ba8178e423_6e306722', u'mariadb'])
2019-04-08 23:03:34,580 22611 INFO promoter changed: [localhost] => (item=[u'e1bc201c57c5f4522d75048ddf3de8ba8178e423_6e306722', u'multipathd'])
2019-04-08 23:03:34,580 22611 INFO promoter changed: [localhost] => (item=[u'e1bc201c57c5f4522d75048ddf3de8ba8178e423_6e306722', u'qdrouterd'])
2019-04-08 23:03:34,580 22611 INFO promoter changed: [localhost] => (item=[u'e1bc201c57c5f4522d75048ddf3de8ba8178e423_6e306722', u'rabbitmq'])
2019-04-08 23:03:34,580 22611 INFO promoter changed: [localhost] => (item=[u'e1bc201c57c5f4522d75048ddf3de8ba8178e423_6e306722', u'rsyslog-base'])
2019-04-08 23:03:34,580 22611 INFO promoter changed: [localhost] => (item=[u'e1bc201c57c5f4522d75048ddf3de8ba8178e423_6e306722', u'base'])
2019-04-08 23:03:34,580 22611 INFO promoter changed: [localhost] => (item=[u'e1bc201c57c5f4522d75048ddf3de8ba8178e423_6e306722', u'openstack-base'])
2019-04-08 23:03:34,580 22611 INFO promoter changed: [localhost] => (item=[u'current-tripleo', u'cron'])
2019-04-08 23:03:34,581 22611 INFO promoter changed: [localhost] => (item=[u'current-tripleo', u'etcd'])
2019-04-08 23:03:34,581 22611 INFO promoter changed: [localhost] => (item=[u'current-tripleo', u'haproxy'])
2019-04-08 23:03:34,581 22611 INFO promoter changed: [localhost] => (item=[u'current-tripleo', u'iscsid'])
2019-04-08 23:03:34,581 22611 INFO promoter changed: [localhost] => (item=[u'current-tripleo', u'mariadb'])
2019-04-08 23:03:34,581 22611 INFO promoter changed: [localhost] => (item=[u'current-tripleo', u'multipathd'])
2019-04-08 23:03:34,581 22611 INFO promoter changed: [localhost] => (item=[u'current-tripleo', u'qdrouterd'])
2019-04-08 23:03:34,581 22611 INFO promoter changed: [localhost] => (item=[u'current-tripleo', u'rabbitmq'])
2019-04-08 23:03:34,581 22611 INFO promoter changed: [localhost] => (item=[u'current-tripleo', u'rsyslog-base'])
2019-04-08 23:03:34,581 22611 INFO promoter changed: [localhost] => (item=[u'current-tripleo', u'base'])
2019-04-08 23:03:34,581 22611 INFO promoter changed: [localhost] => (item=[u'current-tripleo', u'openstack-base'])
2019-04-08 23:03:34,581 22611 INFO promoter
2019-04-08 23:03:34,581 22611 INFO promoter TASK [Check if doc...

Read more...

Revision history for this message
Quique Llorente (quiquell) wrote :

Un a lot of containers re missing there

Revision history for this message
Quique Llorente (quiquell) wrote :

Fix at promoter https://review.rdoproject.org/r/#/c/20039/

Changed in tripleo:
status: Fix Committed → In Progress
Revision history for this message
Quique Llorente (quiquell) wrote :

We were missing creteria for fedora28 promotions so the containers were not fully pushed the f28 standalone job is now part of the criteria

Changed in tripleo:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.