[Murano][GLARE] There is no Core Library in Murano if Murano using GLARE backend

Bug #1572785 reported by Serg Melikyan
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Committed
Critical
Denis Egorenko
8.0.x
Won't Fix
High
MOS Maintenance
Mitaka
Fix Released
Critical
Denis Egorenko
Newton
Fix Committed
Critical
Denis Egorenko

Bug Description

Upstream bug: https://bugs.launchpad.net/puppet-murano/+bug/1572787

Detailed bug description:
Deployment finished successfully, but murano does not have core Library installed and therefor not functional at all.

Side Note: looks like deployment has misconfigured networking or something else because keystone is not available by url from openrc

Steps to reproduce:
1) Login to primary controller
2) Edit openrc: replace value for OS_AUTH_URL with something which will return 404 immediately
3) Execute murano package-list
root@node-3:~# murano package-list
Authorization Failed: The resource could not be found. (HTTP 404)
4) Run puppet module which installs and imports core library
puppet apply --debug /etc/puppet/modules/osnailyfacter/modular/murano/murano.pp
5) Result:
Debug: Executing '/usr/bin/murano package-list'
Debug: Executing '/usr/bin/murano package-import /var/cache/murano/meta/io.murano.zip --is-public --exists-action u'
Notice: /Stage[main]/Main/Murano::Application[io.murano]/Murano_application[io.murano]/ensure: created

Everything reports about success but nothing actually was imported to the Murano, there is no core library and Murano does not work at all.

UPD: Murano was configured to use glance-glare service for package backend.

Revision history for this message
Serg Melikyan (smelikyan) wrote :

root@node-3:~# source ./openrc
root@node-3:~# keystone endpoint-list
Authorization Failed: The resource could not be found. (HTTP 404)

description: updated
Revision history for this message
Victor Ryzhenkin (vryzhenkin) wrote :

Still don't know, why this one can be happened.
For clients - yes, this was a known issue.
But for murano installation this can not be true.

Look at this: https://github.com/openstack/puppet-murano/blob/stable/liberty/lib/puppet/provider/murano_application/murano.rb#L35
Here we using `auth_murano` to executing package-import command.

And auth_murano is defined here: https://github.com/openstack/puppet-murano/blob/stable/liberty/lib/puppet/provider/murano.rb#L78

Which finally using creds from murano.conf file, not from openrc
https://github.com/openstack/puppet-murano/blob/stable/liberty/lib/puppet/provider/murano.rb#L38

Finally, I'm using our release ISO with MOS 8.0 and can not reproduce it.

[root@fuel ~]# cat /etc/nailgun/version.yaml
VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "8.0"
  api: "1.0"
  build_number: "570"
  build_id: "570"
  fuel-nailgun_sha: "558ca91a854cf29e395940c232911ffb851899c1"
  python-fuelclient_sha: "4f234669cfe88a9406f4e438b1e1f74f1ef484a5"
  fuel-agent_sha: "658be72c4b42d3e1436b86ac4567ab914bfb451b"
  fuel-nailgun-agent_sha: "b2bb466fd5bd92da614cdbd819d6999c510ebfb1"
  astute_sha: "b81577a5b7857c4be8748492bae1dec2fa89b446"
  fuel-library_sha: "c2a335b5b725f1b994f78d4c78723d29fa44685a"
  fuel-ostf_sha: "3bc76a63a9e7d195ff34eadc29552f4235fa6c52"
  fuel-mirror_sha: "fb45b80d7bee5899d931f926e5c9512e2b442749"
  fuelmenu_sha: "78ffc73065a9674b707c081d128cb7eea611474f"
  shotgun_sha: "63645dea384a37dde5c01d4f8905566978e5d906"
  network-checker_sha: "a43cf96cd9532f10794dce736350bf5bed350e9d"
  fuel-upgrade_sha: "616a7490ec7199f69759e97e42f9b97dfc87e85b"
  fuelmain_sha: "d605bcbabf315382d56d0ce8143458be67c53434"

Serg, could you adjust conditions for it?

Changed in mos:
status: Confirmed → Incomplete
Revision history for this message
Denis Egorenko (degorenko) wrote :

I agree with Victor. Can you please provide puppet logs, murano logs and murano.conf?

Changed in mos:
importance: Undecided → High
no longer affects: mos
Revision history for this message
Victor Ryzhenkin (vryzhenkin) wrote :

Raised priority to critical for 9.0 and 10.0 due to GLARE is default package storage backend since MOS 9.0

description: updated
tags: added: area-puppet murano
summary: - Package import during MOS deploy silently fails
+ [Murano][GLARE] There is no Core Library in Murano if Murano using GLARE
+ backend
Revision history for this message
Ivan Berezovskiy (iberezovskiy) wrote :
Revision history for this message
Denis Egorenko (degorenko) wrote :

So, after digging into that problem we have two problems here:

1) upstream puppets don't use parameter 'packages_opts/packages_service'; that's why core library package was uploaded to murano-db - fixed by upstream patches 309139 for master and 315080 for mitaka;

2) when we want to use glare and upload package to glare backend we need to wait for Swift or RadosGW, so we need to patch fuel-library too.

The problem here is that we need depends on both object storage only when we have enabled Glare for Murano. I'm not sure, that current TBD in library has such possibility to have condition on cross-depends task. So,
i see 2 solutions: 1) always wait for Swift or RadosGW - if such task are present we always will wait till then will be finished; 2) Create separate task for uploading Murano core library package - like it done for uploading cirros image.

May be someone else has some ideas for improvement this situation?

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-library (master)

Fix proposed to branch: master
Review: https://review.openstack.org/315197

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Related fix proposed to puppet-modules/puppet-murano (mos-8.0)

Related fix proposed to branch: mos-8.0
Change author: Denis Egorenko <email address hidden>
Review: https://review.fuel-infra.org/20559

Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Related fix merged to puppet-modules/puppet-murano (mos-8.0)

Reviewed: https://review.fuel-infra.org/20559
Submitter: Ivan Berezovskiy <email address hidden>
Branch: mos-8.0

Commit: 4fa78a28259e2fdbdcaacdea38da2cc8279dfea2
Author: Denis Egorenko <email address hidden>
Date: Thu May 12 10:27:42 2016

Fix issue with importing packages with enabled Glare

Current provider is only work with standard storing packages in
murano database (package_service=murano by default) and doesn't
handling parameter package_service in get_murano_credentials function.
In situation when this parameter changed (for example to 'glance')
we can import murano package properly.

Also fix wrong regexp on checking existing packages.

As we have stable/8.0 upstream puppets under our custom tag hence
we need to backport this change too and bump then tag in library.

Also this patch will not be landed for stable/liberty in upstream
puppet-murano, because there is no support for Glare, only in library.

Change-Id: I7c8d207d278bf6ce6cddbb9e1448303bf1f88eb6
Closes-bug: #1572787
Related-bug: #1572785
(cherry picked from commit b36421130e4afcc55ff84c5bcf5d23ba07e2b361)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-library (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/315689

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-library (stable/8.0)

Fix proposed to branch: stable/8.0
Review: https://review.openstack.org/316008

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (master)

Reviewed: https://review.openstack.org/315197
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=cb516fbe546d3cd599ff068547005eb06dc9ea85
Submitter: Jenkins
Branch: master

commit cb516fbe546d3cd599ff068547005eb06dc9ea85
Author: Denis Egorenko <email address hidden>
Date: Wed May 11 21:18:21 2016 +0300

    Create separate task for uploading Murano package

    When we have enabled Glare for Murano, Core library package will
    be uploaded to current Glance backend - Swift or RadosGW - hence
    we must be sure that both already prepared and ready to use.

    Change-Id: If0a684aa9b0410e28a82b1501633c0ebdb64dc67
    Closes-bug: #1572785

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (stable/mitaka)

Reviewed: https://review.openstack.org/315689
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=5fe4a2daf1a115946909102fcf4ffb24690a3bec
Submitter: Jenkins
Branch: stable/mitaka

commit 5fe4a2daf1a115946909102fcf4ffb24690a3bec
Author: Denis Egorenko <email address hidden>
Date: Wed May 11 21:18:21 2016 +0300

    Create separate task for uploading Murano package

    When we have enabled Glare for Murano, Core library package will
    be uploaded to current Glance backend - Swift or RadosGW - hence
    we must be sure that both already prepared and ready to use.

    Change-Id: If0a684aa9b0410e28a82b1501633c0ebdb64dc67
    Closes-bug: #1572785
    (cherry picked from commit e8dd56f5f8ac639ebfef95c151225bc408ed2be2)

Revision history for this message
Alex Kholkin (akholkin) wrote :

Verified on #390 ISO
9.0-mos.all

tags: added: on-verification
Revision history for this message
Ekaterina Shutova (eshutova) wrote :
tags: removed: on-verification
Revision history for this message
Denis Meltsaykin (dmeltsaykin) wrote :

Won't fix for 8.0, since Glare is not the default storage backend in MOS 8.0.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on fuel-library (stable/8.0)

Change abandoned by Andreas Jaeger (<email address hidden>) on branch: stable/8.0
Review: https://review.opendev.org/316008
Reason: This repo is retired now, no further work will get merged.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.