[Glance] Enabled 'show_image_direct_url' by default in glance-api.conf

Thanks Alexey!

In upstream this parameter was always set to False https://review.openstack.org/#/c/11040/ and I don't know why it was required to hardcode it to True.

It affects all Mirantis versions since 5.1 release. Prior python-glanceclient 1.0.0 user had to use v2 api (with option --os-image-api-version 2) to see direct url, but with all latest versions v2 has become the default version.

I made a fix for this bug, but I'm not sure that it is complete, it is essential that this would fix checked MOS-puppet team.

Alexey, before just changing this setting we need to understand why it was set to true. Looks like that was done on purpose and come components are using (or used) that.

I just spoke to Alex Didenko who committed https://review.openstack.org/#/c/11040/. According to him that CR was a part of synching fuel puppet manifests with upstream manifests. So actually setting show_image_direct_url to True was not introduced by that CR but was actually present in Fuel versions prior to 5.0. See for example https://github.com/stackforge/fuel-library/blob/stable/4.1/deployment/puppet/ceph/manifests/init.pp#L64.

So it looks like ceph needs or needed show_image_direct_url parameter set to True. We need to work with Ceph team and figure out if we could change this setting.

But based on the fact that this behavior is present in all prior Fuel versions starting 4.1 I am changing the priority to High.

Feedback from ceph team:

The parameter is needed for copy-on-write cloning of images. I.e. if a volume is created from an image, with COW enabled this process is very fast, because it just creates a snapshot from image pool to volumes pool. Without this feature enabled, it will download the the whole image file from 'images' pool and then upload to 'volumes' pool. I suppose users wouldn't be happy in this case.

Ok, but if users using swift service their cloud can be hacked. I'am sure that users wouldn't be happy too. If you want, I can provide an exploit using the vulnerability that would demonstrate the dangers of using this configuration. And I have a question to the MOS Puppet team: It is possible to use different values of this parameter, depending on the type of backend during enviroment deploy?

In any event, intentional disregard for such vulnerabilities (EoP) in the product - a very bad thing.

We are not disregarding, we are looking for a smart way to fix it, I mean trying to fix without performance/functional degradation

Ok, i want to find a better solution too, but I'am sure that security is more important than performance.

Actual fix for swift without ceph functional degradation

[UPDATED] Actual fix for swift without ceph functional degradation.
For master branch in fuel-library.

Ok, here is the final conclusion based on today investigation:

1. Ceph creates direct utl in the form rbd://cluster-id/images/image-id/snap, for example rbd://97e39c49-9528-4f1a-bcf9-b5ec60d0fd1d/images/c3de19f7-691e-4ed2-825f-fca1ddd9669d/snap, user could just read the image using this URL and could not do any harm like deleting or reading other images. So deployments backed by ceph are not affected by this issue.

2. With Swift it's possible to create temporary URL that doesn't contain glance user name and password and set the TTL to infinity. That would fix the security issue but requires additional testing and most likely changes to glance.

So the recommendation for 7.0 is to disable show_image_direct_url for Swift-backed deployments (see the patch proposed by Alexey Galkin above). The cons of this solution is 2x slower copying of images (it will be performed through glance but not directly).

My last fix set 'show_image_direct_url' = False only with Swift-backed deployments. In this way if we using Ceph-backend then option 'show_image_direct_url' have a 'True' value. Are you sure that in this case will be a loss performance?

Update fix with considering notes from MOS Puppet Team.

Patch looks good, but I have suggestion to improve noop tests:
http://paste.openstack.org/show/473846/

Fixes are on review https://review.openstack.org/#/c/227206/ and https://review.openstack.org/#/c/227213/

I'm not sure it's a good idea - to post this fix on review at now... but now nothing can be changed. I tested this fix (deployed swift without ceph), and now launched the Tempest tests to make sure that the cloud operating normally

Tempest passed tests without any errors. Add to subscribers Mike Fedosin (developer from MOS Glance team), probably it will soon be able to offer another fix for glance (without make changes in puppet scripts).

Found a new bug on the basis of current: https://bugs.launchpad.net/mos/+bug/1499754 (This bug will have a private status until security bug will not be on the public)

Here's the patch that fixes all urls by changing their scheme from 'swift+http(s)' to 'swift+config'.

Usage:
1. Get admin token and insert it in the code to OS_AUTH_TOKEN variable.
2. Set correct url and port of glance-registry server to OS_REGISTRY_HOST and OS_REGISTRY_PORT, respectively.
3. Run the script with 'python fix_urls.py'

Targeted to 7.0-mu-1 as security issue. The fix for stable/7.0 is https://review.openstack.org/#/c/227213/, it has been merged just now.

Set to Won't Fix for 5.1.1-updates and 6.0-updates as there is no way to deliver Fuel fixes in maintenance updates for 5.1.1 and 6.0. To fix the issue on existing environments one could follow instruction in tech bulletin we published - http://content.mirantis.com/rs/451-RBY-185/images/Mirantis-Technical-Bulletin-22.pdf

Fix was verified on my QA environment with MOS 7.0 MU1,

Steps To Verify:
1. Login to controller node.
2. Execute the following command: 'cat /etc/glance/glance-api.conf | grep show_image_direct_url'

Observed Result:
root@node-16:~# cat /etc/glance/glance-api.conf | grep show_image_direct_url
#show_image_direct_url = false
show_image_direct_url = True
# show_image_direct_url. (boolean value)
root@node-16:~#

verified on:

    VERSION:
      api: '1.0'
      astute_sha: 959b06c5ef8143125efd1727d350c050a922eb12
      build_id: '168'
      build_number: '168'
      feature_groups:
      - mirantis
      fuel-agent_sha: 2750600e946e96701099dfef5a7d69017f2e9956
      fuel-createmirror_sha: e34a0dd080fe6c133bcc75a00c31b27934ba51bc
      fuel-library_sha: 0d210dc3851ac6c74b3914ef4df0139defc8d117
      fuel-nailgun-agent_sha: 3e9d17211d65c80bf97c8d83979979f6c7feb687
      fuel-nailgun_sha: a5f4c44d08715f3be6b200a69032b30d347ac911
      fuel-ostf_sha: 9690a2de829d3b063ed1e64b0b10dde39f711dc0
      fuel-upgrade_sha: 1e894e26d4e1423a9b0d66abd6a79505f4175ff6
      fuelmain_sha: 266f9b374934c64629a84b5146632cc0de70ac91
      fuelmenu_sha: 06bbcebed6c8d0b0f9279e2997d2f958c800e98c
      network-checker_sha: a57e1d69acb5e765eb22cab0251c589cd76f51da
      openstack_version: 2015.1.0-8.0
      production: docker
      python-fuelclient_sha: e685d68c1c0d0fa0491a250f07d9c3a8d0f9608c
      release: '8.0'
      shotgun_sha: 25dd78a3118267e3616df0727ce746e7dead2d67

Steps to verify:

1) Deploy env with swift backend.
2) Execute the following command: 'cat /etc/glance/glance-api.conf | grep show_image_direct_url'

Excepted result:
#show_image_direct_url = false
show_image_direct_url = False
# show_image_direct_url. (boolean value)

Actual:
root@node-5:~# cat /etc/glance/glance-api.conf | grep show_image_direct_url
#show_image_direct_url = false
show_image_direct_url = False
# show_image_direct_url. (boolean value)

3) Use this command 'glance image-show $(glance image-list | grep TestVM |awk {' print $2 '})'

Excepted result:

response should not contain links form: swift+http://services%3Aglance*

Actual:

root@node-5:~# glance image-show $(glance image-list | grep TestVM |awk {' print $2 '}) | grep direct_url
/usr/lib/python2.7/dist-packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
/usr/lib/python2.7/dist-packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning

Set this bug's status to Won't fix (wontfix-munotapplic) for 5.0 branch since we don't provide MU for 5.0 branch

Here is the CR for 6.1 https://review.openstack.org/#/c/287978/

Test cases are automated:
https://mirantis.testrail.com/index.php?/cases/view/836636
https://mirantis.testrail.com/index.php?/cases/view/843822

Verified on MOS 6.1 + mu6 updates.

Used scenario described in comment #24.