SLO multipart-manifest=put ignored with temp-url

Bug #1553714 reported by Stephen von Takach
14
This bug affects 3 people
Affects Status Importance Assigned to Milestone
OpenStack Object Storage (swift)
In Progress
Medium
Unassigned

Bug Description

Attempting to create a static large object with a temp url writes the multipart manifest to the file instead of creating the SLO.

Example request
https://storage101.dfw1.clouddrive.com/v1/MossoCloudFS_abf330f5-5f4e-48be-9993-b5dd073bf525/acasignage/14572570233147368240.mp4?multipart-manifest=put&temp_url_sig=2904fc2a6f7b036ec30126eeb7aac509defa0f7f&temp_url_expires=1457257390

Does not create a SLO.
Is there any reason this should not work?
I'm aware dynamic large objects are a security risk and no longer work with temp urls however this would be useful

Occurring on Swift 2.3.0 and whatever Rackspace is running (2.4.0+)

Revision history for this message
Rahul U Nair (rahulunair) wrote :

Could you post the logs while you try to create the object.

Changed in swift:
status: New → Incomplete
Revision history for this message
Stephen von Takach (speve) wrote :

Please see the request screen shot included.

This is the final request details

Request URL:https://storage101.dfw1.clouddrive.com/v1/MossoCloudFS_abf330f5-5f4e-48be-9993-b5dd073bf525/acasignage/1457395228580204742.mp4?multipart-manifest=put&temp_url_sig=55b03bef4f28ea2185974cb41db6deee4adaab35&temp_url_expires=1457395641
Request Method:PUT
Status Code:201 Created
Remote Address:174.143.184.158:443

Response Headers
---------------------------
Access-Control-Allow-Origin:http://localhost:9000
Access-Control-Expose-Headers:cache-control, content-language, content-type, expires, last-modified, pragma, etag, x-timestamp, x-trans-id, etag, location, x-timestamp, x-trans-id
Content-Length:0
Content-Type:text/html; charset=UTF-8
Date:Tue, 08 Mar 2016 00:02:25 GMT
Etag:65d574d00e3b8cce5df47fbe345217dc
Last-Modified:Tue, 08 Mar 2016 00:02:23 GMT
X-Trans-Id:tx7399500ede2644adb98b8-0056de168edfw1

Request Headers
-------------------------
Accept:*/*
Accept-Encoding:gzip, deflate, sdch
Accept-Language:en-AU,en;q=0.8,en-US;q=0.6,de;q=0.4
Connection:keep-alive
Content-Length:337
Content-Type:text/plain
Host:storage101.dfw1.clouddrive.com
Origin:http://localhost:9000
Referer:http://localhost:9000/about
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.116 Safari/537.36

Query String Parameters
-------------------------------------
multipart-manifest:put
temp_url_sig:55b03bef4f28ea2185974cb41db6deee4adaab35
temp_url_expires:1457395641

Request Payload
-------------------------
[{"path":"acasignage/1457395228580204742.mp4/p1","etag":"8f397b9f136b8ba3156d663a01fb1b40","size_bytes":2097152},{"path":"acasignage/1457395228580204742.mp4/p2","etag":"4c166259ccb2e7d47a9cfe8747798ddc","size_bytes":2097152},{"path":"acasignage/1457395228580204742.mp4/p3","etag":"8ca2b97af9d4b23e96f37a9d5944d527","size_bytes":1951786}]

I'm waiting on the logs from Rackspace. They take up to 24 hours to appear.
I do all my testing on Rackspace, however we have a number of clients using OpenStack internally that experience this same issue.

Revision history for this message
Stephen von Takach (speve) wrote :

So these are the logs when the above request occurs:

103.224.210.78 - - [08/03/2016:10:53:59 +0000] "OPTIONS /v1/MossoCloudFS_abf330f5-5f4e-48be-9993-b5dd073bf525/acasignage/1457434318335865968.mp4?multipart-manifest=put&temp_url_sig=c2909b4168622c4d52d29e804986a538479f98b5&temp_url_expires=1457434739 HTTP/1.0" 200 0 "http://localhost:9000/about" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.116 Safari/537.36"

103.224.210.78 - - [08/03/2016:10:54:00 +0000] "PUT /v1/MossoCloudFS_abf330f5-5f4e-48be-9993-b5dd073bf525/acasignage/1457434318335865968.mp4?temp_url_expires=1457434739&temp_url_sig=c2909b4168622c4d52d29e804986a538479f98b5 HTTP/1.0" 201 0 "http://localhost:9000/about" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.116 Safari/537.36"

As you can see the multipart-manifest=put is silently removed from the PUT request - however as you can see from the pre-flight options request, and the request above, the multipart-manifest=put is being sent.

Changed in swift:
status: Incomplete → New
Revision history for this message
Christopher Bartz (bartz) wrote :

The tempurl middleware does not preserve query params, thus the multipart-manifest query param is removed from the pipeline.

See https://review.openstack.org/#/c/333331/ .

Changed in swift:
status: New → Confirmed
importance: Undecided → Medium
Revision history for this message
François Leurent (131-ubuntu) wrote :

I'm glad this ticket is easy to find. I'm here after googling "create static large object using tempurl". Is there any update here ?

Revision history for this message
clayg (clay-gerrard) wrote :

This script duplicates the issue in a swift development environment:

https://gist.github.com/clayg/73d08f73c92076046d39b3cce74be3a3

summary: - static large object creation with temp url
+ SLO multipart-manifest=put ignored with temp-url
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.