Make FileStore layout easier for users, devs, and admins
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| Dmedia |
Triaged
|
High
|
Unassigned | ||
Bug Description
So this is about making some small changes to the layout of files and directories inside the ".dmedia" control directory.
Currently there are 3 directories that get created for temporary files:
imports/
writes/
transfers/
And a directory where a corrupted file is moved to if FileStore.verify() detects corruption:
corrupted/
And then there are 1024 directories named with the first 2 characters of a file's ID (content-hash):
2A/
2B/
2C/
And so on. The problem is these 1024 directories are sort of overwhelming in the top-level control directory. It also makes it difficult to grant read access to just the canonically named files (say when serving with Apache). There is no reason to give read access to the temporary files, so you don't want to give access to all of .dmedia/
So taking some inspiration from the .bzr control directory, this is going to be the new layout:
corrupt/
files/
partial/
tmp/
README.txt
That's all that will be in the top level control directory. Not too overwhelming, plus a handy README.txt to help users figure out what the hell this ".dmedia" is... hopefully before, say, they accidentally delete it. "transfers" is being renamed to "partial" (clearer, I think), and "writes" and "imports" are being consolidated into "tmp".
The 1024 directories and the files inside them will all be inside the files/ directory, like this:
files/2A/
files/2B/
files/2C/
And so on. This is important because now you can allow access to .dmedia/files/ without granting access to any of the temporary files. This has already been done in the new standalone filestore:
http://
By the way, if anyone has wondered, the reason there has been less activity in dmedia lately is because I've been working on the standalone filestore in order to get the hashing protocol reviewed and finalized as quickly as possible. The new filestore is currently Python3 only (because PySkein is Python3 only). dmedia will switch to the standalone filestore when its ready, but that also means we either need to port dmedia to Python3 (currently held back by PyGI), or port PySkein to Python2 (probably easier).
