In https://review.openstack.org/#/c/98043 we're going to preserve the hierarchy so that we can have multiple nova-conductor, etc, logs spread out across the various nodes we've test-deployed too, which will permit https://review.openstack.org/#/c/97947/2/toci_devtest.sh to have useful logs for F20 jobs in logstash. YAY.
However we still have the Ubuntu logs all tarred up which stops logstash seeing them.
We can fix this by untarring the logs, but a deep hierarchy will cause infra problems due to poor inode utilisation on the static server(s). This is complicated by us needing two hierarchies - /var/log and /mnt/state/var/log
Discussion with clarkb on IRC, we came up with:
/$instance/root containing /var/log and
/$instance/mnt containing /mnt/state/var/log
Which would avoid concerns on namespace clashes and eliminate 4 pointless inodes per node. We may find there are too many subdirs under /var/log/ etc and need to take further flattening work, but just rsyncing those two paths across (and compressing the contents of the hierarchy e.g. nova-conductor.log -> nova-conductor.log.gz) should give us a good starting point.
The log files from the ubuntu tar ball are already being extracted e.g. see
http://
and we're currently stripping out the /var/log/upstart path with tar
--strip-