Comment 2 for bug 1451947

Revision history for this message
David Britton (dpb) wrote :

http://paste.ubuntu.com/10991033/

unit-ceph-2[32498]: 2015-05-05 17:16:23 INFO unit.ceph/2.mon-relation-changed logger.go:40 #007Caution: invalid backup GPT header, but valid main header; regenerating
unit-ceph-2[32498]: 2015-05-05 17:16:23 INFO unit.ceph/2.mon-relation-changed logger.go:40 backup header from main header.
unit-ceph-2[32498]: 2015-05-05 17:16:23 INFO unit.ceph/2.mon-relation-changed logger.go:40
unit-ceph-2[32498]: 2015-05-05 17:16:23 INFO unit.ceph/2.mon-relation-changed logger.go:40 Warning! Main and backup partition tables differ! Use the 'c' and 'e' options
unit-ceph-2[32498]: 2015-05-05 17:16:23 INFO unit.ceph/2.mon-relation-changed logger.go:40 on the recovery & transformation menu to examine the two tables.
unit-ceph-2[32498]: 2015-05-05 17:16:23 INFO unit.ceph/2.mon-relation-changed logger.go:40
unit-ceph-2[32498]: 2015-05-05 17:16:23 INFO unit.ceph/2.mon-relation-changed logger.go:40 Warning! One or more CRCs don't match. You should repair the disk!
unit-ceph-2[32498]: 2015-05-05 17:16:23 INFO unit.ceph/2.mon-relation-changed logger.go:40
unit-ceph-1[32533]: 2015-05-05 17:16:24 INFO unit.ceph/1.mon-relation-changed logger.go:40 #015Building dependency tree... 0%#015#015Building dependency tree... 0%#015#015Building dependency tree... 50%#015#015Building dependency tree... 50%#015#015Building dependency tree
unit-ceph-1[32533]: 2015-05-05 17:16:24 INFO unit.ceph/1.mon-relation-changed logger.go:40 #015Reading state information... 0%#015#015Reading state information... 0%#015#015Reading state information... Done
unit-ceph-1[32533]: 2015-05-05 17:16:24 INFO unit.ceph/1.mon-relation-changed logger.go:40 Traceback (most recent call last):
unit-ceph-1[32533]: 2015-05-05 17:16:24 INFO unit.ceph/1.mon-relation-changed logger.go:40 File "/usr/sbin/ceph-disk", line 2768, in <module>
unit-ceph-1[32533]: 2015-05-05 17:16:24 INFO unit.ceph/1.mon-relation-changed logger.go:40 main()
unit-ceph-1[32533]: 2015-05-05 17:16:24 INFO unit.ceph/1.mon-relation-changed logger.go:40 File "/usr/sbin/ceph-disk", line 2746, in main
unit-ceph-1[32533]: 2015-05-05 17:16:24 INFO unit.ceph/1.mon-relation-changed logger.go:40 args.func(args)
unit-ceph-1[32533]: 2015-05-05 17:16:24 INFO unit.ceph/1.mon-relation-changed logger.go:40 File "/usr/sbin/ceph-disk", line 1390, in main_prepare
unit-ceph-1[32533]: 2015-05-05 17:16:24 INFO unit.ceph/1.mon-relation-changed logger.go:40 zap(args.data)
unit-ceph-1[32533]: 2015-05-05 17:16:24 INFO unit.ceph/1.mon-relation-changed logger.go:40 File "/usr/sbin/ceph-disk", line 1013, in zap
unit-ceph-1[32533]: 2015-05-05 17:16:24 INFO unit.ceph/1.mon-relation-changed logger.go:40 with file(dev, 'wb') as dev_file:
unit-ceph-1[32533]: 2015-05-05 17:16:24 INFO unit.ceph/1.mon-relation-changed logger.go:40 IOError: [Errno 123] No medium found: '/dev/sdb'
unit-ceph-1[32533]: 2015-05-05 17:16:24 ERROR unit.ceph/1.juju-log cmd.go:247 mon:3: Unable to initialize device: /dev/sdb
unit-ceph-1[32533]: 2015-05-05 17:16:24 INFO unit.ceph/1.mon-relation-changed logger.go:40 Traceback (most recent call last):
unit-ceph-1[32533]: 2015-05-05 17:16:24 INFO unit.ceph/1.mon-relation-changed logger.go:40 File "/var/lib/juju/agents/unit-ceph-1/charm/hooks/mon-relation-changed", line 312, in <module>
unit-ceph-1[32533]: 2015-05-05 17:16:24 INFO unit.ceph/1.mon-relation-changed logger.go:40 hooks.execute(sys.argv)
unit-ceph-1[32533]: 2015-05-05 17:16:24 INFO unit.ceph/1.mon-relation-changed logger.go:40 File "/var/lib/juju/agents/unit-ceph-1/charm/hooks/charmhelpers/core/hookenv.py", line 508, in execute
unit-ceph-1[32533]: 2015-05-05 17:16:24 INFO unit.ceph/1.mon-relation-changed logger.go:40 self._hooks[hook_name]()
unit-ceph-1[32533]: 2015-05-05 17:16:24 INFO unit.ceph/1.mon-relation-changed logger.go:40 File "/var/lib/juju/agents/unit-ceph-1/charm/hooks/mon-relation-changed", line 196, in mon_relation
unit-ceph-1[32533]: 2015-05-05 17:16:24 INFO unit.ceph/1.mon-relation-changed logger.go:40 reformat_osd(), config('ignore-device-errors'))
unit-ceph-1[32533]: 2015-05-05 17:16:24 INFO unit.ceph/1.mon-relation-changed logger.go:40 File "/var/lib/juju/agents/unit-ceph-1/charm/hooks/ceph.py", line 327, in osdize
unit-ceph-1[32533]: 2015-05-05 17:16:24 INFO unit.ceph/1.mon-relation-changed logger.go:40 osdize_dev(dev, osd_format, osd_journal, reformat_osd, ignore_errors)
unit-ceph-1[32533]: 2015-05-05 17:16:24 INFO unit.ceph/1.mon-relation-changed logger.go:40 File "/var/lib/juju/agents/unit-ceph-1/charm/hooks/ceph.py", line 375, in osdize_dev
unit-ceph-1[32533]: 2015-05-05 17:16:24 INFO unit.ceph/1.mon-relation-changed logger.go:40 raise e
unit-ceph-1[32533]: 2015-05-05 17:16:24 INFO unit.ceph/1.mon-relation-changed logger.go:40 subprocess.CalledProcessError: Command '['ceph-disk-prepare', '--fs-type', u'xfs', '--zap-disk', u'/dev/sdb']' returned non-zero exit status 1
unit-ceph-1[32533]: 2015-05-05 17:16:24 ERROR juju.worker.uniter.operation runhook.go:86 hook "mon-relation-changed" failed: exit status 1

This looks like the operative failure. I'd suggest looking into possible hardware failures on that disk. I'll add ceph on here so perhaps the maintainers of that project could have a chance to sound in.