mdadm assembled the other superblock thus failing to boot

Bug #1164008 reported by Austin
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
mdadm (Ubuntu)
Confirmed
Low
Unassigned

Bug Description

I just upgraded a working 10.04 LTS server to 12.04 LTS. The server has two non-boot drives in a software RAID1 configuration. This array worked fine in 10.04 LTS.

When the machine boots I should get one active array, /dev/md0 consisting of /dev/sdb1 and /dev/sdc1. Instead, I get dumped to a busybox session and after quitting that and booting I get two inactive arrays: md0 (consisting of /dev/sdc1 as spare) and md127 (consisting of /dev/sdb as spare). If I run mdadm --detail /dev/md0 or mdadm --detail /dev/md127 I get that the device does not appear to be active.

Once I've finished booting, I can assemble my array manually by running mdadm --stop /dev/md0 and mdadm --stop /dev/md127 and then assembling the array with mdadm --assemble /dev/md0 /dev/sdb1 /dev/sdc1.

Below are the results of some troubleshooting:
$ lsb_release -rd
Description: Ubuntu 12.04.2 LTS
Release: 12.04

$ apt-cache policy mdadm
mdadm:
  Installed: 3.2.5-1ubuntu0.2
  Candidate: 3.2.5-1ubuntu0.2
  Version table:
 *** 3.2.5-1ubuntu0.2 0
        500 http://us.archive.ubuntu.com/ubuntu/ precise-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     3.2.3-2ubuntu1 0
        500 http://us.archive.ubuntu.com/ubuntu/ precise/main amd64 Packages

$ cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : inactive sdc1[1](S)
      1953513408 blocks

md127 : inactive sdb[2](S)
      1953514496 blocks

unused devices: <none>

$ mdadm --examine /dev/sdb1
/dev/sdb1:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : a4176cee:096f0ba6:40663db4:2b8b6d2c
  Creation Time : Thu Jul 7 09:32:01 2011
     Raid Level : raid1
  Used Dev Size : 1953513408 (1863.02 GiB 2000.40 GB)
     Array Size : 1953513408 (1863.02 GiB 2000.40 GB)
   Raid Devices : 2
  Total Devices : 2
Preferred Minor : 0

    Update Time : Wed Apr 3 10:32:36 2013
          State : clean
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0
       Checksum : d685ab7e - correct
         Events : 722

      Number Major Minor RaidDevice State
this 0 8 17 0 active sync /dev/sdb1

   0 0 8 17 0 active sync /dev/sdb1
   1 1 8 33 1 active sync /dev/sdc1

$ mdadm --examine /dev/sdc1
/dev/sdc1:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : a4176cee:096f0ba6:40663db4:2b8b6d2c
  Creation Time : Thu Jul 7 09:32:01 2011
     Raid Level : raid1
  Used Dev Size : 1953513408 (1863.02 GiB 2000.40 GB)
     Array Size : 1953513408 (1863.02 GiB 2000.40 GB)
   Raid Devices : 2
  Total Devices : 2
Preferred Minor : 0

    Update Time : Wed Apr 3 10:32:36 2013
          State : clean
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0
       Checksum : d685ab90 - correct
         Events : 722

      Number Major Minor RaidDevice State
this 1 8 33 1 active sync /dev/sdc1

   0 0 8 17 0 active sync /dev/sdb1
   1 1 8 33 1 active sync /dev/sdc1

Revision history for this message
Austin (ninjamonic) wrote :

I'm digging into this some more and it seems that mdadm in Ubuntu 12.04 is detecting an old unused superblock on /dev/sdb that mdadm in Ubuntu 10.04 ignored.

$ mdadm --examine --scan --verbose
ARRAY /dev/md1 level=raid1 num-devices=2 UUID=5fb6928e:89f09cde:d0aebb6f:e3c24c31
   spares=1 devices=/dev/sdb
ARRAY /dev/md0 level=raid1 num-devices=2 UUID=a4176cee:096f0ba6:40663db4:2b8b6d2c
   devices=/dev/sdc1,/dev/sdb1

When we examine /dev/sdb, we see there's an old superblock there (note the update date is Thu Jul 7 09:00:21 2011).

$ mdadm --examine /dev/sdb
/dev/sdb:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : 5fb6928e:89f09cde:d0aebb6f:e3c24c31
  Creation Time : Fri Jul 1 10:00:58 2011
     Raid Level : raid1
  Used Dev Size : 1953513408 (1863.02 GiB 2000.40 GB)
     Array Size : 1953513408 (1863.02 GiB 2000.40 GB)
   Raid Devices : 2
  Total Devices : 2
Preferred Minor : 1

    Update Time : Thu Jul 7 09:00:21 2011
          State : clean
 Active Devices : 1
Working Devices : 2
 Failed Devices : 1
  Spare Devices : 1
       Checksum : 57d747d3 - correct
         Events : 24

      Number Major Minor RaidDevice State
this 2 8 16 2 spare /dev/sdb

   0 0 8 33 0 active sync /dev/sdc1
   1 1 0 0 1 faulty removed
   2 2 8 16 2 spare /dev/sdb

Revision history for this message
Austin (ninjamonic) wrote :

It looks like zeroing the superblock on /dev/sdb with mdadm --zero-superblock /dev/sdb fixed the issue with the array not assembling on boot.

Revision history for this message
Dimitri John Ledkov (xnox) wrote :

I'm glad it's fixed for you, but I do wonder how many other people hit this issue.

summary: - mdadm devices not recognized correctly at boot
+ mdadm assembled the other superblock thus failing to boot
Changed in mdadm (Ubuntu):
status: New → Confirmed
importance: Undecided → Low
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.