Adventures in AppleRAID

Vincent Danen

December 07, 2013

Actually, it's not much of an adventure so much as a reminder. I'm used to doing software RAID on Linux with mdadm and am fully comfortable with the commandline tools -- in fact, I prefer them to any GUI tools that may or may not be out there (for example, if I were to have a choice of using mdadm on the commandline versus using Anaconda to setup a RAID array, I would take the commandline).

Anyways, I use Apple's software RAID on OS X as well. With mdadm I'm fully confident. With Apple RAID I'm not as much, primarily because I've not had the opportunity to put it through it's paces, and also because Disk Utility is absolutely horrendous for dealing with setting the RAID arrays up.

Anyways, I was unlucky enough in mid-2012 to buy a few of those Seagate drives that chirped. Harmless, once I realized the drives were not going to die, but annoying nonetheless. Suffice it to say, I've gotten used to it. Anyways, 3TB drives are $130CAD a piece now which is pretty cheap, and these drives have been running for the last year and a half pretty much non-stop. Between price, requiring some larger drives as offline archives (to which use I can put these chirpy drives), and really feeling like I should test the Apple RAID (last test I believe was in 10.7 and now the system is running 10.9), I figured it was a good time to give it a go.

Simple procedure: shut down the computer, pull out one drive, pop in a new drive, fire it back up. Because I have a RAID0+1 (so three RAID arrays, two RAID mirrors and one RAID stripe), I could pull two drives, one from each mirror array, and rebuild two at a time but I prefer to be cautious.

Upon bootup I get the "you've inserted an uninitialized disk" notice (to which I tell it to ignore the drive), get a happy degraded RAID warning from SMARTReporter, and am ready to get it back up and running. The required commands are surprisingly simple:

% sudo diskutil list
Password:
/dev/disk0
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *512.1 GB   disk0
   1:                        EFI EFI                     209.7 MB   disk0s1
   2:                  Apple_HFS fenris                  511.3 GB   disk0s2
   3:                 Apple_Boot Recovery HD             650.0 MB   disk0s3
/dev/disk1
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:                                                   *3.0 TB     disk1
/dev/disk2
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *3.0 TB     disk2
   1:                        EFI EFI                     209.7 MB   disk2s1
   2:                 Apple_RAID                         3.0 TB     disk2s2
   3:                 Apple_Boot Boot OS X               134.2 MB   disk2s3
/dev/disk3
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *3.0 TB     disk3
   1:                        EFI EFI                     209.7 MB   disk3s1
   2:                 Apple_RAID                         3.0 TB     disk3s2
   3:                 Apple_Boot Boot OS X               134.2 MB   disk3s3
/dev/disk4
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *3.0 TB     disk4
   1:                        EFI EFI                     209.7 MB   disk4s1
   2:                 Apple_RAID                         3.0 TB     disk4s2
   3:                 Apple_Boot Boot OS X               134.2 MB   disk4s3
/dev/disk5
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *3.0 TB     disk5
   1:                        EFI EFI                     209.7 MB   disk5s1
   2:                  Apple_HFS G-DRIVE                 3.0 TB     disk5s2
/dev/disk6
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:                  Apple_HFS Stuff                  *6.0 TB     disk6

In this case, we've removed the first physical drive in the Mac Pro's four bays (I have an SSD as a boot device in there as disk0), so it's disk1 we're interested in. Make a note of the disk identifier (the one with no partitions on it).

Next we want to see what our RAID information looks like because we need to know which array this drive needs to be added to, and we need to know the unique identifer (UUID) of the RAID array:

% sudo diskutil appleraid list
AppleRAID sets (3 found)
===============================================================================
Name:                 Stuff_1
Unique ID:            A50AAB2A-D2DE-44DD-BAF0-7577072C5937
Type:                 Mirror
Status:               Online
Size:                 3.0 TB (3000248958976 Bytes)
Rebuild:              automatic
Device Node:          -
-------------------------------------------------------------------------------
#  DevNode   UUID                                  Status     Size
-------------------------------------------------------------------------------
0  disk2s2   3BF8BC8C-CE09-447C-8A86-ED51E8515C3E  Online     3000248958976
1  disk3s2   A117C08C-B375-469C-A654-EAA902E5A317  Online     3000248958976
===============================================================================
===============================================================================
Name:                 Stuff
Unique ID:            F2429656-695B-4F8C-B870-B7334DC3340D
Type:                 Stripe
Status:               Online
Size:                 6.0 TB (6000497786880 Bytes)
Rebuild:              manual
Device Node:          disk6
-------------------------------------------------------------------------------
#  DevNode   UUID                                  Status     Size
-------------------------------------------------------------------------------
0  -none-    C96F3052-8EBD-45D5-9A25-29EE23957CA9  Online     3000248893440
1  -none-    A50AAB2A-D2DE-44DD-BAF0-7577072C5937  Online     3000248893440
===============================================================================
===============================================================================
Name:                 Stuff_2
Unique ID:            C96F3052-8EBD-45D5-9A25-29EE23957CA9
Type:                 Mirror
Status:               Degraded
Size:                 3.0 TB (3000248958976 Bytes)
Rebuild:              automatic
Device Node:          -
-------------------------------------------------------------------------------
#  DevNode   UUID                                  Status     Size
-------------------------------------------------------------------------------
-  -none-    F3921C5D-0B94-4264-9619-0291A5147F4E  Missing/Damaged
1  disk4s2   12511D3B-A21C-4BB7-B68B-DAA5E2AC0277  Online     3000248958976
===============================================================================

There's a few bits of info here. The first is the uuid of our RAID mirror that it's in a degraded state (the RAID volume name is Stuff_2, above, with the uuid noted below the name). The second bit of information we need is the uuid of the failed drive (in this case, the drive I removed). We will need to remove this missing device from the array after we add the new one.

% sudo diskutil appleraid add member disk1 C96F3052-8EBD-45D5-9A25-29EE23957CA9
Started RAID operation
Unmounting disk
Repartitioning disk1 so it can be in a RAID set
Unmounting disk
Creating the partition map
Adding disk1s2 to the RAID Set
Finished RAID operation
% sudo diskutil appleraid list
AppleRAID sets (3 found)
===============================================================================
Name:                 Stuff_1
Unique ID:            A50AAB2A-D2DE-44DD-BAF0-7577072C5937
Type:                 Mirror
Status:               Online
Size:                 3.0 TB (3000248958976 Bytes)
Rebuild:              automatic
Device Node:          -
-------------------------------------------------------------------------------
#  DevNode   UUID                                  Status     Size
-------------------------------------------------------------------------------
0  disk2s2   3BF8BC8C-CE09-447C-8A86-ED51E8515C3E  Online     3000248958976
1  disk3s2   A117C08C-B375-469C-A654-EAA902E5A317  Online     3000248958976
===============================================================================
===============================================================================
Name:                 Stuff
Unique ID:            F2429656-695B-4F8C-B870-B7334DC3340D
Type:                 Stripe
Status:               Online
Size:                 6.0 TB (6000497786880 Bytes)
Rebuild:              manual
Device Node:          disk6
-------------------------------------------------------------------------------
#  DevNode   UUID                                  Status     Size
-------------------------------------------------------------------------------
0  -none-    C96F3052-8EBD-45D5-9A25-29EE23957CA9  Online     3000248893440
1  -none-    A50AAB2A-D2DE-44DD-BAF0-7577072C5937  Online     3000248893440
===============================================================================
===============================================================================
Name:                 Stuff_2
Unique ID:            C96F3052-8EBD-45D5-9A25-29EE23957CA9
Type:                 Mirror
Status:               Degraded
Size:                 3.0 TB (3000248958976 Bytes)
Rebuild:              automatic
Device Node:          -
-------------------------------------------------------------------------------
#  DevNode   UUID                                  Status     Size
-------------------------------------------------------------------------------
-  -none-    F3921C5D-0B94-4264-9619-0291A5147F4E  Missing/Damaged
1  disk4s2   12511D3B-A21C-4BB7-B68B-DAA5E2AC0277  Online     3000248958976
2  disk1s2   6A381F4D-759C-4CFB-BB38-1EA22C7D58B8  0% (Rebuilding)3000248958976
===============================================================================

Or, sudo diskutil appleraid add member disk# [uuid-of-raid] As you can see from the above, we added the new drive (disk1) to the RAID array named Stuff_2. And after we did so, performed another list operation, and you can see three drives associated with that array -- one is online, one is missing, and one is rebuilding. Now we need to remove the missing one as we won't be adding it back:

% sudo diskutil appleraid remove F3921C5D-0B94-4264-9619-0291A5147F4E C96F3052-8EBD-45D5-9A25-29EE23957CA9
Started RAID operation
Removing disk from RAID
Finished RAID operation

Or, sudo diskutil appleraid remove [uuid-of-failed-drive] [uuid-of-raid].

And that's it. Honestly, it took me longer to type this out than it did to rebuild the RAID array. Sadly, in that time it's only progressed to 3% rebuilt. The downside of such large drives is the reconstruction time is a little ridiculous. Anyways, once you've run these commands in Terminal, if you were to fire up Disk Utility, you'd see a nice little GUI progress bar that, for me, indicates it will take 4hrs to rebuild.

Honestly, for anyone who has fought with Disk Utility to manage anything RAID in OS X, you'll appreciate how simple the diskutil commandline tool makes things.

Leave a Comment

Comments use MarkDown. Need help? MarkDown Cheatsheet