I've set up a RedHat server with RAID1, i.e. 2 mirrored drives (640Gb SATA), is it possible to "swap out" a drive weekly as a complete backup? I'm thinking something like:

D0: Always stays in the server
D1: Swapped out and replaced by D2
D2: Swapped out and replaced by D3
D3: Swapped out and replaced by D1

We've got it replicating (mySQL db) to an offsite server but we'd like to have a fairly current complete mirror of the server in the fireproof locally.

Hot pulling a drive in a RAID 1 is not a guaranteed way of ensuring the backup is good. If you pull it while unwritten data is still in memory, or is in the middle of a write, the backup will likely be bad. At a minimum, run the sync command to flush the cache.

Swapping the drives after the OS has been shut down should work fine.

RAID is not generally a backup solution, it's protection from downtime in case of a drive failure.

Your RAID would be working all the time if you did that resychronizing and such. Probably wouldn't be good to always have your RAID status as "Degraded" in a production server. Use the extra drives as backup drives.

Points taken, if I power down and then pull the rebuild of the newly inserted disk should happen automatically shouldn't it?

OK. So looks like a bad idea, except as an occasional operation.

I've had issues even trying the technique as an occasional operation. The raid array never accepted the old hard drive back as one of it's own, and constantly complained. Eventually I beat it into submission, but it isn't right.

If that's the case what's the point of being able to replace a drive?

It's perfectly doable, but the common distros do not include any kind of reasonable front-end software to issue the lower-level RAID commands for you. If you study the manpage for mdadm, then you can work them out for yourself, but there's still some silliness with /sys that's not at all obvious to most folks.

I have written RAID-1 auto-repair scripts (for business clients) that truly make this a simple matter of unplug old drive and plug-in new drive. Works great.

But this is not something that most distro's would want, because it could easily result in somebody's USB/eSATA drive getting erased on insertion and made into a RAID member instead of whatever the user might have intended. That's why it's not an automatic thing.

EDIT: RAID appliances generally do have automation like that, but the scripts for it are often proprietary to the vendor.

Cheers

That's mostly over the top of my head, but I think I get the gist of it. You're talking about a script running in the OS that would rebuild the RAID?

Whilst the mirror is repairing itself, you won't have any redundancy and the performance will be bad. That okay?

Yeah, depending on how long it takes. We're a 6 day a week business with hardly any transactions done on day 6 (Sunday), so we could swap drives Friday night I guess.

The "rebuild" part is trivial and automatic: it's part of the in-kernel RAID code. The script, on the other hand, is needed to tell the in-kernel RAID code what to do, and with which drive(s) to do it. That part gets hairy rather quickly, especially for folks unfamiliar with Linux's /sysfs and hot plug weirdnesses.


If it's a >= 3 drive RAID1, then there's still redundancy while the one new drive is sync'd.

Cheers

Would that be a RAID1 array + hot plug spare?

By the sounds of it this is waht I want (from Wikipedia):

Alternatively, a new disk can be substituted so that the inactive disk can be kept in much the same way as traditional backup. To keep redundancy during the backup process, some controllers support adding a third disk to an active pair. After a rebuild to the third disk completes, it is made inactive and backed up as described above.

Not sure how to make that happen here, but it sounds like what I'm after.

So if the script was set to always rebuild say Drive 1, and anyone allowed into the server room knew that that was the only drive they were allowed to swap out, it could be done?

No. It'd be 3 drives all with the same content. If you yank 1 drive then the 2 remaining drives will still be in the machine.

Hmm. And how does drive 3 get the same content on it? RAID1 is 2 drives mirrored, it's as basic as that isn't it?

If you're already doing a write to one other drive then it isn't any harder to do it to more.

So in effect RAID1 + 1. I didn't remember seeing anything in the config section of my raid controller (on board raid by LSI, Fujitsu server) that would do similar, or would this be an OS level thing?

You seem to be under the mistaken impression that "RAID1" means *two* drives. It doesn't. It means one or more drives, all with exactly the same mirrored contents.

Eg. Commercial RAID vendors often use 4-drive RAID1 for data safety (and for faster read performance). All four drives are treated 100% equally.

So for the original proposal in this thread, it would be a *three* drive RAID1. Pull any drive out, and the system continues functioning. Pull a second drive out, and it still works fine.

Plug a removed drive back in, and the system will automatically resync it, while the other drives continue to work and provide full redundancy.

Mmm.. so I suppose what could be done here, is to initially create the RAID1 with all four drives installed, mirroring all of them together in RAID1.

Then remove one (or two, if you must) for off-site backup. The array will continue to operate, with full redundancy, but will claim to be "degraded" (missing a drive).

Later (days, months, seconds), pull out another drive to take off-site, and insert an original off-site drive as a replacement. The RAID array will automatically resync the inserted drive, provided it was one of the original RAID members.

No fancy scripts needed until you bring in a drive that was never part of the original RAID1 setup. Or at least that's how I remember it working, without actually configuring/testing it here today.

Cheers

OK, makes sense now. Will maybe try it, or maybe too chicken .

Thanks BTW