#326979 - 22/10/2009 18:17
Re: Quiet external RAID box?
[Re: Cris]
|
carpal tunnel
Registered: 30/10/2000
Posts: 4931
Loc: New Jersey, USA
|
I've had my Drobo for almost a year now. For a year, it's been running happily with four 750GB drives. Data keeps growing, so they eventually became full. Full to the point where the Drobo starts throttling transfer speeds so you know it's time to throw in some more/bigger drives. Well, I put the Drobo to the test and it failed!
I replaced one 750GB drive with a 1.5TB Samsung. Immediately the array started rebuilding (relayout in Drobo-speak). After 3 days it actually completed the relayout successfully. So, it was time to insert another 1.5TB drive in place of an old 750GB. Similar to the first time, relayout began without an issue. Seamless and predictable. This time relayout took 2 days, but after completion the Drobo went berserk. Rebooting, relayout for 30 seconds, rebooting, relayout, reboot, etc.
Luckily, my crucial data is also uploaded to Mozy so I wasn't sweating it too much. I contacted Drobo support and opened a trouble ticket. I was pleasantly surprised to get a response within a couple hours on a weekend. However, by that time I had already removed the 2nd 1.5TB Samsung and shoved a 1.5TB Seagate in its place under the assumption that the Samsung was faulty.
I wasn't shocked when the Seagate relayout finished and the Drobo started its little dance all over again.... this time corrupting the filesystem.
Removing the 2nd 1.5TB drive made the the Drobo workable again in both instances. Of course, with only 3 drives there's no redundancy.
I was able to recover the filesystem (HFS+) only by disabling journaling and running repair from Disk Utilities. Running repair with journaling on constantly failed. Sheesh.
After speaking with Drobo Support on the phone and through their trouble ticket system, they sent me out a replacement Drobo. It arrived today. It's failing in an even more outrageous way where it decides to reboot randomly during relayout.
Now my filesystem is completely hosed, but I was able to get almost everything off the array while waiting for the new Drobo to arrive. I don't know how this story will end, but I'm very skeptical of Drobo devices at this moment. Since two Drobo boxes are failing in a similar fashion with the same disk pack, obviously my disks are screwed up in some way. But, they've only ever been inside my Drobo where they've been serving me well.
Anticipating this day, I've started looking at Drobo alternatives. The most attractive solution looks like running a ZFS raidz array in Ubuntu. But this doesn't allow the array to grow by replacing single disks in the array. This is the main advantage Drobo has, but as you can see it didn't deliver in my case. Is there anything else to consider?
_________________________
-Rob Riccardelli 80GB 16MB MK2 090000736
|
Top
|
|
|
|
#326981 - 22/10/2009 18:25
Re: Quiet external RAID box?
[Re: robricc]
|
carpal tunnel
Registered: 12/11/2001
Posts: 7738
Loc: Toronto, CANADA
|
Please keep us updated. I don't know about anyone else, but I'll be following this thread like a hawk.
I haven't been able to find anything at all that is even remotely comparable to the Drobo products. The only other multi-disk systems that are even close to the DroboPro (which is what I've been looking at) are rack-mounted arrays from Promise. They don't have the user-friendliness nor auto-expansion features though.
|
Top
|
|
|
|
#326983 - 22/10/2009 18:35
Re: Quiet external RAID box?
[Re: hybrid8]
|
carpal tunnel
Registered: 30/10/2000
Posts: 4931
Loc: New Jersey, USA
|
I plan to keep it updated, but I feel like I've reached the end of the line with this array. I'm almost certain I have to rebuild. I agree that Drobo is in a class of its own. The supposed features are untouchable. The only thing that comes close is unRAID, but that's not nearly as elegant. We have a Drobo Pro at work by the way. I'm really rethinking that decision now, but it's given us no issues.
_________________________
-Rob Riccardelli 80GB 16MB MK2 090000736
|
Top
|
|
|
|
#326994 - 23/10/2009 11:10
Re: Quiet external RAID box?
[Re: robricc]
|
carpal tunnel
Registered: 30/10/2000
Posts: 4931
Loc: New Jersey, USA
|
Below is my final post on the subject at drobospace.com. I would just link it under normal conditions, but you can't read their forum unless you have a Drobo serial number. When I last posted here, I had just put a Seagate 1.5 in place of a Samsung 1.5 and it was doing a relayout. Not surprisingly, toward the end of the relayout, the Seagate did the exact same thing. Constant rebooting.
After speaking with Drobo Support, which has been nice and responsive, I was sent a replacement Drobo which arrived yesterday. While I was waiting for the replacement, I was able to copy all my data off the old Drobo. I started-up the replacement Drobo by putting in the 3 drives that have been working in the old one. Hoping the replacement Drobo would properly complete the relayout, I inserted the Seagate 1.5 once the replacement Drobo was up and running.
I will admit that I was surprised when the replacement Drobo started rebooting randomly in the middle of relayout less than 10 minutes into the process. This time, the replacement Drobo completely hosed my filesystem. I don't know if the corruption would have eventually been fixable, but I let Mac OS Disk Utility try to repair it for 4 hours before I finally pulled the plug.
There's no way around it, the Drobo failed me in two key aspects. One is the ability to increase the capacity of an existing array. The second is protecting my data. I think it's inexcusable, but there's not much I can do about it.
Last night I decided to end this pain and reset/reformat the Drobo with the exact same disk pack that it was unable to relayout properly. Naturally, the reset/reformat went on without a hitch and I now have the healthy, larger array I was after.
I don't know if I'm still a fan of Drobo. The features are great compared to traditional RAID arrays, but those features completely failed to deliver when put to the test. Right now, my data is being copied back to my old Drobo. If I decide to keep the Drobo as my primary storage, you can bet I'll have a full backup before attempting a disk swap in the future. I really suggest you do the same. There's not much more to say, really. The Drobo fell flat on its face when the marketing hype was put to the test. I know many people have been using these things as designed with no problems, but anyone reading this must keep my experience in the back of their minds. Never take the device that's holding your data for granted. If your data doesn't exist in more than one physical location, it may as well not exist at all.
_________________________
-Rob Riccardelli 80GB 16MB MK2 090000736
|
Top
|
|
|
|
#326997 - 23/10/2009 11:51
Re: Quiet external RAID box?
[Re: robricc]
|
carpal tunnel
Registered: 12/11/2001
Posts: 7738
Loc: Toronto, CANADA
|
Not that it's any less serious, but this all sounds like one or more software issues. Did you happen to notice if the new Drobo was running an updated software rev compared to your original?
Within IT circles, something this important/serious needs to be bulletproof. There's really little room for error, especially when it comes to integrity of the base filesystem. If it can't build out a new disk it should pop a warning and fail safe, allowing people without as much foresight as you to go and immediately make a backup.
Now imagine software issues in life-supporting apparatus. It's a dangerous world in the Adobe/Apple/Microsoft generation where "good enough" is usually what's good enough.
The Drobo has been around long enough that one would think any problems such as the one you've noticed would have been found n-times over and corrected by now. If it's something new, it would sure be nice to find out what the issue was. Do you know if the Drobo logs any of its rebuild process to internal flash or somewhere else where DataRobotics can look for clues to try and address this? I'm not a fan of the "send out a replacement and keep your fingers crossed it works this time" policy.
|
Top
|
|
|
|
#326998 - 23/10/2009 12:04
Re: Quiet external RAID box?
[Re: hybrid8]
|
carpal tunnel
Registered: 30/10/2000
Posts: 4931
Loc: New Jersey, USA
|
The Drobo device does keep detailed logs internally. You can request a copy of the current log through Drobo Dashboard. The file is then dumped to your desktop in an encrypted text file. A simple PHP script exists to decrypt the log file for Drobo v1 and v2 units, but the Drobo Pro uses some other encryption that nobody managed/decided to crack yet.
I've sent Data Robotics numerous log files. At one point, they wanted me to remove the drive in bay 3/4 which was a WD 750 that had been working fine for months. When I asked if the logs revealed anything wrong with the drive, the response I got was "Not exactly."
When I look at the logs, I do see various read/write errors on bay 3/4, but the cause of this could be anything. The hard drive passes the Drobo's internal diagnostics and it passed brief WD diagnostics when I had it outside of the Drobo.
I think software is to blame for this corruption and it probably started happening long before I introduced new drives to the system. My guess is that the Drobo has been working in a degraded state for quite some time. The initial cause could really be anything. Some freak power outage, not shutting the Mac down properly, etc.
For what it's worth, my old Drobo and the replacement they sent are both running the latest firmware.
_________________________
-Rob Riccardelli 80GB 16MB MK2 090000736
|
Top
|
|
|
|
#327000 - 23/10/2009 12:25
Re: Quiet external RAID box?
[Re: robricc]
|
carpal tunnel
Registered: 12/11/2001
Posts: 7738
Loc: Toronto, CANADA
|
Encrypted log files? I can possibly understand the customers-only forum if it's part of their official support infrastructure. But encrypted log files? I suppose they may be extremely paranoid about another company copying their technology, but they seem to be taking a somewhat overly opaque stance with customers about the customer's own problems.
|
Top
|
|
|
|
#327001 - 23/10/2009 12:31
Re: Quiet external RAID box?
[Re: hybrid8]
|
carpal tunnel
Registered: 30/10/2000
Posts: 4931
Loc: New Jersey, USA
|
The forums used to be hosted by the same folks that do the Slingbox forum. Then Drobo took the forums in-house and closed it up. They've been accused of having a heavy hand with moderating, but I haven't really seen any evidence of that.
_________________________
-Rob Riccardelli 80GB 16MB MK2 090000736
|
Top
|
|
|
|
#327004 - 23/10/2009 13:07
Re: Quiet external RAID box?
[Re: robricc]
|
carpal tunnel
Registered: 08/06/1999
Posts: 7868
|
Wow, even the massive HP SANs I worked with didn't dump encrypted logs. They were a pain to sort through without a parser, but a customer could open them up if they wanted to.
So far, still no problems with the ReadyNAS. It's been expanded once by putting in a 4th drive, and completed the initial expansion in under an hour. I'll probably be doing a large drive swapout here soon, in prep for ripping all my DVDs to it.
|
Top
|
|
|
|
#327005 - 23/10/2009 13:12
Re: Quiet external RAID box?
[Re: drakino]
|
carpal tunnel
Registered: 30/10/2000
Posts: 4931
Loc: New Jersey, USA
|
I wasn't aware the ReadyNAS was able to expand volumes like the Drobo. What model do you have?
I'm very wary of storage devices made by consumer-grade networking companies after my D-Link DNS-323 imploded.
_________________________
-Rob Riccardelli 80GB 16MB MK2 090000736
|
Top
|
|
|
|
#327007 - 23/10/2009 13:52
Re: Quiet external RAID box?
[Re: robricc]
|
carpal tunnel
Registered: 17/01/2002
Posts: 3996
Loc: Manchester UK
|
I've gone from 2 drives to 3 recently (1TB to 2TB). I just put the drive in one evening and by the following morning it had emailed me to tell me it had finished resizing. I've got the ReadyNAS NV+.
_________________________
Cheers,
Andy M
|
Top
|
|
|
|
#327008 - 23/10/2009 14:17
Re: Quiet external RAID box?
[Re: robricc]
|
carpal tunnel
Registered: 08/06/1999
Posts: 7868
|
I have the NV+ as well, and as far as I know, " X-RAID" is a feature in all of the ReadyNAS products. It is a traditional RAID setup still, so rules about identical drive sizes apply. Though X-RAID2 in the newer units will open up the new space once enough disks in the array have been upgraded. Digging deeper into how to recover from disaster on a ReadyNAS, I did find it would be possible to plug all the drives into a normal Linux machine and get the data back. So even if the device does implode and a replacement can't be found, I should be able to get my data back. I was initially concerned about this, as the NV+ has a hardware raid chip in it to keep speed up, and those can sometimes be very proprietary and closed off.
|
Top
|
|
|
|
#327009 - 23/10/2009 14:17
Re: Quiet external RAID box?
[Re: andym]
|
carpal tunnel
Registered: 30/10/2000
Posts: 4931
Loc: New Jersey, USA
|
These devices don't appear as a local drive which is an issue with Mozy Home... unless they support iSCSI. The NV+ doesn't, though some other models do.
ReadyNAS devices are also pretty expensive compared to Drobo. This wouldn't be as much of an issue if I didn't already own a Drobo.
_________________________
-Rob Riccardelli 80GB 16MB MK2 090000736
|
Top
|
|
|
|
#327010 - 23/10/2009 14:20
Re: Quiet external RAID box?
[Re: robricc]
|
carpal tunnel
Registered: 08/06/1999
Posts: 7868
|
As for iSCSI, there is an addon to add it to older ReadyNAS units. I personally haven't tried it, so I can't speak about how well it works. http://readynasfreeware.org/ has the addon, along with some other useful ones.
|
Top
|
|
|
|
#327011 - 23/10/2009 14:31
Re: Quiet external RAID box?
[Re: drakino]
|
carpal tunnel
Registered: 12/11/2001
Posts: 7738
Loc: Toronto, CANADA
|
Too bad I can't afford (or justify if I could) a ReadyNAS 3200. 12 drive capacity would be super nice. Though not as nice if all the drives had to be the exact same size.
WIth same-size drive requirements, that means you'd have to replace, for example, you current 1.5TB disks if you ever wanted to add any 2 TB disks in the future.
The ReadyNAS Pro Business supports iSCSI out of the box but it streets for as much as the DroboPro MSRP ($1500) unpopulated. It also supports only 6 drives.
I need the storage capacity and redundancy, but I don't need the product to be NAS, that's why the DroboPro appeals to me. But going back to that bullet-proof comment from earlier... Always feeling uncomfortable or downright uncertain about the integrity of some of your data, let alone the entire array, would be too much.
I'm also liking the snapshot feature of the Pro Business model, but I don't think I'd have the external capacity to snapshot the entire array. I suppose it wouldn't be too difficult to automate backups of specific shares or directories on one of the lesser products (NV+, Pro Pioneer).
I'll have to keep an eye on this space over the next couple of months, while for now I just try to scrounge as much free drive space as I can with the disks I have lying about.
|
Top
|
|
|
|
#327012 - 23/10/2009 15:22
Re: Quiet external RAID box?
[Re: hybrid8]
|
carpal tunnel
Registered: 08/06/1999
Posts: 7868
|
It looks like the 3200 supports the newer XRAID-2, so you wouldn't need to swap out all the drives for larger ones, just a few to gain new space.
As for snapshots, thats generally the term applied to creating a new share that represents a normal share, but frozen in time. New changes to the share don't show up in the snapshot. Typically this is used for allowing backups to occur to stable data, or as a basic revision control setup. Space for allowing snapshots to be turned on would be carved out of the RAID disk space, and wouldn't be reliant on an external storage box.
|
Top
|
|
|
|
#327014 - 23/10/2009 17:19
Re: Quiet external RAID box?
[Re: drakino]
|
carpal tunnel
Registered: 12/11/2001
Posts: 7738
Loc: Toronto, CANADA
|
Ah, gotcha. I understood that it supported making external snapshots. Didn't know it could do it internally. I'm going to do a little light reading on XRaid-2. When I finally pull the trigger on a storage appliance I don't want to have any regrets - even if they' come only 6 months later without warning.
|
Top
|
|
|
|
#327028 - 24/10/2009 07:58
Re: Quiet external RAID box?
[Re: robricc]
|
carpal tunnel
Registered: 20/05/2001
Posts: 2616
Loc: Bruges, Belgium
|
Unless I'm seriously mistaken, I believe that most Synology and Qnap NAS products can also extend their free space by adding more drives, and, in case all drive bays are full, by swapping the disks one by one by a larger disk and letting the NAS rebuild each time. I pretty sure I've read about this feature more than a year ago on both the forums of those brands.
_________________________
Riocar 80gig S/N : 010101580 red Riocar 80gig (010102106) - backup
|
Top
|
|
|
|
#327030 - 24/10/2009 11:07
Re: Quiet external RAID box?
[Re: BartDG]
|
carpal tunnel
Registered: 29/08/2000
Posts: 14493
Loc: Canada
|
My buddy's older QNAP box certainly does not work that way.
Stick a 1.5TB drive into it in place of a 750GB, and the new capacity of the RAID remains unchanged.
And on that box, they used a basterdized hack of ext3 internally, with extents from ext4, so the result was incompatible with resize2fs. That's the command needed to take over the new extra space.
We ended up ripping the drives out, plugging them into a real Linux box, formatting them there to full capacity, copying the data back onto them, and then put them back in the QNAP.
-ml
|
Top
|
|
|
|
#327058 - 26/10/2009 11:08
Re: Quiet external RAID box?
[Re: robricc]
|
carpal tunnel
Registered: 30/10/2000
Posts: 4931
Loc: New Jersey, USA
|
I decided to upgrade the Drobo again this weekend to all 1.5TB drives. This took me from 2.7TB to 4.07TB. I chose to do it now because I already have my data backed up at this point.
This time around, the Drobo expanded the volume after replacing each 750 without any drama.
_________________________
-Rob Riccardelli 80GB 16MB MK2 090000736
|
Top
|
|
|
|
#327063 - 26/10/2009 13:00
Re: Quiet external RAID box?
[Re: mlord]
|
carpal tunnel
Registered: 20/05/2001
Posts: 2616
Loc: Bruges, Belgium
|
My buddy's older QNAP box certainly does not work that way. The QNAP website mentions this (in their current feature set): Online RAID Capacity Expansion The storage capacity of a RAID configuration can be expanded by replacing the hard drives with larger ones. All the data will be kept and seamlessly moved to the newly installed hard drives. There is no need to turn off the server during the process.
Isn't that exactly what Rob is looking for? Edit: this is the spec sheet for the Qnap TS-509 Pro : here or here for a Synology example (DS-409+).
Edited by Archeon (26/10/2009 19:17)
_________________________
Riocar 80gig S/N : 010101580 red Riocar 80gig (010102106) - backup
|
Top
|
|
|
|
#327131 - 28/10/2009 08:33
Re: Quiet external RAID box?
[Re: BartDG]
|
carpal tunnel
Registered: 20/05/2001
Posts: 2616
Loc: Bruges, Belgium
|
_________________________
Riocar 80gig S/N : 010101580 red Riocar 80gig (010102106) - backup
|
Top
|
|
|
|
#327134 - 28/10/2009 10:22
Re: Quiet external RAID box?
[Re: BartDG]
|
carpal tunnel
Registered: 08/03/2000
Posts: 12338
Loc: Sterling, VA
|
I'm not so sure that it's doing the same thing the Drobo does. Yes, it looks like Qnap might be a little friendlier than your standard RAID, by letting you do an in-place expansion, but from everything I've read of the examples you're posted, the assumption is still that you are going to replace all the drives in your current RAID. It also assumes the usual RAID drawbacks: that all your current drives are identical and that all the future drives will be identical as well. The idea of Drobo (at least in theory) is that, to begin with, you can just throw whatever hard drives you have in there. And when you want to expand, you can take out any single hard drive and replace it with another of any size and of any manufacturer (except Seagate ). So I could start with a 250GB, a 300GB, a 500GB, and a 1TB drive in my Drobo, and when it started to fill up, I would replace the smallest drive (the 250GB) with whatever I wanted and be done.
_________________________
Matt
|
Top
|
|
|
|
#327136 - 28/10/2009 11:09
Re: Quiet external RAID box?
[Re: Dignan]
|
carpal tunnel
Registered: 12/11/2001
Posts: 7738
Loc: Toronto, CANADA
|
I'm not so sure that it's doing the same thing the Drobo does. There aren't any other products on the market, as far as I'm aware, that do what the Drobo does as far as volume management and expansion.
|
Top
|
|
|
|
#327141 - 28/10/2009 12:02
Re: Quiet external RAID box?
[Re: Dignan]
|
carpal tunnel
Registered: 25/12/2000
Posts: 16706
Loc: Raleigh, NC US
|
There is no requirement in any RAID system that I'm aware of that all of the drives be identical. You will only get as much space as if they were all the same size as the smallest one, though.
What the Drobo does isn't exactly groundbreaking. It's been doable for years, if not decades, given a logical (extent-based) volume manager and a filesystem that supports online resizing. All they've really done is wrap some scripts around it.
_________________________
Bitt Faulk
|
Top
|
|
|
|
#327142 - 28/10/2009 12:22
Re: Quiet external RAID box?
[Re: wfaulk]
|
carpal tunnel
Registered: 08/03/2000
Posts: 12338
Loc: Sterling, VA
|
There is no requirement in any RAID system that I'm aware of that all of the drives be identical. You will only get as much space as if they were all the same size as the smallest one, though. Of course, you're right. I was simplifying even though it was unnecessary I just wanted to make it clear that while it appears that those Qnap devices appear to make expansion (and migration, which is pretty cool) easier than usual for a RAID device, it's still limited by the traditional characteristics of a RAID. They aren't doing what Rob was trying to do with his Drobo. I've been lusting after the Drobo for a long while, Rob. I'm definitely concerned by your experience. However, I suppose that the Drobo doesn't replace good backups. It's always good to have two copies of every file that's important to you.
_________________________
Matt
|
Top
|
|
|
|
#327144 - 28/10/2009 12:59
Re: Quiet external RAID box?
[Re: Dignan]
|
carpal tunnel
Registered: 12/11/2001
Posts: 7738
Loc: Toronto, CANADA
|
A RAID is supposed to be a fault tolerant "live" working repository for your files, not a backup. A backup, regardless of medium (even if it's to a RAID volume) should never be used "live" - only a backup (or restore) operation should touch those volumes. When not performing a backup or restore, the medium should preferably be moved off site. At least put somewhere safe, away from your live volumes. If I move to something like the ReadyNAS Pro, I'm not likely to have enough external volumes for a 1:1 backup, but all my critical assets will be covered, including music. Things like TV recordings I'll have to leave to chance.
|
Top
|
|
|
|
#327145 - 28/10/2009 13:46
Re: Quiet external RAID box?
[Re: hybrid8]
|
pooh-bah
Registered: 12/02/2002
Posts: 2298
Loc: Berkeley, California
|
So can you explain what drobo does that's different? I can put x random drives in my readynas and get the capacity of the smallest drive * x-1.
The only difference I've ever seen is that it's DAS instead of NAS. Fine if that's what you want, but not earth shattering.
|
Top
|
|
|
|
#327146 - 28/10/2009 13:56
Re: Quiet external RAID box?
[Re: matthew_k]
|
carpal tunnel
Registered: 08/06/1999
Posts: 7868
|
The Drobo work by creating RAID sets across the drives in a mix of RAID 5 or RAID 1, depending on the size and number of disks. For example (stealing from wikipedia), a Drobo filled with a 500, 400, 200 and 100 gb disks would create a 100gb raid 5 across all 4 disks, then a 3 disk raid 5 across the 500, 400 and 200. At this point, the 200 and 100 disks are full, so it switches to making RAID 1 raids on the 500 and 400. With this particular setup, 100GB on the 500GB disk would simply be lost.
To calculate the usable space on a Drobo array, you would add up all the disk capacity (1200gb in this example), then subtract the largest size drive (500gb), leaving the usable capacity (700gb).
It's basically the same trick the ReadyNAS can do with XRAID2. In either case, it's always best to have 2 of the largest capacity drives in the RAID to ensure no wasted space.
*edit* Oh, one thing that bothers me with the way the Drobo works is that it asks you during setup to set the maximum size of the RAID, and allows this number to be higher then what is possible with the disks in the array. This can lead to some odd issues where a computer sees an 8TB drive, but can only write to 3.5TB or so of it. It's a way to work around file system expanding limits I suppose, but it still seems strange.
Edited by drakino (28/10/2009 14:02)
|
Top
|
|
|
|
#327150 - 28/10/2009 15:54
Re: Quiet external RAID box?
[Re: drakino]
|
carpal tunnel
Registered: 12/11/2001
Posts: 7738
Loc: Toronto, CANADA
|
Try Drobo's interactive capacity calculator: http://www.drobo.com/resources/drobolator.phpThis way you can see the exact effect of adding and removing drives of any size up to 2TB (ea) and with single or double drive failure protection.
|
Top
|
|
|
|
|
|