I have a pair of Western Digital Caviar Black 1-TB drives (one for data, one for backup) that at present usage rate are going to be full in another 6--8 weeks.

I plan to replace these drives with a pair of 2-TB units. (The 1-TB drives will be "recycled" as replacements for two 300 GB drives. I don't know what I'll do with the left over 300s.)

I've been looking at this drive, but am not wedded to it. The use to which these drives will be put is primarily data archiving. There will be no operating system or executables on them. One of them (the backup) will be used as an eSATA external drive. They will not be used in any kind of RAID setup. Speed is not important, long service life and reliability is.

Of tangential relevance is that the drives will NOT be purchased from Amazon even if they're cheaper. There are literally hundreds of horror stories of how Amazon ships hard drives, rattling around loose in oversized boxes and the DOA rate from Amazon is truly staggering. From Newegg the horror stories are about drives shipped by Newegg's default carrier, UPS, many reviewers say to spend the extra $10 and have them delivered by FedEx.

Is this a good choice of drive for my purposes?

tanstaafl.

I'm rather keen on the 64MB cache drives from WD right now, but not the older 32MB cache drives.

And with the caveat that one must run the wdidle3 program (once) on them to fix the stupid 8-second head park default.

The DOA rate from newegg is way too high for me to consider purchasing from them.. some of that does go down to shipping, but other stuff I've purchased from newegg has arrived with zero padding in a huge box. Not so harmful for a video card, perhaps, but a hard drive..???

Fedex vs. UPS wouldn't matter much there: equally bad when the product isn't packaged correctly.

Buy locally at a slight premium, if possible.

Cheers

Is there an advantage to the 64MB cache drives other than read/write speed? These drives will be used but seldom, and when they are it will be for a continuous data write of about 5 GB, and continuous data reads of about 1.5 GB. In other words, pretty much streaming contiguous data both writing and reading. With that in mind, would the 8-second head park default be a problem worth fixing?

And somewhere I have gotten it in mind that the 64MB cache drives are primarily set up for RAID operation, and not suitable for non-RAID applications because the error correcting routine on the RAID systems is not as robust, but is biased towards swapping out questionable drives even though the errors detected are correctable. Or am I misunderstanding this entirely?

tanstaafl.

Odd, Amazon's packing approach in the US must be very different to their UK operation. I have never received anything from Amazon that wasn't very well packed. Even books are shrink wrapped to a piece of cardboard that is then glued to the bottom of the box.

Every once in a while I get a package from Amazon that wasn't packed well, but that's certainly the exception and not the rule.

They're a half generation newer than the older 32MB/cache drives, with bugs fixed and theoretical reliability improved. Apparently they are variable spin-speed, too.


I use my 2nd-gen 1.5TB drive exclusively for streaming (MythTV recording / playback), and bumping the 8-second park time up to 30 seconds was very worthwhile for it. I suppose it may depend upon the I/O pattern of the specific software being used, but an 8-second default seems almost brain-dead to me.


Nope. There are special firmware drives just for RAID, with something like "RE" in the name.

Cheers

Oh, and don't forget to align the first partition on an even 8-sector multiple (8 * 512 = 4096 bytes) to match the internal sector size of 4096 bytes.

Cheers

Oh.. the main difference is that they have 500GB platters inside, instead of 333GB platters. Scuttlebutt has it that they're actually the same 333GB platters as the older generation, and the higher capacity comes from using the 4096-byte native sector size, rather than the traditional 512-byte native sector size.

Would you recommend buying one of those WD Advanced Format 4K drives Mark? (I'm guessing yes, since AFAIK, all the 64MB versions are 4K drives) Are there any negatives to expect in the short term?

The 64MB-cache WD "Green" drives are my current best recommendation for 3.5" mechanical drives, with the caveat that the 8-second default head park time has to be fixed (change it to 15 or 30 seconds).

These are quiet, low-power (despite the "green" name, they really do consume less power than the drives of old), and give very high performance. Yes, high-performance, despite the 5400rpm spin speed. These are nearly as fast as the current WD Black (7200rpm) drives for most stuff.

Edit: I have a single 1.5TB model in the MythTV box, and it is darned near silent in operation, yet can keep up with multiple simultaneous recordings and playback streams without dropping a frame (I've tried 8 streams with it).

And thus far, they do seem reliable enough. Much better than recent Seagate drives, for example.

Cheers

One more note: the 1TB Black and 1.5TB Green drives that I have here, are the first 3.5" drives that I have actually paid for since the 1990s. Normally I use vendor freebies for everything.

So I'm voting with my wallet on these ones.

I do purchase 2.5" drives as well, though. There, my current preference is also WD, and for the same reasons: quiet, very low power usage, and very, very fast.

Cheers

Thanks Mark.
I think those 64MB drivers are sweet also, the only downside is the come at a hefty price premium. I can buy a 32MB 2TB WD drive for about 100 euro here. The 64MB version costs 180 euro. I'm not sure those extra 32MB's and half a new generation software update are worth it.

Also, what is your stance on Samsung HD's? They are usually a bit cheaper than WD, a bit faster and run cooler. (is there a catch?)

Mmm.. that's odd. Here in Canada, they are pretty much the same price, never more than a 5% difference.


Samsung was at the top of my shopping list when I went out to get a 1.5TB drive. I like the specs, and people who have them seem to like them a lot.

But none of our local retailers stock them, and I didn't want to online/mail order a drive due to the high failure rates of shipped single/retail drives. So I ended up with the WD Black and Green drives instead.

Current pricing (CDN$) from a shop near to us here:
$120 Western Digital Caviar Green (WD20EARS) 2000GB (2TB) SATA 3 Gb/s 64MB (OEM)
$200 Western Digital Caviar Black (WD2001FASS) 2000GB (2TB) SATAII 7200RPM 64MB Cache (OEM)

They no longer even list the older 32MB-cache Green drives.
There are some smaller 1TB drives in stock with 32MB caches, though:

$65 Western Digital Caviar Green (WD10EARS) 1000GB (1TB) SATA 3 Gb/s 64MB (OEM)
$73 Western Digital Caviar Blue (WD10EALS) 1000GB (1TB) SATAII 7200RPM 32MB (OEM)
$90 Western Digital Caviar Black (WD1001FALS) 1000GB (1TB) SATAII 7200RPM 32MB Cache (OEM)
$100 Western Digital Caviar Black (WD1002FAEX) 1000GB (1TB) SATA3 7200RPM 64MB Cache (OEM)

I stand corrected. I've now also found the WD20EARS model for 114 euro The 180 euro model is the WD2001FASS. There also seems to be a RE4 GP WD2002FYPS which costs 240 euro and a RE4 WD2003FYYS model which costs 280 euro! All for the same 2TB! Ok, so those last two models are RAID editions, but if the only difference is still that TLER option, then that's really expensive.

Thanks for the list Mark, but I'm only interested in the 2TB models. In fact, if larger models than that would exist I'd probably go for one of those. I've heard rumours about Seagate releasing a 3TB and 4TB model by the end of this year and I really hope the other manufacturers will follow soon. (because I won't ever buy Seagate again - a couple of bad experiences in the past... ).
But I hear there's a problem with drives bigger than 2TB, it seems these are too large for the Bios'es and so it's impossible to boot from them. EFI should fix this though. I wonder if that is the reason why we've been stuck at that 2TB barrier for this long. Could you provide some info on that Mark? Is the info I currently have correct? And if it is, would it possible to use eg. a 3TB drive in an 'older', non-efi system as a simple storage drive?

(sorry for hijacking your thread Tanstaafl!)

A 2TB drive could present as many as (2 * 1024 * 1024 * 1024 * 1024 / 512) logical 512-byte sectors to the host. If you're keeping track, that's 2^32, or the maximum that can be represented by a 32-bit sector number.

The standard MS-DOS partition table layout, used by most PCs, has a 32-bit field for starting LBA (sector number). Similarly, a second 32-bit field is used for the sector-count for a partition.

Any drive with more than 2^32 sectors (2TB) might be unable to boot from sectors beyond the first 2TB, because the standard MS-DOS partition table has no way to encode higher capacity values in a way that the BIOS understands.

There are lots of ways to work around that. Eg. boot from a partition within the first 2TB, and then let the operating system use a different scheme to identify where the rest of the disk partitions are located.

But disk drive makers, like most computer hardware companies, are reluctant to build things that require any kind of special handling for the typical Windows machine.

So, we've been "stuck" at 2TB for (only) the past year because of that limitation.

Cheers

Please! Hijack all you want. You've opened up all kinds of new information.

Can you or Mark address the TLER thing in more detail? I'm confused bout that, it's where I got the (apparently) mistaken idea bout RAID-specific drives not being suitable for regular desktop use.

tanstaafl.

Nor have I. But apparently hard drives are the exception. Read the Amazon customer reviews on any of their larger hard drives, and the great majority of the bad reviews (1-star and 2-star) express amazement at how badly packaged the drive was and many are not surprised to find their drives DOA. Even the good reviews express gratitude that the drive arrived in working condition despite the packaging.

tanstaafl.

Time Limited Error Recovery is a drive feature. When enabled, the drive is supposed to not try very hard to self-correct media errors, so that it is guaranteed to always either succeed or fail within a known, short, time interval. This is useful for RAID1 arrays, where data is replicated (mirrored) in other places (and also for other mirroring RAID types).

But it's a bad feature when the drive is the only drive with the data --> for this, one normally wants the drive to try really hard to recover the data, even if it takes 10-20 seconds to do so.

For some peculiar reason, WD is treating this as a permanently enabled/disabled feature, dependent upon different firmware versions. Rather than just having the feature present in all firmware, disabled until the operating system sends a command to enable it.

Weird, but that's how they do it.

Thanks!


It's got something to do with conflict as to whether error handling should be undertaken by the hard drive or by the RAID controller. WD had added a 7 second delay on RE disks so the RAID controller would have the time to pick up and act upon them. WD claims this is necessary, and sells these RE drives at a considerable price premium. The stupid part is that there is a small application floating around on the web which can also set this TLER switch on non RE disks, effectively making them RE disks at considerably lower cost. Of course WD also claims there are other difference to the RE series like use of better materials etc, but afaik that has never been proven. It is a fact though that, by taking this step, WD has made building a RAID array considerably more expensive for users - if they choose to actually by the real RE drives that is.
There is a wiki entry for this.

Note that for most of us, the non-TLER firmware is probably best, even for RAID applications -- the non-TLER firmware will give the most reliable error-protection for our data.

[EDIT]
There should be no real performance difference between TLER and non-TLER except when handling media errors (but see my subsequent post below).
[/EDIT]

The folks who need TLER are companies like VISA, who don't appreciate it when their entire transaction processing system comes to a halt for for 10-20 seconds just because one drive in the array has a bad sector..

There may also exist other forms of Raid-Enhanced (RE) firmware for drives, with or without the TLER feature tagged on.

These drives differ from desktop drives in that they often have a much smaller sector Read-Ahead value/algorithm than usual, so that they perform better for server-style applications that jump all around on the disks.

The theory there is that they are often at the end of a much slower (than the drive) network connection, shared by many clients, so they'll not normally be doing large/long sequential reads.

On a desktop drive, long sequential reads (eg. application loading, or reading/writing media files) are much more common and well worth optimizing for.

Many people firmly believe that "Enterprise" and "RE" drives also have different hardware internals -- longer lasting bearings and better vibration dampening etc. This is undoubtedly true for some models, but I find it difficult to believe when comparing two otherwise identical spin-rate drives from the same manufacturer.

Cheers

The article states "RE disks are only suitable for RAID arrays and Caviar are only suitable for non-RAID use."

So I am assuming that Mark's recommendation of the WD20EARS would be a suitable drive for my purposes. Is that correct? That is, the WD20EARS is NOT TLER, and for non-raid applications I don't want TLER.

tanstaafl.

Not a delay, but rather a time-limit: The idea is that no read/write operation will ever take more than 7 seconds to succeed or fail on those drives.

In the absence of media errors, all operations will normally take only a few milli-seconds on these drives or on regular drives.

That's only half correct. RE/TLER disks are indeed only suitable for RAID (mirroring) uses. But regular "Caviar" drives can also be used in RAID arrays without trouble, as they have been for decades now. The main difference is how long each type takes to report (or recover from) a bad-sector.


I think so.
And practically NOBODY really wants TLER (outside of big enterprise applications), even if they think they do (see post above).

My concerns now are a little bit of uncertainty about the Advanced Format but from what I've read (thanks, Archeon, for the Wikipedia link) it looks like that will be transparent to Vista; and a concern about the number of Newegg reviewers who were not enchanted with the WD20EARS. (See picture) About 25% of them experienced DOAs or failures within the first week or so, and the percentage is actually higher because a fair portion of the reviews reported multiple failures in the same review. Balancing that is the thought that the a pissed-off person whose drive arrives DOA or fails after a week with loss of data is much more likely to write a review than is someone who buys the disk, pops it into his computer and it works with no problems. (Both of my WD1001FALS Caviar Black 1-TB disks fell into that latter category, and I didn't write a review.)

So, as a result of this thread, I have learned a lot about large capacity drives, TLER, RAID, Desktop, Advance Format, maximum addressable space, sector size, etc. Mark, any time you want me to explain disk drives to you, let me know. (J u u s s t kidding!)

So now the decision. Do I want to go with the WD20EARS "Green" drives, or the WD2001FASS "Black" drives. The Blacks are 50% more expensive but have a 66% longer warranty and are higher performance, 7200 RPM with 64MB cache. With present systems, there seems to be no advantage to the 4096K sectoring, and in random writes a significant disadvantage. Additionally, the Blacks are not bleeding-edge technology with Advanced Format, variable speed, and automatic head parking every 8 seconds as are the Greens. The Blacks are considered to be the very top of the WD desktop lineup. Yet, the Greens run quieter and cooler, use less power, and cost less. With the Advance Format sectoring, the Greens get by with three platters, the Blacks take (I think) four. Less complexity is a good thing.

So is it worth it to spend the extra $120 for two Blacks? Gaaah! Look at me, dithering over this as if it were really important and would make a big difference in my life a year from now. I just have to keep in mind the way SWMBO and I keep things running smoothly: I make all the important decisions, she makes the small ones. So she'll decide things like should we retire and move to Mexico, what house will we buy, things like that. I decide things like what should be our policy in Afghanistan and Iraq, how will we fix the oil spill in the Gulf of Mexico, things like that. Works out well.

So, Mark, tell me: EARS or FASS? The money isn't that big a factor...

tanstaafl.

I know you're using a drive for backup, but I'd also recommend using an online backup service like Mozy or Carbonite. Even with a backup drive you're still prone to damage and theft. I recently upgraded to Mozy Unlimited from the free account I've been using for a couple years (to back up important documents and financial stuff). I can't tell you how much of a relief it is to know that my wedding videos and photos are also outside my home (encrypted before they left).

I've gone through the restore process for one of my clients with Mozy. I had recommended it to him, and he was extremely happy he had it when a pipe burst and drenched/destroyed the laptop sitting underneath it (including the hard drive).

On a separate note, I recently jumped on a Newegg deal for a 2TB Samsung external drive (getting rid of stock, I assume). I love having all the space, but I wouldn't keep any important data on it or I'd add it to my Mozy backup. I use it exclusively as a media drive. I do love having the space, though! Price was the only factor for me, as I didn't care how fast or even how reliable the drive was, so I definitely grabbed the $99 2TB drive.

*edit*
However, I'm transferring about 23GB of files to that drive, and getting an average of about 37MBps, so it's not too slow.

The fly in the ointment there is "online". Perhaps I don't understand how it works, so if I'm wrong here, please tell me.

Let's say I have 1.5 TB of data to back up. That comes to 12 Tb (Tera-bits) of data: 1,649,267,441,664 of data that I would have to upload at a rate of 1.84 Mb/sec (see attached photo), or to make the math a bit less confusing, at 1,929,380 bits per second (1024 bits/kilobit, 1024 kilobits/megabit). That would take 854,817 seconds, or 237 hours, or roughly 10 days of continuous uploading. Assuming my upload speed remains constant (I have seen it as high as 1.98 Mb/sec, as low as .37) and assuming my internet connection even remains connected for 10 straight days which in my experience (this is Mexico!) would be unprecedented. I suppose the trick would be to do it piecemeal, a couple of hours a day, in which case the backup would take about four months to complete.

My 6 Mb/sec download speed is the envy of all my friends and neighbors here, and is the fastest and most expensive internet connection available in this part of Mexico. (I think it costs me about $50 USD/month, but that includes cable television which I don't watch. Ever.) It is not, however, reliable. At the provider's request I am now keeping a log of the status of my cable modem at the time of the outages.

And to digress even further... There must be something fundamental about RAID arrays that escapes me. If I understand correctly (and I probably don't) if a hard drive in a RAID fails, you just replace the bad drive and the RAID controller rebuilds the data... unless a second drive fails before the rebuild is completed, at which point all the data in the array is lost. That seems pretty scary to me. I feel much safer having my data duplicated on a separate external hard drive. Even if my computer catches on fire I will still have my backup. Another thing I don't care for with RAID is if I do something stupid and mistakenly change a file (I did that on a very important, very large Excel file - left a column out of a sort and then saved the file) that file is changed in your RAID and there's no going back. My backup may be a few days out of date, but at least I still have the original file.

Probably I'm blowing smoke here, because a lot of people who know a whole lot more about computers than I ever will know swear by RAID systems,. I probably just don't understand them.

Your point about losing the data through theft, though, gives me pause. Crimes against persons where I live are very rare. Crimes against property is another matter. OK, you've convinced me. I won't do the on-line backup, but I am going to start putting my external backup drive in a hidden storage place where I keep my valuable papers (passport, FM3 visa, etc.)

Thank you for bringing that thought to my attention.

tanstaafl.

I need to toss a 2TB drive into my ReadyNAS as I'm just about out of space on the 4 1.5TB drives I have in there now. Reading the reviews on the various 2TB models has got me a little worried though. Lots of failures from every brand.

t doesn't help either that the ReadyNAS drive compatibility list isn't kept as up to date as one would hope. Drive manufacturers EOL some models rather quickly. As Mark pointed out, I can't get any of the 32MB cache WD models locally either.

You can have dual redundancy in a RAID setup as well, so you can have up to two drives fail. But... The fundamental part is that RAID is not a backup solution. It's for live data. I keep my notebook backed up and really important files like photos I further keep duplicate live copy on the RAID and a second backup on my web server. I don't however have the bulk of my RAID array backed up. Though I should.