Driving Disks into the future

Robin Harris of ZDnet "Storage mojo" has written a series of posts on factors affecting the future of Disk Storage. These are my reactions to these quotes and trends, especially in flash memory.



Flash getting "70% cheaper every year" - hence more attractive:

"Every storage form factor migration has occurred when the smaller size reached a capacity point that enabled the application, even though it cost more per megabyte."

"With flash prices dropping 70% a year and disks 45%, the trend is inexorable: flash will just get more attractive every year."

The problems with RAID and Big Drives:

"There are three general problems with RAID: Economic, Managerial, Architectural"

• RAID costs too much
• Management is based on a broken concept [LUN's]
• Parity RAID is architecturally doomed

"The big problem with parity RAID is that I/O rates are flat as capacity rises. 20 years ago a 500 MB drive could do 50 I/O per second (IOPS), or 1 IOPS for every 10 megabytes of capacity. Today, a 150 GB, 15k drive, the ne plus ultra of disk technology, is at 1 IOPS for every 750 MB of capacity. Big SATA drives are at 1 IOPS per several gigabytes. And the trend is down."

What a "Web Business" wants from Storage Vendors:

What a Web Business wants [Don MacAskill of 'smugmug']:

• External DAS for the database servers .. and dual-controller arrays [simplified recovery after server death]
• Spindle love. Typical array has 14.
• No parity RAID. RAID 1+0.
• 15k drive love. Speed is good.
• Love drive enclosures with odd numbers of drives. Makes keeping one hot spare easy.
• Love big battery-backed up write caches in write-back mode. Because super-fast writes are “. . . easily the hardest thing in a DB to scale.”
• Disable array read caching: array caches are small compared to the 32 GB of RAM in the servers. reserve all array cache for writes.
• Disable array pre-fetching: the database knows better than the array.
• Love configurable stripe and chunk sizes. 1 MB+ is good.

"Don should be the ideal array customer: fanatical about protection; lots of data; heavy workload, not afraid to spend money. Yet he isn’t completely satisfied, let alone delighted, by what’s out there. A lot of the engineering that goes into arrays is wasted on him, so he’s paying for a lot of stuff he’ll never use, like parity RAID, pre-fetch and read caching."

And 'the future of Storage'

The future of storage:

"The dominant storage workload of the 21st century. Large file sizes, bandwidth intensive, sequential reads and writes."



"(OLTP) Not going away. The industry is well supplied with kit for OLTP. It will simply be a steadily shrinking piece of the entire storage industry. OLTP will keep growing, just not as fast as big file apps."



"Disk drives: rapidly growing capacity; slowly growing IOPS. Small I/0s are costly. Big sequential I/0s are cheap. Databases have long used techniques to turn small I/Os into larger ones. With big files, you don’t have to."



"The combination of pervasive high-resolution media, consumer-driven storage needs, expensive random I/0s and cheap bandwidth point to a new style of I/O and storage. The late Jim Gray noted that everything in storage today will be in main memory in ten years. A likely corollary is that everything analog that is stored today will be digital in 10 years."