IT question about disc arrays and RAID

[drift_g35]

I am starting to become more of a productive role with my company, working towards being a DB Admin. I asked about a month ago what you suggested the next step be and someone mentions that keeping up to date of arrays was very important. Unaware at how much a whole set up cost I asked my CEO if I could borrow one to play with and learn how they work (I dont know much about them). He laughed and said that our array setup cost about $100,000 with a controller and everything.

So... What I need to know is what less expensive ways can I get some training on this stuff. I was looking online but I dont even know where to start. Does anyone have any suggestions?

Thanks!

[Gman]

You don't need one to learn about them. Most of what you need to know is high-level as to the pro's and con's; how much storage is lost to have redundancy (RAID 1 = 50% vs. RAID5 = 1 disk, etc.), % of reads vs. writes, rebuild times for failed disks, etc.

The management tools for an array will likely change depending on if it's internal storage vs. an external array.

Some basics can be performed with a desktop that has a controller. I have a home server that backs up all of my systems, so I'm not concerned so much about data retention. I used to run HDDs in a RAID 0 configuration for the extra performance, but SSD has made that unnecessary.

[DOC]

3.5" diskettes all the way.

[esaabye]

For DBA's it is all about IOPS which is a function of the disk speed and the number of them in an aggregate. In the old days we would spend lots of time breaking the BD into table groups and speading across different disk types based on performance needs.

Now we buy an san with data progression and let the controllers move blocks around so that the more popular data is on the outside enge of the fastest spindles. Now we are starting to get SSD and large flash arrays with tons of caching at 'reasonable' pricing thus further removing the planning needs.

Take a look at some of the larger san provider sites (EMC, Dell, HP) and some of the niche like Nimble and read the cool-aid. They will all spin it differently but in the end it is a function of speed of disk times number of disk.

[Holger Danske]

Learn the RAID levels and pros/cons. Agree with Essabye it is about IOPs (and latency). You can buy little 4 disk JBOD's with RAID built in for a couple hundred bucks. Also, my buddy was telling me about unraid. They make arrays and offer a free raid sw solution. I have not checked them out, but you may want to look into it if you want a cheap way to learn. If you have any DBA's or Network admins that have been around the block then pick their brains.

[mitch]

One small nit: if you have a raid controller in front of your disks, you can't specify where your data goes at a granularity smaller than the LUN. The old "outside of the spindle" tricks haven't been valid for a number of years. It's still the way to go for making linux raid devices from a bunch of internal disks, though. Google "linux" and "raid" and you should get the software raid how-to. If you have a spare disk you can wipe - there's no reason you can't partition it into 4 or 5 partitions and mirror them to themselves just as a means of learning some of the commands.

Otherwise, this is an exciting time to be in storage. You can now get 100k IOPS on your desktop quite easily with some of the various flash units out there today. I know there are various companies producing well over 1M I/O per second sustained at fiber speed (in this case, newish 8Gb stuff from Qlogic).

If you want something to play around with at home, some of the various projects like OpenFiler and the like are pretty slick. In general it's like horsepower. Sure you can "do raid". How much money do you have?

[ChadAmberg]

If you REALLY want to spend a few bucks and get something to play with, look on Ebay for an older HP or Dell server with a raid controller and at least 4 disks. Probably get it for a couple hundred bucks. With the 4 disk, you can experience RAID 0, 1, and 5 in both 3 and 4 disk configurations.

[james_bond_007]

What these guys said ^^^^^^
and get somewhat familiar with the hardware architecture...

If you will be working with enterprise systems, learn the differences of SAS and SATA, what an SAS Expander is and does, backup strategies and schedules, practice actually restoring data (you won't believe how many people religiously backup data, only to find, after a crash when they go to restore, that NOTHING was ever backed u), hot spares and how to use them, fault tolerances of the various RAID protocols, why S/W RAID is a poor performer, other systems than RAID (ex. Just A Bunch of Disks JBOD), their tradeoffs with RAID (speed/fault tolerance/storage-to-media ratios), what are volumes/spanned volumes, how to increase a volume by adding more disks, etc. ,

Enterprise is all about spending extra $$$ for
1) Preventing downtime
2) Not/Never losing data...at all...even if a user deletes it, you need to be able to recover it (within a reasonable time frame...ymmv)
3) How to get back up as quickly as possible after a crash

[Ashton]

What these guys said ^^^^^^
and get somewhat familiar with the hardware architecture...

If you will be working with enterprise systems, learn the differences of SAS and SATA, what an SAS Expander is and does, backup strategies and schedules, practice actually restoring data (you won't believe how many people religiously backup data, only to find, after a crash when they go to restore, that NOTHING was ever backed u), hot spares and how to use them, fault tolerances of the various RAID protocols, why S/W RAID is a poor performer, other systems than RAID (ex. Just A Bunch of Disks JBOD), their tradeoffs with RAID (speed/fault tolerance/storage-to-media ratios), what are volumes/spanned volumes, how to increase a volume by adding more disks, etc. ,

Enterprise is all about spending extra $$$ for
1) Preventing downtime
2) Not/Never losing data...at all...even if a user deletes it, you need to be able to recover it (within a reasonable time frame...ymmv)
3) How to get back up as quickly as possible after a crash

Define "enterprise"... In my experience it's 2500+ seats. Which makes everything you said correct and applicable. But no 2500+ seat company is going to hire someone for a DBA that doesn't know anything about RAID configurations. For the jobs he could get (without the knowledge) none of those are applicable. Because they can't afford it.

[james_bond_007]

Define "enterprise"... In my experience it's 2500+ seats. Which makes everything you said correct and applicable. But no 2500+ seat company is going to hire someone for a DBA that doesn't know anything about RAID configurations. For the jobs he could get (without the knowledge) none of those are applicable. Because they can't afford it.

I've seen enterprise used in 2 ways:
1) # of users (similar to what you describe) and
2) Style or class of data management - smaller companies often use more sophisticated/robust methods, if they or their customers are in a position to pay to offset the expense. I have seen this with smaller financial clients, government contractors managing sensitive data, clients that are may be audited by various private/government agencies

What I was trying to imply was the he should get familiar with the pros/cons (I.E "LEARN ABOUT") of different approaches, but not that he become an expert at this point. Learning about the higher end stuff will allow him to better understand 1) What CAN be done, if needed and 2) What the trade-offs are by NOT doing them.

About the only thing I recommended him to do is to restore things once in a while, to make ABSOLUTELY sure things are being backed up as planned.

I was also generalizing...I don't know what type of Job he is after at this point.

Reading through some more, I would also say that learning to use/program SQL would be a useful skill, in all cases.