As part of a project we are building a iSCSI storage server. It has 16 500GB SATA disks and to provide redundancy we needed some sort of RAID controller. So we went with the well known 3ware controllers. More specifically a 9550SXU 16ML. The server itself is running CentOS 5 with 2 Dual-Core Intel Xeon processors. This post is just to share my experiences with this controller.
First, everybody that uses a ext3 filesystem on top of this controller should upgrade to firmware 9.4.2. It improves the write performance. In our case we went from around 55MB/s to around 75 MB/s for sequential I/O. That is not bad for a simple firmware upgrade.
But we were having one issue. On initial tests with a iSCSI exported logical volume we did a copy of a directory tree (on the same volume) with a total size of 1GB. And this test took around 6 minutes, that is even less then 3MB/s. To be fair, this structure contains a lot of smaller files, different directories. So a copy also involves a lot of metadata activity. So we did the same test on the storage server itself and we got around 3 minutes there. Mmm, and this should be a top end RAID controller, but we are only getting 6MB/s in this test.
So, just for fun I decided to let the RAID controller export each disk individually and use the Linux software RAID and see what performance that would give. Well, for this specific test the time was 2'50". And for all the other tests I did software RAID outperformed the hardware RAID.
So what did we learn here. First, do not let the numbers of all the different hardware RAID controller vendors foul you. They are sequential I/O test. But most day-to-day I/O patterns are different.
Second, always test yourself. Benchmarks found online can be indicators, but always test everything yourself for your specific case. Third, when using RAID always give software RAID a change. It may save you some money.
As a final note. If we mounted the filesytems as ext2 in this specific test the copy would only take 1 minute (in both HW and SW RAID). So do not forget about ext2, it still has it's advantages.
Subscribe to:
Post Comments (Atom)
32 comments:
This bug is related to RHEL 3, but it could of interest:
https://bugzilla.redhat.com/show_bug.cgi?id=121434
Can you please post how you're measuring your microbenchmarks as you're progressing? For example: what were you using to measure throughput (for your firmware upgrade)? I've built a 5x500GB SW RAID system with a single-core P4 machine, and I'd love to be able to track my performance against the monster you've put together.
That is a good question indeed. I'll do a writeup of how I tested all this in the coming days.
Thanks for your feedback!
RAID5 I'm assuming? I'd be interested to see what the results yield on XFS as well.
I have also had horrible performance on 3ware raid cards and was therefore considering LSI. do you have any input on a less expensive card for use with software raid, it seems all of the controllers that support a large number of drives are also hardware raid cards.
I've also been looking for simple SATA controllers with at least 8 ports. There are a couple. But the biggest challenge is finding one that has multilane connectors. So if someone knows any plane SATA controllers with multilane connectors please speak up :-)
I too was very disappointed of that 3ware card (with 8 ports) on an AMD Centos 5 sys. But then I tried with XFS, disabled cpu frequency scaling and installed irqbalance. Now the performance rocks, at least with iozone. (seq. writes ~180MB/s and read ~380MB/s)
You see I am not sure what did the trick but you should give XFS a try.
With 8 disks (I presume) those numbers aren't bad. Unfortunately we are more in the need for performance with other IO patterns then sequential. The array will also be used to host the filesystems of Xen guests (using iSCSI) and this makes it a bit more complicated to use XFS so we will be sticking with ext3 (or use it as ext2 when we need more speed).
Same here (random IO)
I'm in the process of testing various setups and interested in any idea for this.
Once I played AOC, I did not know how to get strong, someone told me that you must have aoc gold.He gave me some conan gold, he said that I could buy age of conan gold, but I did not have money, then I played it all my spare time. From then on, I got some cheap aoc gold, if I did not continue to play it, I can sell aoc money to anyone who want.
I was a new player in Aion, I did not know what is the aion kina. Someone told me that aion online kina is the game gold, he said that I could buy aion kina, but I did not have money, then I played it all my spare time. From then on, I got some aion gold, if I did not continue to play it, I can sell cheap aion kina to anyone who want.
Get the lates information of Electronics Gadgets Tech gadgets Cool Gadgets New Gadgets Best Gadgets Latest Gadgets Cool New Gadgets</a
Yeah, ext2 still have its benefits. I still support it.
Website SEO
Website SEO
SEO company
Nice tips.But I think there will be a problem in installing it.
Thanks for the tip. I will give it a try.
South dakota insurance companies, insurance agents brokers and insurance guide. Comprehensive information about South dakota car insurance, home owner insurance, health insurance, life insurance and more.
South dakota
Disability Insurance, South dakota
Earthquake insurance, South dakota
Estate Planning Insurance And Legal Services, South dakota
Farm, Crop, & Livestock Insurance, South dakota
Financial Guaranty & Life Insurance, South dakota
Fire Insurance, South dakota
Fleet Insurance, South dakota
Flood Insurance, South dakota
Foreign Insurance, South dakota
Franchising, South dakota
Government Insurance Carriers, South dakota
Group Insurance, South dakota
Health Insurance
Well Whattadya know, yet another great site to add to my reader! Google blog search has you pretty well indexed it seems! you have some brilliant contents!
auto
body shop, international labour
organization, cheap laminate
flooring, auto tune up, labour
organizations, cleaning flooring, transmission
repair, union history
What a great post, I actually found it very thought provoking, you just never know sometimes when a golden nugget of information is going to land at your feet, thanks
discount
laminate flooring, towing
services, union local, flooring
options, auto repair help, trade
union, flooring sales
What a great post, I actually found it very thought provoking, you just never know sometimes when a golden nugget of information is going to land at your feet, thanks
Illinois
Substance Abuse Counselor, Kansas
Substance Abuse Counselor, Ohio
Substance Abuse Counselor, Michigan
Substance Abuse Counselor, Oregon
Substance Abuse Counselor
A fantastic read….very literate and informative. Many thanks….what theme is this you are using and also, where is your RSS button ? wholesale
earring jewelry, plastic
exporter, Connecticut Substance Abuse Counseling, Illinois
Substance Abuse Counseling, Kansas
Substance Abuse Counseling, Ohio
Substance Abuse Counseling
Thanks for giving the clear definition about 3ware hardware RAID and Linux software RAID. This is very useful.
Drum Beat Machine for Beat Makers, Make your own beats. The Best Beat Making Software program, download to make your own original hip hop beats. make beats
Property software comes in different versions to meet different needs with absolute precision
Hello!It is my first time visit your website.It is nice,just like our website.Yes,we are the cheapest ghd outlet.All of the Discount GHD Straighteners on sale here are high quality,and free shipping.I don't know whether you have already have a flat iron,we here have theghd uk for your choice.Australia is our main market as well.ghd australia also on sale.If you want to have a differdent hairstyle with less money,then come to our website,we have the confidence that we will bring you a nice mood everyday.Pick your beauty now,It is suitable for both men and women.Come on,your beauty is in our website waits for you.what's more,with a nice hairstyle,i think you will also need a nice christian louboutin clearance sale and Ghd Australia to match your hairstyle.Ghd Australia sincerely for your service!
At last,best flat iron thanks for sharing your articles.May you happy everyday.
ghd uk
ghd uk
Your Escort Agency offers exclusive and most beautiful London escort girls of various nationalities.
Bestescort4U agency provids best London escorts companionship in the UK.
Hot - Collection is a honest and confidential London escort agency which provides genuine London escorts girls for gentlemen of taste.
Welcome to the leading online Mandarin school that makes you fluent in a short time -- all from the comfort of your home or office!
I am a Chinese teacher,you can learn more about Free Chinese language info: Mandarin Chinese teacher | Online Chinese teachers | Native chinese teacher
Well, it was a wonderful and constructive post and I am glad to read this posting. Thanks for sharing it. free software download
I really enjoyed the posting you have share in this article. free software download full version
The information is very good and very useful.
clipping path service provider
clipping path service
background removal service
image masking service
raster to vector conversion service
color corrections service
neck joint service
Shadow creating service
Post a Comment