Omega Owners Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

Search the maintenance guides for answers to 99.999% of Omega questions

Pages: 1 [2]  All   Go Down

Author Topic: Computers - RAID  (Read 439 times)

0 Members and 1 Guest are viewing this topic.

TheBoy

  • Administrator
  • *********
  • Offline Offline
  • Gender: Male
  • Brackley, Northants
  • Posts: 94759
  • Ah, the Silver Bullet
    • Jaaaaaaaag
    • View Profile
Re: Computers - RAID
« Reply #15 on: 04 December 2017, 19:30:22 »

When it comes to speed:

RAID0 - (two disks) slightly faster read & write than non-RAID. Lose one disk, lose ALL your data.
RAID1 - (two disks) faster read, no effect on write speed. Lose one disk, lose nothing.
RAID5 - multiple disks, striped data & parity. Read & write speed may be faster, tolerant of one drive failure.
RAID6 - multiple disks, striped data & parity. Read & write speed may be faster, tolerant of two drive failures.
UnRAID - multiple disks, data is contained entirely on each drive, parity is stored on one or two dedicated drives. Tolerant of one or two failures.

The big difference is that for 'real' RAID (0 through 6) all drives must be an identical capacity. UnRAID will use the full capacity of all drives, even of differing size, as long as the parity drive is largest, because it does not stripe the data across the disk set.

BTW, Synology also make nice NAS boxes, though I have no personal experience :)
Conkers, I'm afraid. Parity stripped RAIDs have abysmal write performance.  Not just slow, abysmal.  Even with a top end SAS controller (well, once the cache fills up). It has to do at least 3 reads, wait for data, compute it, then at least 3 writes. Do not underestimate how massively badly this impacts performance - with good caching (not available on any consumer PC boards, or most cheapo NAS devices), you can get almost usable results if its purely sequential, but very little is...  ...and none is with shared storage.

On most PC boards, even R1 is likely to be slower at both read and write than non-raid, as the compute is done in the driver. Toy level NAS devices suffer similar. As does all the shit like FreeNAS etc (made worse by its near exclusive use of ZFS, and its not a great port, and PC hardware is shite, and the ZIL becomes the bottleneck).  Toy NAS only works with one client - give to 2, performance dies, and you often get timeouts and dataloss, hence can't be recommended for data integrity.  *NOTHING* is faster than local DAS, assuming same drive technology when comparing. Also DAS is more reliable, thus the best integrity.

As to SAS v SATA, they are leagues apart. NCQ helps with SATA (assuming drive, controller and driver (if software RAID, most PC shit will fall into this)), but doesn't even bring into same league as SAS.

I could prove this, as the 2 physical servers currently hosting the OOF VMs are currently on RAID10 on 2.5" SAS drives. The same physical servers also have a RAID10 on 2.5" SATA drives (granted, only 7200rpm spindle drives), used a cheap archival storage (2Tb SAS 10k rpm drives are 600+ each, compared to about 70 for SATA). If I put the OOF primary database on the SATA array, I'd expect it to do the usual Linux thing when its resources can't keep up - start thread blocking, before eventually shitting its pants with a panic. However, straight away, you'd see all the database queries to build a page taking a few seconds, rather than the .2s it takes now if not cached.

SAS SSD takes it to another level again, but I ain't rich enough for them ;D.  But if anybody wants to donate some, preferably HPE in the HPE hotplug carriers (gen8 or later type), >800Gb, my address is:

TheBoy
Brackley


;)
Logged
I don't tolerate bickering, and I'm always grumpy.
And Lizzie Zoom says I'm a heartless bastid...and she's absolutely correct!

Mr Skrunts

  • Over Active Thyroid
  • *********
  • Offline Offline
  • Gender: Male
  • Skruntie Land.
  • Posts: 23677
  • 3.O Elite Saloon with all the toys,
    • 1999 (T) Omega Elite 3.0
    • View Profile
Re: Computers - RAID
« Reply #16 on: 04 December 2017, 22:15:25 »

But if anybody wants to donate some, preferably HPE in the HPE hotplug carriers (gen8 or later type), >800Gb, my address is:

So if money was no object what would the OOF server system consist of. (Purely out of interest) :y
Logged
Ask yourself :  " WHY do I believe what I believe?"

Remember that my opinions expressed here are not representative of the opinions of other members on the OOF Forum.

Kevin Wood

  • Global Moderator
  • *****
  • Offline Offline
  • Gender: Male
  • Alton, Hampshire
  • Posts: 33237
    • 3.2 MV6 (LPG), Westfield
    • View Profile
Re: Computers - RAID
« Reply #17 on: 05 December 2017, 00:34:12 »

The thing is, who gives a damn about performance in a typical domestic setup?

If you don't have the budget for enterprise level storage stuff, the best thing you can do is chuck a load of cheap SATA drives in a NAS or a small Linux box and run software RAID, and keep it backed up.

Any hardware RAID controller costing less than four figures will be shite and, whilst software RAID isn't the be all and end all, when it goes wrong there's a decent chance of recovering it rather than realising it's been gradually corrupting both your live filesystems and your backups for months. Yes, got that tea shirt, thanks, Dell. ::)
Logged
Tech2 services currently available. See TheBoy's price list: http://theboy.omegaowners.com/

TheBoy

  • Administrator
  • *********
  • Offline Offline
  • Gender: Male
  • Brackley, Northants
  • Posts: 94759
  • Ah, the Silver Bullet
    • Jaaaaaaaag
    • View Profile
Re: Computers - RAID
« Reply #18 on: 05 December 2017, 08:34:14 »

The thing is, who gives a damn about performance in a typical domestic setup?

If you don't have the budget for enterprise level storage stuff, the best thing you can do is chuck a load of cheap SATA drives in a NAS or a small Linux box and run software RAID, and keep it backed up.

Any hardware RAID controller costing less than four figures will be shite and, whilst software RAID isn't the be all and end all, when it goes wrong there's a decent chance of recovering it rather than realising it's been gradually corrupting both your live filesystems and your backups for months. Yes, got that tea shirt, thanks, Dell. ::)
My point being that consumer systemboard RAID implementations are semi software, but without the tools to diagnose when things go awry.

Unix and Windows Server software RAIDs are reasonably robust, if somewhat slow, and disk replacement can sometimes be a pain.

In a home environment, I'd recommend keeping it nice and simple, and use far simpler storage - 24/7 availability is not a requirement, all you want is data integrity, and a simple disk system and robust backup regime is the best solution to that IMHO.

If shared storage is needed, I'd be more inclined to share it from a PC using normal Windows/Linux tools than use the cheapest of the consumer NAS devices.  If a NAS is needed, look at the higher end of the consumer range if possible...   ...but a PC running Windows/Linux (depending if your clients are Windows or Unix (as Samba introduces a whole new set of issues to keep on top of)) will likely outperform it.
Logged
I don't tolerate bickering, and I'm always grumpy.
And Lizzie Zoom says I'm a heartless bastid...and she's absolutely correct!

TheBoy

  • Administrator
  • *********
  • Offline Offline
  • Gender: Male
  • Brackley, Northants
  • Posts: 94759
  • Ah, the Silver Bullet
    • Jaaaaaaaag
    • View Profile
Re: Computers - RAID
« Reply #19 on: 05 December 2017, 08:46:15 »

So if money was no object what would the OOF server system consist of. (Purely out of interest) :y
Pretty much what it does now, but more RAM (only a measly 128Gb in each physical), and more storage with more SSD (currently around 28Tb raw spinning media, 4Tb raw SSD), and a bigger backup device (currently LTO5). More 10G networking would be a bonus.

The rest is how I would like it - branded top tier enterprise class server hardware, with branded, top tier disk controllers,  All of which should allow 24/7 reliability to the VMs, even if the VMs themselves need to occasionally be rebooted for patching - for OOF, I don't see the gain in making that HA for the 2 mins a fortnight it needs to reboot.


Most of the outages in the last year have been due to shared storage, where the external storage devices has been unable to cope, or the crap Netgear switches used here (Netgear G724 - defo one to avoid at all costs!) have gone non responsive, in both cases, the result is VM storage corruption.  This is now hopefully resolved by using hyperconverged storage, so the storage is always physically connected to the local hypervisor running that VM.
Logged
I don't tolerate bickering, and I'm always grumpy.
And Lizzie Zoom says I'm a heartless bastid...and she's absolutely correct!

aaronjb

  • Omega Queen
  • ********
  • Offline Offline
  • Gender: Male
  • Northampton
  • Posts: 11284
    • '16 Skoda, '06 BMW 650i
    • View Profile
Re: Computers - RAID
« Reply #20 on: 05 December 2017, 09:14:46 »

In a home environment, I'd recommend keeping it nice and simple, and use far simpler storage - 24/7 availability is not a requirement, all you want is data integrity, and a simple disk system and robust backup regime is the best solution to that IMHO.

Now if you could let me know how I can easily (and with similar expense) back up 32TB (yes, 32TB filled, not capacity) of stuff, that'd be great ;)

The only way I can imagine a robust backup solution for that would be several sets of drives at 1k a pop, cycling them continuously and backing them up pretty much continuously.

Or, y'know, I could just keep my RAID6 array and not worry about it too much ;) (I mean, unless I am very unlucky and lose three drives at once - which could happen..)


Don't forget, TB, there's a bit difference between a corporate environment and Skrunts not wanting to lose his porn collection ;D
Logged
..and it went 'dugga dugga dugga dugga' no more.

Kevin Wood

  • Global Moderator
  • *****
  • Offline Offline
  • Gender: Male
  • Alton, Hampshire
  • Posts: 33237
    • 3.2 MV6 (LPG), Westfield
    • View Profile
Re: Computers - RAID
« Reply #21 on: 05 December 2017, 11:00:03 »

If shared storage is needed, I'd be more inclined to share it from a PC using normal Windows/Linux tools than use the cheapest of the consumer NAS devices.  If a NAS is needed, look at the higher end of the consumer range if possible...   ...but a PC running Windows/Linux (depending if your clients are Windows or Unix (as Samba introduces a whole new set of issues to keep on top of)) will likely outperform it.

The attraction of NAS devices is that they can be "always-on" without consuming a stupid amount of leccy, unlike any usable PC. If all your media, documents, etc. are on a PC it's a real ballache to have to remember to turn it on before you decide to listen to some music in bed using your tablet, etc..

Now if you could let me know how I can easily (and with similar expense) back up 32TB (yes, 32TB filled, not capacity) of stuff, that'd be great ;)

aaronjb has too much porn! :o
Logged
Tech2 services currently available. See TheBoy's price list: http://theboy.omegaowners.com/

aaronjb

  • Omega Queen
  • ********
  • Offline Offline
  • Gender: Male
  • Northampton
  • Posts: 11284
    • '16 Skoda, '06 BMW 650i
    • View Profile
Re: Computers - RAID
« Reply #22 on: 05 December 2017, 11:01:21 »

aaronjb has too much porn! :o

Nah, only half of it is p.. I mean, none of it is porn ;D
Logged
..and it went 'dugga dugga dugga dugga' no more.

Kevin Wood

  • Global Moderator
  • *****
  • Offline Offline
  • Gender: Male
  • Alton, Hampshire
  • Posts: 33237
    • 3.2 MV6 (LPG), Westfield
    • View Profile
Re: Computers - RAID
« Reply #23 on: 05 December 2017, 11:58:27 »

aaronjb has too much porn! :o

Nah, only half of it is p.. I mean, none of it is porn ;D

The other half being those files that nice Damian Green bloke asked you to look after? ;D
Logged
Tech2 services currently available. See TheBoy's price list: http://theboy.omegaowners.com/

TheBoy

  • Administrator
  • *********
  • Offline Offline
  • Gender: Male
  • Brackley, Northants
  • Posts: 94759
  • Ah, the Silver Bullet
    • Jaaaaaaaag
    • View Profile
Re: Computers - RAID
« Reply #24 on: 05 December 2017, 17:04:41 »

In a home environment, I'd recommend keeping it nice and simple, and use far simpler storage - 24/7 availability is not a requirement, all you want is data integrity, and a simple disk system and robust backup regime is the best solution to that IMHO.

Now if you could let me know how I can easily (and with similar expense) back up 32TB (yes, 32TB filled, not capacity) of stuff, that'd be great ;)

The only way I can imagine a robust backup solution for that would be several sets of drives at 1k a pop, cycling them continuously and backing them up pretty much continuously.

Or, y'know, I could just keep my RAID6 array and not worry about it too much ;) (I mean, unless I am very unlucky and lose three drives at once - which could happen..)


Don't forget, TB, there's a bit difference between a corporate environment and Skrunts not wanting to lose his porn collection ;D
But RAID won't back up your data either. Remember RAID is about availability. Virii, controller glitches, human error, smoke etc can all easily reduce 32Tb to SFA with remarkable speed and efficiency ;)
Logged
I don't tolerate bickering, and I'm always grumpy.
And Lizzie Zoom says I'm a heartless bastid...and she's absolutely correct!

TheBoy

  • Administrator
  • *********
  • Offline Offline
  • Gender: Male
  • Brackley, Northants
  • Posts: 94759
  • Ah, the Silver Bullet
    • Jaaaaaaaag
    • View Profile
Re: Computers - RAID
« Reply #25 on: 05 December 2017, 17:11:43 »

The attraction of NAS devices is that they can be "always-on" without consuming a stupid amount of leccy, unlike any usable PC. If all your media, documents, etc. are on a PC it's a real ballache to have to remember to turn it on before you decide to listen to some music in bed using your tablet, etc..
There is a place for always on shared storage. For some it will be cloud, for some it will be NAS type devices.  The cheap ARM based ones (and even Atom based) are pretty dire though, both in performance and integrity.

When something as "good" as, say, the HP gen8 Microserver is available, and pulls 30-35W with a pair of shitty 3Tb Toshiba 3.5" drives (unsurprisingiy one of them has popped up a prefailure warning, *sigh*) is available for around 100 have cashback incentives, its a bit of a non brainer.  And it can run your favourite Windows/Linux OS, which is far more robust than the bastardised toy NAS systems.  Granted, enterprise class it ain't, but its a cheap way to a small, quiet, efficient NAS.  Just don't use that crap onboard controller for RAID5 ;D
Logged
I don't tolerate bickering, and I'm always grumpy.
And Lizzie Zoom says I'm a heartless bastid...and she's absolutely correct!

Kevin Wood

  • Global Moderator
  • *****
  • Offline Offline
  • Gender: Male
  • Alton, Hampshire
  • Posts: 33237
    • 3.2 MV6 (LPG), Westfield
    • View Profile
Re: Computers - RAID
« Reply #26 on: 05 December 2017, 20:11:09 »

The attraction of NAS devices is that they can be "always-on" without consuming a stupid amount of leccy, unlike any usable PC. If all your media, documents, etc. are on a PC it's a real ballache to have to remember to turn it on before you decide to listen to some music in bed using your tablet, etc..
There is a place for always on shared storage. For some it will be cloud, for some it will be NAS type devices.  The cheap ARM based ones (and even Atom based) are pretty dire though, both in performance and integrity.

When something as "good" as, say, the HP gen8 Microserver is available, and pulls 30-35W with a pair of shitty 3Tb Toshiba 3.5" drives (unsurprisingiy one of them has popped up a prefailure warning, *sigh*) is available for around 100 have cashback incentives, its a bit of a non brainer.  And it can run your favourite Windows/Linux OS, which is far more robust than the bastardised toy NAS systems.  Granted, enterprise class it ain't, but its a cheap way to a small, quiet, efficient NAS.  Just don't use that crap onboard controller for RAID5 ;D

Yep, agreed, that's where my money would go but most people want an "appliance". (= they CBA to install software on it, etc.) ::)
Logged
Tech2 services currently available. See TheBoy's price list: http://theboy.omegaowners.com/
Pages: 1 [2]  All   Go Up
 

Page created in 0.253 seconds with 16 queries.