Omega Owners Forum
Chat Area => General Discussion Area => Topic started by: Martin_1962 on 27 September 2009, 18:39:17
-
2 tapes already over 70GB
I have about 20 to do and I do not want to fill up my HDD.
What is the best way of archiving lots of video data, lets say over 600GB?
Thanks.
Dumping to MiniDV would take about 50 tapes, DVD about 150 or so single layers.
So what is the best answer?
Loosy compression is not, it has already lossy compressed to DV compression which is 5:1 over uncompressed.
-
Buy an external harddrive :y
-
I use Pinnacle software to convert tape to DVD/HDD.
Maybe you get a big separate drive say 1 terrabyte and store it on there.
-
External 1TB, or even leave it in but on sleep.
Then I am also looking at a big stack of MiniDV 30 or so of them.
Not applying any more compression unless lossless.
-
2 choices spring to mind. BluRay or Ext Hard Drive. 1TB drive now arround £50 1.5TB starting to get chaaper.
I currently have loads of stuff on 6 x 1TB drives, and if I need it quick I just pop int in a desk top Sata Caddy till I build a storage server.
-
Get a usb hardrive caddy and then use .Rar to squash everything.
eddie
-
BluRay - 1 tape per disc at DV compression.
Burner - waiting for under £50
caddy system - hmm
-
Don't think RAR or ZIP will shrink video material
-
http://www.aria.co.uk/Products?search=32mb+cache&p_order=price_asc&p=
Plenty of choice for large hard drives, get one and put it in a caddy, if you got 2 then consider a NAS Box, more exspensive but could mirror the data for safety.
-
Agreed Skruntie..
external hard disk .. will be faster and reliable..
second option is blue ray..
and imho I would stay away from compression..
-
Specify EXACTLY what you mean by archiving.
If you are talking basics (space versus quality), then DivX/XviD is the way to go (you capture the source in raw AVI, then convert down).
-
Specify EXACTLY what you mean by archiving.
If you are talking basics (space versus quality), then DivX/XviD is the way to go (you capture the source in raw AVI, then convert down).
Moving lots of camera footage off analogue tapes for future use, possibly reediting.
But DIVX is not lossless. I have already used 5:1 compression by using DV encoding.
-
Specify EXACTLY what you mean by archiving.
If you are talking basics (space versus quality), then DivX/XviD is the way to go (you capture the source in raw AVI, then convert down).
Moving lots of camera footage off analogue tapes for future use, possibly reediting.
But DIVX is not lossless. I have already used 5:1 compression by using DV encoding.
You have no choice then Martin, you have to capture in raw AVI.
Raw AVI - around 14GB per hour (with stereo audio at 48KHz).
As for drive space, what's your budget?
-
External HDDs by nature aren't a good long term storage medium.
For 600G of long(ish) term storage of highly compressed data on to a single piece of media, LTO IV is the solution that springs to mind, but don't expect change from £2.50 ;D
-
External HDDs by nature aren't a good long term storage medium.
I'll assume you are talking single drives?
-
External HDDs by nature aren't a good long term storage medium.
I'll assume you are talking single drives?
Portable disks are inherently unreliable, so even using likes of ADG/R6, its still a probability, moreso if disks are left unused for long periods.
-
Portable disks are inherently unreliable, so even using likes of ADG/R6, its still a probability, moreso if disks are left unused for long periods.
A single drive is inherently unreliable, regardless of whether it's portable or not.
Although, as you rightly say, portable is even more susceptible to failure.
For an equal balance of capacity, redundancy, speed, reliability, and features...I'd be looking at RAID 5 minimum....although the OP may not have that much money.
-
Portable disks are inherently unreliable, so even using likes of ADG/R6, its still a probability, moreso if disks are left unused for long periods.
A single drive is inherently unreliable, regardless of whether it's portable or not.
Although, as you rightly say, portable is even more susceptible to failure.
For an equal balance of capacity, redundancy, speed, reliability, and features...I'd be looking at RAID 5 minimum....although the OP may not have that much money.
If I was forced down the portable HDD route, I'd be looking at a system that as a minimum could cope with 2 simultaneous drive failures. I've never seen such a 'portable' device, and if made from seperate cheapo disks, would lack any kind of central controller, probably relying on OS level software fault tolerance. The built in ftdisk.sys in Windows is a hugely, cut-down, limited version of Veritas Volume Manager. VxVM is around £3.5k iirc, so 'cheap' disk solutions end up expensive.
-
Portable disks are inherently unreliable, so even using likes of ADG/R6, its still a probability, moreso if disks are left unused for long periods.
A single drive is inherently unreliable, regardless of whether it's portable or not.
Although, as you rightly say, portable is even more susceptible to failure.
For an equal balance of capacity, redundancy, speed, reliability, and features...I'd be looking at RAID 5 minimum....although the OP may not have that much money.
If I was forced down the portable HDD route, I'd be looking at a system that as a minimum could cope with 2 simultaneous drive failures. I've never seen such a 'portable' device, and if made from seperate cheapo disks, would lack any kind of central controller, probably relying on OS level software fault tolerance. The built in ftdisk.sys in Windows is a hugely, cut-down, limited version of Veritas Volume Manager. VxVM is around £3.5k iirc, so 'cheap' disk solutions end up expensive.
So if £800 bought you 6TB of storage, with hot-swap disks, guaranteed 100% redundancy, Web server, FTP server, Print server, 3x USB 2.0 ports, 1GB of onboard RAM, and only consumed 55W in use (among other things), while measuring 8" high, by 5" wide, by 9" deep (http://i680.photobucket.com/albums/vv166/KillerWatt_1K/OOF/DSC00267.jpg)...you'd be interested then? ;D
-
Storing a video file as one big lump is Dodgy. Any part of it gets corrupted etc then its knackered.
By using .Rar and splitting it up you can use 'par' files to rebuild the broken bits if the recombining complains.
http://www.techsono.com/faq/par.html
In all fairness its main use is for transmitting large files over the net.
I should warn though,using an external usb drive is not foolproof-I recently lost a couple of films this way.
I didn't check to see if the transfer was fully intact before deleting the original!
eddie
-
Raw is 5 times bigger than AVI - I have handled it in the past but DV is good enough, and easier to handle, all edit packages and also encoders drop it down anyway.
Virtualdub (I think does raw)
For now sitting on drive D of my PC, as long as I am not doing any editing I will be OK.
Now thinking that archiving is a nightmare and perhaps making sure I still have a working SLF1 is more important.
Perhaps short term on D drive, when work is better buy a couple of SATA 1TB one external and one internal
-
On my 5th tape now, 130GB already on HDD.
The F1 is enjoying its long run of playback, not bad for video recorder of the year 1982.
Some nice shots of a GWR Pannier tank on screen while typing this
-
Portable disks are inherently unreliable, so even using likes of ADG/R6, its still a probability, moreso if disks are left unused for long periods.
A single drive is inherently unreliable, regardless of whether it's portable or not.
Although, as you rightly say, portable is even more susceptible to failure.
For an equal balance of capacity, redundancy, speed, reliability, and features...I'd be looking at RAID 5 minimum....although the OP may not have that much money.
If I was forced down the portable HDD route, I'd be looking at a system that as a minimum could cope with 2 simultaneous drive failures. I've never seen such a 'portable' device, and if made from seperate cheapo disks, would lack any kind of central controller, probably relying on OS level software fault tolerance. The built in ftdisk.sys in Windows is a hugely, cut-down, limited version of Veritas Volume Manager. VxVM is around £3.5k iirc, so 'cheap' disk solutions end up expensive.
So if £800 bought you 6TB of storage, with hot-swap disks, guaranteed 100% redundancy, Web server, FTP server, Print server, 3x USB 2.0 ports, 1GB of onboard RAM, and only consumed 55W in use (among other things), while measuring 8" high, by 5" wide, by 9" deep (http://i680.photobucket.com/albums/vv166/KillerWatt_1K/OOF/DSC00267.jpg)...you'd be interested then? ;D
Personally, no. Others may be though.
As for the 100% redundancy claim, we both know that is rubbish, particularly if used as a portable device (though getting a bit large to be portable)
-
As for the 100% redundancy claim, we both know that is rubbish, particularly if used as a portable device (though getting a bit large to be portable)
It's not that large J, and while we both know that 100% isn't possible in real life no matter what,......what do you think the chances are of 4 independant hard drives failing all at once?
-
As for the 100% redundancy claim, we both know that is rubbish, particularly if used as a portable device (though getting a bit large to be portable)
It's not that large J, and while we both know that 100% isn't possible in real life no matter what,......what do you think the chances are of 4 independant hard drives failing all at once?
Seen 2 simultaneously fail in a 5 disk Raid 5 when a 3rd disk was being replaced. They don't believe it was electrical issue during hot withdrawal, but flex in the drive bay as the old drive was stiff.
Believe it or not, the server stayed up, and due to Netware caching, we copied off all virtually all of the data :o.
Not had too much luck with hot plug - I once changed a 500Mb disk in a Proliant, and HOT plug was apt - bloody thing started smoking, and the NetWare server promptly abended. Couldn't recover diddly squat off that one :-[
Went through a patch in the early part of millenium when temps were more critical that any glitch on environment control would wipe out 30 odd disks a day, sometimes on same array.
So, yes, it does happen ;D. But I agree, very rare :y
But in Martin Imber's case, I was under impression he wanted a long term archive, and in that case, I simply wouldn't use disks - if you leave on, there is significant risk of corruption, if you switch off, you run high risk of drive sticking.
As an aside, it is most definately possible for 2 disks in a 2 disk mirror to break :-X
-
As for the 100% redundancy claim, we both know that is rubbish, particularly if used as a portable device (though getting a bit large to be portable)
It's not that large J, and while we both know that 100% isn't possible in real life no matter what,......what do you think the chances are of 4 independant hard drives failing all at once?
As an aside, I thought the Netgear stuff could only do R0,1 and 5, and max of 1 online spare? Must admit, not looked for a while (pay little attention to Netgear, as don't have much faith in their firmware writers)
-
As for the 100% redundancy claim, we both know that is rubbish, particularly if used as a portable device (though getting a bit large to be portable)
It's not that large J, and while we both know that 100% isn't possible in real life no matter what,......what do you think the chances are of 4 independant hard drives failing all at once?
Seen 2 simultaneously fail in a 5 disk Raid 5 when a 3rd disk was being replaced. They don't believe it was electrical issue during hot withdrawal, but flex in the drive bay as the old drive was stiff.
Believe it or not, the server stayed up, and due to Netware caching, we copied off all virtually all of the data :o.
Not had too much luck with hot plug - I once changed a 500Mb disk in a Proliant, and HOT plug was apt - bloody thing started smoking, and the NetWare server promptly abended. Couldn't recover diddly squat off that one :-[
Went through a patch in the early part of millenium when temps were more critical that any glitch on environment control would wipe out 30 odd disks a day, sometimes on same array.
So, yes, it does happen ;D. But I agree, very rare :y
But in Martin Imber's case, I was under impression he wanted a long term archive, and in that case, I simply wouldn't use disks - if you leave on, there is significant risk of corruption, if you switch off, you run high risk of drive sticking.
As an aside, it is most definately possible for 2 disks in a 2 disk mirror to break :-X
Hmm so what is the answer then?
-
As for the 100% redundancy claim, we both know that is rubbish, particularly if used as a portable device (though getting a bit large to be portable)
It's not that large J, and while we both know that 100% isn't possible in real life no matter what,......what do you think the chances are of 4 independant hard drives failing all at once?
As an aside, I thought the Netgear stuff could only do R0,1 and 5, and max of 1 online spare? Must admit, not looked for a while (pay little attention to Netgear, as don't have much faith in their firmware writers)
I bought my NV10's BEFORE Netgear got involved ;)
-
Hmm so what is the answer then?
Given the amount of physical data you want to store, hard disk really is the only viable option if you are like most of us (average man, average wage, average family, etc).
However, I'd be using a second disk as a sole backup to the first one as an absolute minimum.
-
Hmm so what is the answer then?
Given the amount of physical data you want to store, hard disk really is the only viable option if you are like most of us (average man, average wage, average family, etc).
However, I'd be using a second disk as a sole backup to the first one as an absolute minimum.
Another Samsung drive when funds permit then.
Half tempted to spread it over a large number of DVDs - but would that kill my burner?
-
Not sure if it's what you're after but a storage solution I have is a Drobo mk2. It's like a RAID enclosure but a lot more simple. I have many many gigabytes of video, either rips from DVDs or downloaded tv. I got the Drobo because I couldn't spare the time for countless DVD burns as backup. :y
It takes upto 4 sata drives, you choose what size goes in - mix & match if you like - and if there's more than 1 it has redundancy for 1 drive failing. I currently have 3 1TB and 1 500GB drive in there so total storage is actually 2.3TB. I did start with a couple of 500GB and 320 GBs, it's not fussed what goes in!
It has firewire 800 and usb connectivity and these days you can get a droboshare free which adds networking too.
Some consider it an expensive box, i have had no problems and the peace of mind and simplicity are priceless. It saves any hassle/thoughts of raid configs and having to have matching drives.
My friend has a DroboPro which takes 8 drives in one 19" rackmount unit and can do redundancy for 2 drives.... that one really is overkill for most home use!
If you want to be burning DVDs then DVD writers are cheap as chips these days in the unlucky event you wear yours out!
just my late night tuppence worth.... toodles!
-
Half tempted to spread it over a large number of DVDs - but would that kill my burner?
At one point in time, some manufacturers were writing it in to the drive firmware that you could only do 1000 burns (although they have stopped that since).
However, as you can buy brand new 16x DVD-RW drives for £16 inc VAT, it doesn't matter whether you knock the 'dangle berries' out of the drive or not.
-
Large capacity drives prices are still coming down.
http://www.aria.co.uk/SuperSpecials/newsletter?productId=37094&utm_source=newsletter&utm_medium=email&utm_content=1.5TB+Samsung+EcoGreen+F2+SATA-2+Hard+Drive&utm_campaign=newsletter300909
-
Large capacity drives prices are still coming down.
http://www.aria.co.uk/SuperSpecials/newsletter?productId=37094&utm_source=newsletter&utm_medium=email&utm_content=1.5TB+Samsung+EcoGreen+F2+SATA-2+Hard+Drive&utm_campaign=newsletter300909
Nice price for the size, shame it's been lumbered with a 5400rpm spindle though.
-
Large capacity drives prices are still coming down.
http://www.aria.co.uk/SuperSpecials/newsletter?productId=37094&utm_source=newsletter&utm_medium=email&utm_content=1.5TB+Samsung+EcoGreen+F2+SATA-2+Hard+Drive&utm_campaign=newsletter300909
Nice price for the size, shame it's been lumbered with a 5400rpm spindle though.
True, but not critical just for storing wads of big files.
-
Large capacity drives prices are still coming down.
http://www.aria.co.uk/SuperSpecials/newsletter?productId=37094&utm_source=newsletter&utm_medium=email&utm_content=1.5TB+Samsung+EcoGreen+F2+SATA-2+Hard+Drive&utm_campaign=newsletter300909
Nice price for the size, shame it's been lumbered with a 5400rpm spindle though.
True, but not critical just for storing wads of big files.
Critical, no.
Noticeable when reading & writing, definately.
-
5400 is the new trend on large drives. The drive manufacturers claim similar performance due to better caching and firmware, but obviously if doing large file copies, the slower speed will eventually show.
-
5400 is the new trend on large drives. The drive manufacturers claim similar performance due to better caching and firmware, but obviously if doing large file copies, the slower speed will eventually show.
Power consumption is usually a little lower, though, which can be important in an always-on application where speed isn't the be all...
Kevin
-
5400 is the new trend on large drives. The drive manufacturers claim similar performance due to better caching and firmware, but obviously if doing large file copies, the slower speed will eventually show.
Power consumption is usually a little lower, though, which can be important in an always-on application where speed isn't the be all...
Kevin
Exactly, which is why it tends to be only on large drives currently (in the consumer section) - many want large capacity, but infrequent access.
The Enterprise market has also gone that way, but that is because there are real savings to be made in energy reduction
-
D: is up to 405GB and new captures along up to 250GB
-
Hehe, I have annother 1TB drive coming on wednesday (Trying a WD one this time - srill 32mb cache tho) current hard drive is filling up far too fast since I bought the Fuji S2000HD camera. Been having a play with the multi shot and continous pic taking options.
-
I noticed Tesco have a WD 1Gb external for under £70, which I thought was quite good for a non mail order place.
I really cannot stress this highly enough, rarely used HDDs make a piss poor backup regime. I've fallen foul of that over the weekend. Fortunately, I can confirm that LTO tapes are more reliable, though obviously out of reach for many.
-
I noticed Tesco have a WD 1Gb external for under £70, which I thought was quite good for a non mail order place.
I really cannot stress this highly enough, rarely used HDDs make a piss poor backup regime. I've fallen foul of that over the weekend. Fortunately, I can confirm that LTO tapes are more reliable, though obviously out of reach for many.
I have learnt the hard way with most types of back up. I have found the best and probably the laziest way for me is the Hard drive, with regular backups to annother machine. My pictices tend to get copied to evey machine that gets booted up (and there are quite a few :-[) Plus I have a dediacted PC just for video clips and pictures.
Am wanting a stand alone PC now where I can sycronise a folder to backup/duplicate/syscronise each of the "My Documents" folders on each machine. Any ides for software to deal with that would be appreciated. TIA. :y
-
I noticed Tesco have a WD 1Gb external for under £70, which I thought was quite good for a non mail order place.
I really cannot stress this highly enough, rarely used HDDs make a piss poor backup regime. I've fallen foul of that over the weekend. Fortunately, I can confirm that LTO tapes are more reliable, though obviously out of reach for many.
I have learnt the hard way with most types of back up. I have found the best and probably the laziest way for me is the Hard drive, with regular backups to annother machine. My pictices tend to get copied to evey machine that gets booted up (and there are quite a few :-[) Plus I have a dediacted PC just for video clips and pictures.
Am wanting a stand alone PC now where I can sycronise a folder to backup/duplicate/syscronise each of the "My Documents" folders on each machine. Any ides for software to deal with that would be appreciated. TIA. :y
I know its extreme, but I had 3 disks fail over the weekend.
Take a look at Home Server, which automatically backs up your PCs. Not ever used it myself, so don't know full ins and outs
-
Another option, though limits storage, and it hammers your net connection, but MS Live Mesh (www.mesh.com). Probably not suitable for your needs though
-
I noticed Tesco have a WD 1Gb external for under £70, which I thought was quite good for a non mail order place.
I really cannot stress this highly enough, rarely used HDDs make a piss poor backup regime. I've fallen foul of that over the weekend. Fortunately, I can confirm that LTO tapes are more reliable, though obviously out of reach for many.
I have learnt the hard way with most types of back up. I have found the best and probably the laziest way for me is the Hard drive, with regular backups to annother machine. My pictices tend to get copied to evey machine that gets booted up (and there are quite a few :-[) Plus I have a dediacted PC just for video clips and pictures.
Am wanting a stand alone PC now where I can sycronise a folder to backup/duplicate/syscronise each of the "My Documents" folders on each machine. Any ides for software to deal with that would be appreciated. TIA. :y
I know its extreme, but I had 3 disks fail over the weekend. Ouch, was that on the OOF server?
Take a look at Home Server, which automatically backs up your PCs. Not ever used it myself, so don't know full ins and outs Cheers Jamie, will look into it. :y :y
-
I have had 3 Maxtors fail on me, one had some ISOs on them, lost a few - if I want them it wil be tapes from loft and recapture.
Other two were work and the last one I actually managed the get the VO repository off intact before total failure.
-
I have had 3 Maxtors fail on me, one had some ISOs on them, lost a few - if I want them it wil be tapes from loft and recapture.
Other two were work and the last one I actually managed the get the VO repository off intact before total failure.
Used to swear by maxtor, but had a bad run on them about 4 years ago, wont touch them with a berge pole now.
Never had an issue with Hitachi drives, allways found them good value for money.
-
I have had 3 Maxtors fail on me, one had some ISOs on them, lost a few - if I want them it wil be tapes from loft and recapture.
Other two were work and the last one I actually managed the get the VO repository off intact before total failure.
Used to swear by maxtor, but had a bad run on them about 4 years ago, wont touch them with a berge pole now.
Never had an issue with Hitachi drives, allways found them good value for money.
All much of a muchness to be honest. Had a hot-plug Hitachi start smoking once :o
-
I have had 3 Maxtors fail on me, one had some ISOs on them, lost a few - if I want them it wil be tapes from loft and recapture.
Other two were work and the last one I actually managed the get the VO repository off intact before total failure.
Used to swear by maxtor, but had a bad run on them about 4 years ago, wont touch them with a berge pole now.
Never had an issue with Hitachi drives, allways found them good value for money.
All much of a muchness to be honest. Had a hot-plug Hitachi start smoking once :o
Before or after you "educated" it with Sammie?
-
I have had 3 Maxtors fail on me, one had some ISOs on them, lost a few - if I want them it wil be tapes from loft and recapture.
Other two were work and the last one I actually managed the get the VO repository off intact before total failure.
Used to swear by maxtor, but had a bad run on them about 4 years ago, wont touch them with a berge pole now.
Never had an issue with Hitachi drives, allways found them good value for money.
All much of a muchness to be honest. Had a hot-plug Hitachi start smoking once :o
Before or after you "educated" it with Sammie?
Or during ::)
-
Used to swear by maxtor, but had a bad run on them about 4 years ago, wont touch them with a berge pole now.
Every drive manufacturer has had a "flawed" production run at some point in time.