Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Mr Skrunts]

It has come to a time that I now have to consider a dedicated storage system.  I am not technically minded enough to know whats best for me so hence I require advice.

Having taken 1000's of pictures & video's over the years and sorted them into various named folders etc, backed them up to other PC's using Windows 95 through to the current Windows 10 Pro 64bit things have got into quite a disarray with some folders having 50+ copies of the same picture.

So the questions are.

Whats the equivalent (or better) than my old fav program XTree Gold.  Loved how you could define what you could copy/move/delete in the way of folder/structures and file types.

Then a NAS/RAID storage system, tried the Motherboard/software contolled type once and got everything screwed up when trying to retrive it.  So now looking at N.A.S. and thats where I give up.

I currently have 6TB of photo's then copied on to a 2nd 6TB drive, possibly as little as 2 TB once sifted & sorted.

Now looking to stOre multiple file typeS, .xls etc photo's, video's & backups from 2 dash cams.

I reckon for the moment 12TB would suffice for a couple more years but do I need to use a mirror setup (and would that be 1 or 2 NAS units?) or a stripe system.  Also bearing in mind I do a lot of searhing through the drives looking for certain items or folders.

I now have a headache and need to sleep.

Here some of the Brands I have seen on Amazon

Synology
Western Digital
TERRAMASTER
QNAP
Seagate
WD-40
ORICO
SSK
Toshiba
Asustor

TIA for any advice.

[LC0112G]

Try Beyondcompare.

It allows you to compare two files or folders. You can then make sure all the photos called "IMG_1234" or whatever are actually the same, and allows you to concatenate all the files into a single directory structure. You can then delete the duplicates.

[STEMO]

Word for today: Concatenate. I'm off to look it up.



Link things together in a chain or series. Committed to memory.

[deviator]

Regards Mirror -v- Striping

Mirror means the data is copied to both drives (you can use more drives in multiples of 2). This means you have a backup, in theory. However 1 drive acts as the master and the other follows it. This means if the data gets corrupted on the master, it's also copied to the slave.

A Stripe can either contain a level of redundancy or not. EG you can stripe 2 discs, but if either disc fails, you've potentially lost the lot. You can do a 3 disc stripe, you'll only gain the storage of 2 of the discs as the third disc provides the redundancy. Stripes are made to provide redundancy and also a speed advantage, as the data is striped over 3 drives, the data can then be read back from 3 drives. However you are unlikely to see this advantage over a network and with one PC only accessing the data.

Personally I think you are over complicating things. If you have 2 x 6TB, both containing the same data. Make one offline. Sort the data on the other one and once you are certain you are happy with it, you can then replicate that to the other drive. Once done, make the drive offline again (IE disconnected from your PC).

Every now and then reconnect the spare drive and copy over the new stuff. I'm sure there is software to make this easy, but if you are happy to work in DOS, I can recommend robocopy, no not John Murphy. This tool can compare the contents of one drive with another and only copy the files that have changed or are new.

[Mr Skrunts]

Personally I think you are over complicating things. If you have 2 x 6TB, both containing the same data. Make one offline. Sort the data on the other one and once you are certain you are happy with it, you can then replicate that to the other drive. Once done, make the drive offline again (IE disconnected from your PC).

Every now and then reconnect the spare drive and copy over the new stuff. I'm sure there is software to make this easy, but if you are happy to work in DOS, I can recommend robocopy, no not John Murphy. This tool can compare the contents of one drive with another and only copy the files that have changed or are new.

Basically that is the current method.

I have an M.2. boot drive, with a 2 TB data/work/backup drive as I learnt it never a good idea to same to a windows system drive.

Then I have the 2 6TB drives, then over and above that I have 3 4 TB External drives as added back up.

My problem is I forget where I put new pictures or forget where the old folders where stored hence I end up with multiple files after backing up.

I did once see some software that would sort and sift duplicates, but not sure that I would trust it to do aproper job.

[TheBoy]

How many devices (PCs) need to access this?

If its usually just 1, stick to local storage.  Consumer grade RAID chips found on some motherboards are hopeless, so don't bother trying to use them, just stick with simple single drives, plus a backup strategy*


*depending how important the data is, this is where it can get complex. For example, if the data is very important, there is no point keeping backup copies in the same house, as they will burn with the PC.

[Mr Skrunts]

Cheers TB

Stratedgy recently has been the 2 drives but different brands, currently Toshiba & WD

Bought 2 4TB SSHD Seagates and an external 4TB Seagate, 1 SSHD the external disnt last 3 months and because of the personal data I dont trust sending them back to Seagate, so as you can imagine I will never buy any of thier drives again.

Plus through the house I have most of my old systems with stuff stored on them that need moving to back up.  I also have a few drives that wont read and come up as "Foreign " in the disk managemnt.  So got plenty of work to be doing over the next few weeks if I dont get any female distractions.

[Entwood]

If using "offline" .. ie not permanently connected storage as a backup, a very simple to use, effective (and free) tool is "SyncToy" ... I use it simply on the 1st & 15th of each month .. and occasionally do an extra one if I've done a lot of file work

Switch on the external HD, wait a couple of moments for win 10 to "See" the extra drives
Run synctoy "All folder pairs" (it is set up with  pre-selected drives and folders )
When finished ... usually about 20 minutes 
exit synctoy
switch of external drive.

Seemples

First time takes a tad longer as it requires setting up the "folder pairs" and "options"  .. I use "echo" so only changes are actually written/deleted