Omega Owners Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

Welcome to OOF

Pages: [1] 2  All   Go Down

Author Topic: What a great day back at work......not!!  (Read 2616 times)

0 Members and 1 Guest are viewing this topic.

bob.dent

  • Omega Lord
  • *****
  • Offline Offline
  • Gender: Male
  • Hertfordshire
  • Posts: 6781
  • Drives better than an Omega
    • Mondeo 2.0TDCI Estate
    • View Profile
What a great day back at work......not!!
« on: 09 May 2007, 13:57:39 »

Had an extra day off work after the Bank Holiday and arriving back at work this morning discovered the hard disc on our bloody file server crashed either Monday or yesterday. All accounts and admin data now corrupt. >:( >:( >:(

Fortunately, I have a good backup from last Friday for the accounts data but still means all work my staff did yesterday and today is lost. The tape backup for the server is also a week old due to a problem with the tape drive, which means that all last weeks admin work is also lost. Finally to add insult to injury, our IT manager is out today and even when he's back in tomorrow it'll probably take a couple of days to sort out.

AAAAAHHHHHHHGGGGGG!! Wish I'd stayed in bed this morning! :'( :'( :'(
Logged
I HAVE THE BODY OF AN 18 YEAR OLD.......I KEEP IT IN THE FRIDGE!

Gwilym

  • Senior Member
  • ****
  • Offline Offline
  • Gender: Male
  • Bridgwater, Somerset, UK
  • Posts: 677
    • View Profile
Re: What a great day back at work......not!!
« Reply #1 on: 09 May 2007, 14:25:40 »

Tomorrow is my first day back since last Weds. I hate to think what is waiting for me.
Logged


White '00 3.0 MV6 Estate (Ex Met Police Firearms Car)

x25xe

  • Senior Member
  • ****
  • Offline Offline
  • Gender: Male
  • Hayes, Middlesex
  • Posts: 915
  • Avater removed due to size
    • View Profile
Re: What a great day back at work......not!!
« Reply #2 on: 09 May 2007, 14:58:15 »

Really feel for your there mate.  When did your IT manager know about the tape drive issue?  Hopefully, you will be able to recover quite quickly.
Logged
1998 2.5 V6 Omega CDX, Leather and Electric Pack!

Kevin Wood

  • Global Moderator
  • *****
  • Offline Offline
  • Gender: Male
  • Alton, Hampshire
  • Posts: 36417
    • Jaguar XE 25t, Westfield
    • View Profile
Re: What a great day back at work......not!!
« Reply #3 on: 09 May 2007, 15:03:26 »

I have dug myself out of holes like this before using forensic data recovery. It's not cheap but depending on what you've lost and what it's worth to you it may be an option. Also depends on the nature of the failure.

I do know what it's like trying to rebuild after something like this because it happened to me 6 months ago. I now backup to another server and to tape drives so I have both a secure backup and one that's very quick to get up and running again.

Tapes are all very well but if I go on holiday nobody bothers to check the backups so I need to fall back on something requiring no user intervention!

Well, I hope you manage to sort it all out soon.

Kevin
« Last Edit: 09 May 2007, 15:04:27 by Kevin_Wood »
Logged
Tech2 services currently available. See TheBoy's price list: http://theboy.omegaowners.com/

x25xe

  • Senior Member
  • ****
  • Offline Offline
  • Gender: Male
  • Hayes, Middlesex
  • Posts: 915
  • Avater removed due to size
    • View Profile
Re: What a great day back at work......not!!
« Reply #4 on: 09 May 2007, 15:08:20 »

Quote
Tapes are all very well but if I go on holiday nobody bothers to check the backups so I need to fall back on something requiring no user intervention!

Well, I hope you manage to sort it all out soon.

Kevin

I know exactly what you mean.  I am going on holiday soon (well away from work at least) and can't rely on anyone here to change the tape each day.  If they do, it won't be dated and the jobs certainly won't be checked.  Will be taking a laptop with a 3G card with me so I can check remotly.

There was a time here when one of my drives (a DLT) failed I was starting the back up at 19:00 and coming back each night at around 00:30 to feed it the second tape.  I did this for about 4 months.  Damn near killed me though!
Logged
1998 2.5 V6 Omega CDX, Leather and Electric Pack!

bob.dent

  • Omega Lord
  • *****
  • Offline Offline
  • Gender: Male
  • Hertfordshire
  • Posts: 6781
  • Drives better than an Omega
    • Mondeo 2.0TDCI Estate
    • View Profile
Re: What a great day back at work......not!!
« Reply #5 on: 09 May 2007, 15:21:16 »

Quote
Really feel for your there mate.  When did your IT manager know about the tape drive issue?  Hopefully, you will be able to recover quite quickly.

I think he knew about it day after it went t*ts up and he's waiting for a replacement unit to arrive. Don't think it's going to be a major problem now, just a couple of days re-inputting once it's up and running again and the backups are restored. :-/
Logged
I HAVE THE BODY OF AN 18 YEAR OLD.......I KEEP IT IN THE FRIDGE!

bob.dent

  • Omega Lord
  • *****
  • Offline Offline
  • Gender: Male
  • Hertfordshire
  • Posts: 6781
  • Drives better than an Omega
    • Mondeo 2.0TDCI Estate
    • View Profile
Re: What a great day back at work......not!!
« Reply #6 on: 09 May 2007, 15:29:44 »

Quote
I have dug myself out of holes like this before using forensic data recovery. It's not cheap but depending on what you've lost and what it's worth to you it may be an option. Also depends on the nature of the failure.

I do know what it's like trying to rebuild after something like this because it happened to me 6 months ago. I now backup to another server and to tape drives so I have both a secure backup and one that's very quick to get up and running again.

Tapes are all very well but if I go on holiday nobody bothers to check the backups so I need to fall back on something requiring no user intervention!

Well, I hope you manage to sort it all out soon.

Kevin

Thanks Kev, hopefully shouldn't be too much of a nightmare as once the server is up and running again we can restore the last good backup tape from a week ago plus I have a good backup of the accounts data from Friday evening. Trouble is, it's unlikely to be up and running again until Friday as I suspect a new hard disk will have to be ordered, so work is going to build up a fair bit. Could have been worse I suppose! :-/
Logged
I HAVE THE BODY OF AN 18 YEAR OLD.......I KEEP IT IN THE FRIDGE!

TheBoy

  • Administrator
  • *****
  • Offline Offline
  • Gender: Male
  • Brackley, Northants
  • Posts: 107023
  • I Like Lockdown
    • Whatever Starts
    • View Profile
Re: What a great day back at work......not!!
« Reply #7 on: 09 May 2007, 19:48:19 »

Not using a fault tolerant disk setup  :o
Logged
Grumpy old man

bob.dent

  • Omega Lord
  • *****
  • Offline Offline
  • Gender: Male
  • Hertfordshire
  • Posts: 6781
  • Drives better than an Omega
    • Mondeo 2.0TDCI Estate
    • View Profile
Re: What a great day back at work......not!!
« Reply #8 on: 10 May 2007, 09:11:46 »

Quote
Not using a fault tolerant disk setup  :o

erm..........a what? :-?

Sorry Jaime, I'm not that well up on IT technicalities, not sure what this is. :-/ My only concerns are when there's something wrong with the damn system. I must admit, I don't have the greatest faith in our IT department as issues always take far too long to get resolved.

**Update** Just asked out IT guy about it and he said we don't use this kind of set up due to the cost and the trade off in processing power.
« Last Edit: 10 May 2007, 09:26:31 by bob.dent »
Logged
I HAVE THE BODY OF AN 18 YEAR OLD.......I KEEP IT IN THE FRIDGE!

Kevin Wood

  • Global Moderator
  • *****
  • Offline Offline
  • Gender: Male
  • Alton, Hampshire
  • Posts: 36417
    • Jaguar XE 25t, Westfield
    • View Profile
Re: What a great day back at work......not!!
« Reply #9 on: 10 May 2007, 09:44:47 »

Quote
**Update** Just asked out IT guy about it and he said we don't use this kind of set up due to the cost and the trade off in processing power.

It doesn't have to cost that much or consume much processing power, TBH. Depends how you implement it. In some cases performance will be boosted because hard drives are the biggest bottleneck in a server and if you've got multiple drives with the same data on, you can read from them in parallel.

Hard drives are the one thing in a server that have a very finite life so it does make sense to build-in some fault tolerance. In the simplest case, you install 2 drives and "mirror" them. Anything that gets written to disc goes to both drives. You effectively maintain 2 drives with exactly the same data on. When one dies, the system switches to using the remaining working drive. You replace the failed drive and the system then rebuilds the "mirror" by copying everything onto the new drive. The server can even be up and running, albeit with reduced performance, during this whole process.

You can get hardware controllers that do all this for you, so it doesn't take any resources from the CPU of the server itself, or you can do it all in software, which means it's much cheaper but does use some resources of the CPU itself.

I really would look into implementing some kind of fault tolerance. A single hard drive failure is too common an occurrence for it to take your server down.

Kevin
Logged
Tech2 services currently available. See TheBoy's price list: http://theboy.omegaowners.com/

bob.dent

  • Omega Lord
  • *****
  • Offline Offline
  • Gender: Male
  • Hertfordshire
  • Posts: 6781
  • Drives better than an Omega
    • Mondeo 2.0TDCI Estate
    • View Profile
Re: What a great day back at work......not!!
« Reply #10 on: 10 May 2007, 09:58:14 »

Quote
Quote
**Update** Just asked out IT guy about it and he said we don't use this kind of set up due to the cost and the trade off in processing power.

It doesn't have to cost that much or consume much processing power, TBH. Depends how you implement it. In some cases performance will be boosted because hard drives are the biggest bottleneck in a server and if you've got multiple drives with the same data on, you can read from them in parallel.

Hard drives are the one thing in a server that have a very finite life so it does make sense to build-in some fault tolerance. In the simplest case, you install 2 drives and "mirror" them. Anything that gets written to disc goes to both drives. You effectively maintain 2 drives with exactly the same data on. When one dies, the system switches to using the remaining working drive. You replace the failed drive and the system then rebuilds the "mirror" by copying everything onto the new drive. The server can even be up and running, albeit with reduced performance, during this whole process.

You can get hardware controllers that do all this for you, so it doesn't take any resources from the CPU of the server itself, or you can do it all in software, which means it's much cheaper but does use some resources of the CPU itself.

I really would look into implementing some kind of fault tolerance. A single hard drive failure is too common an occurrence for it to take your server down.

Kevin

Thanks for the explanation Kev, out IT guy gave me a similar explanation. I'm really not sure why we don't use such a system but we do have to keep our costs to a mimimum at the moment so could be the main reason. When the automatic tape backups are running properly I guess disk failure is not so much of a problem apart from the delay in having to fit and setup a new hard disk.
Logged
I HAVE THE BODY OF AN 18 YEAR OLD.......I KEEP IT IN THE FRIDGE!

TheBoy

  • Administrator
  • *****
  • Offline Offline
  • Gender: Male
  • Brackley, Northants
  • Posts: 107023
  • I Like Lockdown
    • Whatever Starts
    • View Profile
Re: What a great day back at work......not!!
« Reply #11 on: 10 May 2007, 12:30:17 »

Quote
**Update** Just asked out IT guy about it and he said we don't use this kind of set up due to the cost and the trade off in processing power.
Sounds like they are too stupid to either consider it or implement it....
Logged
Grumpy old man

bob.dent

  • Omega Lord
  • *****
  • Offline Offline
  • Gender: Male
  • Hertfordshire
  • Posts: 6781
  • Drives better than an Omega
    • Mondeo 2.0TDCI Estate
    • View Profile
Re: What a great day back at work......not!!
« Reply #12 on: 10 May 2007, 12:49:05 »

Quote
Quote
**Update** Just asked out IT guy about it and he said we don't use this kind of set up due to the cost and the trade off in processing power.
Sounds like they are too stupid to either consider it or implement it....

Certainly sounds like something we ought to have in place, but we are a very small company as is our IT dept which consists of..............just one! He's generally reasonably good with most IT issues but I do question his ability with higher level issues. Also, he's our Field Engineer Manager and half the time he's out of the office dealing with customers. Not an ideal situation, but I don't think our resources would stretch to a dedicated IT dept.
Logged
I HAVE THE BODY OF AN 18 YEAR OLD.......I KEEP IT IN THE FRIDGE!

TheBoy

  • Administrator
  • *****
  • Offline Offline
  • Gender: Male
  • Brackley, Northants
  • Posts: 107023
  • I Like Lockdown
    • Whatever Starts
    • View Profile
Re: What a great day back at work......not!!
« Reply #13 on: 10 May 2007, 12:57:22 »

Quote
Quote
Quote
**Update** Just asked out IT guy about it and he said we don't use this kind of set up due to the cost and the trade off in processing power.
Sounds like they are too stupid to either consider it or implement it....

Certainly sounds like something we ought to have in place, but we are a very small company as is our IT dept which consists of..............just one! He's generally reasonably good with most IT issues but I do question his ability with higher level issues. Also, he's our Field Engineer Manager and half the time he's out of the office dealing with customers. Not an ideal situation, but I don't think our resources would stretch to a dedicated IT dept.
My brothers retail business have fault tolerant disks at both his sites (linked by a wireless link, apparently capable of up to 16km!).

Unfortunately, the disk controller failed in one today. Doh!  Still, I talked him through nicking the controller from his Citrix server and fitting in the failed SQL Server to get his tills up and running again.

He has no IT staff.
Logged
Grumpy old man

bob.dent

  • Omega Lord
  • *****
  • Offline Offline
  • Gender: Male
  • Hertfordshire
  • Posts: 6781
  • Drives better than an Omega
    • Mondeo 2.0TDCI Estate
    • View Profile
Re: What a great day back at work......not!!
« Reply #14 on: 10 May 2007, 13:09:58 »

Quote
Quote
Quote
Quote
**Update** Just asked out IT guy about it and he said we don't use this kind of set up due to the cost and the trade off in processing power.
Sounds like they are too stupid to either consider it or implement it....

Certainly sounds like something we ought to have in place, but we are a very small company as is our IT dept which consists of..............just one! He's generally reasonably good with most IT issues but I do question his ability with higher level issues. Also, he's our Field Engineer Manager and half the time he's out of the office dealing with customers. Not an ideal situation, but I don't think our resources would stretch to a dedicated IT dept.
My brothers retail business have fault tolerant disks at both his sites (linked by a wireless link, apparently capable of up to 16km!).

Unfortunately, the disk controller failed in one today. Doh!  Still, I talked him through nicking the controller from his Citrix server and fitting in the failed SQL Server to get his tills up and running again.

He has no IT staff.

Ahhhhh.........the benefits of having an IT literate brother!! ;)
Logged
I HAVE THE BODY OF AN 18 YEAR OLD.......I KEEP IT IN THE FRIDGE!
Pages: [1] 2  All   Go Up
 

Page created in 0.015 seconds with 17 queries.