Unfortunately we suffered a bizarre incident last night that caused everything here to fail... ...including the alerting system, which meant it went undetected until early this morning.
Clearly we are back up and running, but we will have to go down again for an hour or so later to resync the databases - not quite sure when, as I still have a lot to do getting all the infrastructure up and running.
It looks like we had a near simultaneous failure of 2 of our 3 storage solutions, one being a hyperconverged, high availability one, and the other being a standalone SAN solution. Trying to work back through the logs to see if it genuinely was 3 simultaneous storage server failures (unlikely) or 2 simultaneous hypervisor failures (unlikely), or something common that I haven't yet found (likely).
Sorry, as always, for the outage, and the fact you've probably had to talk to wives/partners or go down the pub!