Exa Networks - Notice history

As all affected VM Fortihgate services have been restored, we will now be closing this status. We will be carrying out a post-mortem to identify the root cause to ensure we do not encounter the same issue when we next upgrade the VM Firewalls. If you are still unable to access the internet, please call our Helpdesk on 0345 145 12345 and the team will be happy to assist. Sorry again for the inconvenience this outage will have caused you this morning.

We have now rolled back the VM FortiGates, and we are seeing services come back online. We will carry out further investigations to identify what part of the upgrade was the root cause to avoid this happening again. If you are still unable to access the internet, please call our Helpdesk on 0345 145 12345 and the team will be happy to assist.

We have identified an issue with the VM Fortigate upgrade carried out yesterday evening. We are in the process of rolling back to the snapshot taken before the upgrade, and we will provide a further update once this has happened.

The service has been restored overnight, and all affected connections will now be online again. From the monitoring, we can see the circuits were back up at 23:40. We apologise for any inconvenience this may have caused you. If you are still unable to access the internet, please call our Helpdesk from 8 am on 0345 145 1234, and the team will be more than happy to help.

The engineering team have been able to confirm the fibre goes the full distance between the data centres in Sheffield and Leeds, so there is confidence that there is no fibre break. They are currently gaining access to the AQL data centre in Leeds to confirm everything is patched correctly; following this, a further update will be provided. The next update from Exa will be posted no later than 8 am.

The engineering team are still working to repair the affected fibre, and we are awaiting a comprehensive update from the suppliers. Further updates to follow as we get more information.

The engineering team arrived on site just after 7pm, and they are working to resolve the issue.

The suppliers have dispatched an engineering team to investigate the affected fibre. We will provide the next update once the engineers have carried out their investigations. We're sorry for any inconvenience that this supplier outage will be causing you.

Following a period of monitoring, we are happy that all the affected services are now stable and operational again. We will now close down this status. Thank you for your patience while we worked to resolve the outage.

The VMs have now all been rebooted, and other than the SurfProtect Real Time Alerts & Website Analytics, all other affected services are now operational again. Our Service Engineers are working to resolve the SurfProtect Real Time Alerts & Website Analytics issue, and we will provide an update on this as soon as we can. If you are still experiencing any issues, please contact our Helpdesk on 0345 145 1234, and the team will be more than happy to help.

Our Service Engineers identified and resolved the underlying issue with the VM Cluster. They are now carrying out a rolling reboot on the individual VMs within the cluster to restore the affected services. We will provide a further update once the work has been completed.

Following further investigation, our Service Engineers have identified an issue with our VM Cluster in Manchester. This cluster hosts the following services, which will also be seeing issues; Email Webspace Exa Hosted Police CyberAlarm Securus SurfProtect Captive Portal PAC Service SurfProtect Real Time Alerts & Website Analytics SurfProtect Website Classification Exa Education/Moodle We will provide updates as soon as we have them, and we are very sorry for any inconvenience this will cause you this morning.

Our Services team have identified and resolved an issue with the Upstream DNS service used for Surfprotect. We have confirmed this is working with a few customers, but if you are still having any issues, please call our Helpdesk from 8 am on 0345 145 1234, and the team will be more than happy to help. Sorry again for the inconvenience this will have caused this morning.

All affected servervices are now working again.

Despite successful internal testing, when the release was rolled out to the production servers, we identified that SurfProtect customers are unable to access the internet. Our Services team have already rolled back the changes, and all customers should be able to get online again. If you are still affected and unable to access the internet, then please call our helpdesk on 0345 145 12345, and the team will be more than happy to investigate.

The supplier's engineer has successfully replaced the at fault component. Service is now stable again.

The supplier engineer and parts are expected to arrive on site at 19:00.

Our supplier has sent engineers to site to replace the affected hardware. We have not yet received an estimate for when they expect this to be resolved by.

The suppliers confirmed they repaired the damaged cross-connect fibre at 10:30. We have monitored the link for the last few hours, and everything is operational again, and all affected customers are back online. We have requested a Reason for Outage report from the suppliers. If you would like to receive a copy of this, please let us know by emailing helpdesk@exa.net.uk, or calling us on 0345 145 1234.

Customers will have seen a small 2-minute disconnection again on their leased line connectivity. Our suppliers are still working to resolve and close the outage, and further disconnections may occur until the outage has been fully closed. Further updates to follow.

From our monitoring, we have seen the affected services come back online. We are awaiting confirmation from our suppliers that the outage has been resolved, and as such, we will classify the circuits as 'At Risk' until the outage is officially closed. We will provide further updates as we receive them.

Engineers are still working to repair the damaged cross-connect, and work will continue throughout the evening till all affected services have been restored. Exa will provide the next update at approximately 7:00

The supplier's engineers have instructed the data centre engineers to assist with resolving the issue with the cross-connect fibre. Work is ongoing, and we will provide further updates as we get them.

The engineers are still working to resolve the outage, and we will continue to chase this evening and provide any updates as we have them. We are sorry for any inconvenience this will be causing your school or business.

The engineers have identified an issue with the cross-connect fibre linking their infrastructure to ours within the data centre. They are working to resolve this now. The next update is due at 18:00.

The engineers are continuing to work to resolve the outage. We will continue to chase and provide updates as soon as we have them.

The engineer is onsite at the data centre and is currently investigating. The next update is due at 14:30.

The engineer is due on site at the data centre at 13:30. We will update the status once the engineers have investigated. The next update is due at 14:00.

The suppliers have dispatched an engineer to attend the data centre where the outage is located. We will update this status when we have an ETA for their arrival.

Our suppliers are investigating, and we have a fault logged against the outage. We are due an update at 13:00, but we will continue to chase and provide updates as soon as we have them.