Exa Networks - Notice history

System Under Maintenance

Core Network - Operational

100% - uptime
Nov 2022 · 100.0%Dec · 100.0%Jan 2023 · 99.62%

Previous page

Next page

Nov 2022
Dec 2022
Jan 2023

Notice history

Jan 2023

SurfProtect Issues
  • Resolved
    Resolved

    This problem has been resolved. Our engineers were performing some DNS maintenance which resulted in the DNS settings for SurfProtect to stop working, along with other services. As the DNS entries for SurfpProtect have a short time to live to allow our engineers to migrate traffic in case of hardware failure, the problem became immediately apparent for the service. The Team will perform a post-mortem to see what went wrong and why our testing didn't catch the problem.

  • Investigating
    Investigating

    We've had reports of customers having problems resolving the DNS for some of the SurfProtect Services. Our team have identified the cause and are working towards a resolution. We expect service back within the next few minutes.

SurfProtect & Core Network Issues
  • Resolved
    Resolved

    We have monitored the affected links and services throughout the day with no further issues or disruptions identified.. We will now close down this incident.

  • Investigating
    Investigating

    Some customers may have just seen a lack of internet connectivity for a few minutes. We lost 2 links in our data centre in Manchester, but all services have now recovered. Our Networking and Services engineers are investigating. If you are struggling to get back online following this issue, please call our Helpdesk on 0345 145 1234 and we will be happy to help.

SurfProtect Server Maintenance
  • Planned
    January 19, 2023 at 10:00 PM
    Planned
    January 19, 2023 at 10:00 PM

    We will be rebooting one of our Surfprotect servers this evening at 10 pm to resolve an issue that is not currently customer-affecting. The reboot is being carried out to ensure this does progress to be customer-affecting. The reboot will take around 5 minutes, during which time customers using that service will have a lack of internet on devices that are using the content filtering service. Our Services engineers will ensure all services are back up and running before closing off the maintenance.

Supplier Core Link Issues
  • Resolved
    Resolved

    The Suppliers have now confirmed the service has been rectified and all services should be functioning normally. Our Networking team have confirmed the link is stable and working and no further issues should be seen.

  • Update
    Update

    The supplier's engineers are still on-site at the affected data centers in London. As soon as we have any updates, we will update this status.

  • Update
    Update

    Our suppliers believe the issue has been caused by a faulty line card. They have their engineers onsite in the London data centre and we will provide updates as we get them. The next update will be by 2 pm.

  • Update
    Update

    We have escalated this with the suppliers, and we are just waiting on an update. As soon as we have any more information, we will update this status. The next update will follow by midday.

  • Investigating
    Investigating

    One of our core links between London and Manchester has just dropped. All traffic has now taken alternate routes within our network, but you may see performance issues until this link is brought back in. Our Networking team is speaking with our suppliers to find out what has happened. We will post a further update by 11 am.

Core Network - Supplier Outage
  • Resolved
    Resolved

    We have had confirmation the fault is now resolved on the core link, we will continue to monitor the link but we no longer consider our core network "at risk".

  • Update
    Update

    Service has now been restored on the affected core link, however, we are awaiting confirmation from our suppliers that the service is no longer at risk. We will continue to monitor the link and provide further updates.

  • Update
    Update

    Engineers are still progressing with the fault, we will continue to provide further updates as they are provided.

  • Update
    Update

    Our suppliers have dispatched engineers who are on-site at the moment and are investigating, we will continue to provide further updates as we receive them.

  • Investigating
    Investigating

    Due to an outage affecting our supplier's network, we have lost one of our core links between London and Leeds. This should not be customer-affecting as all traffic will take alternate routes through our network, but until the link is restored we are classifying our core network as 'at risk'

Dec 2022

SurfProtect Quantum
  • Resolved
    Resolved

    Following the reboot, we monitored the service and everything is back up and running. Sorry for any inconvenience this will have caused the school this morning. If you are still having issues with a lack of internet connectivity when using SurfProtect Quantum, please call our Helpdesk on 0345 145 1234 and we will be happy to investigate.

  • Update
    Update

    We are currently in the process of rebooting the affected server and hope to restore service shortly. Next update to be provided by 08:50.

  • Investigating
    Investigating

    We have identified an issue with one of our SurfProtect proxies. Customers using this affected server will currently have no internet access. Our Services engineers are investigating this as a matter of priority. Further update to follow by 08:45

Emergency Maintenance - Fortigate OS Vunerability
  • Planned
    December 13, 2022 at 5:00 PM
    Planned
    December 13, 2022 at 5:00 PM

    We will be performing emergency maintenance on both our FortiGate VM platforms and physical Fortigates to upgrade them to the latest stable firmware release. On 12th December 2022, Fortinet announced a vulnerability with the SSL VPN functionality on FortiOS. This is rated critical with a severity score of 9.3/10. More information can be found here: https://www.fortiguard.com/psirt/FG-IR-22-398 our engineering and support teams must now proceed to patch the vulnerability by upgrading the firmware to the latest firmware release. A maintenance window has been scheduled for 13/12/2022 between 18:00 - 20:00 to upgrade both our FortiGate VM platforms, and between 17:00 - 20:00 for physical devices to patch the vulnerability. The estimated downtime is approximately ten minutes during the firmware updates. We will ensure that all appliances are fully operational before closing off the maintenance window.

Nov 2022

SurfProtect Maintenance
  • Planned
    November 21, 2022 at 6:00 PM
    Planned
    November 21, 2022 at 6:00 PM

    We will be rebooting one of our SurfProtect proxies this evening at 6 pm, this will result in a 2-5 minute downtime for any customers using this service at that time. The work is being done as a preventative measure to ensure a bug identified earlier today does not lead to a bigger issue. Our Service engineer will ensure the proxies is fully operational before closing down the work.

Loss of connectivity on DSL services
  • Resolved
    Resolved

    Our Service Team have found the issue was affecting customer using Transparent SurfProtect, following work performed by our engineer service has now been restored. This is a problem we have not encountered before, our team will be running a post-mortem and issuing an RFO, as the reason is not yet fully understood.

  • Investigating
    Investigating

    We are currently investigating reports of connectivity loss on some DSL services. We will update this incident by 9:00 with more details

Investigating slow speeds
  • Resolved
    Resolved

    All work has now been completed by the suppliers and any customers who were affected should now see and problems resolved. If you are still seeing any issues with performance, please contact our Helpdesk on 0345 145 1234 and we will be happy to investigate.

  • Update
    Update

    The suppliers have identified the issue and have started to carry out emergency maintenance to resolve the problems. Customers will start to see improvements and we will update once we know the work has been completed.

  • Update
    Update

    Whilst we thought this was initially isolated to our interconnect with our supplier in Manchester, we have since diverted traffic through our interconnect in London and are still experiencing the same problems. We can see that our supplier carried out emergency maintenance last night which seems to have resulted in issues within their network. Further updates from the supplier has confirmed they can see packet loss with the network and they are investigating further. We will update again as soon as we have any further updates.

  • Update
    Update

    One Senior Network Engineer had identified packet loss on one of our interconnects with the suppliers, so we are escalating this to be investigated as a matter of urgency. We have also seen other ISPs reporting the same issue. We will update this post again at 10am.

  • Update
    Update

    We have carried out in house testing and confirm this is not related to SurfProtect. Our networking team are on the phone with our suppliers, and we will update the status by 9.30am.

  • Investigating
    Investigating

    We have received reports of customers experiencing slow speeds on DSL connections. We are currently investigating and will post an update here by 9:30.

SurfProtect Maintenance
  • Planned
    November 16, 2022 at 5:30 PM
    Planned
    November 16, 2022 at 5:30 PM

    To resolve and issue where some customers have seen OU groups not exporting correctly from their domains to SurfProtect, our Services team will be carrying out maintenance work on our customer panels to rectify this. The work will not affected any services of the SurfProtect filtering, but may lead to the panel.exa.net.uk becoming inaccessible for short period of time, we apologise for any inconvenience this may cause.

Post Maintenance Migrations
  • Resolved
    Resolved

    The incident has been resolved

  • Update
    Update

    As part of our migrations, we unfortunately caused a short service outage that will have affected some Surfprotect Transparent customers for approximately 3 minutes. We apologise for any inconvenience caused.

  • Investigating
    Investigating

    Following the large maintenance work performed over the weekend on Surfprotect our team is monitoring the service closely and performing some customer migrations, this work should not have been perceived by customers. Still, we invite customers seeing any issues to contact us. This work could not have been performed over the weekend to ensure we could see the impact of our changes, this work was completed at 8:30 and we will continue to monitor and perform the required adjustments.

Nov 2022 to Jan 2023

Next