r2.man-tcw hardware fault Sunday 26th November 2017 02:30:00


23:59 Our configuration on this device was hitting a possible software bug. The replacement router chassis is now fully back in-service following a configuration adjustment to avoid triggering the bug.

10:11 The replacement router is exhibiting the same issue, we have raised a TAC case with the vendor.

08:41 Router replaced, we are monitoring.

07:11 Engineers are on-site in Manchester and are preparing to replace the device.

Following a reboot of r2.man-tcw to clear a memory leak, it appears the root cause of the issue is a hardware fault. The router is logging memory checksum errors within the forwarding engine and has been taken out of service. We will be performing a complete swap of the chassis in the next few hours.