Dec 09
It seems that the London Internet Exchange (LINX) are currently experiencing problems. Our connections to LINX are no longer carrying traffic and this is having an impact on some clients who would normally cross LINX to reach us. We do not currently have information on the precise cause but the issue has been reported by other IPs connecting at LINX.
We will provide more information as soon as we have it, but unfortunately it seems that the problem is outside our network and outside our control.
Date: 09/12/2009
Time: 17:20
Effect: Increased latency and packet loss for some routes into the network
10th December, 2009 at 06:30 PM GMT
LINX are still reporting that they have ongoing problems and do not know when these will be resolved. For this reason we have started disabling all peering via LINX and traffic will be rerouted via other connections.
This should resolve any outstanding problems with traffic flow via LINX, but this process will take some time to complete. We cannot just switch off a connection like LINX without ensuring that the traffic is properly handled, or there will be negative effects for all clients. Once this process is complete, we will provide further updates.
10th December, 2009 at 06:35 PM GMT
Update from NOC:
At 13:10 we received an update from LINX to say that their emergency maintenance steps had restored stability and they were advising all peers to re-establish sessions. We followed that advice and began restoring our LINX sessions. This process was completed at 13:48, however a small number of our peers have kept their sessions administratively down. This is their choice but may impact traffic if you would normally cross their network to reach us. We have been monitoring the situation this afternoon and have seen traffic levels return to normal. There have been no further reports of problems.
LINX are planning further ‘tidy up’ maintenance tonight, but according to their descriptions of the works, they do not affect our connections. We are hopeful that this is a conclusion to this incident and that LINX will maintain the resumed normal service.
Please bear in mind that the cause of all the problems over the last 24 hours has been outside our network.