Cerberus Networks

Update 16-02-2018 14:00 PM

All links have remained up and operating normally for 24 hours. We have not observed any further disruption to connectivity or unusual behaviour on the network. We are now closing this incident.

Update 15-02-2018 14:00 PM

We have now re-established all links back to the supplier and traffic is gradually converging back to re-distribute load as normal. The network is operating normally. We will continue to closely monitor performance until normal traffic patterns are resumed and we can confirm that all systems are operating normally.

If you experience any recurrence of slow upstream speed or high packet loss on your connection, please contact us urgently on 03452571335 to report the fault.

Update 15-02-2018 12:40 PM

Our supplier has notified us that they have replaced the faulty equipment on their network. However, unfortunately this caused an unforeseen shutdown of one of our interconnects to them. This was not an expected outcome of the maintenance and we have requested a root fault analysis.

As a consequence of the unplanned shutdown of the link, a significant number of PPP sessions were dropped. This caused a short period of downtime on some broadband connections as they re-established their PPP session. However, there is no physical fault present and now that the supplier equipment is replaced and operational, we do not anticipate any further disruption to services.

We apologise for the inconvenience caused. If your service has still not resumed connectivity, please reboot your router. If you still have any problem after that, please contact our support team at support@cerberusnetworks.co.uk or by phone on 03452571335.

Update 15-02-2018 08:10 AM

Our supplier has confirmed that they have identified a faulty card in the equipment that provides one of our interconnects for broadband services. The card was suffering a high-CPU condition which would account for why traffic was passing over the link, but started generating numerous errors when traffic was at substantial rates. The supplier is currently engaged in replacing the faulty card.

At present, we are operating without connectivity to this faulty supplier equipment, though due to earlier measures that we have taken, this should not be affecting our broadband services capacity.

Once the supplier has confirmed that their equipment is working correctly, and verified as such, then we will bring up the relevant links. If this could be service affecting, then we will postpone such work to after business hours.

Update 14-02-2018 16:05 AM

Since the last update, we have been working hard to restore normal functionality and capacity to the broadband network, and to verify this is the case with clients. We have focussed on ensuring reliable and stable connectivity with adequate downstream and upstream bandwidth and very minimal packet loss. We have surveyed customers widely and believe that we have reached this interim target. We will be taking further action between 0600-0700 tomorrow to further optimise traffic flows to ensure no limitations on downstream traffic. This may result in some connections experiencing a PPP session reset, that will cause no more than 30 seconds downtime.

In parallel to this, we continue to work with our supplier partners to resolve the underlying fault. Having eliminated the issue from our network by completely replacing the equipment where the issue was apparent on our network, we will be running through low-level diagnostics of our supplier’s interconnect links to us. This should not cause any significant disruption. We will provide further updates as the work proceeds.

If you are still experiencing unusual connectivity on your broadband service such as very low upstream speed, high latency or packet loss, please email support@cerberusnetworks.co.uk providing as much detail in your description as possible including speed test results, latency and packet loss ratios, as well as the relevant circuitID or telephone number.

Update 14-02-2018 11:45 AM

We have replaced one of our aggregation broadband router over the night but we have observed same issue this morning hence we have applied temporary quick fix as quick resolution.
Our network team is investigating on this issue for permanent fix.

If you continue to have problems with upstream speeds or packet loss on your connection, please email support@cerberusnetworks.co.uk along with speed test and packet drops results. 

We apologise for the disruption to your service.

Update 13-02-2018 03:00 PM

We have identified the fault on our broadband network and have isolated the faulty systems. We continue to work on a final fix for the issue. In the meantime, your broadband connection should work as normal though maximum download speeds may be reduced during peak times. The next updates will be posted in an hour.
 
If you continue to have problems with upstream speeds or packet loss on your connection, please email support@cerberusnetworks.co.uk
 
We apologise for the disruption to your service.