Hopefully bad things don’t come in threes. Graeme Oxby, the CEO of London focused full fibre broadband ISP CommunityFibre, has apologised to customers after the operator suffered a second service outage yesterday, which much like the first one appears as if it could have been related to their Domain Name Servers (DNS). But this outage was much shorter.
In case anybody has forgotten, CommunityFibre was hit by a protracted outage on Monday (here), which lasted for several hours and impacted a sizeable portion of their customer base. But some savvy customers were able to work around it by using a third-party DNS provider (Quad9, Google Public DNS, OpenDNS, Cloudflare DNS etc.) to circumvent the ISPs own domain name system.
The DNS service typically works by converting Internet Protocol (IP) addresses into a human-readable form and back again (e.g. 123.56.32.122 becomes – examplezfakedomainlols.uk). Services like this tend to be provided automatically by your broadband and mobile provider, usually operating seamlessly in the background.
However, it’s not uncommon for ISPs to very occasionally suffer from problems with their DNS servers, which may arise due to a fault or misconfiguration in the system. When this happens, your physical broadband connection may still be live, but many of your requests to online domains will fail (other issues can also cause this, so we’re not 100% certain it was DNS).
Sadly, a similar outage struck CommunityFibre yesterday at around 3:45pm, although unlike the first outage they were able to resolve it within the space of about 30 minutes. Once again, some customers reported that they could work around it by changing DNS provider, but a few others found this didn’t work and opted to use a Virtual Private Network (VPN) instead. The situation prompted the provider’s CEO, Graeme Oxby, to issue a personal apology to his customers.
Graeme Oxby said:
“Unfortunately, I am having to apologise again as we had a network outage for 30 minutes this afternoon. Most customers should have had service restored immediately and we will keep working to get everyone restored.
If you’re still experiencing any issues getting online, please switch the power off to the fibre box for 2 minutes and turn it back on. Then, switch off the power to the router for 2 minutes and turn it back on. Your connection may take 1 to 3 minutes to re-establish.
Please do not attempt any other troubleshooting steps like adjusting any settings or taking out any cables as this may prevent service restoring automatically.
Thank you so much and sorry for any inconvenience.”
We have to give credit to the provider, and it’s CEO, here for taking responsibility and responding in a much more personal way than we’re used to seeing from ISPs in this market. Most providers tend to just fob customers off with a vague notice and then offer no follow up after the event (although it would have been even better to get some explanation for the cause).
However, brief outages like this are of course to be expected in the complexity of modern broadband networks, although in this case CommunityFibre appears as if it may have been dealing with a reoccurrence of the same or a similar / related issue to the one that struck on Monday. Hopefully there won’t be a third event anytime soon.