11 October 2020
A story of industry collaboration during lockdown
We were just starting a several week project with Pinnacom to build-out their network to three London datacentres, linked by our dark fibre ring, when COVID lockdown hit in March. Our staff were locked down in Manchester, and Marek our Technical Director was locked down in France. Our client Pinnacom urgently needed new network services as they were pivoting their business as part of their response to lockdown.
So how did we manage to complete the work within a tight timescale while solving a number of novel problems?
Pinnacom are an ISP providing temporary connectivity for outside broadcast and public facing events in the UK like festivals.
They approached Faelix before lockdown via a recommendation. At the time they only had a POP (point of presence) at one London datacentre, Telehouse, and they were being approached by many broadcasters looking for connectivity for remote broadcasts. Pinnacom realised that their network needed to be more resilient and flexible and because of lockdown they would need to work in a completely different way.
The broadcast marketplace was becoming more fluid: they were being asked for more bandwidth and connectivity; there were more broadcasts happening; and timescales became more demanding.
A year ago most orders were 100Mbit — now there are very few orders below 500Mbit so we need large amounts of redundant connectivity for a short periods of time. We do big installs for shows that are only 30mins long. To be able to set up quickly and then when finished to move on the next location is key for us, and our new network allows us to do that.
We had to build out their new network so that they could meet the increasing demand, and build it fast!
We identified partners to connect to and where to build out to in London: Telehouse North and West, and Interxion (a growing hub for the media industries). So we engaged with Interxion and LINX in order to book space and contact connectivity partners.
We faced a number of problems – our team was locked down, and the supply chain was severely impacted — supplies of fibre patches, optics, racks etc virtually ground to a halt. We turned to colleagues in our community who were working in data centres on other projects in London to help with remote hands, and provide help to solve problems with out-of-the-box thinking. In one case one of Pinnacom’s providers could not provision the NNI that they need for the new build in time, so we temporarily migrated those services to one of our NNIs in Manchester, and long-lined the traffic over our layer-2 interconnects. The operations teams at LINX and Interxion and many others were also extremely helpful.
There was a community feel to the whole of the industry response (to COVID and lockdown). It reminded me of what the industry used to be like in the 2000s where it was very much driven by techies, trying to solve practical problems. That changed as industry became more consolidated and things became a more formal and process driven. But now the personal contact and out-of-the-box thinking came back — communities that LINX, UKNOF, NetMcr bring together fostering a collaboration, and we leaned on that to do this build of several weeks.
And where Pinnacom had been supporting broadcasts from a single location (e.g. a truck with 15 people on board) post-lockdown they’ve connected shows with up to 70 remote contributors who all needed to be connected for short periods of time.
Alex Kostiw said:
We've now got flexibility behind us to create peerings to new contacts either through LINX or direct cross-connect in a media hub such as Interxion or Telehouse West — we can be flexible in our network which is key — when we were dealing mainly with live events we could plan well ahead for them. Since March and in the medium term future there will be no events. The flexibility we have now has allowed us to pivot — we have a 10GB fibre ring between our 3 datacentre POPs, we can handle more connectivity, and be more resiliant. This has given us the confidence to take on these big broadcasts for major UK television shows, and without that resilience it just wouldn't happen.