We are experiencing an outage from our Cogent feed to the Richmond datacenter. Cogent has acknowledged the problem and are investigating at this time. This impacts any servers with IPs starting with 38.
3:51 PM - We have been informed that some construction project in the Richmond area cut through Cogent's fiber line between Sadnston and Richmond. They are working to isolate the location of the cut. We do not have an ETR at this time.
4:37 PM - At this time there is nothing new to report. I am expecting some kind of update in the next two hours.
6:42 PM - At this time technicians have been dispatched to effect repairs but we do not have an ETR at this time. We will be compensating users for the outage time and we will help anyone who wishes to move to a different facility and offer incentives to do so. More information on that will be forthcoming once we are past this particular incident.
7:09 PM - We have received the following update: Fiber Vendor advises that the tech has arrived on site and is conducting tone testing. Splicers are on their way to the site. There is currently no ETA.
8:10 PM - At this time there is nothing new to report. I will continue to provide updates as I get them and either way every couple of hours.
10:40 PM - We have yet to receive anything new to report at this time. We are told the issue is being worked but they have not yet provided an ETR. The vendor site outage info is located here: https://ecogent.cogentco.com/network-status
11:09 PM - I’ve been now told that repairs commenced at 10 PM so crews are on site. There is still not an advertised ETR.
01;20 AM - I just spoke with Cogent again and beyond that repairs are ongoing, they were unable to provide anything meaningful as an update. We are continuing to press them for more information.
3:30 AM - Repairs ongoing.
5:00 AM - Cogents last update from Uniti was about two hours ago and that was simply that repairs are still ongoing. Cogent has requested fresh info. I did mange to get a hold of Uniti and they did share that progress is being made.
5:38 AM - Systems are coming back online at this time. More info will follow.
Affecting Other - Cogent
UPDATE
Cogent Network maintenance finished around 1:30 AM EST on Jan 26th 2025.
Full network communications were confirmed and all servers were back online by that time.
We have been informed by one of our network providers "Cogent" that there is active network maintenance scheduled from 12:30 AM - 6:00 AM EST on Jan 26th 2025.
Unfortunately servers on that network will be unreachable or have intermittent connection during that maintenance window.
While they have it scheduled until 6AM EST they may finish sooner.
Again we are aware of the maintenance but should you have any questions, please open a ticket and we will do our best to address your concerns.
Thank you.
Affecting System - Richmond IPs starting with 38.
At approximately 1400 eastern time we experienced a total outage of our 38. Ip space hosted in Richmond VA. It was later determined that a boring machine cut into a Lumen fiber line. Cogent leases these lines and provides network connectivity to our Richmond facility.
At this time repairs are underway however this is not a simple process as the cable has to be dug up and hundreds of fiber strands spliced together. Optimistically it will likely be morning before we see a restoration of connectivity though officially we have not been given an ETR.
The following update was posted at 0430 eastern time
** The issue you are seeing on this circuit is due to damage to fibers leased by Cogent. ** The owner of the fibers have determined the damage was caused by a boring machine. ** The fiber owner provided the following update "construction team has officially begun excavation efforts to uncover the fiber. This process involves carefully digging around the area to safely expose the damaged fiber cable, ensuring minimal disruption to surrounding infrastructure while preparing for the necessary repairs."
12/6/24 07:17
We just received the following update:
Vendor is onsite working to resolve the issue. Crews are splicing fibers in the south pit and digging in the north pit.
While thin this is good news. Due to the damage, two holes had to be dug, one on each side of the damage. They are already working on splicing the cables in one access point and are working to finishing digging down to the second access point where splices will also need to happen. We are still a ways off, but its moving in the right direction.
12/6/24 09:22
Cogent is saying that the expected return of service is currently at 10 AM eastern time.
12/6/24 10:49
Cogent has updated the ETR to 1 PM eastern time.
