Network Status| Albury Local Internet Network Status checked on 21-Nov-2024 17:35 -ALImon- |
|---|
| Links | |
| Core Router | 0.5 |
| Peering | 1.4 |
| Sydney | 0.5 |
| SDH-NBN-2 | 29.8 |
| Northpoint | 10.2 |
| OTH-Northpoint | 10.9 |
| SDH-Northpoint | 11.3 |
| Routers | |
| LNS | 0.2 |
| Albury Core | 25.6 |
| Albury GTF | 31.0 |
| Servers | |
| nameserver 1 | 0.0 |
|
| nameserver 2 | 0.2 |
| nameserver 3 | 0.2 |
| nameserver 4 | 1.5 |
| Mail Server | 0.1 |
| DataRetention | 0.6 |
| Cameras | 0.6 |
| webserver 1 | 0.2 |
| webserver 2 | 0.0 |
| webserver 3 | 0.2 |
| webserver 5 | 0.7 |
| webserver 6 | 0.8 |
| Firewall | 25.6 |
| Accounts | 31.5 |
| Time Server | 0.2 |
|
| Mel-3 | 1.5 |
| VoIP/Phone | |
| GTF | 0.5 |
| CO1 | 32.8 |
| CO2 | 32.8 |
| CO3 | 32.8 |
| CO5 | 32.8 |
| X108 | 30.5 |
| X114 | 30.3 |
| X201 | 16.9 |
| Environment/Meterological | |
| Darkfighter PTZ | 37.9 |
| AviationCam | 30.0 |
| Weather 1 | 57.5 |
|
| Albury Lightning Sensor | 38.7 |
| Corryong Lightning Sensor | 23.6 |
| Wagga Lightning Sensor | - |
| ADSB feeder | 54.7 |
| AirSupport feeder | 11.2 |
| pyranometer | 31.9 |
| dustmon1 | 102.0 |
| dustmon2 | 41.0 |
| vk2ray-irlp | 36.4 |
| vk2ray-mmdvm | 37.1 |
| vk2ray-telem | 38.3 |
| vk2ray-wlan | 36.3 |
| TTN Gateway | 40.9 |
|
| Key: |
UP, optimal |
Slow |
Slight Loss |
Excess Loss |
Down |
Restored |
Alarm |
DNS Fail |
|
18 Jun 2024
08:30 We're doing part-1 of a major server cluster upgrade.
Various servers will experience a brief outage as they are
migrated to the new environment. Most will be around 2 minutes
downtime, but a few are going to be about an hour.
Completed by 12:30 with only a couple of minor glitches.
Part-2 planned for tomorrow.
06 Jun 2024
12:50 Loss of multiple services on 203.27.67.0/24 subnet. This would
have only affected a small number of customers with specialty
services. Caused by a misconfiguration of a router upstream of
our servers and resulting ARP storm which required physical
corrective action at the datacentre.
Services all restored at 14:36.
23 Jan 2024
00:21 Loss of all NBN connected services.
Upstream advises dark fiber failure. Services restored 01:48
06 Nov 2023
07:07 A number of customers services all went offline at the same time.
Upstream can't see any planned maintenance or notice of fault.
Updated at 9:10 to say major outage at Albury POI. No ETR.
Services restored 10:03, waiting on PIR (Post Incident Report)
22-Dec-2022
07:03 A SAN controller failure has seen several servers go offline. Engineers
got replacement parts quickly. Most services were back running at 8:45
but one server took forever to complete file system consistency checks
and didn't come back online until 9:37.
Auth, Phone and email all affected. Disruption to DNS will have affected
various unrelated services until restoration was complete.
20-Nov-2022
01:40 Loss of connectivity to a small number of hosts (especially rural.net.au)
3rd-party router is showing multiple failures. Engineers working on it.
Router restored 02:30, however only internal traffic is flowing. More
analysis underway. 06:45 corrupted firewall rules rebuilt and all working.
01-Jul-2022
22:30 BGP failure on core router saw pretty much all connectivity lost.
Reload of router by technicians at 23:00 restored service.
Investigations continuing.
02-Jun-2022
19:00 Network connections in WAIT_STATE inexplicably increasing exponentially.
No source identified, performance appears unaffected but not comfortable
leaving it without explanation. Subsystem being restarted after 374 days.
06-Apr-2022
16:40 Remarkably similar outage to 18-Feb. Sudden loss of all connectivity.
Problem identified as failed Redback. Engineers despatched to the
datacentre to install/configure the spare. Services restored 18:39
Waiting on post-incident report.
18-Feb-2022
11:25 Sudden loss of all connectivity to all customers and all servers.
Upstream experienced a "power event" within the datancentre that
has destroyed both primary and secondary power supplies on their
aggregation switch, effectively isolating every part of the network.
Engineers on site installing backup router, and rebuilding configuration.
Seeing customers come back online at 14:12, restoring most of our core
infrastructure including authentication, mail and phones. Some servers
are still unreachable, we have no ETR but it is being worked on as a
priority.
25-May-2021
02:00 Over the last couple of hours, performance issues have been observed
with several hosts, ultimately being identified as a failure of a SAN.
Due to subsequent loss of ISCSI connections, multiple servers have
rebooted. Although most servers were restored soon after, phone, alarms,
monitoring, weather, graphing, some DNS and a few other services were
unable to be restored until 09:00, although all services have degraded
performance.
Nominal performance restored around 14:30.
15-May-2021
09:00 Have lost all our meteorological, albury cameras, lightning tracking,
amateur radio wormhole, IRLP, ADSB, websdr etc feeds due to being out
of town and suffering yet another UN-NOTIFIED NBN outage. At least it
was only 6 hours not the 12 they now suggest it may take! Restored just
before 15:00. How many more times, nbn???
25-Mar-2021
06:30 We are advised there is a "Major incident impacting various POIs in NSW.
Services are experiencing no data flow. POIs include: Albury, Campbelltown,
Coffs Harbour, Gosford, Hamilton, Liverpool, Maitland, Nowra, Tamworth,
Wagga, Berkeley Vale. Engineers are investigating with highest priority.
Previous status reports (by year):
2019
2018
2017
2016
2015
2014
2013
2012
2011
2010
2009
2008
2007
2006
2005
2004
2003
2002
2001
2000
1999
1998
1997
1996
