Neos Systems - Cloud3 NOC Notification

PBX Platform Upgrade

2025-12-03T18:43:24+00:00

Monday evening we will be performing an upgrade to our PBX platform, during this time there could be some disruption with making and receiving calls. The outage window is between 10 to 30 minutes. If you have any questions, please reach out through normal channels.

Hosted Exchange - Some Cloud Service Infrastructure (resolved)

2025-11-27T21:00:00+00:00

Issue has been resolved. Was due to a configuration issue causing degraded access to one of the SANs. Some slowness may be experienced over the next couple of hours as the system settles.

Hosted Exchange - Some Cloud Service Infrastructure

2025-11-27T18:30:00+00:00

We are currently experiencing an issue with hosted exchange, we are currently working through this and expect systems to be up shortly.

Hosted Exchange is experiencing an outage (resolved)

2025-10-21T00:26:00+00:00

Exchange issues are now fully resolved, the backlog of email has now been processed. If you are having issues with emails you are sending stuck in your Outbox, please close out of Outlook and re-open and the emails in this folder should automatically send. We would recommend you check this folder at your earliest convenience.

Note: We will be implementing a permanent fix tonight from 9pm, during this time we don't expect any disruption of services to occur.

Hosted Exchange is experiencing an outage (update)

2025-10-21T00:03:00+00:00

Hosted Exchange is now functioning, however there may be some impact to performance while active and passive nodes synchronize, this includes sending and receiving email. We'll continue to monitoring the system. During this time, all emails have been queued for delivery and will be delivered shortly. If you are still experiencing issues, please reach out using normal support channels.

Hosted Exchange is experiencing an outage

2025-10-20T23:30:00+00:00

We are currently experiencing an outage with Hosted Exchange, this is due to Windows Updates being installed automatically, this has caused the Exchange services to be suspended while the updates are installed. Unfortunately this is happening on all active and backup systems. ETA - 15 to 30 minutes for restore

Hosted Exchange Maintenance

2025-10-17T00:33:15+00:00

We will be performing maintenance on the Hosted Exchange platform from 11 PM to 3 AM on 19/10/2025. We expect that there will be an outage during this time and emails will be queued.

Any questions please feel free to give Neos a call on 0800 022123

Network Upgrade - Transit, VOIP, ISP Connections

2025-07-10T19:22:35+00:00

From 12AM to 6AM (1 hour outage) on Tuesday the 29th of July 2025 one of our upstream providers will be relocating fibers to new equipment affecting transit, VOIP and ISP connections along with associated services. This is to provide modern and a supported network by the upstream. During this time, systems will be moved over to redundant equipment however some breaks may occur in different pats of the network for up to 3 minutes.

Network drops - Internet Connections (resolved)

2025-07-03T00:56:00+00:00

Issue was resolved. Upstream are looking into what caused the fault, please contact us if you would like a PIR.

Network drops - Internet Connections (update)

2025-07-02T02:43:00+00:00

Transit is now running normally, Wellington internet connections have failed over to backup LNS (network). There may be some quality issues e.g. voice or some slowness while on backup systems. We are working with upstream provider to resolve the incident as soon as possible.

Network drops - Internet Connections

2025-07-02T01:50:00+00:00

ISP connections have been dropping since 1:52pm along with some transit. We are working with our upstream provider on the issue, they are currently experiencing the same issue for Wellington users inside their network. The issue seems to be healing, services have moved over to some backup systems. Please contact support@neos.co.nz if you would like a PIR (Post Incident Report).

Network Upgrade - Transit

2025-06-23T02:13:25+00:00

One of our upstream providers will be performing maintenance on their core network in Featherston St Wellington. During this outage (1 hour), our systems will move to alternative paths limiting any downtime experienced. There will be a break of about 30 seconds while systems reconverge to alternative paths. Outage window 03/07/2025 00:01 - 06:00

Network Upgrade - Transit

2025-06-08T21:00:07+00:00

Major Network Outage (Resolved)

2025-06-06T01:16:16+00:00

All internet connections should now be up and running. Please give Neos a call if there are any further issues.

Major Network Outage (Resolved)

2025-06-06T01:00:00+00:00

Major Network Outage (Resolved)

Major Network Outage

2025-06-05T23:48:00+00:00

There is an issue affecting Chorus Services. This issue is affecting services in Wellington, Kapiti, Hutt Valley, Palmerston North and through to Napier. Currently around 90% of connections are offline. We are working with Chorus on a resolution. We are also aware that this is affecting the mobile network, both data and voice.

Chorus have identified the source of the issue and are working on a fix. Issue is affecting multiple service providers and is not isolated to just Neos.

12:23 PM Update: Chorus has implemented a fix and connections are slowing being restored.

Wellington - Internet drops (resolved)

2025-04-18T06:41:00+00:00

This issue was resolved on the 18th of April 2025. Upstream provider resolved a switching MPLS issue on their Wellington switches. We haven't had any reoccurrences since this date.

Wellington - Internet drops

2025-04-16T19:05:00+00:00

The upstream provider made some changes last night at 2am to their LSP switch timings, however it looks like the fault is still reoccurring as we had a small number of sessions drop at 6:30am this morning. we have updated them on this to help them rectify the issue. Note: Other providers are also affected by this fault.

Wellington - Internet Drops

2025-04-16T04:28:00+00:00

We have been advised our upstream provider is going to make some changes on their LSP switches tonight to try and rectify the issue. We'll monitor once the change has been made and provide further updates tomorrow.

Wellington - Internet Drops

2025-04-15T02:15:00+00:00

We had 4 * 1-20 second session drops for 5-10 client sessions at a time who are located on a Wellington BNG (Broadband Network Gateway) Occurring on the 14th of April 2025 close to the hours of 9am, 12pm, 12:30pm and 6pm. And now another one at 2pm today. Our upstream is working with the LSP as it looks like the issue is occurring inside their network. You can keep up to date through https://status.neos.co.nz

Wellington - Internet Drops

2025-04-14T04:00:00+00:00

Our upstream provider has identified some ongoing fibre issues on a particular Wellington BNG (network equipment). We are working with them to resolve the issue as soon as possible. No further dropped sessions have occurred since 12:33pm this afternoon.

Wellington - Internet Drops

2025-04-14T03:20:00+00:00

We haven’t had any re-occurrences of this since 12:33pm today, we are monitoring and still investigating with the upstream provider to ascertain the cause of the fault.

Wellington - Internet drops

2025-04-13T21:00:00+00:00

We are aware of a fault where a number of Wellington based internet connections are dropping for small amounts of time, 10 - 30 seconds. We are working with our upstream providers to identify the fault

Advisory: Upstream VOIP maintenance (low impact)

2025-04-01T23:25:14+00:00

We have been advised that one of our upstream voice providers will be performing maintenance between the hours of 5am and 9am on the 7th of April 2025. We expect there to be no disruption during this time, however this notification is for if a possible outage occurs. We will monitor our systems during this outage window. If you have any questions, please contact us using normal channels.

Network Upgrade Scheduled - 26/03/2025

2025-03-20T20:49:52+00:00

Between the hours of 4:30am and 6:30am on the 26th of March 2025 we will be performing upgrades to our core routing infrastructure in our Wellington datacentre. During this outage window we expect 2 network breaks of about 30 seconds to 3 minutes in length while network is moved between active and backup systems. If you have any questions, please contact us through normal channels.

CloudTalk Upgrade

2024-12-18T19:41:42+00:00

Tonight from 9pm our vendor will be upgrading our CloudTalk/PBX platform to the latest version. We'll be working closely with them during the upgrade and be testing platform once upgraded. During the upgrade window of 9 until 12am, there may be some loss of service, call disconnections or quality issues. If you have an issues after 12am, please get in contract with our support team.

Issue with routing - Internet / Voice with an upstream provider (resolved)

2024-12-04T21:48:00+00:00

The routing issue with upstream provider was resolved at 10:15am, there was a fibre fault between two locations in the North Island. We are waiting for an incident report from one of our upstream providers, if you would like a copy, please contact our support team

Issue with routing - Internet / Voice with an upstream provider

2024-12-04T21:12:00+00:00

We are currently experiencing a fault with routing with an upstream provider. They are currently working on the issue, this may affect Internet Connections and Voice. We will continue to update here

Internet Outage (fixed)

2024-10-15T03:16:00+00:00

This issue has been been resolved, issue was due to a router problem with upstream provider. Working with upstream.

Internet Outage

2024-10-15T01:33:00+00:00

At 2:33 pm today we experienced a drop of PPP sessions on our primary connection. All connections automatically moved over to our secondary connection and investigation of cause is currently underway

DNS Issue in Office365 - May result in delayed email delivery (fixed)

2024-10-13T18:58:00+00:00

This issue has been resolved, Microsoft made the required network changes. Reverting workarounds.

DNS Issue in Office365 - May result in delayed email delivery

2024-10-10T18:10:00+00:00

07:00am: Currently there is an issue with Microsoft DNS servers, we have put a workaround in place but there may be some intermittent issues until Microsoft resolve the issue. These issues may result in delayed email delivery, being unable to access some Azure services etc. If you have any questions, please feel free to get in contact with us.

11:08am: We are seeing routing issues still happening between Microsoft servers, and some emails are still in a queue and haven't been delivered yet.

11:52am: We have had reports from other providers in the industry unable to access MS API. We have tested network connectivity and transit providers are ok.

01:30pm: We have been advised that Microsoft has made some IP address changes overnight, we are working with them to assess what has changed so they can resolve the issue.

02:15pm: We are continuing to work with Microsoft to resolve the routing issue currently being experienced using two different avenues.

02:52pm: Incident has be escalated internally at Microsoft, they are working with the Azure team to identify what is going on.

04:35pm: incident has been escalated to Microsoft's network team, waiting on them to pick this up, hope to have more updates soon.

05:58pm: Things are improving, email queues are starting to clear. Waiting on Microsoft for more details.

06;01pm: Email queues have cleared. We will keep the workaround in place until Microsoft confirms the incident has been resolved and there will be no reoccurrences.

Hosted Exchange Maintenance

2024-09-02T07:28:59+00:00

We are performing routine maintenance on this platform,no outage is expected.

Scheduled router optimization by upstream provider

2024-08-26T00:00:13+00:00

An emergency scheduled outage on our international leg with one of our upstream providers will happen tonight from about 11:00pm. During this time, international connectivity will use an alternative path with about a 30 second drop to some international routes. As some other services use a similar path, some unintended consequences may occur e.g. AK DC connectivity issues, or some internet circuits may experience some loss of service as backup paths are utilized.

Hosted Exchange

2024-08-02T03:35:00+00:00

All services restored

Hosted Exchange

2024-08-02T02:24:00+00:00

Reports of slowness with Outlook

Microsoft 365 Outage - Affecting some Neos apps (resolved)

2024-07-31T21:52:00+00:00

Microsoft has resolved the Office 365 issue, if you continue to have any further issues, please contact support. If you were using the Outlook webapp for Hosted Exchange, you should now be able to use Outlook.

Microsoft 365 Outage - Affecting some Neos apps

2024-07-31T20:52:00+00:00

There is currently an outage with Microsoft and Office 365. Services like Outlook, Teams may not function. If you are using Outlook and Neos Hosted Exchange, you may also be affected. Please use the webmail app https://mail.cloud3.co.nz to work round this issue.

Updates to Hosted Exchange

2024-07-08T02:20:09+00:00

Tonight from 8 PM we will be doing scheduled maintenance on the Hosted Exchange Platform. No outages expected during this time

Upgrade to Hosted Exchange

2024-04-22T00:39:19+00:00

From 9 Pm tonight we will be upgrading the Hosted Exchange platform. We do not expect there to be any outage while this takes place.

If you experience any issues related to this maintenance please let us know.

Hosted Exchange security updates

2024-02-20T07:09:59+00:00

We are performing additional security updates to this platform from 8:30 PM tonight.

Hosted Exchange Updates

2024-02-19T07:24:43+00:00

We are performing additional security updates on this platform. The maintenance window is tonight between 8:30 PM and 12:30 AM.

Updates to Hosted Exchange platform (fixed)

2024-02-18T12:00:00+00:00

Upgrade complete

Updates to Hosted Exchange platform

2024-02-18T07:30:00+00:00

We are performing security patching on our Hosted Exchange platform from 8:30 PM to 1 AM. We expect there to be no impact to email connectivity during this time.

CloudSync Upgrades

2023-12-21T02:58:27+00:00

From 9pm tonight we are performing further updates to CloudSync, we expect this to be completed by midnight. During this time, files may not sync but will be synced once systems are back online.

CloudSync Upgrade

2023-12-20T08:00:29+00:00

Tonight we are upgrading CloudSync to a later version, we expect this to be completed by midnight. During this time, files may not sync but will be synced once systems are back online.

CloudSync Upgrade

2023-12-13T08:10:31+00:00

Tonight 13/12/2023 from 9pm we are upgrading CloudSync to the latest version. We would expect this to be completed by 12am. You can continue to access your files locally and they will be synced when the service comes back online.

Hosted Exchange maintenance

2023-11-06T20:32:31+00:00

We will be performing scheduled maintenance on the Hosted Exchange Platform from 8 PM to 12 AM on Wednesday 8th November. No impact to services is expected during this time.

Hosted Exchange Upgrades

2023-09-03T23:02:33+00:00

Tonight we will be preforming infrastructure, security updates and changes to our Hosted Exchange platform. These will be applied between 8 pm and 12 am tonight. We expect that there will be very minimal impact to users if any. If you experience any issues Tuesday morning, please contact our support team.

Issue with backups (fixed)

2023-08-04T03:49:00+00:00

Issue has been resolved, backups are now running and are back on schedule.

Issue with backups

2023-08-03T20:35:00+00:00

Some users may notice that backups did not run correctly . We have identified a certificate issue on the Cloud storage and are currently working with our provider to resolve the issue.

Issue with Hosted Exchange and Cloud services (fixed)

2023-08-02T00:45:00+00:00

Issue has been resolved. Was related to a node failure in a cluster. If you would like an incident report, please contact your account manager.

Issue with Hosted Exchange and Cloud services

2023-08-02T00:26:01+00:00

We have identified an issue with networking within our Cloud Infrastructure which is causing outages to Hosted Exchange and other Cloud services. Engineers have identified the issue and are currently on route to the Datacentre to resolve these. Further updates once services have been restored.

CloudTalk Upgrade

2023-01-25T22:13:47+00:00

Tonight between the hours or 10pm and midnight we will be applying updates to our CloudTalk PBX platform. We will perform the updates when no calls are active on this particular node, the service refresh will take about 20 seconds and during this time inbound and outbound calls may be limited. If you have any questions, please feel free to contact us.

Primary SAN firmware upgrades - no impact

2023-01-18T23:01:42+00:00

Tonight (between the hours of 10pm and 1am) we will be upgrading the firmware on our primary SAN. As the array is redundant we don't expect any issues during the upgrade. If you have any questions, please feel free to reach out.

(minimal impact) Hosted Exchange Cluster Upgrade

2022-11-15T19:09:34+00:00

Tonight between the hours of 7pm and 5am tomorrow morning we will be performing some upgrades to the Hosted Exchange cluster. During this maintenance window we don't expect any downtime as services will be crossed over to backup systems. If you have any questions, please feel free to contact us.

Planned Outage - Minimal Impact

2022-09-27T22:15:57+00:00

Between the hours of 11/10/2022 19:30:00 and 12/10/2022 03:30:00 an upstream provider will be relocating a core fibre connected to Neos infrastructure as part of a planned water mains upgrade. During this outage window we expect network traffic could be disrupted for 1-3 minutes while internet transit is rerouted to alternative paths. This could also disrupt internet and voice services while they are rerouted to backup links. If you have any questions regarding this please feel free to contact us.

CloudTalk PBX - Maintenance

2022-08-08T02:17:12+00:00

We will be performing maintenance on our CloudTalk PBX cluster at 12AM on Wednesday morning. This maintenance window is required for us to apply the latest security fixes and feature updates to the CloudTalk infrastructure. We expect an outage of about 1 to 3 minutes during the upgrade window. If you have any questions, please feel free to get in contact with us.

Hosted Exchange/Outlook (resolved)

2022-08-07T20:55:00+00:00

Fixes have been applied to the problem server in cluster over weekend, issue resolved.

Hosted Exchange/Outlook

2022-08-05T03:10:00+00:00

We are currently aware of issues on our Hosted Exchange / Manage Outlook service. We have removed the faulty server causing disruption from the cluster and will be applying some fixes on this server later tonight. There may be some further intermittent breaks of outlook connectivity while the server is patched during the outage window. We expect maintenance of the cluster between the hours of 9pm and 12am tonight. If you have any questions, please feel free to contact us.

CloudTalk Voice Platform Upgrade

2022-03-16T02:52:31+00:00

On Tuesday the 22nd of March we will be upgrading our CloudTalk SBC endpoint's with one of our upstream providers. This outage window is between the hours of 11:30pm until 1:00am Wednesday morning. During this time external calls in and out of the platform may be effected for about 20 minutes as the provider migrates services to the new SBC. If you have any questions, please feel free to contact Neos Support using the normal channels.

Currently experiencing internet outage - resolved

2022-03-02T01:54:00+00:00

All services should now be restored. If you are still having problems please contact our support team. Our upstream provider is still trying to determine the root cause of the incident, please send an email to Neos support if you would like to receive a copy of the incident report.

Currently experiencing internet outage

2022-03-02T01:37:00+00:00

Services are slowly being restored, we have been advised by our upstream provider that they are still investigating root cause of the issue. We will provide further updates soon.

Currently experiencing internet outage

2022-03-02T01:00:00+00:00

We are currently experiencing internet issues for clients on our internet connections. This is a national outage and we are working with our upstreams to resolve the issue. More updates to follow.

Hosted Exchange Performance Issues (fixed)

2021-11-30T10:21:00+00:00

All systems are up and running, issue has been resolved.

Hosted Exchange / Manage Outlook Maintenance - Minimal impact

2021-11-30T01:34:39+00:00

Tonight from 10pm we will be performing some maintenance on our Hosted Exchange / Managed Outlook infrastructure. During this time we will swap over to redundant infrastructure so we expect there to be minimal to no impact to end users. If you experience any issues from 11pm please feel free to get in contact with us.

Hosted Exchange Performance Issues

2021-11-30T01:00:00+00:00

Today between 2pm and 3:20pm we noticed high load on one of our Exchange Server nodes causing the system to not process some requests. This caused a number of users to have some connectivity issues with Outlook. We have since identified a bug in the platform and temporarily remediated the fault and are now watching the systems closely. Tonight we have scheduled further remedial work to prevent the issue from reoccurring. If you have any further questions, please feel free to contact us.

Hosted Exchange Not Responding (Fixed)

2021-11-10T03:34:00+00:00

At 16:17 today we received alerts that people were getting disconnected from hosted exchange. We found that a process was using an unusually high amount of memory causing the server to not respond correctly. We switched over to the backup servers which took a few minutes to become active. After a reboot of the primary server process usage is back to normal. We will be investigating the high memory process further. If you have any further questions, please feel free to contact us.

Hosted Exchange Not Responding

2021-11-10T03:17:00+00:00

We are currently investigating hosted exchange 2016 issue

Scheduled Chorus UFB Upgrades

2021-10-28T19:33:16+00:00

On the 2nd, 7th, 8th and 9th of November Chorus will be performing updates on their UFB fibre networks between the hours of 12AM and 6AM. During these outage windows some customers may experience loss of UFB internet services for up to 60 minutes. If you have any questions regarding this, please feel free to get in contact with us.

CloudSync System Upgrade

2021-10-26T23:36:46+00:00

Tonight from 10pm we will be upgrading our CloudSync platform (our file synchronization service) to a later version which will allow for a number of new features. You will still be able to use your local files on your desktop while this upgrade takes place and these modified files will resync once the servers come back online. If you have any questions, please feel free to get in contact with us.

Issue with national transit (resolved)

2021-09-03T02:26:00+00:00

The primary circuit is back up, traffic flow is going back to normal.

Issue with national transit

2021-09-03T01:30:00+00:00

Some users may be experiencing issues accessing some websites, our primary transit has dropped and we are now running on backup circuits. . We are still waiting on an ETA from one of our upstream providers.

Issue with low priority storage - backups and archive (resolved)

2021-09-02T21:30:00+00:00

This has been resolved, we will be preforming extensive monitoring of this infrastructure in the immediate term.

Issue with low priority storage - backups and archive

2021-09-02T20:47:00+00:00

We are investigating an issue with our backup and archive storage. Some backup processes and archive storage from VPS may not be accessable

Intermittent issues with Hosted Exchange (fixed)

2021-08-08T12:05:00+00:00

All systems are running normally. Security updates were performed on our Hosted Exchange cluster causing minor intermittent connectivity issues during the upgrade process. As the cluster is redundant no downtime should have been experienced by end users. We have investigated the cause of these to prevent future re-occurrences.

Intermittent issues with Hosted Exchange (update)

2021-08-08T06:49:00+00:00

Servers are running, we are working on the backup server's in the background to apply some patches

Intermittent issues with Hosted Exchange

2021-08-08T06:37:00+00:00

We are experiencing some connectivity issues with hosted exchange, we are currently investigating

CloudTalk System Upgrade

2021-08-02T21:28:01+00:00

On the 23rd or August 2021 from the hours of 2am to 5am we will be upgrading our underlying CloudTalk PBX infrastructure to a later version. As this infrastructure is redundant we expect a number of small outages of about 1-2 minutes while services transition between redundant nodes. If you have any questions regarding this upgrade, please feel free to contact us.

Webserver Updates

2021-06-02T19:54:24+00:00

Today we are performing security updates to each of our web servers, expect up to a 4 minute outage of your website as we reboot the servers.

Issue with Microsoft Outlook (365) (resolved)

2021-05-12T00:52:00+00:00

We have been advised by Microsoft that the new fix is now being applied globally. If you are still experiencing issues and haven't applied the provided fixes, please restart your computer and open outlook again. If it asks for the update to be applied, please proceed with this.

Issue with Microsoft Outlook (365) (update)

2021-05-11T22:35:00+00:00

Latest update from Microsoft is as follows. We recommend using the workarounds previously provided for the remainder of the day, if you continue to experience issues, please feel free to contact us.

Microsoft: "Current status: We’ve identified the underlying cause of impact and are applying a fix. This fix will reach all affected users incrementally over the course of the next four-to-five hours. Once users receive the fix, they will need to restart their email client to apply the fix. In some circumstances, users may need to restart their client a second time for the changes to take effect. We expect to complete this process and restore service for all affected users by May 12, 2021, at 3:00 AM UTC.

We encourage affected users who are able to do so to leverage the workarounds described above in the "more info" section of this post while we complete the process of fixing this problem."

Issue with Microsoft Outlook (365) (update)

2021-05-11T22:06:00+00:00

We have improved the instructions for a manual workaround fix for outlook experiencing display issues. If you are still having issues viewing or creating email messages in outlook, please go to https://www.neos.co.nz/downloads and follow the instructions listed under "MICROSOFT OUTLOOK FIX". An alternative way is to use the web-based email client provided either by Neos or Microsoft.

Latest update from Microsoft "We’re investigating available data to isolate the root cause. Users with access to alternate protocols like Outlook on the web or Outlook mobile can use those as a potential workaround in the interim. More details can be found under EX255650 in the admin center." we expect further updates at 10:30AM.

Issue with Microsoft Outlook (365) (update)

2021-05-11T21:37:00+00:00

If you are still having issues with outlook not displaying correctly, please do the following Go to https://www.neos.co.nz/downloads and then right click on the "Download Outlook Fix" link and select "Save Link As" and then save it to your desktop. Once the downloaded file is on your desktop, double click, it will come up with a message "Windows protected your PC", click the "More info" link and then select the "Run anyway" button The bat file will downgrade your outlook to an earlier working version.

Issue with Microsoft Outlook (365) (update)

2021-05-11T21:23:00+00:00

If you have a PC that is managed by Neos under a service contract and use our RMM, we are now pushing out a roll back command to get your Outlook up and running. Please except any prompts that outlook may ask you while this roll back happens. We are still waiting on Microsoft to a permanent fix.

Issue with Microsoft Outlook (365)

2021-05-11T20:00:00+00:00

A large number of users are experiencing issues with Outlook where messages are not displayed correctly or users are unable to write into the new message box. This is a worldwide issue due to Microsoft sending a bad outlook update overnight. Microsoft are currently investigating the issue and expect an update in the next hour.

Hosted Exchange Security Updates

2021-04-13T23:07:49+00:00

Tonight from 10pm we will be installing some security patches on to our hosted exchange platform. During this time we expect everything to run as normal as systems will failover to backup infrastructure. If you have any questions, please feel free to get in contact with us.

Webserver Maintenance

2021-04-09T02:04:21+00:00

Tonight (09/04/2021) from 10PM we will be performing maintenance on our one of our webservers (servers that provide websites). On our NZ4 webserver, we expect two separate breaks in service of about 4 minutes as services are restarted. If you have any questions, please feel free to contact us.

CloudTalk PBX Upgrade

2021-03-30T23:33:53+00:00

Tonight between the hours of 10PM and 3AM we will be upgrading our core CloudTalk PBX System. We expect a break in service of about 3 minutes while the components are restarted. This will be done when no calls are in progress on the system so we don't expect any clients to be impacted. If you have any questions, please feel free to contact us.

Node Reboot

2021-03-21T22:15:00+00:00

We had a server node loose connectivity, we are going to the DC to investigate this. Any VMs that were running on this host have been migrated to other servers inside the cluster.

Network Upgrade

2021-03-12T03:49:18+00:00

On Tuesday morning (16/03/2021) between the hours of 1AM to 6AM we will be upgrading our core network equipment and rolling over to a new DC. During this time we expect a small number of breaks of network service of about 5-15 minutes as the two networks converge. This will affect ISP services, VPS network access, colocation, voice services etc.

If you have any questions regarding this please feel free to get in contact with us.

One way audio on some inbound calls (fixed)

2020-08-03T03:17:00+00:00

We have been advised by our upstream provider the issue has been resolved and a permanent fix will be applied after midnight tonight on the Wellington gateway. If you are continuing to have one way audio issues please get in contact with us.

One way audio on some inbound calls (update)

2020-08-03T02:41:00+00:00

We have been doing some more testing, it's looking like the issues have so far been resolved. We haven't had an official update from our upstream provider, once we do we will update you here.

One way audio on some inbound calls (update)

2020-08-03T02:05:00+00:00

We have been advised the issue has returned, we have escalated the issue further to our upstream provider. They are currently investigating again.

One way audio on some inbound calls (update)

2020-08-03T00:50:00+00:00

It's looking like fixes have been applied to a number of Wellington areas, however we are still noticing a few instances of one way audio to some locations. We are continuing to work on this fault with our upstream provider. Further updates to follow.

One way audio on some inbound calls (update)

2020-08-02T23:10:00+00:00

We have been since been advised that multiple providers are affected and the fault is situated in Wellington. The fault is now a P1 and we hope to have it resolved soon.

One way audio on some inbound calls (update)

2020-08-02T22:30:00+00:00

The ticket has been escalated within our upstream provider's network team. We have identified that mainly Spark people calling in are are impacted where the B side (you) are unable to hear the A side (spark) phone call. We are also noticing this with some Vodafone callers. As a work around, if a caller ID is presented, you can redial the number and talk as normal until the issue is fixed. We will be following the issue up within the next hour. Any further updates will be available at https://www.neos.co.nz/status

One way audio on some inbound calls

2020-08-02T21:00:00+00:00

We are experiencing one way audio on some inbound calls for some providers like Vodafone and Spark. We are working on our upstream provider to diagnose the fault.

Network Maintenance (minimal disruption)

2020-07-21T03:10:06+00:00

On Thursday morning the 23rd of July between the hours of 1AM and 6AM we will be migrating one of our upstream network handovers to their new V2 network. During this time we expect a network drop off of about 30 seconds while our system reroutes to backup systems. If you have any questions regarding this, please feel free to contact us.

Network Upgrades

2020-06-03T01:13:30+00:00

On Wednesday the 24th of June between the hours of midnight and 6AM one of our upstream providers will be performing maintenance on our internet legs. During this time our systems will cross over to redundant paths with a dropout of no longer than 30 seconds. If you have any questions, please feel free to get in contact with us.

CloudTalk Upgrade

2020-05-06T23:03:12+00:00

On Wednesday (May 13th) between the hours of 2AM and 6AM we will be upgrading our VOIP CloudTalk PBX platform with our upstream provider, during this time some calls may not connect during the outage window. This new major release supports video conferences, shared messaging and other new features and bug fixes. If you have any questions, please feel free to contact us.

NZVZ2 Upgrade (Auckland) 2

2020-05-06T01:18:04+00:00

Tonight we will be upgrading one of our Auckland servers from 8pm. There is some old email and DNS services on this server and these services will be suspended during the upgrade. If you have any questions, please feel free to contact us. DNS will continue as normal as other DNS servers will pick up the load, Email will be delivered once the servers come back online.

Hosted Exchange Upgrades

2020-05-05T08:18:59+00:00

Tonight we will be upgrading each of our Hosted Exchange servers, we don't expect an client impact during this time due to their cluster setup. If you have any questions, please don't hesitate in contacting us.

NZVZ2 Upgrade (Auckland)

2020-05-05T05:09:31+00:00

Tonight we will be upgrading one of our Auckland servers from 9pm. There is some old email and DNS services on this server and these services will be suspended during the upgrade. If you have any questions, please feel free to contact us. DNS will continue as normal as other DNS servers will pick up the load, Email will be delivered once the servers come back online.

Internet Connections Planned Work

2020-04-21T01:27:32+00:00

Friday morning 24/04/2020 between the hours of 12am and 2am we will be doing some work on our Internet Connection (ISP) backup circuits, this is to resolve an identified fault on our handover links from one of our upstream providers. If you have an internet connection with us, we expect an outage of about 10 to 60 seconds during this maintenance window to test the failover of these systems once the changes has been made. If you have any questions, please don't hesitate in contacting us.

Scheduled Network Upgrade 2 (Low Impact)

2020-04-09T02:37:52+00:00

Thursday morning between the hours of 1AM and 3AM we will be migrating our client internet connections and VOIP services over to a new handover. During the outage window we expect the disruption to be less than 3 minutes as the services are migrated. If you have any questions, please don't hesitate in contacting us.

Scheduled Network Upgrade (Low Impact)

2020-04-08T03:24:58+00:00

We will be upgrading one of our transit links between us and one of our upstream providers on Wednesday the 15th of April with a planned outage window between 1AM and 3AM. During this time we expect a short drop off of internet services for about 30 seconds while they transition to redundant systems. If you have any questions regarding this, please feel free to contact us.

Upstream Planned Outage: Thursday 19-Mar-2020 - Minimal Impact

2020-03-03T00:01:02+00:00

One of our upstream providers will be performing maintenance on their network between the hours of 1 AM and 6 AM in the early hours of the 19th of March. During this time our network will automatically cut over to a diverse path with a network drop of about 30 seconds as it crosses over. If you have any questions regarding this notification, please contact us.

Hosted Exchange Maintenance - Minimal Disruption

2020-03-02T02:28:42+00:00

On Tuesday 3rd March between 8 PM and 4 AM we will be applying Cumulative Updates to the Hosted Exchange cluster. As the Hosted Exchange Platform is redundant, we don’t expect any disruption during this maintenance window. Maintenance is expected to be completed by 4 AM. If you have any questions, please feel free to get in contact with us.

Hosted Exchange Maintenance Tonight - Minimum Disruption

2020-02-27T07:57:32+00:00

We have identified some urgent security patches that need to be applied to the hosted exchange servers from Microsoft and we are wanting to do this as soon as possible, we have opened up a maintenance window tonight to get this done. Tonight between 10pm until 12am we will be performing a failover test and applying security updates to the Exchange cluster. From 10pm we expect a small network break of no more that 1-2 minutes to test the system failover and the remaining time will be applying patches to the nodes in maintenance. If you have any problems, please feel free to get in contact with us.

General System Maintenance

2020-02-26T01:25:09+00:00

Tonight between the hours of 9pm and 6am tomorrow we will be upgrading our Cloud Management System (CloudStack) and Email Cleaning servers (SpamShield). We don't expect any disruption during this time as these services are either not customer facing or inside redundant clusters. If you have any questions, please don't hesitate in contacting us.

CloudSync and Web Server Maintenance

2020-02-25T00:00:53+00:00

Tonight between 10pm to 6am we will be upgrading our CloudSync (file sync) servers. We expect an outage of maybe 1-2 hours during this time. You will continue to be able to access your files locally on your machine and file synchronization will continue once the service resumes.

In this outage window we will also be upgrading our webservers (NZ1-NZ4) for general security updates, the outage shouldn't be any longer than 30 seconds on each server while the server reboots. We don't expect any issues but would recommend you check your website in the morning.

We will also be applying new firmware on our firewalls during this time, we don't expect there to be any outages here as the firewalls are in a redundant cluster.

If you have any questions, please feel free to get in contact with us.

Network Upgrade

2020-02-17T02:39:38+00:00

Early Tuesday morning next week (25/02/2020) in between the hours of 1am to 6am we will be performing some upgrades to increase the capacity and redundancy of our network. Depending on the services you have with us we expect about 1-5 small network drops of about 1 to 30 seconds during the outage window while our network crosses over to redundant paths. Services affected are Internet Services (www), Internet (ISP) Connections and VOIP (phone) services. If you have any questions, please get in touch with us.

(Cancelled) Network Upgrade

2020-02-13T01:08:11+00:00

We have decided to cancel tonight's (Friday morning 14/02/2020) network upgrade. We will likely be rescheduling this for the end of next week. If you have any questions, please don't hesitate in contacting us.

Network Upgrade

2020-02-12T20:48:59+00:00

Tonight (Friday morning 14/02/2020) in between the hours of 1am to 6am we will be performing some upgrades to increase the capacity and redundancy of our network. Depending on the services you have with us we expect about 1-5 small network drops of about 1 to 30 seconds during the outage window while our network crosses over to redundant paths. Services affected are Internet Services (www), Internet (ISP) Connections and VOIP (phone) services. If you have any questions, please get in touch with us.

Exchange Queue Flush

2020-02-11T03:35:46+00:00

Tonight at about 9pm we will be flushing the Exchange queue and rebuilding the queue file. This is to resolve a queue database size issue. We don't expect their to be any impact, at most a small disconnection of 30 seconds while a service is restated on the primary server. Any questions, please get in contact with us.

VOIP Platform Upgrade

2020-01-26T19:25:10+00:00

Tonight between the hours of 10pm and 3am we will be upgrading firmware and services inside our CloudTalk VOIP PBX cluster. During this time there will be two 30 second drops within the outage window where calls won't be allowed while services move between primary and backup systems, no calls will be dropped during this time.

Issue with NZ4 Webserver

2020-01-22T23:32:20+00:00

All services have resumed and working as normal, we will monitor the server closely for the remainder of the day.

Issue with NZ4 Webserver

2020-01-22T23:20:00+00:00

Issue with NZ4 Webserver, not responding to some tasks.

CloudPBX Upgrade

2019-12-15T21:13:36+00:00

Tonight we will be applying a patch to our cloud PBX system. The update will only be applied when there are no calls on the system.

Upstream Maintenance

2019-11-18T19:44:28+00:00

05-Dec-2019 01:00 - 05-Dec-2019 06:00 NZT one of our upstream providers will be performing maintenance on their core network and will have an outage of up to half an hour. During this time our systems will failover to backup links with an outage of no longer than a couple of minutes during crossover. If you have any questions regarding this, please feel free to contact us.

Websites not working (resolved)

2019-10-22T04:00:00+00:00

All Fixed

Websites not working

2019-10-22T01:15:00+00:00

We are currently experiencing issues on one of our web servers where pages are not being served correctly, we are working on the issue now.

2Degrees calling issue

2019-10-17T19:56:51+00:00

This has now been resolved by 2Degrees, if you have any further problems please contact us.

2Degrees calling issue

2019-10-17T00:00:00+00:00

We have been advised that some customers of 2Degrees Mobile are unable to call our numbers on our platform. We are working with 2Degrees to resolve the issue on their end, they expect the problem to be resolved overnight. All other platforms are running as normal, any issues please contact us using the normal support channels.

Voice Platform Drop

2019-10-02T02:00:00+00:00

Today at about 3pm some clients may have had degraded service or lost access to their voice services for a minute or so. This was due to a primary node failing and unexpectedly rebooting. We are in contact with our vendor to look into the issue.

Packetloss at 9:51

2019-10-01T08:50:00+00:00

At 9:51pm tonight one of our upstream internet peers dropped for about 30 seconds and then came back online with major packet loss. We have since dropped this and reverted to backups until our upstream can resolve the issue.

Unable to access some sites at random times this morning

2019-09-29T19:30:00+00:00

This morning we had a number of small outages affecting domestic transit (and also some international services which peer domestically e.g. Microsoft Office). This was caused by a line card in one of our upstream providers routers causing traffic to be re-routed over backup networks. These 30 second outages happened at 08:29, 09:05 and 08:54. We are working with our upstream provider on the issue, please contact us if you have any further network issues.

Online Backup Maintenance

2019-09-16T05:00:00+00:00

This has now been resolved

Online Backup Maintenance

2019-09-15T20:00:00+00:00

Currently our Online Backup server is running in debug mode and not responding. Our provider is working on it currently to identify the issue. Any backups will resume once the server is up and running again. If you have any questions, please contact us via our help desk.

UFB/Fibre Chorus Scheduled Outage

2019-09-10T04:05:47+00:00

This Thursday morning between the hours of 1AM and 6AM UFB connections in the lower North Island area may experience an outage of up to 15 minutes while Chorus installs updates to resolve on their network.

Internet Drop

2019-09-03T01:34:00+00:00

At 1:34pm today we had one of our ISP edge routers lockup and reboot and all of the internet connections on this router failed over to a secondary. Everyone should be reconnected to the internet now, however if you are not please either restart your router on contact our team on 0800 022 123 for support.

Issue with email on NZ4 (resolved)

2019-08-29T02:56:00+00:00

Sending email issue on NZ4 has now been resolved. We can confirm we can send from external/internal devices as well as websites. If you continue to have any issues, let let us know.

Issue with email on NZ4

2019-08-29T02:01:00+00:00

Issue with email on one of our web servers, looking into issue now.

Upgrade and test of fail over systems for UFB/ADSL/VDSL internet (ISP) services

2019-08-21T23:59:25+00:00

On Tuesday morning between the hours of 12am to 6am (out of business hours) we will be upgrading and testing the failover of our internet services ADSL/VDSL and UFB (fibre). During this time there may be 2 outages of about 30 seconds while the circuits failover and failback to the original configuration. If you have any questions please contact us at https://neos.co.nz/contact-neos/

Online Backup Service suspended

2019-08-21T08:59:00+00:00

We have resumed our online backup service, any backups will continue as scheduled. If your PC or Server requires the redistributable to be installed this will be done automatically. In most cases client machines will ask to be rebooted and servers will reboot automatically if there isn't any response. If you have any issues, please contact our support team.

Online Backup Upgrade

2019-08-21T00:32:26+00:00

Today at about 12pm users may have noticed some of their servers or desktops reboot due to a Microsoft Visual basic run-time update, this was an unforeseen issue due to our Online Backup platform upgrade which also updated the clients. Tonight at 9pm we will resume our Online Backup servers which will also resume the upgrade process of the backup clients which may require some servers and desktops be restarted. If you have any questions, please contact our support team.

Online Backup Service suspended

2019-08-21T00:00:00+00:00

We have stopped our Online Backup service as it was pushing out an update which would restart the Client and server machines (server machines automatically) due to a Microsoft Visual Basic Runtime update. We have scheduled this to be resumed at 9pm tonight so all servers can be upgraded automatically.

Issue with Exchange 2016

2019-08-11T22:02:00+00:00

Performance issues with Exchange 2016 has been resolved. Issue is due to a load balancer memory utilization issue, we have put a work around in place while we work with the vendor to find a more permanent solution.

Issue with Exchange 2016

2019-08-11T21:39:00+00:00

Issue with Exchange 2016, some drops may be happening for VPS clients or clients with direct connections

Upstream provider maintenance notification

2019-08-01T03:44:49+00:00

One of our upstream providers will be performing out of hours maintenance on one of our internet circuits on the 5th of August between the hours of 12am and 6am. When they perform the update, our systems will automatically be moved to backup links with an outage of no longer than 30 seconds. If you have any questions regarding this please contact us on our website or through the usual support method.

Exchange 2016 Upgrades

2019-08-01T02:33:58+00:00

From the 31/07/2019 7pm for a maximum of a week we will be upgrading our hosted exchange 2016 platform. We expect very little interruption during this period but some customers that are directly connected may experience a couple of short cut outs while database copies happen. If you have any questions regarding this, please contact us through the standard support channels.

Issues with Hosted Exchange

2019-07-30T02:28:00+00:00

This issue has now been resolved.

Issues with Hosted Exchange

2019-07-30T00:07:00+00:00

Reports of some out going email getting a "relay denied" message. We have added some rules to stop this from happening. If you continue to have issues, please let us know.

Issues with Hosted Exchange

2019-07-29T23:56:00+00:00

The Hosted Exchange platform issues experienced this morning was due to a migration taking place to new hardware and the load balancer not correctly balancing all of the requests while the migration was in progress. We have installed a fix to only point to the primary server until the migration is completed, We will monitor the load of the servers until this has been completed. if you have any questions, please feel free to contact us on our website or though the usual means.

Issues with Hosted Exchange

2019-07-29T23:39:00+00:00

We have updated the hostname record, if you are having issues connecting with outlook, please close out of outlook and go back in.

Issues with Hosted Exchange

2019-07-29T23:27:00+00:00

We are updating host names so connections take a different route to the database servers.

Issues with Hosted Exchange

2019-07-29T23:19:00+00:00

We are making changes on the load balancer to try an reduce any impact

Issues with Hosted Exchange

2019-07-29T22:46:00+00:00

We have rebooted one of the primary database servers and new resources. It is coming up again now.

Issues with Hosted Exchange

2019-07-29T20:00:00+00:00

We are currently experiencing issues with our hosted exchange platform, we are in the process of resolving the issues here.

Core Network Maintenance - Tonight

2019-06-20T01:45:08+00:00

Tonight 20/06/2019 from 11pm we will be performing maintenance on our core and border networks. We expect up to four (4) thirty (30) second outages while the network converges to backup hardware. If you have any questions, please feel free to contact us at https://neos.co.nz/contact-neos/.

Upstream Network Upgrades

2019-06-19T00:39:29+00:00

On the 1st of July one of our upstream providers will be upgrading their network between the hours of 12am and 6am. During this time our network will cross over to backups with a maximum outage of about 30 seconds during the crossover. If you have any questions regarding this please contact us.

There is currently an issue with the 2016 exchange server - Resolved

2019-06-04T11:05:00+00:00

This issue has now been resolved.

There is currently an issue with the 2016 exchange server

2019-06-04T10:47:00+00:00

There is currently an issue with the 2016 exchange server, we are looking into the issue

Network Maintenance - Minimal Impact

2019-05-26T21:40:48+00:00

One of our upstream providers will be upgrading their network on Monday the 10th of June between the hours of 12am and 6am. During this outage window our systems will fail over to a backup circuit with an outage of no longer than a minute. If you have any questions regarding this please feel free to get in contact with us.

Hosted Exchange 2016 Maintenance

2019-05-08T22:55:41+00:00

Tonight between 11pm and 12am we will be upgrading the virtual routers within our Exchange 2016 platform to make them more fault tolerant and will be performing some optimizations to increase the Exchange responsiveness. We expect an the Exchange outage to be no longer than 5 minutes during this maintenance window. Email will be queued and delivered during the maintenance period.

Issue with 2010 exchange server

2019-04-09T21:45:00+00:00

Exchange 2010 platform is back up and running again, fault was due to a piece of software being installed causing the disk to lock up. This has since been removed. As we are migrating people away from this platform to a new one, please contact us if you would like to make your upgrade a priority.

Issue with 2010 exchange server

2019-04-09T21:20:00+00:00

Currently an issue with exchange server, we are rebooting the server now to try and resolve the fault.

Issue with 2010 exchange server not responding

2019-04-07T20:34:00+00:00

Exchange server is now running correct, there is an issue with the backup processes which we are investigating today.

Issue with 2010 exchange server not responding

2019-04-07T20:16:00+00:00

We are rebooting the server now

Issue with 2010 exchange server not responding

2019-04-07T19:47:00+00:00

Issue with 2010 Exchange server not responding, we are investigating the problem

2010 Exchange sending email issue

2019-04-07T19:43:00+00:00

System is not operational

2010 Exchange sending email issue

2019-04-07T19:42:00+00:00

This morning there was a problem with the 2010 Exchange server where it wasn’t delivering email. It looks like it was due to a backup process potentially causing the hub transport to stop. We have killed the backup process and restarted the hub transport and can see email flowing again. We will be investigating what actually caused this particular problem later today.

Planned Outage Notification - Minimal Impact

2019-02-04T02:18:14+00:00

One of our upstream providers will be preforming maintenance on their circuits early Friday morning the 15th of Feb between the hours of 01:00 and 05:00. There may be a short 30 second outage as our system crosses over to our backup peers during the maintenance window. If you have any questions regarding this notification, please feel free to contact us.

Planned Outage Notification - Minimal Impact

2019-01-17T22:39:00+00:00

One of our upstream providers will be preforming maintenance on their circuits early Thursday morning the 24th of Jan between the hours of 01:00 and 05:00. There may be a short 30 second outage as our system crosses over to our backup peers during the maintenance window. If you have any questions regarding this notification, please feel free to contact us.

NZSAN2 Down - Secondary Storage

2018-12-06T20:07:16+00:00

NZSAN2 is up (has been since the outage). Outage was caused by a bad hard drive locking up the entire array, we are unsure why this happened but could replicate the problem with our hard drive testing software. The hard drive was replaced at the time of the outage, a parity check was completed without error. As at 7/12/18 9am the system is 50% through adding the new disk to the RAID6 volume.

NZSAN2 Down - Secondary Storage

2018-11-30T09:55:00+00:00

We are currently experiencing issues with NZSAN2. This Storage system is used for low end archive storage, some backup systems etc. The system is currently rebuilding from two failed hard drives. If you have any questions, please contact us by email or call for urgent assistance.

Exchange issue resolved

2018-11-30T09:45:00+00:00

We have resolved the exchange problem

Issue with exchange, we are looking into issue

2018-11-30T08:00:00+00:00

We are experiencing an issue with exchange and a down storage network, we are investigating the issue

Network Upgrade

2018-11-14T02:05:58+00:00

Tonight from 1AM 15/11/2018 we will be replacing our edge and upgrading our core networking equipment. Within the window of 1am to 6am expect a number of small outages (no longer than 2 minutes each) as we upgrade this infrastructure. If you have any questions, please contact us at support@neos.co.nz

Hosted Exchange 2016 issues

2018-11-05T21:00:00+00:00

From 7am this morning some users were experiencing issues connecting to our Hosted Exchange 2016 platform, on call technicians were contacted and started investigating the problem. The Neos tech found that one of the databases were dismounted due to an EDB size limit imposed by Microsoft Exchange. Neos tech found a fix from Microsoft to work around the issue and patched the required servers. Note: Because this was a software limit reached for a particular database, it affected some users on all servers within the cluster as the databases wouldn't fail over, as the servers themselves were fine no alarm notifications were sent. The patch is now in place, however we are closely monitoring the Exchange cluster and are investigating measures to mitigate any future occurrences of the fault.

Hosted Exchange 2016 issues

2018-11-05T18:00:00+00:00

Some users with hosted exchange may be experiencing issues connecting, we have identified the issue.

Wellington Chours UFB Network Upgrades

2018-10-02T00:03:38+00:00

This is a late notice notification that Chorus are upgrading OLT’s in the Wellington area UFB network. If you have UFB/fibre, between the hours of 11pm 2/10/2018 and 7am 3/10/2018 expect an outage of a maximum 23 minutes while Chorus perform the upgrade. If you have any issues following the upgrade, please contact Neos on 0800 022 123.

CloudSync software upgrade

2018-09-30T20:56:20+00:00

Tonight from 10pm we will be upgrading our CloudSync/OwnCloud server packages. During this time you will be able to access your files as normal, however any changes you make won't be synchronized until the upgrade is complete. If you have any questions regarding the upgrade, please email support@neos.co.nz.

Network Maintenance

2018-08-21T03:28:55+00:00

Thursday morning between 12am and 6am one of our upstream providers will be performing scheduled maintenance on our internet circuits. During this time our systems will cut over to a backup circuit with a transition time of a couple of minutes. If you have any questions regarding this notification, please contact us here https://neos.co.nz/contact-neos/

VOIP calling issue (resolved)

2018-08-12T22:07:15+00:00

The VOIP calling issue has now been resolved, this issue is also related to some websites not being able to be contacted during the outage. We have requested incident reports from our voice and network upstream providers, if you require a copy please contact us here https://neos.co.nz/contact-neos/

VOIP calling issue

2018-08-12T20:33:12+00:00

Currently there is an issue with VOIP calling, our upstream provider is investigating the issue

Backup Power Maintenance

2018-07-25T00:21:40+00:00

On Sunday the 29th of July from 3pm we will be performing backup power maintenance and testing. During this time we will be load testing generator, UPS and backup power transfer. We don't expect any customer impact during this time.

Upstream Router Upgrades

2018-07-05T02:24:14+00:00

Date: Wednesday 25-Jul-2018 Time: 25-Jul-2018 00:00 - 25-Jul-2018 06:00 NZT

One of our upstream providers are upgrading their router hardware on the 25th of July between 12am and 6am. During this time our systems will cross over to a backup network. We expect this to take no longer than 30 seconds to 2 minutes during the 20 minute upstream outage.

If you have any questions, please contact us at support or call on 0800 022 123

CloudSync Services Upgrade

2018-06-21T21:23:08+00:00

We will be upgrading the CloudSync services on Tuesday evening from 10:30pm . During this time CloudSync services will stop, however you will still be able to access your files as normal and any unsynced files will resume once the service is back up and running again.

If you have any further questions regarding the upgrade, please email support@neos.co.nz

Hosted Exchange 2010 Upgrade

2018-06-12T21:30:20+00:00

On Thursday evening we will be upgrading some of the .net framework on one of the Exchange 2010 services. There will be some outages during this outage window of 10:30pm - 11:30pm

Hosted Exchange 2010 maintenance tonight

2018-06-12T04:24:16+00:00

From 10pm tonight we will be upgrading our Hosted Exchange server tonight from 10pm. We expect an outage from 10:30pm

Hosted Exchange 2010 upgrade

2018-06-06T23:18:44+00:00

Tonight from 10:30pm we will be upgrading our older hosted exchange platform. During this time the exchange server will not be available, email will be queued and delivered once up again.

Maintenance on 2010 Hosted Exchange

2018-06-03T10:42:38+00:00

Tonight we are performing maintenance on the Exchange 2010 servers, we don't expect anymore than a 15 minute outage as we move log files between volumes.

Auckland Network Drop

2018-05-29T00:15:55+00:00

Issue now looks to be resolved.

Auckland Network Drop

2018-05-28T23:40:00+00:00

Our Auckland datacentre has experienced small outage of a couple of minutes, this looks like it is due to a broadcast storm. Technicians are looking into the issue now, all services seem to be up and running now. This will effect VOIP and some web services.

Redundancy Testing and Firewall Upgrade

2018-05-10T03:53:47+00:00

Tonight from 1am we will be performing fail over testing on our network. We expect there to be 2-4 short 2 minute network drops in the outage window of 1am to 2am.

We will also be upgrading the firmware on our two firewalls, since these are clustered together we don't expect there to be any noticeable outages.

Core Network Issue

2018-05-08T16:57:36+00:00

Power supplies have been replaced, network has also be optimized for faster fail-over in the event a similar event was to happen.

Replace Core Router Power Supplies

2018-05-08T03:32:23+00:00

Tonight at 1am (01:00 09/05/2018) we will be replacing power supplies in each of our core routers and placing the primary router back into service. We expect any disruption to be minimal during this time.

If you have any queries, please feel free to contact us on 0800 022 123

Core Network Issue

2018-05-08T02:56:25+00:00

Core router failure is due to a failed power supply, we are in the process of replacing this now.

Core Network Issue

2018-05-08T02:21:15+00:00

On backup core router, all services normal.

Core Network Issue

2018-05-08T02:00:00+00:00

We have just had one of our core routers fail, the system has moved over to a redundant router however this took a little bit of time. All systems are now functional however please call if you are experiencing any further issues. We will continue investigating the issue and update here.

Peering Outage (updated)

2018-04-14T20:44:26+00:00

Issue has now been resolved

Peering Outage

2018-04-14T18:25:00+00:00

One of our peers (an internet connection) with and upstream provider has failed, we are currently running on a backup peer. There maybe some internet performance issues until the primary peer issue is resolved.

Network latency to some international destinations (fixed)

2018-03-21T18:04:01+00:00

Please be advised that the repair of the fibre-cut has been completed. Services are expected to be up and running.

Network latency to some international destinations

2018-03-21T11:08:06+00:00

From our upstream provider :

Please be advised fibre team have hauled replacement fibre and splicing will soon begin. Services will begin restoring from around 22:30 AEDT with all circuits expected to be restored by 04:00 AEDT tomorrow morning. A notification will be sent upon restoration of all services. We appreciate your patience throughout the restoration works.

Network latency to some international destinations

2018-03-21T09:08:39+00:00

As per the identified major fibre cut in Mascot, Sydney which is resulting in all our upstream provider's links between Australia and NZ via the Southern Cross Cable being down.

The AU team has advised that field force have located the damage to be 4 metres deep. Excavation work has begun with caution due to nearby power, gas and water pipes.

Temporary solutions are being explored via other conduit types in the area.

Currently there is no update to the provided ETR of 22/03/18 - 02:00 NZT

Network latency to international destinations

2018-03-21T08:56:12+00:00

Fibre technicians and staff are scoping for our provider's infrastructure near the damage as a means to rectify, however new cables need to be hauled & spliced. Hauling is currently underway with splicing to begin as soon as possible to restore services.

Network latency to international destinations

2018-03-21T06:38:39+00:00

As per the identified major fibre cut in Mascot, Sydney which is resulting in all our provider links between Australia and NZ via the Southern Cross Cable being down.

Fibre crews are onsite and planning the fibre replacement/repair. Two cables have been identified so far to of been cut by civil works - a 72 core fibre and a 96 core fibre. The two new fibre cables need to be hauled between splice joints and then spliced at each end.

Due to the nature of the work involved, there is an updated ETR of 22/03/18 - 02:00 NZT.

Network latency to international destinations

2018-03-21T05:41:13+00:00

As per the identified major fibre cut in Mascot, Sydney which is resulting in all provider links between Australia and NZ via the Southern Cross Cable being down.

Traffic management has now been implemented to mitigate the impact of capacity lost whilst the the fibre repairs are carried out.

The estimated ETR is still currently 20:00 NZT with further updates to come.

Network latency to international destinations

2018-03-21T04:44:06+00:00

Fibre technicians arrived on site and identified that the damage has been caused by a third party performing civil works. Hauling is currently underway with splicing to begin as soon as possible to restore services.

Network latency to international destinations

2018-03-21T04:06:53+00:00

Update from our upstream:

We have a major fibre cut in Mascot, Sydney resulting in all provider links between Australia and NZ via the Southern Cross Cable being down.

This is causing international traffic to be rerouted and a loss of international capacity, our engineers are working on changes to minimize the impact of the capacity loss. Customers will be experiencing high latency and packet loss for traffic routing internationally

Fibre contractors have been informed and are en-route to Mascot. The estimated ETR is currently 20:00 NZT, further updates to come.

Network latency to international destinations

2018-03-21T03:40:34+00:00

Update from our upstream provider:

Please be advised that multiple International circuits between Australia and New Zealand and Australia to the USA has failed. Engineers are working to establish the cause of the failure and to bring the links back online. Further updates will be provided within the hour.

Network latency to international destinations

2018-03-21T02:51:58+00:00

Our upstream provider has advised that they have identified a fault with their Australian peering and are working to rectify the issues, there is no ETA currently.

Network latency to international destinations

2018-03-21T02:36:58+00:00

We have noticed network latency to international destinations. We have lodged this with our upstream providers and they are currently investigating. We will drop the link to go to backups if required.

Network Outage (resolved)

2018-03-15T21:39:23+00:00

We have been advised the issue has been resolved. The issue was due to a power outage in Wellington and our upstream is now running on generator power.

Network Outage

2018-03-15T21:18:00+00:00

National back haul has failed and we are now running on backup links. This looks like a wide spread national outage, we have lodged this with our upstream providers to resolve

Issue with Auckland DC

2018-03-15T21:04:55+00:00

Issue with Auckland DC, investigating

Email Cleaning Resolved

2018-03-12T08:34:20+00:00

Upgrade is complete

Email Cleaning Upgrade

2018-03-12T01:17:49+00:00

The email cleaning platform is currently being upgraded. Since it is a clustered environment email will continue to flow as normal however some web access controls may not be available.

Failed memory module (resolved)

2018-03-11T03:19:26+00:00

The failed memory modules have been removed and VMs have been migrated back to this node. We will schedule a memory replacement at a later date.

Failed memory module

2018-03-11T02:07:07+00:00

A memory module has failed on one of our compute nodes (c3vz1.c0), we are migrating machines away from this node while it is repaired. There shouldn't be any impact on users/servers during this time.

Node Clusters

2018-02-27T11:06:43+00:00

All clusters are operating normally.

Hosted Exchange 2010 (update)

2018-02-27T11:05:41+00:00

All systems are back to normal. An email will be sent out at a later date.

Hosted Exchange 2010 (update)

2018-02-27T01:50:55+00:00

We will be rebooted the Exchange 2010 services tonight from 10:30pm to further apply some fixes.

Hosted Exchange 2010 (update)

2018-02-25T16:29:38+00:00

Storage migration is complete, the last step is finalizing the data store which will take a couple of days. During this time there maybe some slight performance impact, but it shouldn't get to a point of the system being unusable.

Hosted Exchange 2010 change platform update

2018-02-24T21:23:59+00:00

The storage migration is almost complete, we hope to change over to the new system tonight all going well. We will schedule a 2.5 hour outage from 11pm 25/02/2018 for cross over.

Hosted Exchange 2010 (update)

2018-02-24T21:22:14+00:00

We are at 92% through the storage migration, we hope to schedule the change over tonight from 11pm

Hosted Exchange 2010 (update)

2018-02-23T06:40:39+00:00

We are migrating the storage volume at the moment, there maybe some slowness at times but all systems are functional.

Hosted Exchange 2010 (update)

2018-02-22T22:12:38+00:00

We will be attempting to migrate the storage to production hardware again this weekend (starting tonight 5:30pm), we don't expect there to be any downtime until cross over which may happen Sunday night which we will schedule closer to the time. The recovery server will stay down until this is done so we don't cause any performance impact the existing infrastructure.

Hosted Exchange 2010 (update)

2018-02-21T02:32:38+00:00

The recovery server with old emails crashed and we are in the process of rebuilding, Once recovered we will be able to remount the old database. We expect this may be in the next day or two.

Hosted Exchange 2010 (update)

2018-02-20T14:37:34+00:00

We have had to abort the migration due to the amount of time it was going to take, we will keep it running in the background but will need to schedule a cross over at a later date.

Hosted Exchange 2010 (update)

2018-02-20T10:44:09+00:00

We are about to start the migration, we will keep you up to date here.

Hosted Exchange 2010 (update)

2018-02-20T10:02:25+00:00

All up and running again. We will be migrating to new infrastructure tonight, expect a 2 - 4 hour outage during this time.

Hosted Exchange 2010 (update)

2018-02-20T08:00:57+00:00

Exchange server has crashed, looking into issue

Hosted Exchange 2010 (update)

2018-02-19T20:24:08+00:00

We are in the process of testing a different type of migration that will allow us to increase the performance of the Exchange 2010 services, if this works we hope to be able to migrate tonight.

Hosted Exchange 2010 (update)

2018-02-18T19:25:56+00:00

The storage migration failed over the weekend, we are in the process of figuring out why this happened and the next steps to take.

Hosted Exchange 2010 (update)

2018-02-16T02:04:44+00:00

We are currently at 79% through the storage migration to new hardware. At this stage we expect to be going live with new infrastructure Sunday evening.

Hosted Exchange 2010 (update)

2018-02-15T22:52:27+00:00

We are at 76% through the storage replication

Hosted Exchange 2010 (update)

2018-02-14T22:20:09+00:00

We are at 58% through the storage replication. We expect everything will be back to normal by or during the weekend.

Hosted Exchange 2010 (update)

2018-02-14T00:24:30+00:00

We are currently at 38% through the storage replication to new hardware. We expect this to be completed by/during the weekend and this will rectify any performance issues users maybe experiencing once complete.

Hosted Exchange 2010 (update)

2018-02-13T04:28:49+00:00

Over the weekend we build a temp Exchange platform for people to access their missing emails, we are in the process of sending out instructions of how to access this and forward the emails you require through to your current email addresses. We are doing this slowly to reduce load on the platform. You may also notice things are running slow at the moment, this is due to the system running on backup infrastructure which has reduced performance. At the same time we are migrating the storage to a higher performance system which will eliminate the slowness once complete. We expect this to happen before the weekend.

Hosted Exchange 2010 (update)

2018-02-10T04:47:46+00:00

Setup of missing emails is complete, you will receive details with how to access soon.

Building power maintenance

2018-02-09T19:34:00+00:00

Today we will have an electrician doing work on power to the building, we will cross over to generator power and expect there to be no down time for our customers.

Hosted Exchange 2010 (update)

2018-02-09T19:32:55+00:00

The copy was successful, we will have a missed email available this afternoon

Hosted Exchange 2010 (update)

2018-02-09T04:48:25+00:00

We have had to reboot the server due to the copy causing too much load, it is all looking stable now but we will migrate the copy to another machine to resolve this

Hosted Exchange 2010 (update)

2018-02-09T03:33:46+00:00

Copy unfortunately failed, we are having to start the copy again from the beginning.

Hosted Exchange 2010 (update)

2018-02-09T02:05:44+00:00

We are currently at 60% of the missing email copy.

Hosted Exchange 2010 (update)

2018-02-09T00:29:06+00:00

35% of copy for missed email

Hosted Exchange 2010 (update)

2018-02-08T23:30:34+00:00

We are at 20% of the copy

Hosted Exchange 2010 (update)

2018-02-08T22:22:46+00:00

We are having to do one more copy of the data for missing emails, we hope this to be completed in the next couple of hours.

Hosted Exchange 2010 (update)

2018-02-08T20:17:25+00:00

The missing database restore is complete, expect access to be provided this morning with instructions.

Hosted Exchange 2010 (update)

2018-02-08T03:45:15+00:00

We are at 75% of the verify for the missing emails. We hope to have them available by tomorrow morning.

Hosted Exchange 2010 (update)

2018-02-08T02:42:43+00:00

We are at 100% through the database recovery for missing emails, we are just running verify now.

Hosted Exchange 2010 (update)

2018-02-07T23:23:16+00:00

Currently we are at 88% through the database recovery for missing emails

Hosted Exchange 2010 (update)

2018-02-07T21:31:06+00:00

Currently we are at 80% through the database recovery for missing emails , it looks like it has moved a lot faster over night and we hope it to be completed soon. We have been advised that some users were experiencing slowness this morning, this was due to a backup running on the database, we have paused the backup for the time being and the load has been reduced. Note: changes with calendar entries and other components along with missing emails will be available in the recovery.

Hosted Exchange 2010 (update)

2018-02-07T00:15:30+00:00

We are about 15% through the database recovery for missing emails, current server performance is better now as user email syncs are more up to date. We will continue to keep you posted with updates.

Hosted Exchange 2010 (update)

2018-02-06T20:10:55+00:00

All of the missing email reports were sent out last night, We are 10% the way through the missing email database recovery and currently load on the server is high at times the but system is stable.

Hosted Exchange 2010 (update)

2018-02-06T12:27:52+00:00

We have sent most of the the missing email reports, the database repair is still running in the back ground. We will have more updates tomorrow morning.

Hosted Exchange 2010 (update)

2018-02-06T09:29:11+00:00

We have been able to extract all missing email data for all days, however we are repairing the database and working on the best way to deliver this. We are also working on lists of missing emails that were received, we should be able to deliver this soon.

Hosted Exchange 2010 (update)

2018-02-06T03:27:42+00:00

The merge of missing emails from 2nd & 3rd has been unsuccessful. We are working on other options to get access to these emails. More updates to follow

Hosted Exchange 2010 (update)

2018-02-05T06:31:43+00:00

We are rebooting the host to help with any of the performance issues we are experiencing. System will be down for 5 minutes while this happens.

Hosted Exchange 2010 (update)

2018-02-05T01:28:55+00:00

Load is now decreasing on Servers and we are seeing improved performance. More changes are planned to increase performance further. The log replay for the merge is still in progress.

Hosted Exchange 2010 (update)

2018-02-05T00:12:32+00:00

Due to the high load on the Servers there will be ongoing performance issues with the delivery of email. You may experience slowness both within Outlook and with new emails coming in.

Hosted Exchange 2010 (update)

2018-02-04T22:34:59+00:00

You may be noticing some slowness and/or emails coming in maybe slower than normal. This is due to the system being under high load, we expect this to be resolved soon.

Hosted Exchange 2010 (update)

2018-02-04T21:12:35+00:00

Hosted Exchange Email is now working. Please note there may be some missing emails from 2nd & 3rd of February which we will merge back in as soon as possible.

Hosted Exchange 2010 (update)

2018-02-04T20:38:55+00:00

We are about to implement a different fix, we will restore an alternative system so customers can use their email and merge missing data once it is recovered. More updates soon.

Hosted Exchange 2010 (update)

2018-02-04T20:10:28+00:00

We are working on another fix which should mean we will be up and running sooner. We will update you again shortly.

Hosted Exchange 2010 (update)

2018-02-04T19:22:11+00:00

We are still working through the log replay. This currently sits at 27%. Based on this we can estimate that the service should be up by 4am 6th February. We will continue to update you on progress throughout the day.

Hosted Exchange 2010 (explained)

2018-02-04T16:28:30+00:00

What Has Happened - Explained

On Saturday evening 6:30pm one of our server nodes inside a cluster failed due to loosing 2 hard drives at a time. This node can loose a maximum of 1 drive which in most cases would be replaced straight away and rebuilt. Because it lost 2 drives the array failed completely causing the server node to fail itself. We take backups every night at 11pm. Due to the length of time between backups we decided to recover the logs from the broken node over the top of the existing backup to minimize any data loss, however this method takes time as it requires us to run all of the missed transactions between the time of the backup and the loss of the server, we decided this was the best approach forward. Due to the number of transactions this is taking longer than expected and as stated before we feel this is better than missing data/emails.

Note: This is a different problem that what we experienced late last year, however we are currently in the process of moving customers away from this platform to newer technology which has more forms of redundancy, reliability and features which eliminates these types of issues.

Hosted Exchange 2010 (update)

2018-02-04T15:25:29+00:00

20% through log replay

Hosted Exchange 2010 (update)

2018-02-04T12:45:00+00:00

Log extract complete, starting log replay.

Hosted Exchange 2010 (update)

2018-02-04T11:23:44+00:00

New copy process is working faster, 10% in 15 minutes. Once this completes we will replay the EDB logs

Hosted Exchange 2010 (update)

2018-02-04T10:20:56+00:00

Log merge is taking longer than expected, we are changing the merge process to speed things up. Next update before 3am.

Hosted Exchange 2010 (update)

2018-02-04T07:57:54+00:00

Log files are just about recovered, machine is ready for them to be applied. Next update at or before 12am

Hosted Exchange 2010 (update)

2018-02-04T05:00:40+00:00

We continue to restore backups and log files. At this stage our estimate is between 9 – 10pm this evening. More updates to follow closer to this time.

Hosted Exchange 2010 (update)

2018-02-04T01:54:46+00:00

We are still in the process of recovering the backups. To avoid loss of data we are restoring log files along with the main database, this will take longer than initial estimates. We plan to have another update before 6pm. Email is queuing and will be delivered when the exchange server is back online.

Hosted Exchange 2010 (update)

2018-02-03T22:28:39+00:00

We are still in the process of restoring the backup. ETA 3pm 04/02/2018 for server recovery

Hosted Exchange 2010 (update)

2018-02-03T21:11:29+00:00

Recovery of the database failed due to another hard drive failing. We are now in the stages of restoring from a backup.

Hosted Exchange 2010 (update)

2018-02-03T20:07:22+00:00

We have identified the issue to be a unclean shutdown database. We are trying to repair the database in the first instance, failing that we will run from backups. No ETA as yet, but we will post updates here.

Hosted Exchange 2010

2018-02-03T19:33:00+00:00

Hosted Exchange 2010 is having some issues with its database, this looks like it maybe related to the failed node that happened last night. We will put updates here

Node Failure

2018-02-03T08:56:44+00:00

All systems are running OK and have been for quite some time. Most services have been migrated off the faulty node, except 2 remaining services including Hosted Exchange 2010 which an in the process of being moved. Systems will continue to function normally while this happens.

Node Failure

2018-02-03T06:21:17+00:00

We have found 2 failed disks on one of our nodes has failed, we are looking at restoring services soon.

Issue with node in cluster

2018-02-03T05:39:41+00:00

We have had an alert a node has crashed. This may be effecting things like Hosted Exchange 2010 and a few others on this node

Fixed Node Failure

2018-02-01T12:37:04+00:00

Node has been fixed and all services running on it restored.

Node Failure

2018-02-01T11:33:11+00:00

One of our nodes have failed, we are working to resolve the issue. Hosted exchange and some other services may not be available while it is down

Email Cleaning Platform

2018-01-26T05:00:00+00:00

You may have noticed email was slow to deliver this afternoon for some of our services, this was due to the Antivirus engine having performance issues on each of our email cleaning servers. It looks like the issue has been resolved, any email that was in the backlog should now be delivered or delivered shortly. We will continue to monitor the servers closely and work with our up stream provider to resolve any further issues if they arise.

Email Cleaning

2018-01-26T03:58:33+00:00

We have identified the fault and are working with our upstream providers to resolve. Email may take a bit longer to get through while we are working on this.

Email Cleaning issue

2018-01-26T03:21:24+00:00

Currently experiencing issues with inbound and outbound cleaning, currently investigating issue.

Resolved Exchange Issue

2018-01-10T06:24:06+00:00

Hosted exchange is now operational

Issue with Exchange

2018-01-10T06:14:07+00:00

There is currently an issue with sending emails with hosted exchange, we are working on the problem. You may notice some drops during this time.

Fixed - Issue calling local numbers from VOIP platform

2017-10-31T01:32:00+00:00

It looks like the issue has now been resolved. We will continue to monitor the platform and work with our upstream to prevent future occurrences.

14:40 - Advised by upstream, issue confirmed resolved.

Issue calling local numbers from VOIP platform

2017-10-31T01:00:00+00:00

Currently there is an outage with outbound calling local numbers, we are investigating the issue. Mobile numbers seem fine. We are working with our upstream to resolve the issue, currently there is not ETA.

14:27 - Update We have been advised that mobiles are also having the issue for some customers.

Wpanel Upgrade

2017-10-16T22:52:51+00:00

We will be upgrading our Hosted Exchange Web Panel, there will be no impact to any customers during this upgrade.

Core Router testing

2017-10-11T03:14:29+00:00

Tomorrow early morning we will be implementing a core router change along with some fail over testing. We expect up to 4 intermittent outages of up to 5 minutes each between the hours or 12:30 am and 5 am.

Regards, Nick.

Core router upgrade

2017-10-09T02:30:23+00:00

We have scheduled an upgrade to each of our core routers between the hours of 2am - 3am Wednesday morning the 11th of October. During this outage window we expect 2 to 3 outages of up to 5 minutes each.

Regards, Nick.

CloudStack Upgrade

2017-10-05T00:48:52+00:00

Tonight at 9pm we will be upgrading our CloudStack management server to the latest security release. This will not impact any of our customer's services, only remote administration for the period of 9pm to 9:30pm.

Nick.

Online Backup Upgrade

2017-09-12T02:52:00+00:00

Online backup has now been upgraded to another server, if you access the backup server directly, you may need to clean your browser cache. Any issues, please contact Neos Systems at support@neos.co.nz

Regards, Nick.

Online Backup

2017-09-11T23:04:00+00:00

We are currently upgrading our online backup servers, we expect this to be completed before the end of the day. Your backups will automatically resume once the upgrade is complete. If you have any further questions, please contact Neos Systems at support@neos.co.nz.

Regards, Nick.