Customer Impact: Customers were unable to access FONE portal starting 05:18 AM UTC until 01:01 PM UTC. Access to managed applications was impacted for customers between 05:18 AM UTC until 03:38 PM UTC.
Description: After an extended period of monitoring, we can confirm stability of services. This incident is resolved.
Posted Apr 23, 2024 - 02:50 UTC
Update
Customer Impact: Customers were unable to access FONE portal starting 05:18 AM UTC until 01:01 PM UTC. Access to managed applications was impacted for customers between 05:18 AM UTC until 03:38 PM UTC.
Description: Access to the portal and managed applications continues to remain stable. Agent synchronizations have been re-enabled for both new and existing agents.
We will continue to monitor the environment for stability and will provide updates as we make progress.
Posted Apr 23, 2024 - 01:16 UTC
Monitoring
Customer Impact: Customers were unable to access FONE portal starting 05:18 AM UTC until 01:01 PM UTC. Access to managed applications was impacted for customers between 05:18 AM UTC until 03:38 PM UTC.
Description: Access to the portal and managed applications continues to remain stable. Agent synchronisations have been re-enabled for both new and existing agents.
We continue to monitor our environment for stability.
Next update time: 11:00 PM UTC.
Posted Apr 22, 2024 - 19:03 UTC
Update
Customer Impact: Customers were unable to access FONE portal starting 05:18 AM UTC until 01:01 PM UTC. Access to managed applications was impacted for customers between 05:18 AM UTC until 03:38 PM UTC. New agents may experience a delay while attempting to get latest configuration and policies from the FONE portal.
Description: Access to the portal and managed applications continues to remain stable. We are working on restoring SmartEdge agent synchronisations calls back to its full health as customers may observe delays in new agents getting the most upto date configurations and policies from the Portal.
Next update time: 07:00 PM UTC
Posted Apr 22, 2024 - 17:24 UTC
Update
Customer Impact: Customers were unable to access FONE portal and managed applications via FONE SmartEdge and reverse proxy services
Description: We have implemented certain configuration changes which has helped restore access to FONE Portal and managed applications. However, synchronisations calls made by SmartEdge agents to the Portal for fetching configuration and policies continue to remain impacted.
Next update time: 05:30 PM UTC
Posted Apr 22, 2024 - 16:56 UTC
Update
Customer Impact: Customers were unable to access FONE portal and managed applications via FONE SmartEdge and reverse proxy services
Description: We have implemented certain configuration changes which has helped restore access to FONE Portal and managed applications. However, synchronisations calls made by SmartEdge agents to the Portal for fetching configuration and policies continue to remain impacted.
Next update time: 05:00 PM UTC
Posted Apr 22, 2024 - 16:14 UTC
Update
Customer Impact: Customers may not be able to access FONE Portal, or managed applications via forward or reverse proxy services.
Description: we have implemented additional configuration changes and we are monitoring. We will provide more updates as we make progress.
Next update time: 04:30 PM UTC
Posted Apr 22, 2024 - 15:46 UTC
Update
Customer Impact: Customers may not be able to access FONE Portal, or managed applications via forward or reverse proxy services.
Description: We continue work on several mitigation steps. We will provide further updates as we make progress
Next update time: 03:30 PM UTC
Posted Apr 22, 2024 - 15:09 UTC
Update
Customer Impact: Customers may not be able to access FONE Portal, or managed applications via forward or reverse proxy services.
Description: We continue to aggressively work towards a resolution. Teams are working on multiple tracks to address the ongoing issues.
We continue to work towards a resolution at the earliest.
Next update time: 03:00 PM UTC
Posted Apr 22, 2024 - 14:29 UTC
Update
Customer Impact: Customers may not be able to access FONE Portal, or managed applications via forward or reverse proxy services.
Description: Our teams continue to work on a remediation path to help stabilise our infrastructure. We will provide further update as we make progress.
Next update time: 02:30 PM UTC
Posted Apr 22, 2024 - 13:52 UTC
Update
Customer Impact: Customers may not be able to access FONE Portal, or managed applications via forward or reverse proxy services.
Description: Our teams continue to work on a remediation path to help stabilise our infrastructure. In addition to introducing a new infrastructure component, we have performed a roll-back of some recent configuration changes to isolate the problem further.
We continue to work towards a resolution at the earliest.
Next update time: 02:00 PM UTC
Posted Apr 22, 2024 - 13:26 UTC
Update
Customer Impact: Customers may not be able to access FONE Portal, or managed applications via forward or reverse proxy services.
Description: We continue work on recovery of our services. We will provide further updates as we make progress.
Next update time: 01:00 PM UTC
Posted Apr 22, 2024 - 12:44 UTC
Update
Customer Impact: Customers may not be able to access FONE Portal, or managed applications via forward or reverse proxy services.
Description: We attempted to switch production traffic to a new set of infrastructure components which were brought online to isolate the problem. However, we continue to observe a re-occurrence of the problem and investigate the cause further. We will provide more updates as we make progress.
Next update time: 12:30 PM UTC
Posted Apr 22, 2024 - 11:54 UTC
Update
Customer Impact: Customers may not be able to access FONE Portal, or managed applications via forward or reverse proxy services.
Description: We continue work on several mitigation steps. We will provide further updates as we make progress
Next update time: 11:00 AM UTC
Posted Apr 22, 2024 - 10:34 UTC
Update
Customer Impact: Customers may not be able to access FONE Portal, or managed applications via forward or reverse proxy services.
Description: We continue work on several mitigation steps. We will provide further updates as we make progress
Next update time: 10:30 AM UTC
Posted Apr 22, 2024 - 10:04 UTC
Update
Customer Impact: Customers may not be able to access FONE Portal, or managed applications via forward or reverse proxy services.
Description: We continue to work on recovery of our services with our cloud service provider. We will provide further updates as we make progress.
Next update time: 10:00 AM UTC
Posted Apr 22, 2024 - 09:33 UTC
Update
Customer Impact: Customers may not be able to access FONE Portal, or managed applications via forward or reverse proxy services.
Description: We continue to work with our cloud services provider to remediate the problem, in parallel to introducing certain optimizations across our environment to help achieve stability.
Next update time: 09:30 AM UTC
Posted Apr 22, 2024 - 08:55 UTC
Update
Customer Impact: Customers may not be able to access FONE Portal, or managed applications via forward or reverse proxy services.
Description: We continue to work with our cloud services provider on a recovery path. We are also working on a parallel remediation track to assist with the recovery efforts.
Next update time: 09:00 AM UTC
Posted Apr 22, 2024 - 08:30 UTC
Update
Customer Impact: Customers may not be able to access FONE Portal, or managed applications via forward or reverse proxy services.
Description: We continue work on recovery of our services. We will provide further updates as we make progress.
Next update time: 08:30 AM UTC
Posted Apr 22, 2024 - 08:07 UTC
Update
Customer Impact: Customers may not be able to access FONE Portal, or managed applications via forward or reverse proxy services.
Description: We continue work on recovery of our services. We will provide further updates as we make progress.
Next update time: 07:49 AM UTC
Posted Apr 22, 2024 - 07:30 UTC
Update
Customer Impact: Customers may not be able to access FONE Portal, or managed applications via forward or reverse proxy services.
Description: While we continue to work with our Vendor for recovering the impacted component, we have initiated internal efforts at re-configuring the impacted component. We will provide further updates as we make progress with our recovery efforts.
Next update time: 07:15 AM UTC
Posted Apr 22, 2024 - 07:20 UTC
Identified
Customer Impact: Customers may not be able to access FONE Portal, or managed applications via forward or reverse proxy services.
Description: We have identified the problem to be related to an underlying infrastructure component being in a degraded state. Our attempts to recover the component has been unsuccessful, and we have engaged our Cloud services vendor to help us with the recovery efforts. We continue to actively work on the recovery efforts.
Next update time: 07:15 AM UTC
Posted Apr 22, 2024 - 06:51 UTC
Update
Customer Impact: Customers may not be able to access FONE Portal, or managed applications via forward or reverse proxy services.
Description: Our investigation continues and we have partnered with our Cloud services provider to accelerate the recovery efforts.
Next update time: 08:00 AM UTC
Posted Apr 22, 2024 - 06:26 UTC
Investigating
Customer Impact: Customers may not be able to access FONE Portal, or managed applications via forward or reverse proxy services.
Description: Our teams are investigating the problem and we will provide further update as we make progress.
Next update time: 06:00 AM UTC
Posted Apr 22, 2024 - 05:39 UTC
This incident affected: Cloud Services (Commercial Cloud, EU Cloud, Proxy & SmartEdge, CASB).