Network Platforms - Storage perfomance Issue – Incident details

Storage perfomance Issue

Resolved
Operational
Started 16 days agoLasted about 2 hours

Affected

Virtual Hosting

Operational from 10:03 AM to 11:40 AM

JB1 - VMWare Nodes

Operational from 10:03 AM to 11:40 AM

JB1 - vSAN

Operational from 10:03 AM to 11:40 AM

Updates
  • Resolved
    Resolved

    Good day,

    Our Ceph storage has now stabilized. Early this morning, we added NVMe disks to the cluster, which unfortunately triggered a resynchronization of the Placement Groups (PGs) in our current Ceph cluster.

    As part of our migration roadmap, we are now at the stage where we need to integrate the new hardware—previously used for the VMware environment—into the new Proxmox cluster. This requires building a new storage pool in the Proxmox cluster using NVMe disks and subsequently migrating VMs from the current SSD pool to the new NVMe pool.

    We are currently consulting with Proxmox to explore ways to minimize the impact on client VMs running on Ceph during this transition. Our goal is to move the VMs to faster storage as soon as possible. However, if we determine that this process cannot be completed with minimal to no disruption, we will reschedule the migration for the upcoming long weekend.

    We will provide further communication once the change schedule has been confirmed.

    Regards,
    Network Platforms

  • Investigating
    Investigating

    Good day

    We are investigating a Performance issue on our Ceph storage affecting all VMs on this Storage.

    Our Engineers are migrating affected VMs of the storage in the meantime.

    Regards

    Network Platforms