Hewlett Packard Enterprise (HPE) released firmware updates for a number of its Serial-Attached SCSI solid-state drives to prevent their failure at exactly 32,768 hours of operation time.

The devices are used in multiple server and storage products for enterprise, such as HPE ProLiant, Synergy, Apollo, JBOD D3xxx, D6xxx, D8xxx, MSA, StoreVirtual 4335 and StoreVirtual 3200.

The abnormal expiration time translates to 3 years, 270 days and 8 hours, a lot less than the normal lifespan of these products. For some of them, the warranty can be extended to up to five years.

Dire notification

The warning came through a customer bulletin from the HPE support center and notes that the SSD failure caused by the bug makes both the drive and the data on it unrecoverable.

Restoring the data would be possible "from backup in non-fault tolerance, such as RAID 0 and in fault tolerance RAID mode if more drives fail than what is supported by the fault tolerance RAID mode logical drive."

In total, HPE lists 20 drives affected by the issue. For the time being, a fix is available for only eight of them since November 22. For the rest of them, the company plans to release a firmware update in the second week of December.

HPE Model Number HPE SKU HPE SKU DESCRIPTION HPE Spare Part SKU HPE Firmware Fix Date VO0480JFDGT 816562-B21 HP 480GB 12Gb SAS 2.5 RI PLP SC SSD 817047-001 11/22/2019 VO0960JFDGU 816568-B21 HP 960GB 12Gb SAS 2.5 RI PLP SC SSD 817049-001 11/22/2019 VO1920JFDGV 816572-B21 HP 1.92TB 12Gb SAS 2.5 RI PLP SC SSD 817051-001 11/22/2019 VO3840JFDHA 816576-B21 HP 3.84TB 12Gb SAS 2.5 RI PLP SC SSD 817053-001 11/22/2019 MO0400JFFCF 822555-B21 HP 400GB 12Gb SAS 2.5 MU PLP SC SSD S2 822784-001 11/22/2019 MO0800JFFCH 822559-B21 HP 800GB 12Gb SAS 2.5 MU PLP SC SSD S2 822786-001 11/22/2019 MO1600JFFCK 822563-B21 HP 1.6TB 12Gb SAS 2.5 MU PLP SC SSD S2 822788-001 11/22/2019 MO3200JFFCL 822567-B21 HP 3.2TB 12Gb SAS 2.5 MU PLP SC SSD S2 822790-001 11/22/2019 VO000480JWDAR 875311-B21 HPE 480GB SAS SFF RI SC DS SSD 875681-001 Week of 12/9/2019 VO000960JWDAT 875313-B21 HPE 960GB SAS SFF RI SC DS SSD 875682-001 Week of 12/9/2019 VO001920JWDAU 875326-B21 HPE1.92TB SAS RI SFF SC DS SSD 875684-001 Week of 12/9/2019 VO003840JWDAV 875330-B21 HPE 3.84TB SAS RI SFF SC DS SSD 875686-001 Week of 12/9/2019 VO007680JWCNK 870144-B21 HPE 7.68TB SAS 12G RI SFF SC DS SSD 870460-001 Week of 12/9/2019 VO015300JWCNL 870148-B21 HPE 15.3TB SAS 12G RI SFF SC DS SSD 870462-001 Week of 12/9/2019 VK000960JWSSQ P06584-B21 HPE 960GB SAS RI SFF SC DS SSD P08608-001 Week of 12/9/2019 VK001920JWSSR P06586-B21 HPE 1.92TB SAS RI SFF SC DS SSD P08609-001 Week of 12/9/2019 VK003840JWSST P06588-B21 HPE 3.84TB SAS RI SFF SC DS SSD P08610-001 Week of 12/9/2019 VK003840JWSST P11329-B21 HPE 3.84TB SAS RI LFF SCC DS SPL SSD P11360-001 Week of 12/9/2019 VK007680JWSSU P06590-B21 HPE 7.68TB SAS RI SFF SC DS SSD P08611-001 Week of 12/9/2019 VO015300JWSSV P06592-B21 HPE 15.3TB SAS RI SFF SC DS SSD P08612-001 Week of 12/9/2019

According to the advisory, the failure is unavoidable unless the new firmware is installed, and SSDs installed at the same time are likely to fail "nearly simultaneously."

Some users experienced this scenario, with multiple vulnerable HPE drives in a server collapsing at about the same time. One user on Reddit says that six of their SSDs died in 15 minutes. Another one writes: "Just had eight VO0480JFDGT drives fail 2 weeks ago in one server. Each failed within minutes of each other. It took over 2 weeks of HPE dropping the ball until they figured it out." HPE did not find on its own about the issue in the firmware but learned about it from a SSD manufacturer, which explains why some products already reached the premature end. Users can check the uptime of their drives with the Smart Storage Administrator (SSA) software, which provides power-on hours for each installed drive. The saving firmware is HPD8. It is available for the VO0480JFDGT, VO0960JFDGU, VO1920JFDGV, VO3840JFDHA drive models (download for VMware ESXi, Windows, Linux). It is also available for MO0400JFFCF, MO0800JFFCH, MO1600JFFCK and MO3200JFFCL (download for VMware, Windows, Linux).

Correction [11/27/2019] The Hewlett Packard Enterprise Company (HPE) was founded on November 1, 2015, from the splitting of the Hewlett-Packard (HP) company. It is business-oriented and has two divisions. One provides server, storage, and networking solutions, as well as consulting and support, while the other is Financial services and provides "investment and lifecyle strategies to accelerate digital transformation." The company trades under its own ticker symbol, HPE.