AIX > Storage > Disk

Welcome to Storage 2.0

XIV storage and Diligent de-duplication technologies can transform your information infrastructure

XIV storage and Diligent de-duplication technologies can transform your information infrastructure
Illustration by Ralph Voltz

Information is one of the top assets of any organization. IT’s mission is to store, protect and secure this information while still providing it 24-7-365 economically and sustainably. Traditionally, this has meant an information infrastructure must be available, reliable, energy efficient, secure and cost effective.

ITIL* and ITSM are in widespread use and help meet these goals. However, they’re not enough in the modern world. We’re witnessing a change in business requirements that relegate these five traditional information infrastructure tenants to a foundation status and adds three new imperatives: flexibility, agility and responsiveness.

IBM maintains advanced virtualization technology should be at the core of this transformation. With this as a backdrop, let’s explore two emerging IBM technologies designed to help move enterprises toward storage 2.0, Diligent* de-duplication and XIV* storage arrays.

Diligent De-duplication

Online storage capacity has grown enormously. Backup systems are strained to the breaking point. Traditional data compression typically yields a reduction in backup data of one and a half to two times. IBM thinks data de-duplication is the solution. Why? Imagine reducing the amount of stored backup information by a factor of 10 to 25.

Data de-duplication solutions aim to filter duplicated data from the backup environment, allowing customers to store more data on a given disk-storage system, which can improve the economics of disk as a backup media. Previous solutions relied upon hashing algorithms, such as MD-5 and/or SHA-1. This approach has been demonstrated to have a statistical chance of error that could result in data loss or corruption.

At its cornerstone, the Diligent Protectier* solution (see Figure 1) was designed to have a revolutionary data de-duplication technology that will enable superior data factoring without compromising performance, scalability or data integrity. The six primary goals of this technology are:

  1. Redundancy elimination—The first fundamental requirement of the technology is to filter out any duplicated incoming data so the system stores each unique data element only once. The target established for the technology is a system-wide capacity increase of 25 times or more for a typical use profile.
  2. Simple, nondisruptive deployment—A true enterprise solution must enable a smooth deployment, such that existing operations may continue uninterrupted as the new technology is introduced.
  3. Hardware agnostic—Solutions built on specialized, proprietary hardware are suboptimal because they eliminate a customer’s ability to protect its software investment and benefit from the hardware commodity curve. A fundamental requirement for Diligent’s solution is that it doesn’t depend on any specialized hardware and can leverage open hardware standards.
  4. Enterprise-class performance—The solution must perform the factoring function at high data-throughput speeds, so the overall solution meets the performance requirements of the high-end data center.
  5. Unmatched scalability—The solution must be “enterprise scale,” allowing storage and management of many PB of storage.
  6. One hundred-percent data integrity—The solution must have no risk of corrupting data based on false data matches at the hash reference level.

Hyperfactor Overview

Hyperfactor* was designed from the top down to allow unmatched data-factoring, scalability, performance and data integrity. At its core, Hyperfactor detects recurring data within sets of data. The common data is merged into a single instance store, enabling dramatic disk-space savings without sacrificing performance or the availability of data for recovery. The net effect of Hyperfactor is only new data elements are stored on disk (virtual tape library). Over time, the impact is a minimum 25-times increase in physical storage capacity.

Hyperfactor is based on a series of algorithms that identify and filter out the elements of a data stream previously stored by ProtecTIER. Hyperfactor finds the data matches without any I/O to the disk, utilizing an efficient RAM-based index, which enables a high data throughput rate. Hyperfactor is a breakthrough on several fronts:

  • It enables scalability up to 1 PB of physical storage (25 PB of represented capacity).
  • Its efficient algorithm enables enterprise class, high throughput backup and recovery.
  • There’s no statistical risk of data corruption.
  • Fine-grain analysis enables high overall factoring ratios.
The XIV Storage Platform provides its customers with an innovative approach to internal data replication that’s an organically conceived extension of the system’s basic architecture.

Kem Clawson is an XIV storage specialist with IBM’s Federal group. He has 36 years of IT experience that began on an IBM 360-40 running OS/MFT. Kem can be reached at

comments powered by Disqus



2019 Solutions Edition

A Comprehensive Online Buyer's Guide to Solutions, Services and Education.

AIX Makes Using RAM Easy

Using the mkramdisk command can create high-speed I/O

IBM Systems Magazine Subscribe Box Read Now Link Subscribe Now Link iPad App Google Play Store
IBMi News Sign Up Today! Past News Letters