Persistent Memory Enhancements : SAP Native Storage Extension and Fast Restart: What’s Possible Today?
This is the first installment in a three-part series on persistent memory enhancements for SAP HANA.
By Wolfgang Reichert07/23/2019
This blog is part of the strategy that discusses the memory enhancements on IBM Power Systems for SAP HANA.
Clients’ data is continuously growing and they are looking for technologies that reduce time to insights at lower total cost of ownership (TCO) and higher availability. SAP and IBM have released new features and technologies to address these client needs.
1. TCO Reduction
A significant part of the server cost is related to the memory (DRAM) because more data the customer will have in memory, higher the infrastructure cost. Depending on the license model, the in-memory footprint of HANA may also affect the SAP software license cost. The existing data tiering options available are only applicable to data that are retrieved rarely. Furthermore, these tiering options require a second LPAR and connectivity to the main HANA instance leading to higher cost and complex operations.
The Native Storage Extension (NSE), a new SAP HANA feature, reduces the in-memory size of the database by utilizing a buffer cache and keeping less frequently used data on disk. This is a built-in functionality in HANA that just needs to be switched on.
2. Reduce Planned DowntimePlanned maintenance—e.g., for SAP HANA or Linux software updates—requires restart of the database. For multi-terabyte databases, the restart and data load time contribute significantly to the overall downtime. The larger the database, the longer it takes to restart.
The new SAP HANA Fast Restart Option dramatically reduces the restart time of the database.
In addition, IBM is currently implementing PowerVM with virtual persistent memory (PMEM ) to further extend the scope of a fast restart.
3. Scale Up to Larger Database SizesUntil now, growing the database required increasing HANA system memory. If the maximum supported HANA memory size per node is exceeded, a scale-out configuration is required.
The Native Storage Extension allows clients to leave less frequently used data on disk, e.g., aged data. The data on disk does not count for the in-memory size, and this allows growing the database significantly over past limits without increasing system memory.
NSE combined with low-latency storage attachment enables scaling up large database size at lower TCO while preserving the overall HANA performance.
SAP HANA Fast Restart Option
Whenever HANA is restarted all the data cleared from the memory need to be reloaded from storage. The larger the database the longer it takes.
Isn’t this inefficient? Why not preserve the data in memory?
Linux offers a temporary filesystem tmpfs that resides in memory. With this new feature, HANA moves large parts of the memory content to tmpfs. At startup, HANA checks if retained memory content is consistent and can be used instead of loading from storage.
There’s no need for additional memory or special hardware. And there’s absolutely no impact on performance or latency at runtime because tmpfs is in DRAM. (The file system just serves an administrative overlay structure to identify memory pages and maps them as tables, columns, etc. at HANA startup. It is similar to the layout in storage.)
To measure the benefits, we loaded 8 TB of data which represents a 16TB HANA system considering that the working area is as big as the data area.
In the standard setup, HANA shutdown takes about 6 minutes. HANA startup including data load takes more than 40 minutes. (Note: While HANA is up and responsive sooner, we wait till the log entry indicating that all data have been loaded.)
With the Fast Restart Option, the startup time is drastically reduced. HANA is fully available within less than 4 minutes. Shutdown is also faster as can be seen in below.
SAP HANA Native Storage ExtensionIn prior HANA releases data growth and data tiering have been addressed by scale-out configurations and/or data archiving.
With HANA 2 SPS04 a new feature has been introduced that allows to keep less frequently used data ("warm data") on storage instead of loading all data into memory. This feature enables a substantial increase in SAP HANA data capacity without impacting performance for high-data volumes. Clients will be able to expand SAP HANA database capacity with warm data on disk up to about four times the size of hot data in memory.
The Native Storage Extension (NSE) is part of the HANA data tiering concept, that is the assignment of data to different storage and processing tiers based on various characteristics of the data (hot, warm and cold data for these tiering systems).
NSE is deeply integrated with the HANA database. It implements a buffer cache for HANA column store tables. Warm data mainly reside on disk and loaded into memory only as required for query processing.
Currently, SAP has established the following sizing rules. As indicated, the rules will be relaxed based on client experience with this new feature.
- HANA system must be scale up (first release restriction)
- May add as much warm storage as desired—up to 1:4 ratio of HANA hot data in memory to warm data on disk
- NSE disk store should be no larger than 10TB (first release restriction)
- Divide volume of warm data by 8—this is size of memory buffer cache required to manage warm data on disk
- Work area should be same size as hot data in memory
- The maximum of supported HANA database size that can be managed by a scale-up system is roughly doubled—without changing the server hardware or software licensing. In the future, even more data on disk may be supported.
- Lab measurements with HANA have shown the following savings: Given a mixed workload and data, 50% of column tables can be marked as warm data without noticeable performance degradation. Thereby the global allocation limit (GAL) can be reduced by 38%—resulting in smaller HANA system size.
Accelerated Storage AttachmentA low-latency storage attachment allows faster paging of warm data as well as quick startup of HANA. For POWER9 systems, the new PCIe3 x8 NVMe Flash Adapter (add-in card) can be used. NVMe is a high-performance software protocol that can read/write flash memory. Compared to a SAS or SATA SSD technology, the NVMe Flash adapter provides more read/write input/output operations per second and larger throughput at lower latency.
The PCIe NVMe cards are available with capacities of 3.2 TB and 6.4 TB. The POWER9 servers are equipped with several PCIe slots; the exact number depends on the model.
The PCIe NVMe storage can be configured as file system mirror for the persistency on external storage devices, as shown in the figure. Hence backup, high-availability configurations and lifecycle management remain unchanged.
The NVMe flash storage can also be used as local disks, assuming that appropriate data replication and/or backup procedures are in place.
SAP HANA Administration Guide for SAP HANA Platform, Version 2.0 SPS 04
The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.