Autonomous workload-driven reorganization of column groupings in MMDBS

Abstract

A current trend to achieve high query performance even for huge data warehouse and business intelligence systems is to exploit main-memory-based processing techniques such as compression, cache-conscious strategies, and optimized data structures. However, update processing and skews in data distribution might lead to degenerations in such densely packed and highly compressed data structures affecting the memory efficiency and query performance negatively. Thus, reorganization tasks for repairing these data structures are necessary but should be carefully applied in order to not impact query execution or even system availability significantly. In this paper, we consider the special problem of tuple layout in banked storage structures. Based on runtime statistics capturing typical access patterns in the current workload, we present a bank reassignment approach that can be piggybacked to maintenance tasks without any administration overhead. We have implemented this approach in IBM Smart Analytics Optimizer (ISAOPT). The results of our experimental evaluation show that a simple automatic restructuring of the considered hybrid row-column-store structures offers opportunities to improve query runtimes when a slight memory overhead is acceptable.