Time-series data aggregation index

HUANG Xiangdong; ZHENG Liangfan; QIU Mingming; ZHANG Jinrui; WANG Jianmin

doi:10.16511/j.cnki.qhdxxb.2016.21.032

PDF(1649 KB)

Journal of Tsinghua University(Science and Technology) ›› 2016, Vol. 56 ›› Issue (3) : 229-236,245. DOI: 10.16511/j.cnki.qhdxxb.2016.21.032

COMPUTER SCIENCE AND TECHNOLOGY

Time-series data aggregation index

{{article.zuoZhe_EN}}

Author information +

History +

Abstract

Time-series data is the key to industrial development, with the aggregation of the data an important step in practice. However, traditional relational databases fail to support vast amounts of time-series data. The NoSQL databases are inefficient and require time-consuming calculation to aggregate of time-series data. This paper presents an efficient index mechanism that supports time-series data aggregation by combining a synopsis table and a segment tree. A query algorithm based on this mechanism introduces the synopsis table into the NoSQL database and builds a segment tree from the synopsis table for archiving that is lbn the size of the original query set. This query algorithm can directly locate a series of index data to be queried without the recursive operations in traditional trees and effectively reduces I/O overhead. This study shows the efficiency of this index mechanism by comparisons with general index mechanisms.

Key words

index / aggregate operation / time-series data / synopsis table / segment tree

Cite this article

EndNote

Ris (Procite)

Bibtex

Download Citations

HUANG Xiangdong, ZHENG Liangfan, QIU Mingming, ZHANG Jinrui, WANG Jianmin. Time-series data aggregation index[J]. Journal of Tsinghua University(Science and Technology). 2016, 56(3): 229-236,245 https://doi.org/10.16511/j.cnki.qhdxxb.2016.21.032

References

[1] Goldstein J, Larson P A. Optimizing queries using materialized views:a practical, scalable solution[J]. Special Interest Group on Management Of Data, 2001, 30(2):331-342.
[2] Lehner W, Cochrane B, Pirahesh H, et al. Applying mass query optimization to speed up automatic summary table refresh[C]//In proceedings of the international conference on data engineering. Heidelberg, Germany:IEEE Computer Society, 2001:1-22.
[3] Chen Y, Chen M, Liu X, et al. MapReduce based aggregate-join query algorithms[J]. Journal of Computer Research & Development, 2013, 50(z1):306-311.
[4] Li Y, Kim G B, Wen L R, et al. MHB-Tree:A distributed spatial index method for document based NoSQL database system[J]. Lecture Notes in Electrical Engineering, 2013, 214:489-497.
[5] Dean J, Ghemawat S. MapReduce:Simplified data processing on large clusters[J]. Operating Systems Design & Implementation, 2004, 51(1):147-152.
[6] Ross K A, Srivastava D, Sudarshan S. Materialized view maintenance and integrity constraint checking:Trading space for time[J]. AcmSigmod Record, 199625(2):447-458.
[7] Li C, Chen J, Jin C, et al. MR-tree:An efficient index for MapReduce[J]. International Journal of Communication Systems, 2014, 27(6):828-838.
[8] Abramova V, Bernardino J. NoSQL databases:MongoDB vs Cassandra[C]//Proceedings of the International C^* Conference on Computer Science and Software Engineering. Porto, Portual:ACM, 2013:14-22.
[9] Ibrahim S, Jin H, Lu L, et al. Adaptive disk I/O scheduling for MapReduce in virtualized environment[C]//IEEE 42nd International Conference on Parallel Processing. Taipei, Taiwan, China:IEEE Computer Society, 2011:335-344.
[10] Zilio D C, Zuzarte C, Lightstone S. Recommending materialized views and indexes with IBM DB2 design advisor[C]//In Proceedings of the International Conference on Autonomic Computing. New York, NY, USA:IEEE Computer Society, 2004:180-187.
[11] Qu Z C, Guo T L. A maintenance strategy of materialized views in distributed environment[J].Applied Mechanics and Materials, 2012, 182-183:2123-2126.
[12] Gal A. Obsolescent materialized views in query processing of enterprise information systems[C]//In Proc Eighth International Conference on Information and Knowledge Management. Kansas City, MO, USA:ACM, 1999:367-374.
[13] Arman N. A materialized view for the same generation query in deductive databases[J].Computer & Information Science, 2012,5(6):1-5.