TY - GEN
T1 - SuperCDC
T2 - 40th IEEE International Conference on Computer Design, ICCD 2022
AU - Wan, Binzhaoshuo
AU - Pu, Lifeng
AU - Zou, Xiangyu
AU - Li, Shiyi
AU - Wang, Peng
AU - Xia, Wen
N1 - Publisher Copyright:
© 2022 IEEE.
PY - 2022
Y1 - 2022
N2 - Content-Defined Chunking (CDC) has been widely applied in data deduplication systems in the past since it can detect much more redundant data than Fixed-Size Chunking (FSC). CDC approach becomes faster and faster to match the improvement of high performance storage systems, and there are two main kinds of acceleration mechanisms: calculation-efficient acceleration and stream-informed acceleration. We observe the opportunity to combine the benefits of these two mechanisms to chase a faster speed, and the challenges of memory overhead and deduplication ratio loss, which are caused by stream histories and the configuration of minimum/maximum chunk size, respectively. Motivated by these observations, we proposed SuperCDC with several corresponding techniques, including hybridizing calculation-efficient processing with a stream-informed design, memory-efficient structure for tracking stream history, and Min-Max chunking for improving deduplication ratio. Evaluations suggest that SuperCDC achieves an up to 4.94× faster chunking speed and 6.23% higher deduplication ratio while saving 99.58% memory space on stream histories compared with the state-of-the-art Gear-based RapidCDC.
AB - Content-Defined Chunking (CDC) has been widely applied in data deduplication systems in the past since it can detect much more redundant data than Fixed-Size Chunking (FSC). CDC approach becomes faster and faster to match the improvement of high performance storage systems, and there are two main kinds of acceleration mechanisms: calculation-efficient acceleration and stream-informed acceleration. We observe the opportunity to combine the benefits of these two mechanisms to chase a faster speed, and the challenges of memory overhead and deduplication ratio loss, which are caused by stream histories and the configuration of minimum/maximum chunk size, respectively. Motivated by these observations, we proposed SuperCDC with several corresponding techniques, including hybridizing calculation-efficient processing with a stream-informed design, memory-efficient structure for tracking stream history, and Min-Max chunking for improving deduplication ratio. Evaluations suggest that SuperCDC achieves an up to 4.94× faster chunking speed and 6.23% higher deduplication ratio while saving 99.58% memory space on stream histories compared with the state-of-the-art Gear-based RapidCDC.
KW - Backup Storage
KW - Content-Defined Chunking
KW - Deduplication
UR - https://www.scopus.com/pages/publications/85145874655
U2 - 10.1109/ICCD56317.2022.00034
DO - 10.1109/ICCD56317.2022.00034
M3 - 会议稿件
AN - SCOPUS:85145874655
T3 - Proceedings - IEEE International Conference on Computer Design: VLSI in Computers and Processors
SP - 170
EP - 178
BT - Proceedings - 2022 IEEE 40th International Conference on Computer Design, ICCD 2022
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 23 October 2022 through 26 October 2022
ER -