Skip to main navigation Skip to search Skip to main content

DEC: An efficient deduplication-enhanced compression approach

  • Zijin Han
  • , Wen Xia
  • , Yuchong Hu*
  • , Dan Feng
  • , Yucheng Zhang
  • , Yukun Zhou
  • , Min Fu
  • , Liang Gu
  • *Corresponding author for this work
  • Huazhong University of Science and Technology
  • Sangfor Technologies Inc.

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Data compression is widely used in storage systems to reduce redundant data and thus save storage space. One challenge facing the traditional compression approaches is the limitation of compression windows size, which fails to reduce redundancy globally. In this paper, we present DEC, a Deduplication-Enhanced Compression approach that effectively combines deduplication and traditional compressors to increase compression ratio and efficiency. Specifically, we make full use of deduplication to (1) accelerate data reduction by fast but global deduplication and (2) exploit data locality to compress similar chunks by clustering the data chunks which are adjacent to the same duplicate chunks. Our experimental results of a DEC prototype based on real-world datasets show that DEC increases the compression ratio by 20% to 71% and speeds up the compression throughput by 17%~183% compared to traditional compressors, without sacrificing the decompression throughput by leveraging deduplication in traditional compression approaches.

Original languageEnglish
Title of host publicationProceedings - 22nd IEEE International Conference on Parallel and Distributed Systems, ICPADS 2016
EditorsXiaofei Liao, Robert Lovas, Xipeng Shen, Ran Zheng
PublisherIEEE Computer Society
Pages519-526
Number of pages8
ISBN (Electronic)9781509044573
DOIs
StatePublished - 2 Jul 2016
Externally publishedYes
Event22nd IEEE International Conference on Parallel and Distributed Systems, ICPADS 2016 - Wuhan, Hubei, China
Duration: 13 Dec 201616 Dec 2016

Publication series

NameProceedings of the International Conference on Parallel and Distributed Systems - ICPADS
Volume0
ISSN (Print)1521-9097

Conference

Conference22nd IEEE International Conference on Parallel and Distributed Systems, ICPADS 2016
Country/TerritoryChina
CityWuhan, Hubei
Period13/12/1616/12/16

Keywords

  • Data Locality
  • Delta Deduplication
  • Storage Systems
  • Traditional Compression

Fingerprint

Dive into the research topics of 'DEC: An efficient deduplication-enhanced compression approach'. Together they form a unique fingerprint.

Cite this