Deduplication-while-Storage: A New Encrypted Deduplication Storage Paradigm

Ensuring data confidentiality while achieving efficient deduplication is critical for cloud storage services. Existing works on encrypted data deduplication operate under a deduplication-before-storage (DbS) model, identifying duplicates only during the data uploading phase. They require the owners of duplicate data or additional key servers to assist in the data uploading phase for enabling encrypted deduplication. This makes it hard to balance data security and deduplication effectiveness. This paper introduces a new paradigm named deduplication-while-storage (DwS), which differs from the existing DbS model by enabling dynamic deduplication throughout the entire storage lifecycle, activated whenever owners of potential duplicate data are available. This paradigm aims to significantly boost deduplication ratio without compromising the security of outsourced data. To instantiate the DwS paradigm, we develop SGX-DwS, a shielded storage system that utilizes Intel SGX to support encrypted data deduplication. SGX-DwS builds on an ownership-based encryption mechanism that enables secure equality checks on ciphertexts with the cooperation of owners of duplicate data. It adopts a two-phase deduplication strategy. The first phase processes duplicates during uploading if a previous uploader of duplicate data is online and willing to assist (with incentives provided for participation), while the second phase addresses remaining duplicates during the storage phase when owners of the potential duplicates are simultaneously online and cooperative. Our experimental results demonstrate that SGX-DwS achieves high deduplication ratio with modest overhead, even in large-scale cloud storage environments. Specifically, the improvement in deduplication ratio achieved by SGX-DwS is up to 92.49% as compared to that in the DbS model.