Communication optimization for RDMA-based science data transmission tools

Weizhe Zhang; Meng Hao; Zhiyong Xu

doi:10.1007/s11227-015-1399-7

Communication optimization for RDMA-based science data transmission tools

Weizhe Zhang^*
, Meng Hao
, Zhiyong Xu

^*Corresponding author for this work

School of Computer Science and Technology, Harbin Institute of Technology

Research output: Contribution to journal › Article › peer-review

2 Scopus citations

Abstract

Big data has raised new challenges to data communication and transmission capacity. RDMA (Remote Direct Memory Access) Transport Protocol can reduce the communication delay of big data through kernel memory bypass technology and zero-copy technology. This paper introduced RDMA technology for the network performance optimization of BBCP (Babar Copy Program), a big data network file copy tool. Also, transmission queue-oriented Tetris Scheduler algorithm, I/O performance optimization and buffer memory optimization technique were introduced to realize RDMA-based file transfer tool, OBCP (Optimized Babar Copy Program). The experiment results verified that the optimization method could decrease the file transmission time to one-tenth of original BBCP and improve the performance by 50 % compared with related works.

Original language	English
Pages (from-to)	3312-3327
Number of pages	16
Journal	Journal of Supercomputing
Volume	72
Issue number	9
DOIs	https://doi.org/10.1007/s11227-015-1399-7
State	Published - 1 Sep 2016
Externally published	Yes

Keywords

BBCP
Big data
Data transmission
RDMA
Tetris scheduler

Access to Document

10.1007/s11227-015-1399-7

Cite this

@article{246d816e5fa24c9bacd48fea0223e33e,

title = "Communication optimization for RDMA-based science data transmission tools",

abstract = "Big data has raised new challenges to data communication and transmission capacity. RDMA (Remote Direct Memory Access) Transport Protocol can reduce the communication delay of big data through kernel memory bypass technology and zero-copy technology. This paper introduced RDMA technology for the network performance optimization of BBCP (Babar Copy Program), a big data network file copy tool. Also, transmission queue-oriented Tetris Scheduler algorithm, I/O performance optimization and buffer memory optimization technique were introduced to realize RDMA-based file transfer tool, OBCP (Optimized Babar Copy Program). The experiment results verified that the optimization method could decrease the file transmission time to one-tenth of original BBCP and improve the performance by 50 \% compared with related works.",

keywords = "BBCP, Big data, Data transmission, RDMA, Tetris scheduler",

author = "Weizhe Zhang and Meng Hao and Zhiyong Xu",

note = "Publisher Copyright: {\textcopyright} 2015, Springer Science+Business Media New York.",

year = "2016",

month = sep,

day = "1",

doi = "10.1007/s11227-015-1399-7",

language = "英语",

volume = "72",

pages = "3312--3327",

journal = "Journal of Supercomputing",

issn = "0920-8542",

publisher = "Springer Netherlands",

number = "9",

}

TY - JOUR

T1 - Communication optimization for RDMA-based science data transmission tools

AU - Zhang, Weizhe

AU - Hao, Meng

AU - Xu, Zhiyong

PY - 2016/9/1

Y1 - 2016/9/1

N2 - Big data has raised new challenges to data communication and transmission capacity. RDMA (Remote Direct Memory Access) Transport Protocol can reduce the communication delay of big data through kernel memory bypass technology and zero-copy technology. This paper introduced RDMA technology for the network performance optimization of BBCP (Babar Copy Program), a big data network file copy tool. Also, transmission queue-oriented Tetris Scheduler algorithm, I/O performance optimization and buffer memory optimization technique were introduced to realize RDMA-based file transfer tool, OBCP (Optimized Babar Copy Program). The experiment results verified that the optimization method could decrease the file transmission time to one-tenth of original BBCP and improve the performance by 50 % compared with related works.

AB - Big data has raised new challenges to data communication and transmission capacity. RDMA (Remote Direct Memory Access) Transport Protocol can reduce the communication delay of big data through kernel memory bypass technology and zero-copy technology. This paper introduced RDMA technology for the network performance optimization of BBCP (Babar Copy Program), a big data network file copy tool. Also, transmission queue-oriented Tetris Scheduler algorithm, I/O performance optimization and buffer memory optimization technique were introduced to realize RDMA-based file transfer tool, OBCP (Optimized Babar Copy Program). The experiment results verified that the optimization method could decrease the file transmission time to one-tenth of original BBCP and improve the performance by 50 % compared with related works.

KW - BBCP

KW - Big data

KW - Data transmission

KW - RDMA

KW - Tetris scheduler

UR - https://www.scopus.com/pages/publications/84924252086

U2 - 10.1007/s11227-015-1399-7

DO - 10.1007/s11227-015-1399-7

M3 - 文章

AN - SCOPUS:84924252086

SN - 0920-8542

VL - 72

SP - 3312

EP - 3327

JO - Journal of Supercomputing

JF - Journal of Supercomputing

IS - 9

ER -

Communication optimization for RDMA-based science data transmission tools

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this