MIPI 2022 Challenge on RGBW Sensor Fusion: Dataset and Report

Qingyu Yang; Guang Yang; Jun Jiang; Chongyi Li; Ruicheng Feng; Shangchen Zhou; Wenxiu Sun; Qingpeng Zhu; Chen Change Loy; Jinwei Gu; Zhen Wang; Daoyu Li; Yuzhe Zhang; Lintao Peng; Xuyang Chang; Yinuo Zhang; Liheng Bian; Bing Li; Jie Huang; Mingde Yao; Ruikang Xu; Feng Zhao; Xiaohui Liu; Rongjian Xu; Zhilu Zhang; Xiaohe Wu; Ruohao Wang; Junyi Li; Wangmeng Zuo; Zhuang Jia; Dong Jae Lee; Ting Jiang; Qi Wu; Chengzhi Jiang; Mingyan Han; Xinpeng Li; Wenjie Lin; Youwei Li; Haoqiang Fan; Shuaicheng Liu

doi:10.1007/978-3-031-25072-9_4

MIPI 2022 Challenge on RGBW Sensor Fusion: Dataset and Report

Qingyu Yang^*
, Guang Yang
, Jun Jiang
, Chongyi Li
, Ruicheng Feng
, Shangchen Zhou
, Wenxiu Sun
, Qingpeng Zhu
, Chen Change Loy
, Jinwei Gu
, Zhen Wang
, Daoyu Li
, Yuzhe Zhang
, Lintao Peng
, Xuyang Chang
, Yinuo Zhang
, Liheng Bian
, Bing Li
, Jie Huang
, Mingde Yao

Ruikang Xu, Feng Zhao, Xiaohui Liu, Rongjian Xu, Zhilu Zhang, Xiaohe Wu, Ruohao Wang, Junyi Li, Wangmeng Zuo, Zhuang Jia, Dong Jae Lee, Ting Jiang, Qi Wu, Chengzhi Jiang, Mingyan Han, Xinpeng Li, Wenjie Lin, Youwei Li, Haoqiang Fan, Shuaicheng Liu

^*Corresponding author for this work

SenseBrain Technology
SenseTime
Shanghai Artificial Intelligence Laboratory
Nanyang Technological University
Beijing Institute of Technology
University of Science and Technology of China
Harbin Institute of Technology
Xiaomi
Korea Advanced Institute of Science and Technology
Megvii Technology Limited

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

3 Scopus citations

Abstract

Developing and integrating advanced image sensors with novel algorithms in camera systems is prevalent with the increasing demand for computational photography and imaging on mobile platforms. However, the lack of high-quality data for research and the rare opportunity for in-depth exchange of views from industry and academia constrain the development of mobile intelligent photography and imaging (MIPI). To bridge the gap, we introduce the first MIPI challenge including five tracks focusing on novel image sensors and imaging algorithms. In this paper, RGBW Joint Fusion and Denoise, one of the five tracks, working on the fusion of binning-mode RGBW to Bayer at half resolution is introduced. The participants were provided with a new dataset including 70 (training) and 15 (validation) scenes of high-quality RGBW and Bayer pair. In addition, for each scene, RGBW of 24 dB and 42 dB are provided. All the data were captured using a RGBW sensor in both outdoor and indoor conditions. The final results are evaluated using objective metrics including PSNR, SSIM [11], LPIPS [15] and KLD. A detailed description of all models developed in this challenge is provided in this paper. More details of this challenge and the link to the dataset can be found in https://github.com/mipi-challenge/MIPI2022

Original language	English
Title of host publication	Computer Vision – ECCV 2022 Workshops, Proceedings
Editors	Leonid Karlinsky, Tomer Michaeli, Ko Nishino
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	46-59
Number of pages	14
ISBN (Print)	9783031250712
DOIs	https://doi.org/10.1007/978-3-031-25072-9_4
State	Published - 2023
Event	Workshops held at the 17th European Conference on Computer Vision, ECCV 2022 - Tel Aviv, Israel Duration: 23 Oct 2022 → 27 Oct 2022

Publication series

Name	Lecture Notes in Computer Science
Volume	13805 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	Workshops held at the 17th European Conference on Computer Vision, ECCV 2022
Country/Territory	Israel
City	Tel Aviv
Period	23/10/22 → 27/10/22

Keywords

Bayer
Denoise
Fusion
MIPI challenge
RGBW

Access to Document

10.1007/978-3-031-25072-9_4

Cite this

Yang, Q., Yang, G., Jiang, J., Li, C., Feng, R., Zhou, S., Sun, W., Zhu, Q., Loy, C. C., Gu, J., Wang, Z., Li, D., Zhang, Y., Peng, L., Chang, X., Zhang, Y., Bian, L., Li, B., Huang, J., ... Liu, S. (2023). MIPI 2022 Challenge on RGBW Sensor Fusion: Dataset and Report. In L. Karlinsky, T. Michaeli, & K. Nishino (Eds.), Computer Vision – ECCV 2022 Workshops, Proceedings (pp. 46-59). (Lecture Notes in Computer Science; Vol. 13805 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-25072-9_4

@inproceedings{db10386d3d434c699f21ef6827bc12e0,

title = "MIPI 2022 Challenge on RGBW Sensor Fusion: Dataset and Report",

abstract = "Developing and integrating advanced image sensors with novel algorithms in camera systems is prevalent with the increasing demand for computational photography and imaging on mobile platforms. However, the lack of high-quality data for research and the rare opportunity for in-depth exchange of views from industry and academia constrain the development of mobile intelligent photography and imaging (MIPI). To bridge the gap, we introduce the first MIPI challenge including five tracks focusing on novel image sensors and imaging algorithms. In this paper, RGBW Joint Fusion and Denoise, one of the five tracks, working on the fusion of binning-mode RGBW to Bayer at half resolution is introduced. The participants were provided with a new dataset including 70 (training) and 15 (validation) scenes of high-quality RGBW and Bayer pair. In addition, for each scene, RGBW of 24 dB and 42 dB are provided. All the data were captured using a RGBW sensor in both outdoor and indoor conditions. The final results are evaluated using objective metrics including PSNR, SSIM [11], LPIPS [15] and KLD. A detailed description of all models developed in this challenge is provided in this paper. More details of this challenge and the link to the dataset can be found in https://github.com/mipi-challenge/MIPI2022",

keywords = "Bayer, Denoise, Fusion, MIPI challenge, RGBW",

author = "Qingyu Yang and Guang Yang and Jun Jiang and Chongyi Li and Ruicheng Feng and Shangchen Zhou and Wenxiu Sun and Qingpeng Zhu and Loy, \{Chen Change\} and Jinwei Gu and Zhen Wang and Daoyu Li and Yuzhe Zhang and Lintao Peng and Xuyang Chang and Yinuo Zhang and Liheng Bian and Bing Li and Jie Huang and Mingde Yao and Ruikang Xu and Feng Zhao and Xiaohui Liu and Rongjian Xu and Zhilu Zhang and Xiaohe Wu and Ruohao Wang and Junyi Li and Wangmeng Zuo and Zhuang Jia and Lee, \{Dong Jae\} and Ting Jiang and Qi Wu and Chengzhi Jiang and Mingyan Han and Xinpeng Li and Wenjie Lin and Youwei Li and Haoqiang Fan and Shuaicheng Liu",

note = "Publisher Copyright: {\textcopyright} 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.; Workshops held at the 17th European Conference on Computer Vision, ECCV 2022 ; Conference date: 23-10-2022 Through 27-10-2022",

year = "2023",

doi = "10.1007/978-3-031-25072-9\_4",

language = "英语",

isbn = "9783031250712",

series = "Lecture Notes in Computer Science",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "46--59",

editor = "Leonid Karlinsky and Tomer Michaeli and Ko Nishino",

booktitle = "Computer Vision – ECCV 2022 Workshops, Proceedings",

address = "德国",

}

Yang, Q, Yang, G, Jiang, J, Li, C, Feng, R, Zhou, S, Sun, W, Zhu, Q, Loy, CC, Gu, J, Wang, Z, Li, D, Zhang, Y, Peng, L, Chang, X, Zhang, Y, Bian, L, Li, B, Huang, J, Yao, M, Xu, R, Zhao, F, Liu, X, Xu, R, Zhang, Z, Wu, X, Wang, R, Li, J , Zuo, W, Jia, Z, Lee, DJ, Jiang, T, Wu, Q, Jiang, C, Han, M, Li, X, Lin, W, Li, Y, Fan, H & Liu, S 2023, MIPI 2022 Challenge on RGBW Sensor Fusion: Dataset and Report. in L Karlinsky, T Michaeli & K Nishino (eds), Computer Vision – ECCV 2022 Workshops, Proceedings. Lecture Notes in Computer Science, vol. 13805 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 46-59, Workshops held at the 17th European Conference on Computer Vision, ECCV 2022, Tel Aviv, Israel, 23/10/22. https://doi.org/10.1007/978-3-031-25072-9_4

MIPI 2022 Challenge on RGBW Sensor Fusion: Dataset and Report. / Yang, Qingyu; Yang, Guang; Jiang, Jun et al.
Computer Vision – ECCV 2022 Workshops, Proceedings. ed. / Leonid Karlinsky; Tomer Michaeli; Ko Nishino. Springer Science and Business Media Deutschland GmbH, 2023. p. 46-59 (Lecture Notes in Computer Science; Vol. 13805 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - MIPI 2022 Challenge on RGBW Sensor Fusion

T2 - Workshops held at the 17th European Conference on Computer Vision, ECCV 2022

AU - Yang, Qingyu

AU - Yang, Guang

AU - Jiang, Jun

AU - Li, Chongyi

AU - Feng, Ruicheng

AU - Zhou, Shangchen

AU - Sun, Wenxiu

AU - Zhu, Qingpeng

AU - Loy, Chen Change

AU - Gu, Jinwei

AU - Wang, Zhen

AU - Li, Daoyu

AU - Zhang, Yuzhe

AU - Peng, Lintao

AU - Chang, Xuyang

AU - Zhang, Yinuo

AU - Bian, Liheng

AU - Li, Bing

AU - Huang, Jie

AU - Yao, Mingde

AU - Xu, Ruikang

AU - Zhao, Feng

AU - Liu, Xiaohui

AU - Xu, Rongjian

AU - Zhang, Zhilu

AU - Wu, Xiaohe

AU - Wang, Ruohao

AU - Li, Junyi

AU - Zuo, Wangmeng

AU - Jia, Zhuang

AU - Lee, Dong Jae

AU - Jiang, Ting

AU - Wu, Qi

AU - Jiang, Chengzhi

AU - Han, Mingyan

AU - Li, Xinpeng

AU - Lin, Wenjie

AU - Li, Youwei

AU - Fan, Haoqiang

AU - Liu, Shuaicheng

PY - 2023

Y1 - 2023

N2 - Developing and integrating advanced image sensors with novel algorithms in camera systems is prevalent with the increasing demand for computational photography and imaging on mobile platforms. However, the lack of high-quality data for research and the rare opportunity for in-depth exchange of views from industry and academia constrain the development of mobile intelligent photography and imaging (MIPI). To bridge the gap, we introduce the first MIPI challenge including five tracks focusing on novel image sensors and imaging algorithms. In this paper, RGBW Joint Fusion and Denoise, one of the five tracks, working on the fusion of binning-mode RGBW to Bayer at half resolution is introduced. The participants were provided with a new dataset including 70 (training) and 15 (validation) scenes of high-quality RGBW and Bayer pair. In addition, for each scene, RGBW of 24 dB and 42 dB are provided. All the data were captured using a RGBW sensor in both outdoor and indoor conditions. The final results are evaluated using objective metrics including PSNR, SSIM [11], LPIPS [15] and KLD. A detailed description of all models developed in this challenge is provided in this paper. More details of this challenge and the link to the dataset can be found in https://github.com/mipi-challenge/MIPI2022

AB - Developing and integrating advanced image sensors with novel algorithms in camera systems is prevalent with the increasing demand for computational photography and imaging on mobile platforms. However, the lack of high-quality data for research and the rare opportunity for in-depth exchange of views from industry and academia constrain the development of mobile intelligent photography and imaging (MIPI). To bridge the gap, we introduce the first MIPI challenge including five tracks focusing on novel image sensors and imaging algorithms. In this paper, RGBW Joint Fusion and Denoise, one of the five tracks, working on the fusion of binning-mode RGBW to Bayer at half resolution is introduced. The participants were provided with a new dataset including 70 (training) and 15 (validation) scenes of high-quality RGBW and Bayer pair. In addition, for each scene, RGBW of 24 dB and 42 dB are provided. All the data were captured using a RGBW sensor in both outdoor and indoor conditions. The final results are evaluated using objective metrics including PSNR, SSIM [11], LPIPS [15] and KLD. A detailed description of all models developed in this challenge is provided in this paper. More details of this challenge and the link to the dataset can be found in https://github.com/mipi-challenge/MIPI2022

KW - Bayer

KW - Denoise

KW - Fusion

KW - MIPI challenge

KW - RGBW

UR - https://www.scopus.com/pages/publications/85150953320

U2 - 10.1007/978-3-031-25072-9_4

DO - 10.1007/978-3-031-25072-9_4

M3 - 会议稿件

AN - SCOPUS:85150953320

SN - 9783031250712

T3 - Lecture Notes in Computer Science

SP - 46

EP - 59

BT - Computer Vision – ECCV 2022 Workshops, Proceedings

A2 - Karlinsky, Leonid

A2 - Michaeli, Tomer

A2 - Nishino, Ko

PB - Springer Science and Business Media Deutschland GmbH

Y2 - 23 October 2022 through 27 October 2022

ER -