Efficient Single-Image Depth Estimation on Mobile Devices, Mobile AI & AIM 2022 Challenge: Report

Andrey Ignatov; Grigory Malivenko; Radu Timofte; Lukasz Treszczotko; Xin Chang; Piotr Ksiazek; Michal Lopuszynski; Maciej Pioro; Rafal Rudnicki; Maciej Smyl; Yujie Ma; Zhenyu Li; Zehui Chen; Jialei Xu; Xianming Liu; Junjun Jiang; Xue Chao Shi; Difan Xu; Yanan Li; Xiaotao Wang; Lei Lei; Ziyu Zhang; Yicheng Wang; Zilong Huang; Guozhong Luo; Gang Yu; Bin Fu; Jiaqi Li; Yiran Wang; Zihao Huang; Zhiguo Cao; Marcos V. Conde; Denis Sapozhnikov; Byeong Hyun Lee; Dongwon Park; Seongmin Hong; Joonhee Lee; Seunggyu Lee; Se Young Chun

doi:10.1007/978-3-031-25066-8_4

Efficient Single-Image Depth Estimation on Mobile Devices, Mobile AI & AIM 2022 Challenge: Report

Andrey Ignatov^*
, Grigory Malivenko
, Radu Timofte
, Lukasz Treszczotko
, Xin Chang
, Piotr Ksiazek
, Michal Lopuszynski
, Maciej Pioro
, Rafal Rudnicki
, Maciej Smyl
, Yujie Ma
, Zhenyu Li
, Zehui Chen
, Jialei Xu
, Xianming Liu
, Junjun Jiang
, Xue Chao Shi
, Difan Xu
, Yanan Li
, Xiaotao Wang

Lei Lei, Ziyu Zhang, Yicheng Wang, Zilong Huang, Guozhong Luo, Gang Yu, Bin Fu, Jiaqi Li, Yiran Wang, Zihao Huang, Zhiguo Cao, Marcos V. Conde, Denis Sapozhnikov, Byeong Hyun Lee, Dongwon Park, Seongmin Hong, Joonhee Lee, Seunggyu Lee, Se Young Chun

^*Corresponding author for this work

Faculty of Computing

Swiss Federal Institute of Technology Zurich
AI Witchlabs Ltd.
University of Würzburg
TCL Research Europe
Harbin Institute of Technology
Xiaomi
Tencent
Huazhong University of Science and Technology
Seoul National University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

12 Scopus citations

Abstract

Various depth estimation models are now widely used on many mobile and IoT devices for image segmentation, bokeh effect rendering, object tracking and many other mobile tasks. Thus, it is very crucial to have efficient and accurate depth estimation models that can run fast on low-power mobile chipsets. In this Mobile AI challenge, the target was to develop deep learning-based single image depth estimation solutions that can show a real-time performance on IoT platforms and smartphones. For this, the participants used a large-scale RGB-to-depth dataset that was collected with the ZED stereo camera capable to generated depth maps for objects located at up to 50 m. The runtime of all models was evaluated on the Raspberry Pi 4 platform, where the developed solutions were able to generate VGA resolution depth maps at up to 27 FPS while achieving high fidelity results. All models developed in the challenge are also compatible with any Android or Linux-based mobile devices, their detailed description is provided in this paper.

Original language	English
Title of host publication	Computer Vision – ECCV 2022 Workshops, Proceedings
Editors	Leonid Karlinsky, Tomer Michaeli, Ko Nishino
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	71-91
Number of pages	21
ISBN (Print)	9783031250651
DOIs	https://doi.org/10.1007/978-3-031-25066-8_4
State	Published - 2023
Event	Workshops held at the 17th European Conference on Computer Vision, ECCV 2022 - Tel Aviv, Israel Duration: 23 Oct 2022 → 27 Oct 2022

Publication series

Name	Lecture Notes in Computer Science
Volume	13803 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	Workshops held at the 17th European Conference on Computer Vision, ECCV 2022
Country/Territory	Israel
City	Tel Aviv
Period	23/10/22 → 27/10/22

Keywords

AI Benchmark
Deep learning
Depth estimation
Mobile AI
Mobile ai challenge
Raspberry pi

Access to Document

10.1007/978-3-031-25066-8_4

Cite this

Ignatov, A., Malivenko, G., Timofte, R., Treszczotko, L., Chang, X., Ksiazek, P., Lopuszynski, M., Pioro, M., Rudnicki, R., Smyl, M., Ma, Y., Li, Z., Chen, Z., Xu, J., Liu, X., Jiang, J., Shi, X. C., Xu, D., Li, Y., ... Chun, S. Y. (2023). Efficient Single-Image Depth Estimation on Mobile Devices, Mobile AI & AIM 2022 Challenge: Report. In L. Karlinsky, T. Michaeli, & K. Nishino (Eds.), Computer Vision – ECCV 2022 Workshops, Proceedings (pp. 71-91). (Lecture Notes in Computer Science; Vol. 13803 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-25066-8_4

Ignatov, Andrey ; Malivenko, Grigory ; Timofte, Radu et al. / Efficient Single-Image Depth Estimation on Mobile Devices, Mobile AI & AIM 2022 Challenge : Report. Computer Vision – ECCV 2022 Workshops, Proceedings. editor / Leonid Karlinsky ; Tomer Michaeli ; Ko Nishino. Springer Science and Business Media Deutschland GmbH, 2023. pp. 71-91 (Lecture Notes in Computer Science).

@inproceedings{f03b2c12432a4c2b9bc0d921b533c014,

title = "Efficient Single-Image Depth Estimation on Mobile Devices, Mobile AI \& AIM 2022 Challenge: Report",

abstract = "Various depth estimation models are now widely used on many mobile and IoT devices for image segmentation, bokeh effect rendering, object tracking and many other mobile tasks. Thus, it is very crucial to have efficient and accurate depth estimation models that can run fast on low-power mobile chipsets. In this Mobile AI challenge, the target was to develop deep learning-based single image depth estimation solutions that can show a real-time performance on IoT platforms and smartphones. For this, the participants used a large-scale RGB-to-depth dataset that was collected with the ZED stereo camera capable to generated depth maps for objects located at up to 50 m. The runtime of all models was evaluated on the Raspberry Pi 4 platform, where the developed solutions were able to generate VGA resolution depth maps at up to 27 FPS while achieving high fidelity results. All models developed in the challenge are also compatible with any Android or Linux-based mobile devices, their detailed description is provided in this paper.",

keywords = "AI Benchmark, Deep learning, Depth estimation, Mobile AI, Mobile ai challenge, Raspberry pi",

author = "Andrey Ignatov and Grigory Malivenko and Radu Timofte and Lukasz Treszczotko and Xin Chang and Piotr Ksiazek and Michal Lopuszynski and Maciej Pioro and Rafal Rudnicki and Maciej Smyl and Yujie Ma and Zhenyu Li and Zehui Chen and Jialei Xu and Xianming Liu and Junjun Jiang and Shi, \{Xue Chao\} and Difan Xu and Yanan Li and Xiaotao Wang and Lei Lei and Ziyu Zhang and Yicheng Wang and Zilong Huang and Guozhong Luo and Gang Yu and Bin Fu and Jiaqi Li and Yiran Wang and Zihao Huang and Zhiguo Cao and Conde, \{Marcos V.\} and Denis Sapozhnikov and Lee, \{Byeong Hyun\} and Dongwon Park and Seongmin Hong and Joonhee Lee and Seunggyu Lee and Chun, \{Se Young\}",

note = "Publisher Copyright: {\textcopyright} 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.; Workshops held at the 17th European Conference on Computer Vision, ECCV 2022 ; Conference date: 23-10-2022 Through 27-10-2022",

year = "2023",

doi = "10.1007/978-3-031-25066-8\_4",

language = "英语",

isbn = "9783031250651",

series = "Lecture Notes in Computer Science",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "71--91",

editor = "Leonid Karlinsky and Tomer Michaeli and Ko Nishino",

booktitle = "Computer Vision – ECCV 2022 Workshops, Proceedings",

address = "德国",

}

Ignatov, A, Malivenko, G, Timofte, R, Treszczotko, L, Chang, X, Ksiazek, P, Lopuszynski, M, Pioro, M, Rudnicki, R, Smyl, M, Ma, Y, Li, Z, Chen, Z, Xu, J, Liu, X , Jiang, J, Shi, XC, Xu, D, Li, Y, Wang, X, Lei, L, Zhang, Z, Wang, Y, Huang, Z, Luo, G, Yu, G, Fu, B, Li, J, Wang, Y, Huang, Z, Cao, Z, Conde, MV, Sapozhnikov, D, Lee, BH, Park, D, Hong, S, Lee, J, Lee, S & Chun, SY 2023, Efficient Single-Image Depth Estimation on Mobile Devices, Mobile AI & AIM 2022 Challenge: Report. in L Karlinsky, T Michaeli & K Nishino (eds), Computer Vision – ECCV 2022 Workshops, Proceedings. Lecture Notes in Computer Science, vol. 13803 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 71-91, Workshops held at the 17th European Conference on Computer Vision, ECCV 2022, Tel Aviv, Israel, 23/10/22. https://doi.org/10.1007/978-3-031-25066-8_4

Efficient Single-Image Depth Estimation on Mobile Devices, Mobile AI & AIM 2022 Challenge: Report. / Ignatov, Andrey; Malivenko, Grigory; Timofte, Radu et al.
Computer Vision – ECCV 2022 Workshops, Proceedings. ed. / Leonid Karlinsky; Tomer Michaeli; Ko Nishino. Springer Science and Business Media Deutschland GmbH, 2023. p. 71-91 (Lecture Notes in Computer Science; Vol. 13803 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Efficient Single-Image Depth Estimation on Mobile Devices, Mobile AI & AIM 2022 Challenge

T2 - Workshops held at the 17th European Conference on Computer Vision, ECCV 2022

AU - Ignatov, Andrey

AU - Malivenko, Grigory

AU - Timofte, Radu

AU - Treszczotko, Lukasz

AU - Chang, Xin

AU - Ksiazek, Piotr

AU - Lopuszynski, Michal

AU - Pioro, Maciej

AU - Rudnicki, Rafal

AU - Smyl, Maciej

AU - Ma, Yujie

AU - Li, Zhenyu

AU - Chen, Zehui

AU - Xu, Jialei

AU - Liu, Xianming

AU - Jiang, Junjun

AU - Shi, Xue Chao

AU - Xu, Difan

AU - Li, Yanan

AU - Wang, Xiaotao

AU - Lei, Lei

AU - Zhang, Ziyu

AU - Wang, Yicheng

AU - Huang, Zilong

AU - Luo, Guozhong

AU - Yu, Gang

AU - Fu, Bin

AU - Li, Jiaqi

AU - Wang, Yiran

AU - Huang, Zihao

AU - Cao, Zhiguo

AU - Conde, Marcos V.

AU - Sapozhnikov, Denis

AU - Lee, Byeong Hyun

AU - Park, Dongwon

AU - Hong, Seongmin

AU - Lee, Joonhee

AU - Lee, Seunggyu

AU - Chun, Se Young

PY - 2023

Y1 - 2023

N2 - Various depth estimation models are now widely used on many mobile and IoT devices for image segmentation, bokeh effect rendering, object tracking and many other mobile tasks. Thus, it is very crucial to have efficient and accurate depth estimation models that can run fast on low-power mobile chipsets. In this Mobile AI challenge, the target was to develop deep learning-based single image depth estimation solutions that can show a real-time performance on IoT platforms and smartphones. For this, the participants used a large-scale RGB-to-depth dataset that was collected with the ZED stereo camera capable to generated depth maps for objects located at up to 50 m. The runtime of all models was evaluated on the Raspberry Pi 4 platform, where the developed solutions were able to generate VGA resolution depth maps at up to 27 FPS while achieving high fidelity results. All models developed in the challenge are also compatible with any Android or Linux-based mobile devices, their detailed description is provided in this paper.

AB - Various depth estimation models are now widely used on many mobile and IoT devices for image segmentation, bokeh effect rendering, object tracking and many other mobile tasks. Thus, it is very crucial to have efficient and accurate depth estimation models that can run fast on low-power mobile chipsets. In this Mobile AI challenge, the target was to develop deep learning-based single image depth estimation solutions that can show a real-time performance on IoT platforms and smartphones. For this, the participants used a large-scale RGB-to-depth dataset that was collected with the ZED stereo camera capable to generated depth maps for objects located at up to 50 m. The runtime of all models was evaluated on the Raspberry Pi 4 platform, where the developed solutions were able to generate VGA resolution depth maps at up to 27 FPS while achieving high fidelity results. All models developed in the challenge are also compatible with any Android or Linux-based mobile devices, their detailed description is provided in this paper.

KW - AI Benchmark

KW - Deep learning

KW - Depth estimation

KW - Mobile AI

KW - Mobile ai challenge

KW - Raspberry pi

UR - https://www.scopus.com/pages/publications/85151132690

U2 - 10.1007/978-3-031-25066-8_4

DO - 10.1007/978-3-031-25066-8_4

M3 - 会议稿件

AN - SCOPUS:85151132690

SN - 9783031250651

T3 - Lecture Notes in Computer Science

SP - 71

EP - 91

BT - Computer Vision – ECCV 2022 Workshops, Proceedings

A2 - Karlinsky, Leonid

A2 - Michaeli, Tomer

A2 - Nishino, Ko

PB - Springer Science and Business Media Deutschland GmbH

Y2 - 23 October 2022 through 27 October 2022

ER -

Ignatov A, Malivenko G, Timofte R, Treszczotko L, Chang X, Ksiazek P et al. Efficient Single-Image Depth Estimation on Mobile Devices, Mobile AI & AIM 2022 Challenge: Report. In Karlinsky L, Michaeli T, Nishino K, editors, Computer Vision – ECCV 2022 Workshops, Proceedings. Springer Science and Business Media Deutschland GmbH. 2023. p. 71-91. (Lecture Notes in Computer Science). doi: 10.1007/978-3-031-25066-8_4