A Baseline for ViCo Conversational Head Generation Challenge

Meng Liu; Shuyan Zhai; Yongqiang Li; Weili Guan; Liqiang Nie

doi:10.1145/3503161.3551569

A Baseline for ViCo Conversational Head Generation Challenge

Meng Liu
, Shuyan Zhai
, Yongqiang Li
, Weili Guan
, Liqiang Nie

Shandong Jianzhu University
Shandong University
Monash University
Harbin Institute of Technology Shenzhen

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Scopus citations

Abstract

ViCo conversational head generation challenge is organized in conjunction with ACM Multimedia 2022. The challenge includes two tasks: 1) generating a vivid talking head according to a reference image and driving audio, and 2) synthesizing a listening head conditioned on the reference image and speaker's behaviors. For talking head generation, we design a three-stage method to generate high-quality and lip-sync talking head videos with natural head poses, which wins 3rd place and People's Selection Award. On the listening head generation track, we adopt MakeItTalk to generate vivid head movements of listeners, which achieves 2nd place in the competition.

Original language	English
Title of host publication	MM 2022 - Proceedings of the 30th ACM International Conference on Multimedia
Publisher	Association for Computing Machinery, Inc
Pages	7013-7015
Number of pages	3
ISBN (Electronic)	9781450392037
DOIs	https://doi.org/10.1145/3503161.3551569
State	Published - 10 Oct 2022
Externally published	Yes
Event	30th ACM International Conference on Multimedia, MM 2022 - Lisboa, Portugal Duration: 10 Oct 2022 → 14 Oct 2022

Publication series

Name	MM 2022 - Proceedings of the 30th ACM International Conference on Multimedia

Conference

Conference	30th ACM International Conference on Multimedia, MM 2022
Country/Territory	Portugal
City	Lisboa
Period	10/10/22 → 14/10/22

Keywords

listener generation
talking head generation

Access to Document

10.1145/3503161.3551569

Cite this

Liu, M., Zhai, S., Li, Y., Guan, W., & Nie, L. (2022). A Baseline for ViCo Conversational Head Generation Challenge. In MM 2022 - Proceedings of the 30th ACM International Conference on Multimedia (pp. 7013-7015). (MM 2022 - Proceedings of the 30th ACM International Conference on Multimedia). Association for Computing Machinery, Inc. https://doi.org/10.1145/3503161.3551569

@inproceedings{dab999dc72f14ebfb3084f0c7aef2956,

title = "A Baseline for ViCo Conversational Head Generation Challenge",

abstract = "ViCo conversational head generation challenge is organized in conjunction with ACM Multimedia 2022. The challenge includes two tasks: 1) generating a vivid talking head according to a reference image and driving audio, and 2) synthesizing a listening head conditioned on the reference image and speaker's behaviors. For talking head generation, we design a three-stage method to generate high-quality and lip-sync talking head videos with natural head poses, which wins 3rd place and People's Selection Award. On the listening head generation track, we adopt MakeItTalk to generate vivid head movements of listeners, which achieves 2nd place in the competition.",

keywords = "listener generation, talking head generation",

author = "Meng Liu and Shuyan Zhai and Yongqiang Li and Weili Guan and Liqiang Nie",

note = "Publisher Copyright: {\textcopyright} 2022 ACM.; 30th ACM International Conference on Multimedia, MM 2022 ; Conference date: 10-10-2022 Through 14-10-2022",

year = "2022",

month = oct,

day = "10",

doi = "10.1145/3503161.3551569",

language = "英语",

series = "MM 2022 - Proceedings of the 30th ACM International Conference on Multimedia",

publisher = "Association for Computing Machinery, Inc",

pages = "7013--7015",

booktitle = "MM 2022 - Proceedings of the 30th ACM International Conference on Multimedia",

}

Liu, M, Zhai, S, Li, Y, Guan, W & Nie, L 2022, A Baseline for ViCo Conversational Head Generation Challenge. in MM 2022 - Proceedings of the 30th ACM International Conference on Multimedia. MM 2022 - Proceedings of the 30th ACM International Conference on Multimedia, Association for Computing Machinery, Inc, pp. 7013-7015, 30th ACM International Conference on Multimedia, MM 2022, Lisboa, Portugal, 10/10/22. https://doi.org/10.1145/3503161.3551569

A Baseline for ViCo Conversational Head Generation Challenge. / Liu, Meng; Zhai, Shuyan; Li, Yongqiang et al.
MM 2022 - Proceedings of the 30th ACM International Conference on Multimedia. Association for Computing Machinery, Inc, 2022. p. 7013-7015 (MM 2022 - Proceedings of the 30th ACM International Conference on Multimedia).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A Baseline for ViCo Conversational Head Generation Challenge

AU - Liu, Meng

AU - Zhai, Shuyan

AU - Li, Yongqiang

AU - Guan, Weili

AU - Nie, Liqiang

PY - 2022/10/10

Y1 - 2022/10/10

N2 - ViCo conversational head generation challenge is organized in conjunction with ACM Multimedia 2022. The challenge includes two tasks: 1) generating a vivid talking head according to a reference image and driving audio, and 2) synthesizing a listening head conditioned on the reference image and speaker's behaviors. For talking head generation, we design a three-stage method to generate high-quality and lip-sync talking head videos with natural head poses, which wins 3rd place and People's Selection Award. On the listening head generation track, we adopt MakeItTalk to generate vivid head movements of listeners, which achieves 2nd place in the competition.

AB - ViCo conversational head generation challenge is organized in conjunction with ACM Multimedia 2022. The challenge includes two tasks: 1) generating a vivid talking head according to a reference image and driving audio, and 2) synthesizing a listening head conditioned on the reference image and speaker's behaviors. For talking head generation, we design a three-stage method to generate high-quality and lip-sync talking head videos with natural head poses, which wins 3rd place and People's Selection Award. On the listening head generation track, we adopt MakeItTalk to generate vivid head movements of listeners, which achieves 2nd place in the competition.

KW - listener generation

KW - talking head generation

UR - https://www.scopus.com/pages/publications/85150982646

U2 - 10.1145/3503161.3551569

DO - 10.1145/3503161.3551569

M3 - 会议稿件

AN - SCOPUS:85150982646

T3 - MM 2022 - Proceedings of the 30th ACM International Conference on Multimedia

SP - 7013

EP - 7015

BT - MM 2022 - Proceedings of the 30th ACM International Conference on Multimedia

PB - Association for Computing Machinery, Inc

T2 - 30th ACM International Conference on Multimedia, MM 2022

Y2 - 10 October 2022 through 14 October 2022

ER -

A Baseline for ViCo Conversational Head Generation Challenge

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this