TY - JOUR

T1 - The Liver Tumor Segmentation Benchmark (LiTS)

AU - Bilic, Patrick

AU - Christ, Patrick

AU - Li, Hongwei Bran

AU - Vorontsov, Eugene

AU - Ben-Cohen, Avi

AU - Kaissis, Georgios

AU - Szeskin, Adi

AU - Jacobs, Colin

AU - Mamani, Gabriel Efrain Humpire

AU - Chartrand, Gabriel

AU - Lohöfer, Fabian

AU - Holch, Julian Walter

AU - Sommer, Wieland

AU - Hofmann, Felix

AU - Hostettler, Alexandre

AU - Lev-Cohain, Naama

AU - Drozdzal, Michal

AU - Amitai, Michal Marianne

AU - Vivanti, Refael

AU - Sosna, Jacob

AU - Ezhov, Ivan

AU - Sekuboyina, Anjany

AU - Navarro, Fernando

AU - Kofler, Florian

AU - Paetzold, Johannes C

AU - Shit, Suprosanna

AU - Hu, Xiaobin

AU - Lipková, Jana

AU - Rempfler, Markus

AU - Piraud, Marie

AU - Kirschke, Jan

AU - Wiestler, Benedikt

AU - Zhang, Zhiheng

AU - Hülsemeyer, Christian

AU - Beetz, Marcel

AU - Ettlinger, Florian

AU - Antonelli, Michela

AU - Bae, Woong

AU - Bellver, Míriam

AU - Bi, Lei

AU - Chen, Hao

AU - Chlebus, Grzegorz

AU - Dam, Erik B

AU - Dou, Qi

AU - Fu, Chi-Wing

AU - Georgescu, Bogdan

AU - Giró-I-Nieto, Xavier

AU - Gruen, Felix

AU - Han, Xu

AU - Heng, Pheng-Ann

AU - Hesser, Jürgen

AU - Moltz, Jan Hendrik

AU - Igel, Christian

AU - Isensee, Fabian

AU - Jäger, Paul

AU - Jia, Fucang

AU - Kaluva, Krishna Chaitanya

AU - Khened, Mahendra

AU - Kim, Ildoo

AU - Kim, Jae-Hun

AU - Kim, Sungwoong

AU - Kohl, Simon

AU - Konopczynski, Tomasz

AU - Kori, Avinash

AU - Krishnamurthi, Ganapathy

AU - Li, Fan

AU - Li, Hongchao

AU - Li, Junbo

AU - Li, Xiaomeng

AU - Lowengrub, John

AU - Ma, Jun

AU - Maier-Hein, Klaus

AU - Maninis, Kevis-Kokitsi

AU - Meine, Hans

AU - Merhof, Dorit

AU - Pai, Akshay

AU - Perslev, Mathias

AU - Petersen, Jens

AU - Pont-Tuset, Jordi

AU - Qi, Jin

AU - Qi, Xiaojuan

AU - Rippel, Oliver

AU - Roth, Karsten

AU - Sarasua, Ignacio

AU - Schenk, Andrea

AU - Shen, Zengming

AU - Torres, Jordi

AU - Wachinger, Christian

AU - Wang, Chunliang

AU - Weninger, Leon

AU - Wu, Jianrong

AU - Xu, Daguang

AU - Yang, Xiaoping

AU - Yu, Simon Chun-Ho

AU - Yuan, Yading

AU - Yue, Miao

AU - Zhang, Liping

AU - Cardoso, Jorge

AU - Bakas, Spyridon

AU - Braren, Rickmer

AU - Heinemann, Volker

AU - Pal, Christopher

AU - Tang, An

AU - Kadoury, Samuel

AU - Soler, Luc

AU - van Ginneken, Bram

AU - Greenspan, Hayit

AU - Joskowicz, Leo

AU - Menze, Bjoern

N1 - Copyright © 2022 The Author(s). Published by Elsevier B.V. All rights reserved.

PY - 2023

Y1 - 2023

N2 - In this work, we report the set-up and results of the Liver Tumor Segmentation Benchmark (LiTS), which was organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI) 2017 and the International Conferences on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2017 and 2018. The image dataset is diverse and contains primary and secondary tumors with varied sizes and appearances with various lesion-to-background levels (hyper-/hypo-dense), created in collaboration with seven hospitals and research institutions. Seventy-five submitted liver and liver tumor segmentation algorithms were trained on a set of 131 computed tomography (CT) volumes and were tested on 70 unseen test images acquired from different patients. We found that not a single algorithm performed best for both liver and liver tumors in the three events. The best liver segmentation algorithm achieved a Dice score of 0.963, whereas, for tumor segmentation, the best algorithms achieved Dices scores of 0.674 (ISBI 2017), 0.702 (MICCAI 2017), and 0.739 (MICCAI 2018). Retrospectively, we performed additional analysis on liver tumor detection and revealed that not all top-performing segmentation algorithms worked well for tumor detection. The best liver tumor detection method achieved a lesion-wise recall of 0.458 (ISBI 2017), 0.515 (MICCAI 2017), and 0.554 (MICCAI 2018), indicating the need for further research. LiTS remains an active benchmark and resource for research, e.g., contributing the liver-related segmentation tasks in http://medicaldecathlon.com/. In addition, both data and online evaluation are accessible via https://competitions.codalab.org/competitions/17094.

AB - In this work, we report the set-up and results of the Liver Tumor Segmentation Benchmark (LiTS), which was organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI) 2017 and the International Conferences on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2017 and 2018. The image dataset is diverse and contains primary and secondary tumors with varied sizes and appearances with various lesion-to-background levels (hyper-/hypo-dense), created in collaboration with seven hospitals and research institutions. Seventy-five submitted liver and liver tumor segmentation algorithms were trained on a set of 131 computed tomography (CT) volumes and were tested on 70 unseen test images acquired from different patients. We found that not a single algorithm performed best for both liver and liver tumors in the three events. The best liver segmentation algorithm achieved a Dice score of 0.963, whereas, for tumor segmentation, the best algorithms achieved Dices scores of 0.674 (ISBI 2017), 0.702 (MICCAI 2017), and 0.739 (MICCAI 2018). Retrospectively, we performed additional analysis on liver tumor detection and revealed that not all top-performing segmentation algorithms worked well for tumor detection. The best liver tumor detection method achieved a lesion-wise recall of 0.458 (ISBI 2017), 0.515 (MICCAI 2017), and 0.554 (MICCAI 2018), indicating the need for further research. LiTS remains an active benchmark and resource for research, e.g., contributing the liver-related segmentation tasks in http://medicaldecathlon.com/. In addition, both data and online evaluation are accessible via https://competitions.codalab.org/competitions/17094.

U2 - 10.1016/j.media.2022.102680

DO - 10.1016/j.media.2022.102680

M3 - Journal article

C2 - 36481607

VL - 84

JO - Medical Image Analysis

JF - Medical Image Analysis

SN - 1361-8415

M1 - 102680

ER -