Skip to main content

Evaluation & Results

The evaluation will be performed using T1/T2-weighted MR images with manual segmentations of WM, GM and CSF. Automatic segmentations will be evaluated against this manual set – a reference standard, by using various measurements, such as Dice Coefficient (DC), Modified Hausdorff Distance (MHD) and Average Surface Distance (ASD).

Besides these overall measurements (DC, MHD and ASD), we will also have local measurements based on major sulci and gyri (coming soon).

We will only provide a rank list for each metric. We will not provide an overall rank list since we think

“Everyone is a winner.”

 

 

 WM GM CSF
TEAM DATE DICE MHD ASD DICE MHD ASD DICE MHD ASD
ASU-BNI 08/05/2017 0.739 9.710 0.894 0.781 8.592 0.677 0.876 9.501 0.302  Details
Authman 08/07/2017 0.838 7.316 0.568 0.872 7.590 0.494 0.917 9.652 0.224  Details
Bern_IPMI 07/24/2017 0.896 6.782 0.398 0.916 6.455 0.341 0.954 9.616 0.127  Details
BIGS2 08/07/2017 0.867 6.810 0.510 0.892 7.518 0.415 0.948 9.705 0.145  Details
BUUA_braintech 08/06/2017 0.885 7.866 0.559 0.887 7.248 0.481 0.922 9.903 0.215  Details
BuzzLightDay 07/24/2017 0.686 12.185 1.325 0.768 42.935 0.875 0.781 12.070 0.640  Details
CAMP_TUM 08/06/2017 0.880 7.325 0.470 0.901 7.636 0.408 0.940 9.570 0.164  Details
CatholicU 07/18/2017 0.827 7.475 0.622 0.868 6.940 0.524 0.915 10.227 0.235  Details
CUmed 08/08/2017 0.876 7.025 0.484 0.896 6.388 0.420 0.935 9.436 0.180  Details
LIVIA 07/12/2017 0.897 6.975 0.376 0.919 6.415 0.338 0.957 9.029 0.138  Details
LRDE 07/25/2017 0.861 6.607 0.523 0.887 5.852 0.458 0.928 9.875 0.201  Details
MSL_SKKU 08/05/2017 0.901 6.444 0.391 0.919 5.980 0.330 0.958 9.072 0.116  Details
NeuroMTL 08/01/2017 0.885 6.874 0.458 0.902 7.108 0.389 0.935 11.065 0.198  Details
nic_vicorob 07/24/2017 0.885 7.154 0.430 0.910 7.647 0.367 0.951 9.178 0.137  Details
QuIIL 08/01/2017 0.877 7.170 0.471 0.894 6.346 0.429 0.929 9.099 0.194  Details
STH 08/01/2017 0.870 6.942 0.513 0.892 6.505 0.431 0.936 9.477 0.177  Details
TU/e IMAG/e 06/25/2017 0.890 6.908 0.433 0.904 6.856 0.375 0.947 9.426 0.150  Details
UofL-BioImaging 08/07/2017 0.867 6.844 0.531 0.895 9.004 0.436 0.942 9.775 0.163  Details
UPC_DLMI 07/31/2017 0.882 6.689 0.446 0.907 6.927 0.384 0.945 9.086 0.151  Details
UPF_simbiosys 05/21/2017 0.885 6.968 0.435 0.903 6.390 0.405 0.929 9.859 0.194  Details
Xidian 08/08/2017 0.765 17.838 0.948 0.767 15.750 0.723 0.876 12.104 0.335  Details
HUST-CBIB 01/18/2021 0.906 6.773 0.364 0.925 6.304 0.311 0.960 8.249 0.109  Details

The above table is in alphabetical order.

 

 

WM GM CSF
RANK TEAM DATE DICE RANK TEAM DATE DICE RANK TEAM DATE DICE
1 MSL_SKKU 08/05/2017 0.901 1 MSL_SKKU 08/05/2017 0.919 1 MSL_SKKU 08/05/2017 0.958
2 LIVIA 07/12/2014 0.897 2 LIVIA 07/12/2014 0.919 2 LIVIA 07/12/2014 0.957
3 Bern_IPMI 07/24/2017 0.896 3 Bern_IPMI 07/24/2017 0.916 3 Bern_IPMI 07/24/2017 0.954
4 TU/e IMAG/e 06/25/2017 0.890 4 nic_vicorob 07/24/2017 0.910 4 nic_vicorob 07/24/2017 0.951
5 UPF_simbiosys 05/24/2017 0.885 5 UPC_DLMI 07/31/2017 0.907 5 BIGS2 08/07/2017 0.948
6 NeuroMTL 08/01/2017 0.885 6 TU/e IMAG/e 06/25/2017 0.904 6 TU/e IMAG/e 06/25/2017 0.947
7 nic_vicorob 07/24/2017 0.885 7 UPF_simbiosys 05/24/2017 0.903 7 UPC_DLMI 07/31/2017 0.945
8 UPC_DLMI 07/31/2017 0.882 8 NeuroMTL 08/01/2017 0.902 8 UofL-BioImaging 08/07/2017 0.942
9 CAMP_TUM 08/06/2017 0.880 9 CAMP_TUM 08/06/2017 0.901 9 CAMP_TUM 08/06/2017 0.940
10 QuIIL 08/07/2017 0.877 10 Cumed 08/08/2017 0.896 10 STH 08/01/2017 0.936
11 Cumed 08/08/2017 0.876 11 UofL-BioImaging 08/07/2017 0.895 11 NeuroMTL 08/01/2017 0.935
12 STH 08/01/2017 0.870 12 QuIIL 08/07/2017 0.894 12 Cumed 08/08/2017 0.935
13 BIGS2 08/07/2017 0.867 13 STH 08/01/2017 0.892 13 UPF_simbiosys 05/24/2017 0.929
14 UofL-BioImaging 08/07/2017 0.867 14 BIGS2 08/07/2017 0.892 14 QuIIL 08/07/2017 0.929
15 LRDE 07/25/2017 0.861 15 LRDE 07/25/2017 0.887 15 LRDE 07/25/2017 0.928
16 BUUA_braintech 08/06/2017 0.855 16 BUUA_braintech 08/06/2017 0.887 16 BUUA_braintech 08/06/2017 0.922
17 Authman 08/08/2017 0.838 17 Authman 08/08/2017 0.872 17 Authman 08/08/2017 0.917
18 CatholicU 07/18/2017 0.827 18 CatholicU 07/18/2017 0.868 18 CatholicU 07/18/2017 0.915
19 Xidian 08/08/2017 0.765 19 ASU-BNI 08/05/2017 0.781 19 Xidian 08/08/2017 0.876
20 ASU-BNI 08/05/2017 0.739 20 BuzzLightDay 08/02/2017 0.768 20 ASU-BNI 08/05/2017 0.876
21 BuzzLightDay 08/02/2017 0.686 21 Xidian 08/08/2017 0.767 21 BuzzLightDay 08/02/2017 0.781

 

 

WM GM CSF
RANK TEAM DATE MHD RANK TEAM DATE MHD RANK TEAM DATE MHD
1 MSL_SKKU 08/05/2017 6.444 1 LRDE 07/25/2017 5.852 1 LIVIA 07/12/2014 9.029
2 LRDE 07/25/2017 6.607 2 MSL_SKKU 08/05/2017 5.980 2 MSL_SKKU 08/05/2017 9.072
3 UPC_DLMI 07/31/2017 6.689 3 QuIIL 08/07/2017 6.346 3 UPC_DLMI 07/31/2017 9.086
4 Bern_IPMI 07/24/2017 6.782 4 Cumed 08/08/2017 6.388 4 QuIIL 08/07/2017 9.099
5 BIGS2 08/07/2017 6.810 5 UPF_simbiosys 05/24/2017 6.390 5 nic_vicorob 07/24/2017 9.178
6 UofL-BioImaging 08/07/2017 6.844 6 LIVIA 07/12/2014 6.415 6 TU/e IMAG/e 06/25/2017 9.426
7 NeuroMTL 08/01/2017 6.874 7 Bern_IPMI 07/24/2017 6.455 7 Cumed 08/08/2017 9.436
8 TU/e IMAG/e 06/25/2017 6.908 8 STH 08/01/2017 6.505 8 STH 08/01/2017 9.477
9 STH 08/01/2017 6.942 9 TU/e IMAG/e 06/25/2017 6.856 9 ASU-BNI 08/05/2017 9.501
10 UPF_simbiosys 05/24/2017 6.968 10 UPC_DLMI 07/31/2017 6.927 10 CAMP_TUM 08/06/2017 9.570
11 LIVIA 07/12/2014 6.975 11 CatholicU 07/18/2017 6.940 11 Bern_IPMI 07/24/2017 9.616
12 Cumed 08/08/2017 7.025 12 NeuroMTL 08/01/2017 7.108 12 Authman 08/08/2017 9.652
13 nic_vicorob 07/24/2017 7.154 13 BUUA_braintech 08/06/2017 7.248 13 BIGS2 08/07/2017 9.705
14 QuIIL 08/07/2017 7.170 14 BIGS2 08/07/2017 7.518 14 UofL-BioImaging 08/07/2017 9.775
15 Authman 08/08/2017 7.316 15 Authman 08/08/2017 7.590 15 UPF_simbiosys 05/24/2017 9.859
16 CAMP_TUM 08/06/2017 7.325 16 CAMP_TUM 08/06/2017 7.636 16 LRDE 07/25/2017 9.875
17 CatholicU 07/18/2017 7.475 17 nic_vicorob 07/24/2017 7.647 17 BUUA_braintech 08/06/2017 9.903
18 BUUA_braintech 08/06/2017 7.866 18 ASU-BNI 08/05/2017 8.592 18 CatholicU 07/18/2017 10.227
19 ASU-BNI 08/05/2017 9.710 19 UofL-BioImaging 08/07/2017 9.004 19 NeuroMTL 08/01/2017 11.065
20 BuzzLightDay 08/02/2017 12.185 20 Xidian 08/08/2017 15.750 20 BuzzLightDay 08/02/2017 12.070
21 Xidian 08/08/2017 17.838 21 BuzzLightDay 08/02/2017 42.935 21 Xidian 08/08/2017 12.104

 

 

WM GM CSF
RANK TEAM DATE ASD RANK TEAM DATE ASD RANK TEAM DATE ASD
1 LIVIA 07/12/2014 0.376 1 MSL_SKKU 08/05/2017 0.330 1 MSL_SKKU 08/05/2017 0.116
2 MSL_SKKU 08/05/2017 0.391 2 LIVIA 07/12/2014 0.338 2 Bern_IPMI 07/24/2017 0.127
3 Bern_IPMI 07/24/2017 0.398 3 Bern_IPMI 07/24/2017 0.341 3 nic_vicorob 07/24/2017 0.137
4 nic_vicorob 07/24/2017 0.430 4 nic_vicorob 07/24/2017 0.367 4 LIVIA 07/12/2014 0.138
5 TU/e IMAG/e 06/25/2017 0.433 5 TU/e IMAG/e 06/25/2017 0.375 5 BIGS2 08/07/2017 0.145
6 UPF_simbiosys 05/24/2017 0.435 6 UPC_DLMI 07/31/2017 0.384 6 TU/e IMAG/e 06/25/2017 0.150
7 UPC_DLMI 07/31/2017 0.446 7 NeuroMTL 08/01/2017 0.389 7 UPC_DLMI 07/31/2017 0.151
8 NeuroMTL 08/01/2017 0.458 8 UPF_simbiosys 05/24/2017 0.405 8 UofL-BioImaging 08/07/2017 0.163
9 CAMP_TUM 08/06/2017 0.470 9 CAMP_TUM 08/06/2017 0.408 9 CAMP_TUM 08/06/2017 0.164
10 QuIIL 08/07/2017 0.471 10 BIGS2 08/07/2017 0.415 10 STH 08/01/2017 0.177
11 Cumed 08/08/2017 0.484 11 Cumed 08/08/2017 0.420 11 Cumed 08/08/2017 0.180
12 BIGS2 08/07/2017 0.510 12 QuIIL 08/07/2017 0.429 12 UPF_simbiosys 05/24/2017 0.194
13 STH 08/01/2017 0.513 13 STH 08/01/2017 0.431 13 QuIIL 08/07/2017 0.194
14 LRDE 07/25/2017 0.523 14 UofL-BioImaging 08/07/2017 0.436 14 NeuroMTL 08/01/2017 0.198
15 UofL-BioImaging 08/07/2017 0.531 15 LRDE 07/25/2017 0.458 15 LRDE 07/25/2017 0.201
16 BUUA_braintech 08/06/2017 0.559 16 BUUA_braintech 08/06/2017 0.481 16 BUUA_braintech 08/06/2017 0.215
17 Authman 08/08/2017 0.568 17 Authman 08/08/2017 0.494 17 Authman 08/08/2017 0.224
18 CatholicU 07/18/2017 0.622 18 CatholicU 07/18/2017 0.524 18 CatholicU 07/18/2017 0.235
19 ASU-BNI 08/05/2017 0.894 19 ASU-BNI 08/05/2017 0.677 19 ASU-BNI 08/05/2017 0.302
20 Xidian 08/08/2017 0.948 20 Xidian 08/08/2017 0.723 20 Xidian 08/08/2017 0.335
21 BuzzLightDay 08/02/2017 1.325 21 BuzzLightDay 08/02/2017 0.875 21 BuzzLightDay 08/02/2017 0.640

 

 

 

Print Friendly, PDF & EmailEmail or Print Friendly