Evaluation
Last updated: 3 July 2012
An online tool for comparing automated segmentations with reference ones is available from either the Virtual Skeleton Database evaluation page or from Kitware/MIDAS. The tool computes the following segmentation performance metrics for both edema and active tumor:
- Dice
- Jaccard
- Sensitivity
- Specificity
- Average closest distance
- Hausdorff distance
- Cohen's kappa
Using the on-line evaluation tool at the Virtual Skeleton Database
Automated segmentations should be uploaded directly to the evaluation page. A user account is required - if you don't have one, you can create one from the BRATS2012 starting page. The following guidelines should be followed:- Use the MHA filetype to store your segmentations (not MHD)
- Keep the same labels as the provided truth.mha: 1 for edema, 2 for active tumor, and 0 for everything else
- In order to match your segmentations with the correct ground truth, name your segmentations according to this template: VSD.your_description.###.mha, where the ### is the ID of the corresponding FLAIR scan in the Virtual Skeleton Database. If you downloaded the training data from the Virtual Skeleton Database, then this ID will be apparent from the FLAIR file name in the subdirectory corresponding to the subject in question (e.g., file name VSD.Brain.XX.O.MR_Flair.687.mha corresponds to ID 687). Otherwise, please refer to the following table to match the subject (last column) with the correct FLAIR ID (second column): OpenDocument Spreadsheet file format, Microsoft Excel file format,, or PDF.
