Usage

Quick start

Fill in the configuration file XCAT_dvf_processing.cfg (see here: The configuration file) and call the main script with the config file as a parameter.

python XCATdvfProcessing.py XCAT_dvf_processing.cfg

A detailed description of how to configure the processing is given below.

Processing steps

The processing pipeline can be divided into three separate parts, namely

By setting the corresponding parameters in the first section of the configuration file, the different steps are enabled or disabled. It is highly recommended to run each individual step and check the results before continuing with the next.

[PROCESSING_STEPS]
doPreProcessing       = 1
doBatchPostProcessing = 0
doBatchWarping        = 0

Pre-processing

The most important output of the pre-processing is the signed distance map that defines the sliding regions in the first time point. To calculate this, first a rough segmentation using the information provided by the XCAT DVF-text file is generated. This is an incomplete segmentation (since organs are defined by surfaces), the sliding region is thereafter calculated by a level-set evolution.

Note

Note that the level-set evolution can take some time to complete.

[PREPROCESSING]
xcatAtnFile       = <Complete/Path/to/the/XCAT/attenuation/file.bin>
xcatDVFFile       = <Complete/Path/to/the/XCAT/DVF/file.txt>     # This can also be a g-zipped file ending with .txt.gz
outDir            = <Path/to/where/the/results/of/the/pre-processing/step/will/be/written/>
outDistMapImgName = <fileNameOfTheSignedDistanceMap.nii.gz>      # will be written to [PREPROCESSING][outDir]
outXCATAtnImgName = <fileNameOfTheXCATAttenuationFile.nii.gz>    # will be written to [PREPROCESSING][outDir]
numVoxX           = <imageDimensionInX>
numVoxY           = <imageDimensionInY>
numVoxZ           = <imageDimensionInZ>
spacingX          = <voxelSpacingInX>
spacingY          = <voxelSpacingInY>
spacingZ          = <voxelSpacingInZ>

# Optional, comment if not using
outXCATCTImgName           = <fileNameOfTheXCATImageInHU.nii.gz>   # will be written to [PREPROCESSING][outDir]
numberOfLevelSetIterations = <numberOfLevelSetIterations>          # Values depend on the field of view, the image resolution etc.
                                                                   # we so far had good experiences with values between 5000 and 7500
outLevelSetImgName         = <fileNameOfTheLevelSetImage.nii.gz>   # will be written to [PREPROCESSING][outDir]
saveLevelSetImage          = 1                                     # Set to 1 if you would like the level-set image to be saved, to 0 otherwise

Batch DVF post-processing

The batch processing uses the signed distance map from the previous step to calculate the sliding-preserving inversion of the XCAT DVF. The output of this step are the forward and backward DVFs that can are used as the new ground-truth deformations.

One final pre-processing step is carried out for the first iteration, which calculates the spatial gradient of the signed distance map at the first simulated time point. Only the output file names need to be specified and if these files do not exist in the pre-processing output directory these will be automatically generated.

[BATCH_POSTPROCESSING]
xcatDVFFilePattern   = <Complete/Path/and/pattern/of/all/XCAT/DVF_files_vec_frame1*.txt>   # Use the wildcard * to enable listing all DVF files, g-zipped text files ending with .txt.gz are also supported
outDir               = <Path/to/where/the/results/of/the/batch-dvf-processing/step/will/be/written/>
tmpDir               = <Path/to/where/temporary/filest/of/the/batch-dvf-processing/step/will/be/written/>
outDistMapDxImgName  = <fileNameOfTheSDT_gradientInX.nii.gz>    # maybe something like this: distMap_dx.nii.gz
outDistMapDyImgName  = <fileNameOfTheSDT_gradientInY.nii.gz>    # maybe something like this: distMap_dy.nii.gz
outDistMapDzImgName  = <fileNameOfTheSDT_gradientInZ.nii.gz>    # maybe something like this: distMap_dz.nii.gz
corruptedFilesDir    = <Path/to/where/potentially/corrupted/XCAT/text/files/will/be/written/>
numProcessorsDVFInv  = <NumberOfProcessorsUsedByNiftyReg>       # Integer value

# Optional, comment if not using
niftyRegBinDir       =  <Path/to/where/the/niftyReg/binaries/are/installed/>

XCAT DVF text files that cannot be completely processed will be moved to the path defined in [BATCH_POSTPROCESSING][corruptedFilesDir]. Incomplete processing can be caused by corrupted XCAT-text files or by other errors occurring during the processing.

Batch warping

In this step the deformation vector fields produced in the previous step are used to transform/warp the first time point to all other time points.

[BATCH_WARPING]
warpedOutImgBaseName     = imgWarpedByDVF_          # prefix of the transformed output images
outDir                   = <Path/to/where/the/results/of/the/bath-warping/step/will/be/written/>
dvfPostFix               = __dvf_dvfCor_Nto1.nii.gz  # This usually does not need to be changed

# Optional, comment if not using
scaleLungIntensity       = 1            # Set to 1 if you would like the lung intensity to change according to the Jacobian of the deformation
referenceImgName         = <Complete/path/to/the/image/that/you/would/like/to/warp.nii.gz>
lungLowerThreshold       = -760         # If using HU values this should be a reasonable lower threshold for lung tissue
lungUpperThreshold       = -730         # If using HU values this should be a reasonable upper threshold for lung tissue
additionalResampleParams = -pad -1000.0 # Set the padding value of the resampling to -1000

Note

The parameter dvfPostFix usually does not need to be changed. You should get a list of all post-processed DVF files by doing an ls [BATCH_POSTPROCESSING][outDir]*[BATCH_WARPING][dvfPostFix] on the command line. Replace the elements in square brackets with the parameters in the corresponding sections of your configuration file.

General notes

  • Image files are always saved in nifti image format. To save space it is highly recommended to use the compressed g-zipped file format. This can be done by selecting the file extension .nii.gz instead of .nii.

  • Each processing step has its own output directory

  • Some computations in the later parts of the processing pipeline depend on previous results. Some results are thus expected in the output directories.

  • End parameters that define a path with the separator /

  • Please use the forward slash / as a path separator also on Windows systems

  • To be more resourceful with the original XCAT DVF text files, these can be compressed using gzip. No decompression is required before running, however, the parameters in the config file need to reflect the different file ending, i.e. either .txt or .txt.gz.

The configuration file

The contents of the complete configuration file is given below.

#
# Configuration file for XCAT DVF-processing
#
# Usage: This file configures the python script XCATdvfProcessing.py. The path to this configuration file needs to be 
#        given to this script as an input parameter. 
# 
#  1) The DVF processing consists of three steps which can be switched on and off in the section [PROCESSING_STEPS].
#     Only the proecessing steps selected will be performed. The steps are described in further detail below ( 2) - 4) )
#  2) The pre-processing step as defined in section [PREPROCESSING]
#     2.1) An attenuation file generated by the XCAT simulator (.bin file format) is converted to the nifti image file 
#          format (.nii.gz file format)
#     2.2) Optionally a CT-like file is generated from the attenuation file and the corresponding coefficients
#     2.3) A sliding segmentation is created from a DVF text file generated by the XCAT simulator
#     2.4) A signed distance map is generated based on 2.3)
#  3) The batch-processing of the XCAT-generated deformation vector fields (DVFs) [BATCH_POSTPROCESSING]
#     3.1) Each DVF file will be split into sliding regions according to the segmentation in 2) 
#     3.2) An invertible DVF will be generated for each region and composed into a single DVF
#     Note: The gradient of the signed distance map (2.4) is required. If the defined output files do not exist
#           these will be generated automatically. The output directory for these gradient files is the output
#           directory of step 2), [PREPROCESSING].
#  4) Warping of the first XCAT image based on the generated DVFs
#     4.1) If no reference image is defined, the XCAT attenuation map of step 2.1) will be used
#     4.2) If a CT-like image was generated, lung intensities can be changed according to their volume change. 
#          To do this, set [BATCH_WARPING][scaleLungIntensity = 1], define the CT-like image as the reference
#          image, and select reasonable lung threhold values which are used to generate a lung mask.
#


[PROCESSING_STEPS]
doPreProcessing       = 1
doBatchPostProcessing = 0
doBatchWarping        = 0 


[PREPROCESSING]
xcatAtnFile       = Q:/Data/ICR/XCAT2019_10/phantom/phtm_atn_1.bin
xcatDVFFile       = Q:/Data/ICR/XCAT/2019_10/DVF/dvf_vec_frame1_to_frame2/dvf_vec_frame1_to_frame2.txt
outDir            = Q:/Data/ICR/XCAT2019_10/preProcessing/
outDistMapImgName = distMap.nii.gz
outXCATAtnImgName = phtm_atn_1.nii.gz
numVoxX           = 256
numVoxY           = 256
numVoxZ           = 281
spacingX          = 2.0
spacingY          = 2.0
spacingZ          = 2.0

# Optional, comment if not using
outXCATCTImgName           = phtm_HU_1.nii.gz  
numberOfLevelSetIterations = 7500
outLevelSetImgName         = levelSetOut.nii.gz
saveLevelSetImage          = 1


[BATCH_POSTPROCESSING]
xcatDVFFilePattern   = D:/debugData/xcat_intensityScaling/dvf_vec_frame1*.txt
outDir               = D:/debugData/xcat_intensityScaling/dvfNifti/
tmpDir               = D:/debugData/xcat_intensityScaling/tmpXCAT/
outDistMapDxImgName  = distMap_dx.nii.gz
outDistMapDyImgName  = distMap_dy.nii.gz
outDistMapDzImgName  = distMap_dz.nii.gz
corruptedFilesDir    = D:/debugData/xcat_intensityScaling/corruptedFiles/
numProcessorsDVFInv  = 4

# Optional, comment if not using
niftyRegBinDir       =  D:/development/niftyreg/cmicLab/install_release/bin/


[BATCH_WARPING]
warpedOutImgBaseName     = imgWarpedByDVF_
outDir                   = D:/debugData/xcat_intensityScaling/warpedByDVF/
dvfPostFix               = __dvf_dvfCor_Nto1.nii.gz

# Optional, comment if not using
scaleLungIntensity       = 1
referenceImgName         = D:/debugData/xcat_intensityScaling/preProcessing/phtm_HU_1.nii.gz
lungLowerThreshold       = -760
lungUpperThreshold       = -730
additionalResampleParams = -pad -1000.0

Other (potentially useful) functions

The functionality described below is used as part of the post-processing framework. However, some functions may be useful as on its own.

Converting XCAT binary output to nifti images

The XCAT binary images can be converted to nifti image format by calling file convertXCATBinaryFile.py from the command line alongside with some parameters that are required to fill in the nifti header information correctly. When run without any parameters, it will produce the following output:

Define:

  • the input image (.bin XCAT attenuation),

  • the directory to where you would like to save the nifti image - the file name will be the same as the .bin file,

  • size (nx, ny, and nz) and

  • spacing (dx, dy, and dz).

  • (optional) 0/1 if you would like the image intensities to be converted into HU values.

Note

Note that the image will have the origin set to (0,0,0).

Converting XCAT text deformation vector fields to nifti images

The XCAT software writes the output DVF to a text file. This package contains a python script to convert this into a nifit image file that can be used with niftiy-reg for instance: convertXCATDVFTextFile.py.

When the script is called without parameters, the following help message will appear:

python convertXCATDVFTextFile.py
>> Tool to convert XCAT DVF text output files into nifti images.
>>   Output: outputDir/XCAT_outFileName__dvf.nii.gz       DVF image with a displacement vector (in voxels).
>>
>> Usage: convertXCATDVFTextFile.py pathXCATDVFFile pathToNiftiImage outputDir nx ny nz dx dy dz
>>         - pathXCATDVFFile       -> Path to the DVF text file output by XCAT
>>         - pathToNiftiImage      -> Path to an image showing the XCAT anatomy in nifti image format
>>         - outputDir             -> Path to where the output will be saved
>>         - nx ny nz              -> Number of voxels in x, y, and z direction (optional, defaults to 256 x 256 x 161)
>>         - dx dy dz              -> Voxel spacing in x, y, and z direction (optional, defaults to 2 x 2 x 2)
>>         - topZ0                 -> Number of z-slices (from superior) in the lung-segmentation that is forced to zero, optional.

Define:

  • The DVF text file as an input,

  • A nifti image (CT-like/attenuation) converted as described in the section above

  • The number voxels of the original simulation (nx, ny, and nz)

  • The voxel spacing (dx, dy, and dz)

Note

Note that when using the generated DVFs with the nifty-reg package, the DVF defines relative displacements (and not absolute deformations).