AI 2D TAG

In this mode, you can use an AI to segment your 2D images, or if you want, you can train an AI on your previously segmented images.

This module work with individual images to automatically compute their associated TAG values. The images can be part of a 3D volume, but the 3D information is not used by the AI, each image is treated independently.

Note:

This mode work with the Python language. So, you need to install Python on your system. For help on that subject, please consult the page: www.TomoVision.com/support/sup_sliceO_python.html

From the Graphic Interface

The main "AI with Python" interface has 3 tabs: Train, Test and Predict. Each of theses has a "Config" button that will open the configuration menu, and one (or more) "Compute" button that will open the appropriate page.

The Configuration Menu

Each of the 3 main pages of the "AI Python" interface has a "Config" button. This will open the configuration menu. Depending on the main pages you are in, the configuration menu will have between 2 and 7 sub-pages. For training, you need access to all the pages, for "Use", you only need 2.

The config "File" page

Watch a Video Presentation

This page is used to specify the images that will be used either for training or testing your AI.


Progress bar	Show the progress of the program when analyzing directories
Add slices	Files can be added to this list in 3 different ways: Simply drag&drop a directory on the interface. The program will recursively parse this directory and all its sub-directories. Each directories that contain segmented images will be add as a separate line in the interface or use the "From Path" or "From Browser" options.
From Path	You can achieve the same results by specifying the directory in the input line and clicking the "Add from path" button.
From Browser	You can add files with the "Medi Browser" interface.
Clear List	Clear the list.
Dir List	For each directory specified that contain segmented files, you will see a line of information specifying the number of studies in that directory, the number of series, the modality of the images in the directory, the resolution of these images, the total number of images and the total number of segmented images, the number of TAGs present in these segmented images and a list of these TAGs. You can also select and de-select any of the directories by clicking on the associated button in front of each line. Please note that only the segmented images (those that have an associated ".tag" file) are used. If a directory does not contain any segmented images, it will not have a line in the interface. A double click on a line in this list will open the associated files in sliceOmatic.
Files preview	A preview of all the selected files is available. Clicking an image will select it and highlight the dir it came from in the directory list. A double click on an image will open that image in sliceOmatic.
Files status	Show the number of images found in the different selected directories. Also, the different TAGs found in these directories (along with their prevalence) will be displayed.
Exit	Exit the configuration interface.

The config "Preproc" page

Watch a Video Presentation

This page is used to control any pre-processing that need to be applied to the images. You also use it to select the classes used in your AI.

Note:

The term "Classes" is used in AI terminology to specify the number of different tissues segmented. In sliceO, I use the term "TAG" to designate the same thing. The only difference is that in AI, the background (TAG-0) is also a valid class, while in sliceO I usually did not consider the background as such. So keep in mind that in this module, when we are talking about the number of classes, we are counting the background as a valid class.

Input TAG mapping

In this section, there is one box for each TAG present in the selected images. The title of the box (ex: "Map TAG-1 to") let you know what TAG is being mapped. Each of these can be mapped to a "Class" for the AI.

At first, there is as many classes as there are TAGs in the selected images. But since multiple TAGs can be mapped to the save class, some of the classes can be disabled if no TAG mapped to them. So, if there is a TAG in the images you do not want to use in the AI, just map it to the background.

This mapping is only used for training and testing. When predicting, only the grey values of the image is used.

Class mapping

Each classes used in the AI can be mapped to a TAG value. Each classes can also be assigned a label. So, for example, if you map "Class 1" to the "TAG-1", when predicting the segmentation of an image, the tissue associated to Class 1 will be tagged in sliceO using TAG-1.

This mapping is only used for predicting and testing.

Resolution

Select the resolution of the images you want to use in your AI. If an image is smaller in one axis than the selected resolution, it will be padded to the selected resolution. If both axes are different, the image will be rejected.

TAG mapping

In sliceOmatic, you can assign any of the 256 available TAGs to any tissues. In AI, we have the equivalent "classes". This interface will enable you to map the TAG values to found in the selected images to different classes used in the AI (input). This will be used while training the AI. Also, you can specify how the AI classes will be mapped back to TAG values when the AI will be used to segment (or "Predict") the images (output).

You can also disable some of the tissues and re-name the different classes. These names will automatically be applied to the "output" TAGs when you use the AI to segment images.

Modality

Select the modality of the images you want to use in your AI. The selected modality will also limit the choices you will have in the "Intensity Normalizations" step.

Normalization

The AI expect floating point pixels values between 0 and 1. While medical images use integer values. So, we need to "normalize" the pixel values.

For CT images, all HU values under the minimum will be assigned a value of 0, and all HU greater than the maximum will be assigned a value of +1 in the normalized images. All HU values in-between will be linearly mapped between 0 and 1.

For other modalities, you have a choice of "Min/Max Normalization" or "Z-Score Normalization".

Mapping status

An overview of the current classes and how they are mapped, along with their label and the ratio of pixels (with the associated weights) is presented here. The "Weight" values are used in the different "Weighted" metrics.

Exit

Exit the configuration interface.

Note:

The background (TAG-0) is always enabled and is always mapped to Class 0 and always labelled "Background".

Note:

The Ratio fo each class is computed as: Nb of pixel of a class / Nb of pixels in all classes.

The weight of each class is computed as: Sqrt( 1.0 / Ratio ) / tot

Where: tot = Sum ( sqrt( 1.0 / Ratio) ) for all classes.

Note:

When you select a "Weight" from the "Weights" page, the input/output mapping used to comput these weights will overwrite your current selection.

The config "Augment" page

Watch a Video Presentation

This page enable you to select and control the different augmentations that are applied to the images before they are used for training. The augmentations will be applied to all the "training" images before each epoch. Only the "Training" images are augmented, the images reserved for validation and testing are never augmented.


	Original Image	Rotated	Scaled	Translated


	Distorted	Flipped	With Noise	With "Pseudo MR"

Augmentations

You can enable/disable the different augmentations, and specify the intensity of each of these from this interface.

You have access to 9 augmentations. Each of these is controlled by a selected value "val":

•Rotation:	A rotation of the image, center on the center of the image, of a random angle going from -val to +val.
•Scaling:	A random scaling of the image, ranging from -val % to +val %.
•Translate Horizontal:	An horizontal translation of the image ranging from -val % to +val % of the width of the image.
•Translate Vertical:	A vertical translation of the image ranging from -val % to +val % of the height of the image.
•Distort:	A grid of 8x8 bi-cubic splines are superimposed on the image. Each of the control points of these splines are randomly jiggled by a factor scaled by the value selected in the interface. The resulting splines are used to distort the image.
•Flip Horizontal:	If a random value is over a threshold (fixed by "val"), the image is flipped horizontally.
•Flip Vertical:	If a random value is over a threshold (fixed by "val"), the image is flipped vertically.
•Perlin (Noise):	We use a Perlin noise function of an high frequency to simulate noise in the image.
•Perlin (Pseudo MR):	We use a Perlin noise function of a low frequency to simulate the intensity variations present in MR images.

Demo Original

The slider under the image is used to change the demo frame. All the images used for training are available.

Demo Augmented

The slider under the image is used to show the range of augmentation.

In normal training images, each time an image is augmented, a random value between -1 to +1 is assigned to each augmentation of each image.

In this demo however, we use the slider to show the range of possible augmentation. The slider simulate the random variable, with a range of -1 (complete left) to +1 (complete right).

Exit

Exit the configuration interface.

The config "Metrics" page

Watch a Video Presentation

When you train the AI, 2 sets of "metrics" are used: The Loss Metrics is the one used to compute the AI weights. The other group of metrics are just computed to give you an idea of what is going on. They are only there to provide you some feedback on the accuracy of the AI, they are optional and can be all off if you want.

At this time, you can only have 1 "Loss" metric when training your AI. You can however have as many "Feedback" metrics as you want.

These metrics will be reported back to you when you train your AI. They can also be used when you will select the weights you want to save for your AI. During training you will be presented with 2 graphs: One for the training and validation loss, and one for the training and validation values of the feedback metrics.


Loss Metrics	Select one of the metrics as a "Loss" function to train the AI.
Feedback Metrics	Select any metrics you want from this list to get additional information on the AI training progress.
Text Window	This window reports on the metrics selections and the description of each metrics.
Description	This will cause the description associated with each of the selected metrics to be displayed in the text window.
Edit Loss	The different metrics are all Python files that you can edit if you want. You can also add your own metrics. Just follow the prescribed syntax and add the python file of your new metric in the Python directory. Then click the "re-scan dir" to cause sliceO to re-analyze the directory and add your metrics. The syntax will be described later at the end of this section.
Edit Feedback
Re-scan dir
Exit	Exit the configuration interface.

The config "Split" page

Watch a Video Presentation


Testing	A portion of the images you selected in the config "File" page can be reserved for testing.
Testing Status	Report on the number of available images
Validation	You have access to 3 validation methods: •Training/Validation Split. Probably the simplest, a part of the available images (those left after the images reserved for testing are removed) is reserved for validating the AI at each epoch. The same images are used for validation at each epoch. •K-Fold Cross-Validation. The available images are split in "n" groups using the ratio defined in the K-Fold interface. For example if the K-Fold is 1/5, then n=5. We do a complete training of all the epochs reserving the images of one of these groups validation. We then redo the complete training using another group for validation. We do this n times. After that, we assume that we have a good idea of the validation since by now all the images have been used for that, we do a last training without any validation. In the "Train Save" page, the graphs will show the mean value of the n+1 pass for training metrics and the n pass with validation for validation metrics. •Leave one out Cross-Validation. This is the extreme case of K-Fold where n is equal to the number of images.
Validation Status	Report the number of images that will be used for training and for validation.
Exit	Exit the configuration interface.

The config "Model" page

Watch a Video Presentation

In this page, you will select the model you want to use for the AI.


Model List	Select a model from the list.
Model Feedback	Information on the selected model will be displayed here.
Model Summary	Call the Keras function "Summary" for the selected model.
Edit Model	The different models are all Python files that you can edit if you want. You can also add your own models. Just follow the prescribed syntax and add the python file of your new model in the Python directory. Then click the "re-scan dir" to cause sliceO to re-analyze the directory and add your model. The syntax will be described later at the end of this section.
Re-scan Python dir
Exit	Exit the configuration interface.

The config "Weights" page

Watch a Video Presentation


Weight List	This list will display all the available weights for the selected model. Select one of these from the list.
Weight Feedback	Information on the selected weightsl will be displayed here.
Weight Description	Cause the description of the weights to be displayed in teh text window.
Edit Description	The weight description is a text file assoicated with the weights. It has the same name as the weights but with the ".txt" extension. It can be edited with a simple text editor. If you click "Edit", this file will be opened in Notepad.
Re-Scan weights	Re-scan the directory if you have saved some new weights for this model.
Exit	Exit the configuration interface.

The "Train" page


Text Feedback
Config	Open the Configuration window
Train	Open the "Train" Window.
Save	Open the "Save" Window.

The Train "Train" page

Watch a Video Presentation

Once you used the config pages to select the files you want to train on, and you selected the preprocessing steps, the augmentations, the validation split and the model you want to use, you are ready to start training your model to obtain the AI weights.

This is the page that will enable you to train the AI.

The only things that remain for you to select are the learning rate, the number of epochs and the batch size. After that, you click on "Train" and wait (and wait... and wait...) for the results!

Once started, the training will:

- Pre-load all the images used for training and validation in memory.

Then, for each step it will:

- Reset or pre-load the weights (if you selected weights).

- Make a scrambled list of the images.

Then, for each epoch it will:

- From the scrambles list, select the files used for training and augment them.

- Call the Keras function "train_on_batch" for the training images.

- Call the Keras function "test_on_batch" for the validation images.

- Report the results of this epoch and save the current weights, the current metrics and a script associated with these.

The number of "steps" is dependant on the validation technique. A simple Train/Validation split us done in a single step. K-Fold and Leave-one-out cross validation require multiple steps.

Note:

The first call to "train_on_batch" is always longer than the others. Do not be allarmed if the red dot in the progress bar stop for a few secconds.

Note:

The "ETA" information is computed at the end of each epoch. So you need to wait for at least one epoch to finish before having an idea of how long the complete training will take.

Note:

If you stop the training, or if the computer (or sliceOmatic) crash. You will have the option to re-start the process from where it stopped the next time you press "Start"


Progress Bar	Report on the training progression.
ETA	Estimated time of completion (computed after each epoch).
Step Feedback	Current step number.
Epoch Feedback	Current Epoch number.
Batch Feedback	Currently processed batch.
Text Feedback	textual information about the training progression.
Loss Graphic	A graphic showing the tarining and validation loss values as a function of the epochs.
Validation Graphic	A graphic showing the training and validation values of all the selected feedback metrics
Learning Rate	The learning rate used in training the AI. the default value is 0.001
Max Epoch	The number of epoch used for the training.
Batch Size	The number of images used in each training and testing batchs. If the AI failed becuase of a lack of GPU memory, you can decrease this value.
Early Stopping	Optima training is usually achieve when the validation loss stop decreasing. You can enable this option to stop the training automatically when this happen. The "Patience" parameter will let the AI train for a number of additional epochs to make sure we did indeed have achieve the minimum validation loss.
Start	Start the training.
Pause/Resume	You can pause and resume the training.
Stop	Stop the training.
Exit	Exit the "Training" interface.

The Train "Save" page

Watch a Video Presentation

After having trained your AI, you want to save the weights you just computed.

This page will show you the weights of all the epochs of the training. You need to select one of these and save it. It will then be transfered from the AI_Temp folder to the sub-folder of the selected model with the name you specified. A "description" file will also be saved along the weights in a file with the same name but with the ".txt" extension.

To help you select the optimal epoch for the weights, you have access to plots of the loss and feedback values along with their numerical values.


Epoch Selection	Select the desired epoch from the list.
Name	Name use for the saved weights. A default value is proposed, but you can change it as you which.
Description	The description associated with the saved weights. Again, a default value is proposed, but you can edit this test to your liking.
Epoch Feedback	Numerical values of the different metrics at the end of the selected epoch.
Loss Graphic	Plot of the training and validation loss as a function of the epochs. The yellow vertical bar represent the selected epoch.
Validation Graphic	Plot of the training and validation feedback metrics as a function of the epochs. The yellow vertical bar represent the selected epoch.
Save	Save the selected weights to the model's sub-folder.
Exit	Exit the "Save" interface.

The "Test" page

This page is used to test your AI against a reserved portion of the selected images.

The program will go through all the "test" images, and call the Keras "test_on_batch" function for each of them. It will report the different metrics for each images and once all the images are tested, compute the mean and standard deviation of each of the metrics.

These results will also be saved in a Excel compatible file in the AI_Temp directory.

You will also be able to view each of the tested images. You can view the original segmentation, the predicted segmentation and a mix of both. in the "both" mode, the TAG that are un-correctly predicted will be full color, while the correct prediction will be darker. You also have control over the display options for the match, false positive and false negative values of each TAG.


	Original segmentation	Predicted segmentation	Both in mode "Over"	Both in mode "Grey"


Text Feedback
Config	Open the Configuartion window
Test	Open the "Test Progress" Window.

The Test "Test" page

Watch a Video Presentation

Progress Bar

Show the progression of the testing

Text window

Report on the metrics for each image and the mean values.

Results box

Once the testing is done, this box will display the numerical results of the different metrics. The results will be sorted according to the "Loss" metrics, from the worst value (top) to the best value (bottom). The line matching the displayed test image will be highlighted in yellow.

Test image

Show the test images.

Note:

If you double click on an image, both the "true" and "predicted" TAG values will be saved as ".tbg" and ".tcb" files in the same dir as the grey level image. Then that image will be opened in sliceOmatic. You can then use the "Multi TAG" tools to see the TAGs for the image.

Image selection

You can display any of the images used for testing. the arrow keys will advance to the first image, the previous image, the next image, the last image.

You can aslo use the slider to select images. The images are sorted from the one with the worst to the best "Loss" metrics.

Source selection

You can display either the original segmentation, the prdicted segmentatino, or a mix of both.

Display Mode

This is the same controls as the "2D Color Scheme" tool ins liceOmatic. the "F1" to "F4" keys can also be used as shortcuts.

TAG Color

For each TAG computed by the AI, you enable/disable and change the color of 3 group of pixels:

•The pixels that have this TAG value in both the original and predicted images.

•The pixels that have this TAG value in th epredicted image only (False Positive)

•The pixels that have this TAG value in the Original image only (False Negative)

Start

Start the computation of the tests.

Pause/Resume

Pause/resume the computation.

Stop

Stop the computation.

Exit

Exit this page.

The Predict page

This is the page where the magic happens!

From the AI list, select the AI you want to use. Alternatively, you can use the Config page to select a model and its assciated weights.

Select the desired images in sliceOmatic and click on the "Compute" button.

Note:

In AI terminology "Predicting" is the act of computing the segmentation of an image with an AI.


AI Selection	Select the desired AI from the list. This will automatically set the Model and the associated Weights.
Config	Open the Configuartion window
Compute	Open the "Predict Progress" Window.

The Predict "Compute" page

Watch a Video Presentation


Progress Bar	Show the progression of the predicting
Text Feedback	Report some info on the progression of the prediction.
Start	Start the computation of the prediction.
Pause/Resume	Pause/resume the computation.
Stop	Stop the computation.
Exit	Exit this page.

The Python files

You can add your own metrics and models to this mode. You just have to follow the syntax described here:

The Python directory.

All the Python files used by sliceOmatic's AI module are stored in the Python directory. By default this is in "c:\Program Files\TomoVision\SliceO_Python". The location of this directory can be changed through the sliceOmatic's "Config" interface.

The Metrics Python files.

For my metrics files, I use names that start with "metrics ", but the name is not really important. What is important is that the file contain either a "metrics_2D" function, a "loss_2D" function or both.

You can have 3 types of metrics functions

The "Loss" metrics:

- The Loss functions compute a loss metrics used to train the AI.

The value of the loss function must decrease when we are approach perfect results.

The name of the loss function must be: "loss_2D_TAG" and have 2 arguments: "y_true" and "y_pred", 2 tensors, as described in Keras doc.

You must define a flag for the function: loss_2D_TAG.flag =... (The choices are between: "mean" "weighted" and "class" )

You must define the name of the function: loss_2D_TAG.__name__ =...

You must define a version for the function: loss_2D_TAG.__version__ =... (The version syntax is: "major.minor". ex: "2.3")

You must define the description of the function: loss_2D_TAG.description =...

import keras as kr

# ----------------------------------------------------

# The function must be called "loss_2D"

# ----------------------------------------------------

def loss_2D( y_true, y_pred ) :

# --- We simply call the Keras function ---

return( kr.losses.categorical_crossentropy( y_true, y_pred ) )

# ----------------------------------------------------

# Give a name for this metrics

# ----------------------------------------------------

loss_2D.__name__ = "Categorical CrossEntropy"

# ----------------------------------------------------

# Give a name for this metrics

# ----------------------------------------------------

loss_2D.description = "Call the Keras \"categorical_crossentropy\" function:\n" \

"Computes the crossentropy loss between the labels and predictions."

You can have 2 types of metrics: "global" or "Per class".

The "global" metrics:

- The Metrics functions compute a metrics used as feedback when training or testing.

The name of the metrics function must be: "metrics_2D_TAG" and have 2 arguments: "y_true" and "y_pred", 2 tensors, (as described in Keras doc.)

You must define a flag for the function: metrics_2D_TAG.flag =... (The choices are between: "mean" "weighted" and "class" )

You must define the name of the function: metrics_2D_TAG.__name__ =...

You must define a version for the function: metrics_2D_TAG.__version__ =... (The version syntax is: "major.minor". ex: "2.3")

You must define the description of the function: metrics_2D_TAG.description =...

from keras import backend as K

import AI_Var # --- AI_Var is the SliceOmatic module ---

def dice_coef(y_true, y_pred):

y_true_f = K.flatten(y_true)

y_pred_f = K.flatten(y_pred)

intersection = K.sum(y_true_f * y_pred_f)

epsilon = 1e-6

return (2. * intersection + epsilon) / (K.sum(y_true_f) + K.sum(y_pred_f) + epsilon)

# ----------------------------------------------------

# The function must be called "metrics_2D"

# ----------------------------------------------------

def metrics_2D_TAG( y_true, y_pred ) :

dice=0

# --- I want the number of classes and their weights ---

weights = AI_Var.sliceO_Var_Get_Float("$AI_CLASS_WEIGHT")

# print(weights.shape)

# print(weights)

num_classes = weights.shape[0]

fact = 1.0 / num_classes

# --- compute average Dice coef ---

for index in range(num_classes):

dice += fact * dice_coef(y_true[...,index], y_pred[...,index])

return dice

# ----------------------------------------------------

# Set the "Weighted" flag

# ----------------------------------------------------

metrics_2D_TAG.flag = "mean"

# ----------------------------------------------------

# Give a name for this metrics

# ----------------------------------------------------

metrics_2D_TAG.__name__ = "Dice Coeff (Mean)"

# ----------------------------------------------------

# Give a version for this metrics

# ----------------------------------------------------

metrics_2D_TAG.__version__ = "2.0"

# ----------------------------------------------------

# Give a name for this metrics

# ----------------------------------------------------

metrics_2D_TAG.description = "Compute the mean DICE coefficient of all classes"

The "per class" metric:

The name of the metrics function must be: "metrics_2D_TAG" and have 2 arguments: "class_idx" and "name", the index and name of the target class, for "Per class" metrics.

The "per class" metrics return a "global" metric function for the desired class.

You must define the name of the function: metrics_2D_TAG.__name__ =...

You must define a version for the function: metrics_2D_TAG.__version__ =... (The version syntax is: "major.minor". ex: "2.3")

You must define the description of the function: metrics_2D_TAG.description =...

You must also define a "flag" that must contain the string "class". (metrics_2D_TAG.flag = "class")

import tensorflow as tf

from keras import backend as K

import AI # --- AI is the SliceOmatic module ---

# ----------------------------------------------------

# The function must be called "metrics_2D"

# ----------------------------------------------------

def metrics_2D(class_idx: int, name: str, epsilon=1e-6):

def dice_coef(y_true: tf.Tensor, y_pred: tf.Tensor):

# Extract single class to compute dice coef

y_true_single_class = K.flatten(y_true[...,class_idx])

y_pred_single_class = K.flatten(y_pred[...,class_idx])

intersection = K.sum(y_true_single_class * y_pred_single_class)

return (2. * intersection + epsilon) / (K.sum(y_true_single_class) + K.sum(y_pred_single_class) + epsilon)

dice_coef.__name__ = f"dice_coef_{name}" # Set name used to log metric

return dice_coef

# ----------------------------------------------------

# Set the "Per Class" flag

# ----------------------------------------------------

metrics_2D.flag = "class"

# ----------------------------------------------------

# Give a description for this metrics

# ----------------------------------------------------

metrics_2D.__name__ = "Dice Coeff"

# ----------------------------------------------------

# Give a name for this metrics

# ----------------------------------------------------

metrics_2D.description = "Compute the a DICE coefficient for each classes"

The Model Python files.

Again, I use the prefix "model " for my models files but the name is no important. The important is that the file contain a "model_2D_TAG" function.

That function receive 4 arguments: dim_x, dim_y, dim_z and num_classes. dim_x and y are the X and Y resolution of the images we will train this model with and num_classes is the number of classes. The "dim_z" argument is not used in 2D models. The function must define the layers of your model and return the "model" variable returned by the Keras "Model" function.

You must define the name of the function: model_2D_TAG.__name__ =...

You must define a version for the function: model_2D_TAG.__version__ =... (The version syntax is: "major.minor". ex: "2.3")

You must define the description of the function: model_2D_TAG.description =...

from tensorflow import keras

from keras.models import Model

from tensorflow.keras import layers

# =============================================================================

# ==================== Model 2D Function ======================================

# =============================================================================

# ----------------------------------------------------

# The function must be called "model_2D"

# ----------------------------------------------------

def model_2D_TAG( img_dim_x, img_dim_y, num_classes ) :

inputs = keras.Input( shape=(img_dim_x,img_dim_y,1,) )

### [First half of the network: downsampling inputs] ###

# Entry block

x = layers.Conv2D( 32, 3, strides=2, padding="same")(inputs)

x = layers.BatchNormalization()(x)

x = layers.Activation("relu")(x)

previous_block_activation = x # Set aside residual

# Blocks 1, 2, 3 are identical apart from the feature depth.

for filters in [64, 128, 256]:

x = layers.Activation("relu")(x)

x = layers.SeparableConv2D(filters, 3, padding="same")(x)

x = layers.BatchNormalization()(x)

x = layers.Activation("relu")(x)

x = layers.SeparableConv2D(filters, 3, padding="same")(x)

x = layers.BatchNormalization()(x)

x = layers.MaxPooling2D(3, strides=2, padding="same")(x)

# Project residual

residual = layers.Conv2D(filters, 1, strides=2, padding="same")(

previous_block_activation

)

x = layers.add([x, residual]) # Add back residual

previous_block_activation = x # Set aside next residual

### [Second half of the network: upsampling inputs] ###

for filters in [256, 128, 64, 32]:

x = layers.Activation("relu")(x)

x = layers.Conv2DTranspose(filters, 3, padding="same")(x)

x = layers.BatchNormalization()(x)

x = layers.Activation("relu")(x)

x = layers.Conv2DTranspose(filters, 3, padding="same")(x)

x = layers.BatchNormalization()(x)

x = layers.UpSampling2D(2)(x)

# Project residual

residual = layers.UpSampling2D(2)(previous_block_activation)

residual = layers.Conv2D( filters, 1, padding="same" )(residual)

x = layers.add([x, residual]) # Add back residual

previous_block_activation = x # Set aside next residual

# Add a per-pixel classification layer

outputs = layers.Conv2D( num_classes, 1, activation="softmax", padding="same" )(x)

# Define the model

model = Model( inputs, outputs )

return model

# ----------------------------------------------------

# Give a name for this model

# ----------------------------------------------------

model_2D_TAG.__name__ = "U-Net"

# ----------------------------------------------------

# Give a version for this model

# ----------------------------------------------------

model_2D_TAG.__version__ = "2.0"

# ----------------------------------------------------

# Give a name for this model

# ----------------------------------------------------

model_2D_TAG.description = "Standard\xF1 U-Net\xF0 model as described in:\n" \

"\xF1\001U-Net: Convolutional Networks for Biomedical Image Segmentation.\xF0\002\n" \

"Olaf Ronneberger, Philipp Fischer, Thomas Brox.\n" \

"MICCAI 2015"

The Test Python files.

By default, I test for the presence of Python, Numpy, Tensorflow and Keras. If you use another module in your Python code, you can add a new test file. SliceO will use this to test that the desired module is indeed loaded when it start the AI module.

The file must contain a "version" function that return the version number for the target module.

It must also contain a "__name__" and a "__version__" definition.

import numpy as np

# ----------------------------------------------------

# The function must be called "version"

# ----------------------------------------------------

def version():

print( "numpy vers: ", np.__version__ )

return np.__version__

# ----------------------------------------------------

# Give a name for this test

# ----------------------------------------------------

version.__name__ = "NumPy vers: "

# ----------------------------------------------------

# Give a version for this test

# ----------------------------------------------------

version.__version__ = "2.0"

Accessing sliceOmatic variables from Python.

You can access any of the sliceOmatic variables from your python code.

The functions for this are:

sliceO_Var_Get_Int( Variable_Name )

sliceO_Var_Get_Float( Variable_Name )

sliceO_Var_Get_Str( Variable_Name )

return one (or an array of) value(s), depending on the variable.

sliceO_Var_Set_Int( Variable_Name, Value )

sliceO_Var_Set_Float( Variable_Name, Value )

sliceO_Var_Set_Str( Variable_Name, Value )

Assign the value "Value" to the variable "Var_Name". At this time only variable with one value can be "Set" (no arrays).

From the Display Area

There is no display area interaction specific to this mode.

From the Keyboard

The following commands are mapped to keyboard keys as a shortcut:


	Key map	Action

	F1	Set the color scheme mode for all windows: F1=Grey, F2=Mixed, F3=Tint and F4=Over
	F2
	F3
	F4

From the Command Line

System Variables defined in this library:

$AI_RES_X	(I₁₆)
$AI_RES_Y	(I₁₆)
$AI_RES_Z	(I₁₆)
$AI_CLASS_NB	(I₁₆)
$AI_CLASS_LABEL	A(W)	* $AI_CLASS_NB
$AI_CLASS_RATIO	A(F₃₂)	* $AI_CLASS_NB
$AI_CLASS_WEIGHTS	A(F₃₂)	* $AI_CLASS_NB

Tversky metrics parameters:

	$AI_TVERSKY_ALPHA	(F₃₂)
	$AI_TVERSKY_BETA	(F₃₂)

TAG re-mapping:

$AI_TAG_MAP_IN_NB	(I₃₂)
$AI_TAG_MAP_IN_VAL	A(U₈)	* $AI_CLASS_NB
$AI_TAG_MAP_OUT	A(U₈)	* $AI_CLASS_NB

Training Parameters:

	$AI_TRAIN_STEP_MAX	(I₃₂)
	$AI_TRAIN_STEP_CUR	(I₃₂)
	$AI_TRAIN_EPOCH_MAX	(I₃₂)
	$AI_TRAIN_EPOCH_CUR	(I₃₂)
	$AI_TRAIN_BATCH_MAX	(I₃₂)
	$AI_TRAIN_BATCH_CUR	(I₃₂)
	$AI_TRAIN_LR	(F₃₂)
	$AI_TRAIN_EARLY_FLAG	(I₃₂)
	$AI_TRAIN_EARLY_NB	(I₃₂)

Python Variables:

	$PYTHON_FLAG	(U₃₂)
	$PYTHON_MODULE_PATH	(W)
	$PYTHON_INTERPRETER_PATH	(W)
	$PYTHON_SLICEO	(P)
	$PYTHON_TEMP	(W)

Commands recognized in this mode:

Python: interpreter "path"

Python: path "path"

Python: temp "path"

A2T: file read "path"

A2T: file "path" (on|off)

A2T: preproc resolution x [y]

A2T: preproc modality value

A2T: preproc min value

A2T: preproc max value

A2T: preproc norm ("HU" | "min-max" | "z-score")

A2T: class nb value

A2T: class in id value

A2T: class out id value

A2T: class label id "label"

A2T: augment "name" (on|off) value

A2T: metrics loss "name"

A2T: metrics feedback clear

A2T: metrics feedback "name"

A2T: split test value

A2T: split mode value

A2T: split split value

A2T: split kfold value

A2T: model cur "name"

A2T: model modality value

A2T: model class value

A2T: weights cur "name"

A2T: weights load

The config "File" page

Progress bar

Add slices

From Path

From Browser

Clear List

Dir List

Files preview

Files status

Exit

The config "Preproc" page

Input TAG mapping

Class mapping

Resolution

TAG mapping

Modality

Normalization

Mapping status

Exit

The config "Augment" page

Original Image

Rotated

Scaled

Translated

Distorted

Flipped

With Noise

With "Pseudo MR"

Augmentations

Demo Original

Demo Augmented

Exit

Exit the configuration interface.

The config "Metrics" page

The config "Split" page

Exit the configuration interface.

The config "Model" page

Model Summary

Call the Keras function "Summary" for the selected model.

Edit Model

Re-scan Python dir

Exit the configuration interface.

The config "Weights" page

Information on the selected weightsl will be displayed here.

Weight Description

Cause the description of the weights to be displayed in teh text window.

Edit Description

The weight description is a text file assoicated with the weights. It has the same name as the weights but with the ".txt" extension. It can be edited with a simple text editor. If you click "Edit", this file will be opened in Notepad.

Re-Scan weights

Re-scan the directory if you have saved some new weights for this model.

Exit the configuration interface.

Text Feedback

Config

Train

Save

The first call to "train_on_batch" is always longer than the others. Do not be allarmed if the red dot in the progress bar stop for a few secconds.

The "ETA" information is computed at the end of each epoch. So you need to wait for at least one epoch to finish before having an idea of how long the complete training will take.

If you stop the training, or if the computer (or sliceOmatic) crash. You will have the option to re-start the process from where it stopped the next time you press "Start"

Exit the "Training" interface.

Epoch Selection

Name

Description

Epoch Feedback

Loss Graphic

Validation Graphic

Save

Exit

Original segmentation

Predicted segmentation

Both in mode "Over"

Both in mode "Grey"

Text Feedback

Config

Open the Configuartion window

Test

Open the "Test Progress" Window.

Progress Bar

Show the progression of the testing

Text window

Report on the metrics for each image and the mean values.