Operators/Tools/Interactive/LabelImageEditor: Difference between revisions

From MiToBo
Jump to navigationJump to search
(Screenshot added)
 
(17 intermediate revisions by the same user not shown)
Line 3: Line 3:


== Label Image Editor ==
== Label Image Editor ==
[[File:MTBOperatorControlFrame-LabelImageEditor.png|500px|right|link=]]
[[File:LabelImageEditor-MTBOperatorControlFrame.png|400px|right|link=]]


The ''Label Image Editor'' is available since release version 1.8.8 of MiToBo.<br/>
The ''Label Image Editor'' is available since release version 1.8.8 of MiToBo.<br/>
In MiToBo/MiToBo plugins releases 1.8.13.1 and 2.1.2 it got rich feature updates.


===== Latest News =====
===== Latest News =====
The Label Image Editor has been released for user-friendly editing of label images.
The editor got latest updates in MiToBo/MiToBo plugins 2.1.2 (April 2021).


===== Name of Plugin/Operator =====
===== Name of Plugin/Operator =====
Line 19: Line 20:


===== Main features =====  
===== Main features =====  
* subsequently displays images provided in an input folder to the user
* images provided in an input folder are subsequently displayed to the user
* regions, i.e. connected components, can be removed by simple mouse-clicks
* regions, i.e. connected components, can be removed and joined by simple mouse-clicks
* additional boundaries can be added by drawing free-hand lines with configurable line width
* region labels can be adjusted to enhance visibility
* region re-labeling is supported to guarantee unique labels
* holes within regions can be filled
* missing regions in terms of small holes can automatically filled in
* support for up to 65535 labels


===== Usage =====
===== Usage =====
The editor is dedicated to the analysis of label images. In a label image each region or connected component, respectively, is assumed to be marked with a unique intensity or color value, i.e. all of its pixels should share this value.
The editor provides various editing functions to remove or join regions within such images.<br><br>
Note that the editor can also handle label images with one-pixel wide black boundaries between regions.<br><br>
The editor takes as input a directory from where the images are loaded one after the other.<br>
Once an image is displayed to the user edit operations can be performed. Afterwards the changes are saved to a new image file and the next label image is read from the given input directory.
<br>
To run the LabelImageEditor perform the following steps:
To run the LabelImageEditor perform the following steps:
* install MiToBo by following the instructions on the [[Installation]] page
* install MiToBo by following the instructions on the [[Installation]] page
* run MiToBo and start the operator runner by selecting the menu item '''''MiToBo Runner''''' from '''''Plugins -> MiToBo'''''  
* run MiToBo and start the operator runner by selecting the menu item '''''MiToBo Runner''''' from '''''Plugins -> MiToBo'''''  
* in the selection menu navigate to 'de.unihalle.informatik.MiToBo.tool.interactive' and select the operator '''''LabelImageEditor'''''
* in the selection menu navigate to '''de.unihalle.informatik.MiToBo.tool.interactive''' and select the operator '''''LabelImageEditor'''''
This will bring up the operator window of the LabelImageEditor.
This will bring up the operator window of the LabelImageEditor (see figure on the right, top) where the basic configuration parameters can be added. Clicking the "Run" button will open the editor window with the first image loaded (figure on the right, bottom).
 
<br>
* Input data:<br>The operator reads the contents of a given input image directory and displays all images one after the other to the user. <br>The user can edit the images, i.e. remove labeled regions, by clicking with the left mouse button somewhere into the region to remove.<br> Clicking the "Next" button at the top of the editor window will save the current image to disk and proceed with the next image in the folder. <br><br>
* Output data:<br>For each input image the operator will save a corresponding output image, either in the same folder or a specific output folder if provided.<br> Output images will share the name of the corresponding input image extended with substring "-edited".<br> Note, all images are relabeled automatically before saving to ensure consecutive labels in the resulting edited images. <br> Consecutive labels are a pre-requisite for some other MiToBo tools which can be used in further processing steps, like the [[Applications/FeatureColorMapper|FeatureColorMapper]].


* Input data:<br>The operator analyzes all images in the given input folder, expecting images to have the extension ".tif".<br> If the images contain more than one channel, the first channel is used. In addition, for each image<br> either a label image of pre-segmented cell regions or a set of region boundaries is expected to be available in the given mask folder.<br> These files should have the same basenames like the corresponding images, but either end on "-mask.tif" in case of label images,<br> or on ".zip" or ".roi", respectively, in case of region sets. Note that the operator currently only accepts ImageJ 1.x ROI sets as input.<br><br> If label images are used as segmentation masks, the labels of individual cells in the label images are used as unique identifiers for the cells, i.e., are used as labels in the various plots. If the segmentation data is given in terms of ROIs a unique identifier is derived from the order of the cell boundaries in the ROI set. Note that in the latter case a mask image is written to the output directory as additional output where the cells are marked by their identifiers to allow for easier assessment of the results.<br><br>The operator is able to automatically consider different groups of cells in its analysis, e.g., generate plots of the cluster distributions for each group individually.<br> However, the group membership of each image has to be encoded in its filename. In detail, the operator expects the file names to obey the following structure: <code>groupName_imageID.tif</code><br>In particular, there must not occur more than a single underscore in each filename and the ''imageID'' must be unique for each image of a group.<br>If the image names do not follow these requirements, all images are treated as a single group.<br><br>
<br>
* Output data:<br>The operator displays the cluster distributions for each group of cells as stacked bar plots and box-whisker plots. In addition, it writes several files to the given output folder:
** ''*-features.txt'': feature data for each image
** ''*-features.tif'': image stack visualizing the feature data
** ''*-features-config.ald'': configuration of the operator in this run
** ''*-clusterDistro.txt'': cluster distributions per image
** ''*-clusters.tif'': pseudo-colored image illustrating the cluster distribution per image
** ''AllImagesClusterStatistics.txt'': cluster distribution raw data for all images
** ''AllImagesSubspaceFeatures.txt'': if PCA is applied to the cluster distributions prior to the distance calculations, this file contains the subspace feature vectors
** ''AllImagesPairwiseDistanceData.txt'': matrix of pairwise Euclidean distances for distribution vectors, can be examined, e.g., with ''Multidendrograms'' (see below)
** ''*-distribution.png'': for each cell group a stacked bar plot is saved showing the cluster distribution for each cell of the group<br><br>  
* Parameters:
* Parameters:
{|class="wikitable"
{|class="wikitable"
Line 45: Line 53:
|Description
|Description
|-
|-
|''Image directory''
|''Input Directory''
|directory where the input image data can be found
|directory from where the images to process will be loaded
|-
|''File Filter''
|Optional string to filter image files to be processed. If the string is empty all files are processed. <br>If a non-empty string is provided, files with names not containing the pattern will be skipped.<br> For example, if the string ".tif" is provided, only images ending with ".tif", i.e. in TIFF format, will be considered.<br> The string "cell" would select only images containing the word "cell" somewhere in their names.<br> Note that as string any valid Java regular expression is admissible. See for example [http://www.vogella.com/tutorials/JavaRegularExpressions/article.html this website] for more details on regular expressions.
|-
|''Output directory''
|optional; edited result label images will be saved to this directory; if not provided the result images will be stored in the input folder
|}
 
[[File:LabelImageEditor-EditorWindow.png|500px|right|link=]]
 
<br>
The following table lists all available operations available in the editor window:
{|class="wikitable"
|style="width: 15%;"|Operation
|How to do it
|-
|''remove a region''
|click with left mouse button somewhere into the region to be removed;<br>the region will then be removed by setting all its pixels to a value of zero
|-
|-
|''Mask directory''
|''join two regions''
|directory where the label images or region boundary files can be found
|press 'Shift' and left-click into the first region to remember its label, then release 'Shift' and left-click into the second region;<br>the label of the first region will be transferred to the second region; black pixels between both parts can be removed afterwards using the "Fix borders" button
|-
|-
|''Mask format''
|''draw a new boundary''
|format of the segmentation data files: LABEL_IMAGE = images with unique labels for each cell and a value of zero for the background / IJ_ROIS = set of ImageJ 1.x ROIs, one ROI for each cell
| keep left mouse button pressed and drag the mouse, this will add a one-pixel wide free-hand line to the label image; note that you can configure the line width via the 'Options' button
|-
|-
|''Output and working directory''
|''fill-in a missing region<br> (new in version 2.1.2)''
|directory to which the result files and intermediate data is written
|press 'Ctrl' and 'Shift' and left-click into the missing region area; be sure that the area is fully enclosed by other regions, open background sections cannot be filled properly
|}
 
<br>
The buttons in the editor window have the following functions:
{|class="wikitable"
|Button
|Function
|Keyboard Shortcut
|-
|-
|''Calculate features''
|''Next''
|if disabled the operator expects the features to be already present in the input directory and skips the (time-consuming) feature calculations;<br> this option is helpful if the features have already been calculated ones and only the parameters of the clustering should be changed
|save current changes and jump to next image
|''Alt + n''
|-
|-
|''Feature directory''
|''Skip''
|directory where the features should be saved or - in case they are already available - from where they are read; the directory can be the same as the output and working directory
|cancel current image, don't save anything and jump to next image
|''Alt + s''
|-
|-
|''Tile size x/y''
|''Contrast''
|size of the sliding window used for feature calculations, should be chosen according to the resolution of the input images
|adjust labels to optimize visibility, i.e., shift labels into upper third of gray-scale range with maximal distances between them
|''Alt + c''
|-
|-
|''Tile shift x/y''
|''Relabel''
|shift of the sliding window, if the shift is smaller than the tile size sliding windows overlap
|relabel the image, i.e., make sure that every region gets a unique label
|''Alt + r''
|-
|-
|''Distance''
|''Fix Borders''
|pixel-pair distance in co-occurence matrix calculations
|remove all background pixels with more than one foreground label in their neighborhood; the size of the neighborhood to be considered can be configured via the 'Options' button, and sometimes it is necessary to re-run the command several times to remove all non-border pixels between two region parts
|''Alt + b''
|-
|-
|''Set of directions''
|''Fill Holes''
|directions to be considered in co-occurence matrix calculations
|fill background regions enclosed by a single region with the region label
|''Alt + h''
|-
|-
|''Isotropic calculations''
|''Undo''
|the texture features are derived from co-occurence matrixes; if this flag is enabled features for different directions are averaged,<br> otherwise all individual directions are preserved (resulting in larger, but also more informative feature vectors)
|undo the last action; IMPORTANT: only the most recent action can be undone!
|''Alt + u''
|-
|-
|''Number of feature clusters''
|''Options''
|number of clusters in first stage, i.e., number of expected structural patterns in the images
|opens a window with configuration options, e.g., for setting the line width in drawing or the maximum boundary width
|''Alt + o''
|-
|-
|''Do PCA in stage II?''
|''Quit''
|allows to enable/disable PCA on the cluster distribution vectors prior to the pairwise distance calculations; by default enabled
|save current changes and quit the editor window
|''Alt + q''
|}
|}


===== Additional Tools =====
<br><br>
The hierarchical clustering in stage II of our approach as described in the paper has been done using the [http://deim.urv.cat/~sgomez/multidendrograms.php MultiDendrograms] software.<br> In principal every hierarchical clustering tool can be applied. <br>
'''Hint:''' The ''Undo'' function has currently only a history of length one. If you accidentally deleted too many regions or want to undo more than one action, either run the editor once again on the complete directory or copy the corresponding image to a separate folder and run the editor on that folder. Results of a former run will be overwritten without further inquiry.
The basis for the hierarchical cluster analysis is the file ''AllImagesPairwiseDistanceData.txt'' to be found in the output directory upon termination of an analysis run. It contains a matrix of pairwise Euclidean distances of the (optionally dimension-reduced) cluster distribution vectors of all cells. The file can directly be loaded by MultiDendrograms, for other tools format convertion might be necessary.<br>


You can download the latest version of MultiDendrograms from its webpage: [http://deim.urv.cat/~sgomez/multidendrograms.php]
=== Updates ===
 
----<br>
===== Sample data =====
For testing the ''ActinAnalyzer2D'' operator we provide some sample data:
* [http://www.informatik.uni-halle.de/mitobo/downloads/actin_examples_update.zip ActinAnalyzer2D sample data for MiToBo >= 1.8]
* [http://www.informatik.uni-halle.de/mitobo/downloads/actin_examples.zip ActinAnalyzer2D sample data for MiToBo <= 1.7]


(With release 1.8 the handling of file names and automatic deduction of mask file names changed, thus, image names in the sample data archive had to be updated.)
'''April 2021'''
* The editor got some updates in MiToBo 2.1.2 and MiToBo-plugins 2.1.2, respectively. Background regions can now be filled-in, more labels are supported and some parameters be configured by the user.


The archive contains the following sub-folders:
'''January 2019'''
* ''imageData'': test images that were used in the ICPR publication mentioned above
* The editor got a comprehensive update in MiToBo 1.8.13.1 and MiToBo-plugins 1.8.13.1, respectively. There is now much more editing functionality available, e.g., filling holes or drawing of free-hand lines.
* ''maskData'': corresponding label images
* ''featureData'': pre-calculated features for the cells (re-calculating the features may require up to an hour, depending on the machine used)
* ''resultData'': sample results calculated on the given data


In addition, in the archive a file with a sample parameter configuration for the operator can be found. The parameters are those used for producing the sample results. Once the operator has been started the file can be loaded via the '' 'File' '' menu and its entry '' 'Load Settings' ''. Note that you need to set the image and mask directories, and also the feature directory according to your local file system structure and the place to where you extracted the zip file.
'''March 2018'''
* The LabelImageEditor has been released in MiToBo 1.8.8 and MiToBo-plugins 1.8.8, respectively.<br><br>
<p>


For more information on the data and the morphological analysis of the cells, see<br> <br>
=== Technical Details ===
''Anne Zirkel, Marcell Lederer, Nadine Stöhr, Nikolaos Pazaitis, and Stefan Hüttelmaier''<br>
'''''IGF2BP1 promotes mesenchymal cell properties and migration of tumor-derived cells by enhancing the expression of LEF1 and SNAI2 (SLUG)'''''<br>
''Nucleic Acids Res. Jul 2013; 41(13): 6618–6636. Published online May 15, 2013. doi: 10.1093/nar/gkt410'', [http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3711427/ Article]
 
=== Updates ===
----<br>
----<br>
 
The label editor relies on analyzing and manipulating the grayscale values of segmented regions in a given image.<br> In detail, the most important functions are implemented as follows:
'''December 2016'''
* deleting regions:<br> the grayscale value at the mouse-click position is determined and all pixels in the image sharing this value are set to black
* In MiToBo 1.8.4 and MiToBo-plugins 1.8.3, respectively, which have recently been released, some bugs in the Actin Analyzer plugin were fixed, see latest news at the top of this page.
* joining regions:<br> upon clicking into the first region its grayscale value is internally stored;<br> when the user clicks into the second region which is to be merged with the first one<br> the grayscale value at the click position is extracted and all pixels with this value are relabeled with the formerly stored grayscale value of the first region;<br> important: after doing the join both joined regions share the same grayscale value, however, are not connected to each other; hence, to get a valid label image again you should run the "Fix Borders" function
 
* relabel image:<br> the image is thresholded with a threshold of zero to get a binary image of background and labeled regions;<br> then a sequential component labeling is run on the image which assigns consecutive grayscale labels to all connected components in the foreground of the image
'''October 2016'''
* adjust contrast:<br> the image is relabeled, however, the target labels are equidistantly spread over just the upper two third of the available grayscale range;<br> the underlying motivation for this is given by the observation that low grayscale values are hard to distinguish from the background, thus, brighter labels improve visibility
* Sample data archive has been updated due to some changes in file name handling.
* fix borders:<br> this functions seeks to eliminate borders between regions with identical labels which may for example occur after joining two regions;<br> for removing borders for each background pixel in the image the grayscale values in a 7x7 neighborhood of the pixel are analyzed,<br> and if there is only a single grayscale value found and more than half of the analyzed neighborhood pixels share this value, the corresponding background pixel gets the value as well;<br> the neighborhood is currently fixed to 7x7, thus, borders which are wider cannot be closed, and it has to be noted that sometimes the function has to be run several times to remove all pixels belonging to a single border
 
'''July 2014'''
* Released first version of Actin Analyzer as published in ICPR 2014.

Latest revision as of 15:26, 9 April 2021



Label Image Editor

LabelImageEditor-MTBOperatorControlFrame.png

The Label Image Editor is available since release version 1.8.8 of MiToBo.
In MiToBo/MiToBo plugins releases 1.8.13.1 and 2.1.2 it got rich feature updates.

Latest News

The editor got latest updates in MiToBo/MiToBo plugins 2.1.2 (April 2021).

Name of Plugin/Operator

de.unihalle.informatik.MiToBo.tools.interactive.LabelImageEditor
(available since MiToBo version 1.8.8)

Main features
  • images provided in an input folder are subsequently displayed to the user
  • regions, i.e. connected components, can be removed and joined by simple mouse-clicks
  • additional boundaries can be added by drawing free-hand lines with configurable line width
  • region labels can be adjusted to enhance visibility
  • region re-labeling is supported to guarantee unique labels
  • holes within regions can be filled
  • missing regions in terms of small holes can automatically filled in
  • support for up to 65535 labels
Usage

The editor is dedicated to the analysis of label images. In a label image each region or connected component, respectively, is assumed to be marked with a unique intensity or color value, i.e. all of its pixels should share this value. The editor provides various editing functions to remove or join regions within such images.

Note that the editor can also handle label images with one-pixel wide black boundaries between regions.

The editor takes as input a directory from where the images are loaded one after the other.
Once an image is displayed to the user edit operations can be performed. Afterwards the changes are saved to a new image file and the next label image is read from the given input directory.


To run the LabelImageEditor perform the following steps:

  • install MiToBo by following the instructions on the Installation page
  • run MiToBo and start the operator runner by selecting the menu item MiToBo Runner from Plugins -> MiToBo
  • in the selection menu navigate to de.unihalle.informatik.MiToBo.tool.interactive and select the operator LabelImageEditor

This will bring up the operator window of the LabelImageEditor (see figure on the right, top) where the basic configuration parameters can be added. Clicking the "Run" button will open the editor window with the first image loaded (figure on the right, bottom).


  • Input data:
    The operator reads the contents of a given input image directory and displays all images one after the other to the user.
    The user can edit the images, i.e. remove labeled regions, by clicking with the left mouse button somewhere into the region to remove.
    Clicking the "Next" button at the top of the editor window will save the current image to disk and proceed with the next image in the folder.

  • Output data:
    For each input image the operator will save a corresponding output image, either in the same folder or a specific output folder if provided.
    Output images will share the name of the corresponding input image extended with substring "-edited".
    Note, all images are relabeled automatically before saving to ensure consecutive labels in the resulting edited images.
    Consecutive labels are a pre-requisite for some other MiToBo tools which can be used in further processing steps, like the FeatureColorMapper.


  • Parameters:
Name Description
Input Directory directory from where the images to process will be loaded
File Filter Optional string to filter image files to be processed. If the string is empty all files are processed.
If a non-empty string is provided, files with names not containing the pattern will be skipped.
For example, if the string ".tif" is provided, only images ending with ".tif", i.e. in TIFF format, will be considered.
The string "cell" would select only images containing the word "cell" somewhere in their names.
Note that as string any valid Java regular expression is admissible. See for example this website for more details on regular expressions.
Output directory optional; edited result label images will be saved to this directory; if not provided the result images will be stored in the input folder
LabelImageEditor-EditorWindow.png


The following table lists all available operations available in the editor window:

Operation How to do it
remove a region click with left mouse button somewhere into the region to be removed;
the region will then be removed by setting all its pixels to a value of zero
join two regions press 'Shift' and left-click into the first region to remember its label, then release 'Shift' and left-click into the second region;
the label of the first region will be transferred to the second region; black pixels between both parts can be removed afterwards using the "Fix borders" button
draw a new boundary keep left mouse button pressed and drag the mouse, this will add a one-pixel wide free-hand line to the label image; note that you can configure the line width via the 'Options' button
fill-in a missing region
(new in version 2.1.2)
press 'Ctrl' and 'Shift' and left-click into the missing region area; be sure that the area is fully enclosed by other regions, open background sections cannot be filled properly


The buttons in the editor window have the following functions:

Button Function Keyboard Shortcut
Next save current changes and jump to next image Alt + n
Skip cancel current image, don't save anything and jump to next image Alt + s
Contrast adjust labels to optimize visibility, i.e., shift labels into upper third of gray-scale range with maximal distances between them Alt + c
Relabel relabel the image, i.e., make sure that every region gets a unique label Alt + r
Fix Borders remove all background pixels with more than one foreground label in their neighborhood; the size of the neighborhood to be considered can be configured via the 'Options' button, and sometimes it is necessary to re-run the command several times to remove all non-border pixels between two region parts Alt + b
Fill Holes fill background regions enclosed by a single region with the region label Alt + h
Undo undo the last action; IMPORTANT: only the most recent action can be undone! Alt + u
Options opens a window with configuration options, e.g., for setting the line width in drawing or the maximum boundary width Alt + o
Quit save current changes and quit the editor window Alt + q



Hint: The Undo function has currently only a history of length one. If you accidentally deleted too many regions or want to undo more than one action, either run the editor once again on the complete directory or copy the corresponding image to a separate folder and run the editor on that folder. Results of a former run will be overwritten without further inquiry.

Updates



April 2021

  • The editor got some updates in MiToBo 2.1.2 and MiToBo-plugins 2.1.2, respectively. Background regions can now be filled-in, more labels are supported and some parameters be configured by the user.

January 2019

  • The editor got a comprehensive update in MiToBo 1.8.13.1 and MiToBo-plugins 1.8.13.1, respectively. There is now much more editing functionality available, e.g., filling holes or drawing of free-hand lines.

March 2018

  • The LabelImageEditor has been released in MiToBo 1.8.8 and MiToBo-plugins 1.8.8, respectively.

Technical Details



The label editor relies on analyzing and manipulating the grayscale values of segmented regions in a given image.
In detail, the most important functions are implemented as follows:

  • deleting regions:
    the grayscale value at the mouse-click position is determined and all pixels in the image sharing this value are set to black
  • joining regions:
    upon clicking into the first region its grayscale value is internally stored;
    when the user clicks into the second region which is to be merged with the first one
    the grayscale value at the click position is extracted and all pixels with this value are relabeled with the formerly stored grayscale value of the first region;
    important: after doing the join both joined regions share the same grayscale value, however, are not connected to each other; hence, to get a valid label image again you should run the "Fix Borders" function
  • relabel image:
    the image is thresholded with a threshold of zero to get a binary image of background and labeled regions;
    then a sequential component labeling is run on the image which assigns consecutive grayscale labels to all connected components in the foreground of the image
  • adjust contrast:
    the image is relabeled, however, the target labels are equidistantly spread over just the upper two third of the available grayscale range;
    the underlying motivation for this is given by the observation that low grayscale values are hard to distinguish from the background, thus, brighter labels improve visibility
  • fix borders:
    this functions seeks to eliminate borders between regions with identical labels which may for example occur after joining two regions;
    for removing borders for each background pixel in the image the grayscale values in a 7x7 neighborhood of the pixel are analyzed,
    and if there is only a single grayscale value found and more than half of the analyzed neighborhood pixels share this value, the corresponding background pixel gets the value as well;
    the neighborhood is currently fixed to 7x7, thus, borders which are wider cannot be closed, and it has to be noted that sometimes the function has to be run several times to remove all pixels belonging to a single border