This page contains guidance about the partial annotation process on APEER as well as tips and tricks on how to make the most out of your annotation time

29 seconds Partial Annotation summary

  • Some image regions can be left non-annotated
  • Only annotated pixels are used for training the ML segmentation algorithm
  • The background needs to be explicitly annotated
  • Objects need to be fully annotated (no "scribble" annotations)
  • Ensure that individual objects are separated by at least one pixel of background class (to separate them with instance segmentation)
  • The more accurate the annotations, the better the resulting model and segmentation
  • Borders between objects or between classes are the most important locations to annotate accurately

The Details

What is Partial Annotation?

On APEER we decided to implement partial annotation because we are used to microscopy datasets with a large number of objects per image. In such cases it's just not efficient to annotate all objects of a single image for ML training. It's rather more beneficial to include some objects from different images, which span multiple experiments. This approach allows to include objects of all sizes, shapes and colors which you expect to segment. Such a dataset allows us to train a much more robust segmentation model.
To distinguish between non-annotated regions and regions which don't contain any class that you want to be segmented we introduced the background class. The background is a special class which tells the neural network, that the pixels it covers are not occupied by any of the classes that you want the model to recognize. It is important that you provide background annotations if there are any regions which you don't want to be segmented (most common use case). Non-annotated pixels are completely ignored during model training, thus allowing you to partially annotate.

How to annotate partially

We have made partial annotation efficient by automatically cropping the background class wherever it overlaps with any other class. This process allows you to annotate efficiently:
  • annotate several objects in a region (with brush tool)
  • draw background class around the annotated objects (with brush tool and "autofill holes" active), covering as much background as possible
  • the background class is automatically cropped where other annotations exist
  • ensure that the background class doesn't cover any non-annotated objects
  • if you annotate all objects in your image, you can draw one big background annotation around the entire image to crop all objects at once (with brush tool and "autofill holes" active)
  • select image from list of images (right panel) for training
If there are any regions where you want to make sure that the algorithm interprets them as background, it can make sense to create additional background annotations. These are examples of regions where additional background annotations can help the algorithm to learn how to segment more precisely:
  • narrow background lines between objects helps separating them
  • background annotations on image features which look similar to your classes helps preventing them from getting (wrongly) segmented


  • Each class needs at least annotations which cover 5000 pixels of that class. This sounds a lot, but its for instance just 5 annotations of size 10*100 pixel. Expect to annotate significantly more than that to train a robust algorithm (in hard cases a few hundred annotations per class).
  • The background class does not require any annotations, since there are some use cases which don't require any background. An example of a use case without background is a microscopic image of a composite material comprising two phases. If both phases need to be segmented, both need to be annotated and there is no background. Such cases are rare and in most cases background annotations are required.

Best Practices

  • The APEER ML segmentation algorithm is powerful, but requires a significant number of annotations to learn all the different sizes, shapes and colors that your classes come in. Segmentation tasks vary significantly in complexity. It is hard to predict how many objects must be annotated to achieve the desired accuracy and robustness. From our experience you should be ready to invest the time to annotate several hundred objects. But don't try annotating the whole dataset at once. Run your first training after creating about 50 annotations per class.
  • Most likely you will not produce a highly accurate and robust segmentation with your first training run. Don't give up! Developing a high-performance algorithm is an iterative process.
  • After your first training create more annotations in regions of low performance and iteratively improve your model (see Improve your segmentation)
  • Annotate accurately. You are the teacher of the ML segmentation algorithm and you teach by providing annotations. If you create inaccurate annotations, the algorithm will learn to segment inaccurately.
If you have any questions about the ML Toolset on APEER please reach out to us at [email protected]
Last modified 5mo ago