The Particle Tracking Resource Page

How to Track Particles Using StAT

Graham Milne, July 2008

gmilne@u.washington.edu

Introduction

This document describes how to set up and use the StAT particle tracking package for LabVIEW, a set of programs I developed during my PhD at the University of St Andrews. The package is built upon National Instrument's IMAQ Vision software toolkit, and requires a copy of LabVIEW and NI Vision to run.

These programs were created to study the behaviour of silica and polymer microspheres viewed through a microscope, however the software has been successfully used to track biological cells as well as macroscopic objects in conventional video sequences.

In the colloidal community, a large number of particle tracking solutions have already been produced. For certain experiments, these alternative algorithms will produce better results than are possible using the StAT package. For more details on alternative particle tracking programs see the Particle Tracking Resource Page at http://faculty.washington.edu/gmilne/tracker.htm.

We chose to develop a package in LabVIEW due to our need to be able to simultaneously track multiple particle species, something that is difficult to do, if not impossible, using established solutions. StAT uses National Instrument's proprietary feature tracking tools, which provides the multi-species tracking capability of Additionally, by using the LabVIEW environment the package can easily be integrated with laboratory equipment for real-time experimental analysis.

StAT comes in two principle varieties, the standard program which is designed to process pre-recorded videos or image stacks, and the real-time version, which is designed to be integrated with National Instruments video capture hardware for real-time analysis in the lab.

Requirements:

You will need a PC running Windows XP or Vista. Because of file name/path issues, this program cannot (to my knowledge) run effectively on Max or Linux systems, although remedying the associated problems in the code may not take you too long. You will need to install a copy of LabVIEW 8.2. In addition, you will also need to have a full copy of the NI Vision add-on, version 8.0. I am working to make the package backwards compatible with earlier versions of LabVIEW.

Particle tracking can require a lot of RAM due to the large arrays of data that can be produced, so it is a good idea to have a liberal amount of memory on your computer if you plan to track long sequences with a high density of particles. Additionally, particle tracking works best if your images are uncompressed, so it is probably a good idea to have a fair bit of hard disk space fitted.

Section 1. Using the standard version of StAT

The top-level program in the package is simply called "StAT.vi" and can be found in the StAT.llb library. In Fig. 1 you can see what the front-end of the program looks like.

Fig. 1. Screen capture of the front-end of StAT. Clicking on the image will take you to a full-size screen grab.

1.1 Loading your movie or image stack

On the StAT front panel, below the StAT logo, you can see the 'File Path Information' section. This is where you need to enter the location of your file(s) on your hard disk. Currently StAT supports avi movie files and numbered bitmap image stacks. Support for tiff and png image stacks will be added soon. Handling mpeg movies with LabVIEW can be problematic and this feature has currently been disabled. In the meantime, there are a large number of widely available products such as Quicktime Pro or Adobe Premiere that can be used to convert your video into a format that is compatible with StAT.

1.1.1 Numbered bitmap sequence

Fig. 2. To load a numbered bitmap image stack, enter the location of the first image of your stack in this box.

StAT currently supports 8-bit greyscale (monochrome) bitmaps only. If you happen to load an RGB sequence, it will attempt to load the red portions of each image. The numbered image sequence, or stack, must be numbered appropriately in order for StAT to work. Image filenames should be zero-padded, as shown below in Fig. 2.

Fig. 3. For StAT to work properly, your image stack filenames MUST be properly zero-padded as shown in this image.

Uncompressed bitmaps are optimal for tracking but they do tend to take up a lot of space on a harddisk. Typically, a 640*480 monochrome bitmap weighs in at about 300kB, so you can see that for stacks containing several thousand images, quite a lot of storage space is required.

1.1.2 Tracking from avi files

Fig. 4. To load an avi file click the 'avi' tab and type in the location on disk of the movie you wish to process.

StAT will run happily with most common types of avi file. LabVIEW will automatically take care of issues regarding codecs, however in general I recommend you use a generic codec (eg cinepak) and do not compress your avi files during the recording of the video. To maximise the accuracy of the tracking it is best to have original images. Compression can introduce artifacts and reduces the effectiveness of sub-pixel measurement.

Again, your avi files must have 8-bit depth. For colour movies, StAT extracts the red colour plane only. This restriction on bit depth is due to LabVIEW's feature matching algorithms and cannot be avoided.

Once you have entered your video file location, there are several options you can set on the front panel of StAT to fine-tune tracking performace before you run the program.

1.2.1 Generating Test Images

Before you begin trying to analyse your experimental image sequences, it is useful to familiarise with the various aspects of using StAT. To assist you, I’ve made a LabVIEW program that will generate a series of artificial ‘microscope’ images, which you can then play with in StAT until you understand all of the basic features.

To run the tester, open the VI ‘StAT_Tester.vi’ which is located in the StAT.llb library.

Fig. 5. The front panel of the program that generates a bitmap stack of images for testing StAT. Clicking on the image links to a full-scale screengrab.

The program simulates (crudely) microscopic images of polymer microspheres moving freely with Brownian motion. You can vary:

a) The number of images in the sequence.

b) The number of particles.

c) The amplitude of the Gaussian white noise added to the image (to simulate CCD shot noise)

d) Maximum limit on random particle movement between frames (pixels).

StAT_tester.vi has its own built-in particle template but if you want you can create your own and load it in on the front panel.

Have a go! To get the program started, hit the white arrow, , in the top left-hand corner of the front panel. This is LabVIEW's general 'run' button. Once you have generated your test image stack, you can then start playing with StAT.

Type the location of the first file in your image stack into the bitmap file path tab on the StAT front panel. If you want you can use the browse button, , to navigate to your file.

Make sure the 'New template(s)?' button is set to 'on': (illuminated). When you hit the run botton, , StAT will launch. The first thing it will do is throw up a window asking you to define a 'template image' for your chosen particle species (Fig. 6.)

Fig. When you run StAT, if you have selected to define a new template, the program will display a window asking you to draw a box around a typical particle (click to open a full-size screen grab).

Fig. While processing your image sequence, the program will highlight the tracked spheres with red markers. This way it is easy to spot errors (click for full-size screen grab),

Fig. After processing the collected data, the program will display a plot of the measured trajectories (click for full-size screen grab).

1.2.2 Start and end frames

Fig. 6. You can select a section of your video for tracking by entering the start and end frames in these boxes.

You may not necessarily want to track your entire video sequence. With these two boxes, you can select a specific part of your video sequence.

If you haven't yet defined your particle templates (see later), you MUST chose your 'starting frame' so that it contains a typical particle. This is because the program will ask you to define your target particle(s), and it will use the frame number you define here during this process.

The program will automatically detect the highest frame number in the sequence stored on your hard drive. If you type in a value for 'End frame' that exceeds the actual highest frame number in the real sequence, the program will modify the 'End frame' value to suit. As a consequence, if you are not sure how many frames are in your sequence, you can just type in a really big number in this box and the program will take care of sorting out the correct numbering!

1.2.3 Number of particle species

Fig. 7. An typical image from an experiment conducted at St Andrews. Polymer and silica spheres of similar radius (but different refractive index) are suspended in water. We needed to be able to track the two species of microsphere independently. With conventional colloidal tracking approaches, this would be difficult, but with feature matching it is easy.

As stated, the chief motivation for using a feature tracking approach rather than more conventional centroid-tracking methods is the capability of tracking multiple particle species. In the case of colloidal microspheres, imaged using conventional illumintation (rather than fluorescene), appearance is determined primarily by the spheres' size and refractive index (figure 7). The default number of species in StAT is 1, but you can increase it arbitrarily. Of course, program execution time increases approximately linearly with the number of species you ask it to search for.

1.2.4 Tracker Settings

Fig. 8. Tracker core settings.

Generally, when tracking colloidal microspheres, we do not need to worry about particle rotation. The ability of the tracker to measure rotation as well as lateral displacement has been retained for future applications, but we can ignore it here. Generally, you will want to set "Match Mode" to "Shift Invariant".

By FAR the most important parameter here is the "Minimum Match Score". When the tracker searches the images for your particles, it identifies possible candidate particle locations. These are awarded a score out of 1000, based on the candidate location's mathematical similarity to a particle template that you will define when the program launches. Due to noise, focus drift and a host of other factors, 'correct' particles will often return a 'match score' quite a bit less than 1000. On the other hand, artificats such as uneven lighting, dirty optics or ghost images can generate locations in the image that return a non-zero match score.

You need to set a value that allows you to find your particles accurately while avoiding imaging artifacts. Typically I find a score of 650-700 works quite well for most of my experiments, although the score you should set will be dependent strongly on the nature of your experiment.

Obviously, if you are producing a lot of erroneous matches, try raising the Minimum Match Score. If you aren't tracking anything (including the particles you want), try lowering it a bit.

Fig. 9 . Trajectory determination options.

'Minimum consecutive count' refers to the minimum number of consecutive frames a particle must be tracked for before it is classed as a true particle. Trajectories that occupy sequences shorter than this minimum value are discarded. This helps to restrict the influence of ghost matches on the final data.

When a particle is lost, which can be due to a multitude of reasons (such as occlusion, out-of-focus drift, or uneven background illumination) the program will keep its presence in location and try to look for it in consecutive screens. Once the particle has been lost for more than a certain number of frames, defined by the 'lost forever' parameter, the program will stop looking for it. This is an important feature for saving computation time and memory in long and busy sequences.

'Max drift distance' refers to the radial range in which the program looks for a match in the next frame for a given particle. It is measured in pixels and should be chosen by the user so that the particle can easily be caught, without making it too large so that the 'lost particle' feature fails to execute. The choice of range depends strongly on the experiment. In particular, it will depend on the frame rate of the video capture and the nature of the motion of the particles. Particles moving in an applied fluid flow will need a larger search range than those moving due to Brownain motion.

'Directed search' relates to the manner in which the program searches for lost particles. With the feature turned off, the program looks for the lost particle at the location it was lost at. With 'directed search' enabled, the program extraploates a polynomial fit on the previous known locations to guess a possible location for the re-appearance of the particle. This is useful, for instance, in cases where the particles are moving due to a microfluidic flow.

FInally, you can choose two different principle tracking regimes. The first, 'simple tracker', is ideal for tracking particles that do not change appearance significantly throughout the image sequence. For instance, colloidal microspheres moving in a 2D plane (lying on the bottom of a microscope slide, for example.)

If the particle you are trying to track has an appearance that evolves as the image sequence progresses, you can try using the 'advanced tracker' option. Currently, this option is only effective for tracking one particle at a time, although it can also be used for tracking particles that evolve in a similar manner - for instance particles on a 2D plane that gradually move in or out of focus due to mechanical slip of the microscope stage.

The advanced tracker works by taking the highest scoring match in a particular frame, and using this to define a new template. I have included sub-pixel resolution re-sampling to make this as accurate as possible, however it is inevitable that during this template re-defining, a slight error will always creep in. Over the sequence, this will result in 'template drift', which results in a template that is no longer centred over the particle of interest. This effect is illustrated below in figure ??

FIGURE TEMPLATE DRIFT

If you scroll down on the StAT front panel, there is a variable, Frames between template updates. This variable controls the frequency with which the template is updated. If you use a lower frequency (higher number of frames), the impact of template drift will be reduced. The number you choose here will depend on how rapidly your particle's appearance evolves when compared with the video frame rate - the number should be chosen low enough so that the particle is not lost due to evolving too much in appearance from the current template.

Fig. 11 . Scroll down for some extra options. (click for full-size screen grab).

1.2.4 Output Settings

Fig. 11 . Output settings. These control the nature of the output plot.

These buttons control the nature of the output plot generated by StAT. It is likely that you may not be happy with the aesthetics of the plot produced by the main StAT tracker program. As an example, using the randomly generated colour scheme can produce plots that are quite ugly.

Rather than running the entire tracking package multiple times, once you are happy with the tracker performance, you can use the post-processing tools to clean up the final plot for publication and generate movies for presentations. These tools are discussed later, in this section.

Figure 12 shows the different variations of plots available. Note if you select 'multicoloured' and have only one particle species, the program generates a different random colour for each particle. If you have more than one species, each particle in a given species will have its trajectory plotted in the same colour.

If Labelled start/end points? is enabled, the program will plot large coloured spots on the ends of the trajectory. The colour of the spots is significant. There are four possible colours, with the following meanings:

Green - start point

Red - particle was lost before end of sequence

Blue - particle was tracked until the end of the sequence

Yellow - particle collided with or was occluded by another particle

If Blank background on plot? is enabled, the trajectories will be plotted on a white background, otherwise they will be plotted against the frame identified by the number entered under Starting frame.