0
$\begingroup$

I have a time lapse movie from a specimen and would like to trace the cell wall movements, but I'm not sure how to. By eye, following the cell wall is fairly easy but it seems computationally quite difficult. I was looking into Caffe and deep learning networks, but the tutorials seem more about finding a pattern in images, not so much about tracing data within a movie. If someone can provide me some pointers on how to go about my data that would be great.

Here is a link to an example movie: Example Movie

I'm only interested in the movements of the outer wall, as indicated here:

Wall edges

$\endgroup$
  • $\begingroup$ Can I please ask you to grab a still from that video (even if it is via screenshot) and annotate it with the areas you are trying to track? I suppose it's the semitransparent "groove" that appears to be dilating and contracting periodically (?). But would it be th outer wall? The inner wall? The whole thing? It is semitransparent and therefore might not be straightforward but you could try tracking it with a normalised cross correlation based method (not perfect, but it is a start). $\endgroup$ – A_A Sep 13 '16 at 23:09
  • $\begingroup$ Edited. The outer wall is what I'm interested in. $\endgroup$ – Geo Vogler Sep 13 '16 at 23:30
1
$\begingroup$

Deep learning might have something to offer in the long term but at the expense of training the network with very detailed information about what constitutes an "outer wall". A deep learning network could then iterate all frames and markup those blocks that are very likely to constitute the "outer wall".

In the short term, a technique based on normalised cross correlation might offer more satisfactory results. The idea in this case is simple: Define an area of the image (let's call it a Region of Interest (ROI)) that you want to track and then try to "find" that area within the next frame and the next frame and so on.

The primary assumption here, of course, is that the ROI doesn't change significantly from frame to frame. And in this particular case, this takes us back to what exactly does an "outer wall" mean?

Here is an example of tracking a point on the "outer wall":

Using Blender to track the outer wall

And here is the data of the track (frame num, X, Y).

This is the best tracking I could do out of approximately 8 attempts (I will come back to this) and it was done using Blender.

Blender is using the normalised cross correlation method to track "markers" from a video feed. Depending on the type and number of markers tracked, you can do a number of things such as stabilise video or solve for the camera movement and position in order to synchronise a "virtual" and a real scene.

In this particular case, I am only tracking Location. I could track Location, Rotation, Scale or any combination of those but I figured that this phenomenon is happening more or less on a narrow depth of field. So, just tracking for location will not distort the results.

The tracked coordinates are the green and red timeseries, green for a "Y" coordinate and red for a "X" coordinate.

As you can see, it's not a simple linear movement. The tracked marker seems to be prescribing a tiny little ellipse around its position.

And this brings us to the main "problem" you are going to have to deal with: WHAT is an outer wall? Here are a few observations:

  1. The "outer wall" displacement varies across it's length. The "outer wall" of this membrane (?) is roughly parallel to the width of the video. So, the recorded displacement, really depends on WHERE you are going to track. If I tried to track somewhere else along the length of the track I would have had a different profile.

  2. The "WHERE are you going track?" question is made harder, because the "outer wall" is semi-transparent and furthermore, its transparency throughout it's "pumping" cycle seems to depend on the phase of it. So, there are points in the cycle that there are very clearly defined edges (which is good for accurate tracking) but gradually, these points vanish and introduce drift to the tracking. Gradually, the marker will "slip" from the target it is supposed to be tracking and start tracking a slightly adjacent image patch that happens to look similar.

Therefore, what I would suggest is to set-up a similar script that opens a video, establishes a marker patch and then tracks the patch frame-by-frame, using information that is readily available in many different scientific platforms (for example, for another example).

BUT!, having now established tracking, you will have to go ahead and manipulate the successive frames of the video in order to preserve the features. This could be a simple contrast enhancement with a steep window, or a histogram equalisation or a normalisation step or possibly the addition of a high pass filter to make the edges more pronounced and improve tracking.

Finally, if you would like to replicate this tracking, using your video, you will need:

  • Blender
  • Some demo on tracking. This one is very good.
  • Specific parameters
    • This tracking was produced with: Tracking Settings(Preset, Blurry Footage, Motion Model: Loc, Match: Previous Frame). You can match either on a frame-by-frame mode, or to a keyframe, in which case, you get displacements relative to that particular frame. Might be useful for you if you want to measure everything with respect to a particular point in the cycle.
  • A script to extract the tracking data to a CSV file. Like this one.

(Of course, you can track as many markers as you like and their data will be put in a separate file).

Hope this helps.

$\endgroup$
  • $\begingroup$ Excellent comments! I will recapitulate these steps for my own understanding. Also, your remarks are spot on: it is difficult to identify the edge in the expanded (dilated) state, and the diameter differs depending on the X position. My goal here is to identify as many points along the edge as possible, to generate an average diameter. This is tedious when done manually, and also subjective. Hence the demand for automatic this. The issue really seems to create the best marker patches for edges (where the deep learning would come in). $\endgroup$ – Geo Vogler Sep 14 '16 at 23:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.