How to reliably detect the state (opened or closed) of sliding doors from a statically-positioned video stream

Question

I have a mounted (static) camera that has a view of a set of sliding doors. Since doors are rectangular in nature, and they don't have many degrees of freedom, this seems like a problem that should be relatively easy to solve reliably. Here are some sample views (the door outlines in red I drew for illustration purposes, they are not detected by an algorithm):

First, here is a view of the sliding doors being closed:

Next, here is a view of the sliding doors being opened:

Finally, here is a view of the sliding doors opened but with occlusions:

My original thought was something around a hough transform, but I feel like there is likely a better solution to be had, given the knowledge that the doors can really only slide along one axis.

Reiterating my question, how can one reliably detect whether sliding doors are opened or closed from a video stream like the one in my images?

Answer 1

To better deal with occlusions, my idea would be to separate this problem into detecting if:

1. the 1st door is in position fully opened (1)
2. the 1st door is in position fully closed (2)
3. the 2nd door is in position fully opened (3)
4. the 2nd door is in position fully closed (4)

To tackle either of these problem, I would apply the following algorithm let's say with (1) :

1.1 Take a picture from the camera as a background image.

1.2 Crop it to keep only the part where the 1st door should be when it's fully opened, this is easy as the camera is static.

1.3 Do the same with the current image feed from your camera.

1.4 Calculate the absolute difference of those two croped images.

1.5 Calculate the average value of that absdiff (optionnaly divided by the surface of the croped part to normalize it).

1.6 Decide from that value if the door is fully opened or not ( low value mean it's probably opened, low value mean it's probably not), using a threshold value for example.

For steps 1.1-1.4, especially for updating the background image, you could use a different background subtraction algorithm.

Once you can correctly decide, without much occlusions, if the statements (1-4) are true or not, you could, for dealing with occlusions, this works best assuming that both doors are always silmutaneously in the same open/close state:

5.1 keep the (normalized) absolute difference average of each (1-4) part

5.2 use some algorithm to decide if the doors are both open or closed

I'll illustrate step 5.2 with your last image as example, lets say 1st door is the one in the bottom of the picture (the only one we see well):

• We have a high absdiff average value for both "Door 2 fully closed" and "Door 2 fully opened" as the man is in front of both places. So both (3) and (4) are said as false(or true depends how you implement it), which is unlikely.

• We have a high absdiff average value for (2) but a low one for (1).

-> So a possible algorithm for 5.2 could be using the difference between the absdiff average value from (1) and (2), and (3) and (4) as a threshold.

I think that, plus some noise removal techniques, will do the job.

Do not hesitate to ask me in comment if my solution doesn't appear clear to you.

Edit: didn't saw the date of the post, I don't know why it showed up so late in my feed, I let this answer in case it'll be useful to someone else.

Answer 2

I think tracking motion of something like corner or handle of the window would work. Consider following procedure: 1. Track the corner of the windwo 2. If position of the corner changes more than X pixel(you determine X), change the STATE I suggest to use object tracking algorithms. I think the best candidate point would be corners of the window. For corner tracking the best algorithm I know is KLT (OpenCV implementation) and Kalman filter. I think the following algorithm would work, and since these algorithms updates frame by frame, they can handle environmental changes of light and obstacles.