4
$\begingroup$

I am trying to build a system to detect bullet holes in paper. I've read many StackOverflow threads but not a single one satisfied me. Current methods, which I am using, are good but not perfect and not fully reliable. It often misses couple of shots or has problems with calibration when live testing.

System requirements:

  • Realtime detection,
  • Has very low input lag between bullet making hole in paper and detection,
  • minimal screen size - 2.5m x 1.5m,
  • works with .22, 9x19, 5.56x45, 7.62x39 etc. ammunition,
  • returns X and Y coordinates of detected bullet hit.

What I am currently using:

  • Arducam IMX477 12,3MPx HQ + usb 2.0 interface to simulate webcam,
  • Camera lens - which is terrible :P but i needed zoom function, visible light filter on camera and IR lights - to be independent from enviroment lights.
  • Python detection:
    • Image subtraction - monitoring changes,
    • Thresholding an image from camera - to eliminate noise,
    • Detection - OpenCV.findContours or analysis of numpy array in search of high values in some region of image.

Camera is located 4.5-5m from screen. I'm detecting black areas that appear on following frames by watching for changes between these frames and finding maximum if some big spike (2 times greater than noise) is detected.

Here is some code example and nice plots that I've made:

import cv2
import time

video_stream = cv2.VideoCapture('C:\Projekty\HoleDetection\data\IRNEW\WIN_20220204_14_26_23_Pro_Trim.mp4')
prev_frame = None
i = 0
start = float(time.time())
while video_stream.isOpened():
    ret, frame = video_stream.read()
    if not ret:
        break
    frame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

    if prev_frame is not None:
        subtracted_frame = cv2.subtract(prev_frame, frame)
        # cv2.imshow('subtr', cv2.resize(subtracted_frame, (1280, 720)))

        ax_0_max = subtracted_frame.max(axis=0)
        ax_1_max = subtracted_frame.max(axis=1)
        curr_max = ax_1_max.max()

        if prev_max * 2 < curr_max:
            print('Detected shot')
            x = ax_0_max.argmax()
            y = ax_1_max.argmax()
            print(f'X: {x}, Y: {y}')
            # keypoints.append(cv2.KeyPoint(float(x), float(y), 10))
        prev_max = curr_max
    else:
        prev_max = frame.max()
    prev_frame = frame
    # frame = cv2.drawKeypoints(frame, keypoints, np.array([]), (0, 0, 255), cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)
    # cv2.imshow('im', cv2.resize(frame, (1280, 720)))
    i += 1
    if cv2.waitKey(1) == ord('q'):
        break
end = float(time.time())
cv2.destroyAllWindows()

print(f'Time {end-start} for {i+1} frames. FPS: {(i+1)/(end-start)}')

Whole movie analysis:

Whole movie

Keyframe where first bullet hole appeared:

First shot

This is how it looks like from camera perspective. Little bit blurry and fish lens effect. It's not ideal. It works almost perfect (or maybe just good) with 9x19 ammunition, but has problems with smaller calliber (.22, .223Rem, 5.56x45) - changes are less noticeable because of smaller holes.

Notice one unfortunate thing - I can't put anything behind the screen because it's shelled with live ammunition.

I would like to hear from you how to improve this system. Maybe use some kind of sensors that are capable of detecting bullet holes (if there is such a thing) and more reliable than current methods or maybe there is some better python method available

$\endgroup$

3 Answers 3

4
$\begingroup$

I can't really comment on the machine vision part, other than any question that asks "How do I do <some signal processing task> in <some language>" is fairly naive. The way you do any signal processing task in any language is to first learn how to do the signal processing in math. Full stop. Then go and implement that in the language of your choice.

All the major computing languages share all the major math, signal processing, and machine vision libraries. And for the ones that don't, there's usually a way to put a thin wrapper around the library of your choice and use it anyway. So if you want to do it in Python vs. C++ or Rust or Fortran or whatever -- that's just a detail. The important part is how you do it in the first place.

For your problem at hand -- I think you just need more contrast, and possibly more pixels. Work on your lens, your camera, and your lighting to make the bullet holes more distinctive. If those are 9mm holes and you want to go even smaller, then you need to have a high pixel count camera, and you need optics on it to match. You probably want your bullet holes to be at least five pixels in diameter, and more is always better.

You may want to rethink your "no backlight" rule. E.g., if you set the camera up to view the front of the target from the left, out of range, then couldn't you set up lights behind the target and to the right, so they shine right into the camera but are out of the path of the bullets?

$\endgroup$
3
  • $\begingroup$ GIGO is always a problem with image processing problems. Algorithms can do much, but if you don't start with decent input, it's hard to make it decent via algorithms alone. $\endgroup$
    – Peter K.
    Commented Mar 25, 2022 at 17:57
  • 1
    $\begingroup$ And it's hard to just look at something and decide if it's good enough. Our ancestors were experimenting with neural-net based vision 500 million years ago, but we've only been migrating that to machine vision for the last 50 or 60 years. Thinking that some machine vision task will work OK because we can see it often leads to disappointment. $\endgroup$
    – TimWescott
    Commented Mar 25, 2022 at 19:29
  • 3
    $\begingroup$ Yes. A guy I know who researches (deeply) into learning systems calls NNs "neutral notworks". :-) $\endgroup$
    – Peter K.
    Commented Mar 25, 2022 at 19:38
1
$\begingroup$

You will get better results using an infrared camera and lock-in thermography. The infrared reflection of bullets is much more different than the possibilities with 'normal' light. So you will get more details.

$\endgroup$
1
$\begingroup$

A couple of comments from a practical aspect, I can't comment on the electronics side - close grouping of shots will be an issue unless you an automatic paper roller feeder system which "closes up" the paper periodically. Other substrates will be an issue in the long term such as latex rubber anti splinter screens.

Placement of the camera needs to be considered - certain target systems do not have the space or orientation to get the camera in place. It will all depend on the training being undertaken.

Thermal systems have their issues with accuracy, again down to close grouping on rapid fire practices.

As a benchmark - gold standard accuracy is half a calibre i.e. approx. 4.5mm for a 9mm round. This is the best that that a conventional acoustic target will perform at.

You could consider a machine learning "light screen" in front of the shooting surface.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.