3
$\begingroup$

I came across this demo by Fox sports:

https://youtu.be/N9SxFf5WCb0

Where they show a player's view in a 360 space. Very smooth and nice.

I was wondering how do they achieve it without any cameras attached to the player?

One approach that came into my mind was using a multi-camera setup and convert 2D images to 3D using a stitching mechanism as a post processin. Is it what they do?! What are other approaches that we can do it?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

It seems to be at a stadium which uses 12 cameras and Replay-Technologies special signal processing called FreeD: https://www.youtube.com/watch?v=N1kgt2VDjdM

The image you show seems to use 2-3 cameras, but the technology can work with multiple camera combinations from the 12 in 360'.

It's a particularly stunning real-time effect, and there are also very many 2D to 3D apps available for example in after effects and on google scholar.

Here is a list of replay-technologies videos: https://www.youtube.com/results?search_query=replay+technologies+freed

https://www.tvtechnology.com/news/bringing-the-matrix-to-the-nfl

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks. Looks like they use it only on still images. Do you have any ideas how we can do it (just a single still image) differently? I guess we can't avoid the 2D to 3D stitching part. Right? $\endgroup$
    – Tina J
    Aug 21, 2020 at 15:09
  • 1
    $\begingroup$ In realtime and using algorythms? I've seen a couple of realtime 2D into 3D effects although they can pan the camera by only about 5 degrees. Stereoscopic imaging is what it is, some super expensive camera tried to use one lense and light phase but it's a bit of a weird forgotten technology. $\endgroup$
    – bandybabboon
    Aug 21, 2020 at 19:16
0
$\begingroup$

What makes it interesting is that it is not the bullet-time effect of Matrix vintage, since the view is centered on the subject. In other words, an attempt is made to make it look like the viewer is at the center looking outwards (as if it was standing there wearing a GoPro), rather moving around the center in succession.

Of course it cannot be too accurate - because the cameras are physically far away and the scene content occludes some of the view. But if people get used to it, I don't see it that far off when players will have helmet-mounted cameras.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Yeah exactly. With multiple cameras, I was expecting the player itself would cause occlusion. But here is not the case. How was it possible?! $\endgroup$
    – Tina J
    Aug 22, 2020 at 18:12
  • 2
    $\begingroup$ Thinking aloud: stereo (or space carving, or a combinatin of both, or even coaxial lidar) gives an idea of the depth and extent of the subject. Space carving also individuates the unoccluded portions in each nearby view that are occluded in current view. Then it's a matter of infilling. Or maybe these days they just trained a gigantic deepnet to produce something that looks reasonable on these data. :-) $\endgroup$
    – Francesco Callari
    Aug 22, 2020 at 21:39
  • $\begingroup$ Ok! Too technical for me to understand! haha $\endgroup$
    – Tina J
    Aug 23, 2020 at 2:33
  • 2
    $\begingroup$ Apologies, wasn't sure at what level to address. If you are interested in the 3D geometry aspects of computer vision, a delightfully readable introduction is in A. Criminisi's Ph.D. thesis on visual metrology: microsoft.com/en-us/research/wp-content/uploads/2016/02/… . A more in-depth treatment is found in Hartley's and Zisserman's classic: b-ok.cc/book/998045/419e4f . "Space carving" is a less-known term (though the basic idea is quite simple), a classic introduction is in Kutulakos and Seitz cs.toronto.edu/~kyros/pubs/00.ijcv.carve.pdf , $\endgroup$
    – Francesco Callari
    Aug 23, 2020 at 20:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.