13

Note: Please see comments below @bobthechemist's answer for why question was edited.

I've made a video of two teams playing soccer. Now I would like to translate each soccer-player to a (x,y) coordinate and follow them during the match. I know that Mathematica has some image feature tracking and am looking for a suggested workflow for image analysis and an example of how to apply it to a series of images.

Has anybody experience with video and image recognition? I've made a video of two teams playing soccer. Now I would like to translate each soccer-player to a (x,y) coordinate and follow them during the match.

J. M.'s missing motivation
  • 124,525
  • 11
  • 401
  • 574
Michiel van Mens
  • 2,835
  • 12
  • 23

2 Answers2

29

Well, I suspect this question will get closed because it is a bit broad, but I've played around with image tracking and thought I'd show what I've done in case it's helpful.

I was interested in learning some physics, and have the following video:

enter image description here

To do the tracking, I smoothed out the images and converted them to black and white, which allows for the colored balls to who up on a black background. Note that I have already defined the symbol images which was the series of graphics that made up the video.

imagesbw = Binarize[MedianFilter[Image[ImageData[#][[All,All,1]]],4]]&/@images;

enter image description here

We lose the black ball, which is unfortunate, but my "physics question" was to see if I could predict the movement of the missing black ball from the information that is available. I started by collecting the position of the three visible balls using ComponentMeasurements which in my case finds four balls, one of which is the one in the upper right hand corner that doesn't move, so I'll ignore it. I can then get tracking information from the ComponentMeasurements output. Note I did play around with ImageFeatureTrack, but I didn't have much luck with it.

movement = ComponentMeasurements[#, "BoundingDiskCenter"] & /@ imagesbw;
tracking = 
 Graphics@Riffle[{Red, Orange, Blue, Red}, 
   Point[Cases[# /. movement, x_ /; Dimensions[x] == {2}]] & /@ 
    Range[4]]

Mathematica graphics

There's some messiness in the data, so the line above deletes some of the points. Now I want to predict where the 8-ball is heading from the information in the point plot above. I approach this problem by assuming conservation of momentum assuming an elastic collision, ignoring friction and acceleration. There's a reason why my name isn't bobthephysicist.

There's probably no real reason to go through my ugly code to solve this problem, especially since I did so by surfing around Mathematica.SE so the solutions are already here. At the end of the day, I get the following graphic, which is a pretty decent fit, although I will admit I did need to incorporate a fudge factor in to the solution. My assumptions, as well as my not knowing the weights of the four objects, makes this task a bit daunting.

enter image description here

So in summary, tracking objects in Mathematica is possible, but it's not for the faint of heart. I think the process is summarized in the following steps:

  • Import video as individual frames, possibly "downsampling" the video to make the data set manageable.
  • Manipulating the images to remove extraneous information (colors, stationary objects, etc.)
  • Use some tool such as MorphologicalComponents to identify the information of interest to you.
  • Transform the output of the previous step into a usable format

That's my 2 cents -

Mr.Wizard
  • 271,378
  • 34
  • 587
  • 1,371
bobthechemist
  • 19,693
  • 4
  • 52
  • 138
  • very nice...learned a lot – ubpdqn Nov 07 '13 at 11:51
  • Thanks. This exactly what I want to know. Can you write something about what you have done after reading the images and how you manipulated all the frames. – Michiel van Mens Nov 08 '13 at 08:22
  • @MichielvanMens if you import the first gif in my answer via images=Import["http://i.stack.imgur.com/sJPxW.gif"] then the code I've written will produce all of the results save for the last. My 2nd bullet point about manipulating images (which is my imagesbw symbol) is going to vary significantly based on the application and your best bet would be to start with the Image Processing tutorial in Mathematica to see what might work for your purposes. – bobthechemist Nov 08 '13 at 12:29
  • @MichielvanMens As for the code that I used to "solve" my physics problem; it is ugly and opaque, includes a fudge factor or two, and wouldn't really help this answer all that much. (Which is one way of saying I'd be embarrassed to post the code.) – bobthechemist Nov 08 '13 at 12:33
  • @bobthechemist Do you think perhaps you could edit this question such that its scope is that of this existing answer? I would like to see this reopened. – Mr.Wizard Aug 15 '16 at 01:57
  • @Mr.Wizard I can take a shot at it, although classes start for me this next week which means I've got a couple months of lesson plans to prepare over the next 168 hours. – bobthechemist Aug 15 '16 at 12:10
  • @Mr.Wizard let me know if this is more in line with what you are thinking. – bobthechemist Aug 16 '16 at 15:57
  • Yes, I think that will do. I have reopened the question. – Mr.Wizard Aug 16 '16 at 18:00
2

What you are trying to do is called object tracking, and it is an active research area in computer vision. There are many algorithms for detecting and tracking multiple moving objects. I don't know of any examples in Mathematica, but here is one in MATLAB using background subtraction and Kalman filters.

Dima
  • 129
  • 3