2

Michael Trott's blog shows that Mathematica can convert images into parametric curves

Is it possible to handle video or real life GIF into animated cartoon GIFs similarly? e.g.: cartoon gif like below:

enter image description here,

PS:

If there is any working solution, I'd like to try it on the following example:

enter image description here

user6043040
  • 633
  • 3
  • 9
  • 1
    Apply http://mathematica.stackexchange.com/q/60433/484 or http://mathematica.stackexchange.com/q/11524/484 to each frame of the video? –  Nov 28 '16 at 10:14

1 Answers1

3

At WTC2016 Yuzhu Lu gave a talked titled Graphics 3D. The notebook is not yet available in the Wolfram Library Archive's Conference Proceedings.

On the last slide he showed how you can write a custom image processing/rendering in OpenGL with FEPrivate`AddSurfaceAppearanceDefinition and FE`Evaluate. This can be used to render images in Mathematica instead of the regular front-end rendering. This means it does not need to load and then process the video/image but processes the video/image when loading it (only one pass).

Therefore, if you have coded an OpenGL rendering that can produce the stick players from a frame of the video then you can render the video using it. This was demonstrated with an edge detection OpenGL render on both an image and a short video. The edge detection render that was used in the presentation is below.

FE`Evaluate@Evaluate[
 FEPrivate`AddSurfaceAppearanceDefinition["My Edge3",
  {{"position", "ATTRIB_VERTEX"}, {"vertexTextureCoordIn", "ATTRIB_TEXTURECOORD"}},
  {{"TMatrix","tranformationMatrix",Automatic,"TRANSFORMMATRIX"},
   {"Color","color",Yellow,"COLOR"},
   {"Texture2D","texture",Automatic,"TEXTURE"},
   {"Size","size",360,"NUMBER"}},
  {"varying vec4 texCoordOut; uniform vec4 color; uniform sampler2D texture; uniform float size; 
    void main() { const float offset=1.0/size;vec4 
    c=texture2D(texture,texCoordOut.xy); vec4 
    edge=texture2D(texture,texCoordOut.xy+vec2(-offset,-offset))+texture2D(texture,texCoordOut.xy+vec2(-offset,0.0))+texture2D(texture,texCoordOut.xy+vec2(-offset,offset))+texture2D(texture,texCoordOut.xy+vec2(0.0,offset))+texture2D(texture,texCoordOut.xy+vec2(offset,offset))+texture2D(texture,texCoordOut.xy+vec2(offset,0.0))+texture2D(texture,texCoordOut.xy+vec2(offset,-offset))+texture2D(texture,texCoordOut.xy+vec2(0.0,-offset));
    gl_FragColor=8.0*(c - 0.125*edge);gl_FragColor.a = 1.0; gl_FragColor *= color;}",
   "attribute vec4 position; attribute vec4 vertexTextureCoordIn; uniform mat4 tranformationMatrix; varying vec4 texCoordOut; void main() { gl_Position = tranformationMatrix * position; texCoordOut = vertexTextureCoordIn;}"}]];

This "My Edge3" was then used to render an image and a short video. The image example below.

img = ExampleData[{"TestImage", "Lena"}];
Manipulate[
 Graphics[{System`SurfaceAppearance["My Edge3", "Color" -> Dynamic@c],
    Texture[img], 
   Polygon[{{-1, -1}, {1, -1}, {1, 1}, {-1, 1}}, 
    VertexTextureCoordinates -> {{0, 0}, {1, 0}, {1, 1}, {0, 1}}]}, 
  ImageSize -> {360, 360}, Frame -> True], {c, White}]

Mathematica graphics

A function ApplyEdges was created but this may only work in a future version has he was using an internal build.

ApplyEdges[img_] := 
  RawBoxes[ToBoxes[img] /. 
    RasterBox[arg_, opts___] :> {System`SurfaceAppearance["My Edge3"],
       RasterBox[arg, opts]}];

This was then used passed images and .avi video Imports.

If I recall correctly it was mentioned that better support for this would be available in a future version.

Hope this helps.

Edmund
  • 42,267
  • 3
  • 51
  • 143