8

As Vitaliy Kaurov's answer mentioned, anyone can use a method based on Neural Network (NN) to give a try? Then the bounty is deserved.

Suppose I have a empty paper like

Then I write some text on the paper,then the result paper have been distorted,rotated and translated.like

How to adjust the result papper according the original empty paper? As you see,some empty transverse line can align this two images. The result like following is expected.

Actually I think the ImageAlign can help me(it cannot align a distorted image as its documentation).

ImageAlign[template, img, TransformationClass -> "Similarity"]

But it is a very slow and give me a poor result.Can anybody give a better solution for this?

yode
  • 26,686
  • 4
  • 62
  • 167
  • 1
    Can you post images with clear external borders - this could help significantly. – Vitaliy Kaurov Aug 13 '17 at 07:37
  • @VitaliyKaurov But it don't have a clear external borders...The post have all information I have. – yode Aug 13 '17 at 07:40
  • 2
    ImageAlign works with the following transformations (as per the help): Translation, Rigid, Similarity, Affine, Perspective. The transformation of your paper looks to be nonlinear and/or spatially varying, so it is unlikely to work. – bill s Aug 13 '17 at 14:13
  • @bills Yes,I have read that before I post this question. – yode Aug 13 '17 at 14:58
  • "The result like following is expected." --- how did you get that result? – Vitaliy Kaurov Aug 13 '17 at 22:07
  • @VitaliyKaurov By Photoshop – yode Aug 14 '17 at 00:10
  • @bills gives you the important hint. To register these two images onto each other perfectly, you will need a non-linear image registration algorithm. This is afaik currently not supported by Mathematica. – halirutan Aug 14 '17 at 19:50
  • @halirutan of course you are right, but as ImageMultiply[template, ImageAlign[template, img, TransformationClass -> "Perspective"]] may demonstrate, even an unrestricted linear transform works not too bad for the given example, and also is not really slow. – UDB Aug 25 '17 at 19:00
  • @UDB Why you think that is not bad? – yode Aug 26 '17 at 06:21
  • @yode try ImageMultiply[#1, ImageAlign[#1, #2, TransformationClass -> "Perspective"]] &[Import@"https://i.stack.imgur.com/euyxY.jpg", Import@"https://i.stack.imgur.com/fymre.jpg"] and tell us what you see. – UDB Aug 26 '17 at 13:31
  • You should get Taliesin Beynon in here to give you a hand (or xslittlegrass). Or google [WSS17] "NetTrain" or look at the stuff on community. Lots of people working with neural nets there. – b3m2a1 Sep 15 '17 at 07:17
  • @Taliesin Beynon Bounties wait you here. (also @xslittlegrass) – yode Sep 15 '17 at 07:31

1 Answers1

7

I will give just a start, which I think is not bad already. First just for faster computing times I resize your images:

i1 = ImageResize[Import["https://i.stack.imgur.com/nerv4.jpg"], 400];
i2 = ImageResize[Import["https://i.stack.imgur.com/czNNK.jpg"], 400];

Next FindGeometricTransform

{e, t} = FindGeometricTransform[i1, i2]

enter image description here

Then you can go both ways which give you same result, but 1st is faster:

ImagePerspectiveTransformation[i2, t, DataRange -> Full]

or

ImageForwardTransformation[i2, t, DataRange -> Full]

enter image description here

The same result could be obtained as

pts = ImageCorrespondingPoints[i1, i2];
{cpe, cpt} = FindGeometricTransform @@ pts
ImagePerspectiveTransformation[i2, t, DataRange -> Full]

You can try playing with ImageFeatureTrack, ImageLines, and some manual manipulation of extracted features, for example manual specification or editing points/lines after ImageCorrespondingPoints/ImageLines before they go into FindGeometricTransform. You could also try training a Neural Network (NN) on a bunch if similar simulated transformations. Advantage of NN is it could potentially be very fast.

Vitaliy Kaurov
  • 73,078
  • 9
  • 204
  • 355
  • As I know,those matrix just can process linear transformation.. – yode Aug 14 '17 at 01:37
  • 1
    @yode, why do you believe the distortion in your images cannot be approximated by a linear fractional transformation? – J. M.'s missing motivation Aug 14 '17 at 01:53
  • @J.M. I just juess,I think that cannot be solved by a matrix transformation.I think we need a new solution. And I think this answer is a good start. – yode Aug 14 '17 at 01:57
  • I'm confused still.If we use a NN method. How to get those trianning image. I mean if I just trainnig the image posted in the question with different transform. Then we can deal with other image with different transform? – yode Sep 14 '17 at 06:52
  • @yode you have to generate set of random images where you do know the transform. – Vitaliy Kaurov Sep 14 '17 at 09:22
  • @yode actually, a correction, you have to generate set of random images where you start from wanted result (normal image, output of NN) and deform it somehow (input of NN). – Vitaliy Kaurov Sep 15 '17 at 07:46
  • 1
    I don't very familar thos NN thing..I have to give you still. – yode Sep 20 '17 at 19:21
  • I note after I update the image, your method don't work anymore or I have missed some thing? – yode May 18 '18 at 06:41