4

In this nice tutorial about CNNs, the authors build a single-layer CNN. The initial convolution weights are set randomly, according to a uniform distribution.

By the end of this scetion, the authors note that the randomly initialised kernel behaves very similar to an edge detector and give the following input and output as example.

In/Output

Why does the randomly initialised kernel behave like an edge detector?

Emiswelt
  • 143
  • 4

2 Answers2

3

Take this with a grain of salt, but I think this is simply not true. You can evaluate it with the code I just wrote:

enter image description here

Only 2, probably 3 of the 25 look like edge filters to me. The result of an edge filter looks like this:

enter image description here

Martin Thoma
  • 18,880
  • 35
  • 95
  • 169
  • This is a very bad answer... All the results that you just showed are very similar with the following differences:

    • Some have a flipped sign, meaning that the light edges become dark

    • Some have a constant that is added to them.

    However, the overall behaviour is similar to an edge detector. Your images just prove that the original question is right.

     – DomDev Jan 31 '19 at 05:08
  • 1
    "All the results that you just showed are very similar " - that is exactly what I wanted to show :-) – Martin Thoma Jan 31 '19 at 05:54
  • "However, the overall behavior is similar to an edge detector." - I don't think that the top images look similar to the bottom image. – Martin Thoma Jan 31 '19 at 05:55
  • Because your bottom image is a vertical edge detector. If you compute the horizontal edge detector, then compute their norm, it will be very similar to the other images (with an added bias and a possible negative factor). The reason is that the random initialization will obviously not be biased toward a specific direction. – DomDev Feb 17 '19 at 15:06
  • "the random initialization will obviously not be biased toward a specific direction" - that is exactly what my answer shows – Martin Thoma Feb 17 '19 at 20:22
  • You only show that it doesn't behave like a directionnal edge detector. But it still behaves like a directional-agnostic edge detector, such as the Laplacian or the norm of the gradient. – DomDev Feb 20 '19 at 03:45
  • By this logic, the given structure can not possibly behave different from an edge detector. But when most people say "edge detector" they mean "directional edge detector" – Martin Thoma Feb 20 '19 at 05:58
  • "when most people say "edge detector" they mean "directional edge detector"". This statement is false! The Laplacian and gradient norm are very well known edge detectors. Plus, look up for Edge-detection using deep neural networks, they do not care about the direction, only about the presence or absence of an edge. Another is Matlab's edge detection examples, where you see that edges are not directional. https://www.mathworks.com/discovery/edge-detection.html – DomDev Feb 20 '19 at 18:13
0

The answer by Martin is not right. In his examples, he only shows that the deep network does not behave like a directional edge detector. In fact, his example below is a smoothed vertical derivative (probably Sobel).

However, we clearly see that it behaves like a direction-agnostic edge detector, such as a Laplacian or the norm of a gradient. Also, we can see that all images look the same, with the only major differences being that some are inverted and others have an additional constant.

In summary, the network behaves like an direction-agnostic edge detector. A bad edge detector, but still an edge detector. This helps explain why deep learning edge detection methods converge a lot faster and with simpler architectures than other problems such as saliency and skeleton extraction.

DomDev
  • 101
  • 1