Why would the brain flip the images perceived by your eyes?

Question

The following is a common scientific statement, which you don't have to google long for to find:

The eye views images upside-down in the manner of a camera lens, but our brains reinterpret this input to allow us to see things the correct way up.

My first question is quite straightforward: Is this statement valid?

I don't understand why you should come to the conclusion that your brain should 'flip' the image. Wouldn't your brain be able to cope just fine without flipping it? If anything I would suspect the brain to not flip the image, if there is no reason to do so.

In order for this statement to be valid, I would expect a scientific theory/experiment from which can be concluded that the brain does process the vision in such a way that, after it being processed all subsequent processing occurs on the 'inverted' image. That seems to me to be the most logical interpretation of 'to flip'.

Answer 1

It is not meaningful to talk about your brain processing something as 'right-side up"' or 'upside-down'. The 'images' in your brain are just collections of neural activations, and not actual pictures. Thus they cannot have an orientation. The only meaningful way to test your question is to try flipping the input the brain receives and seeing if it can cope.

Fortunately, the brain is capable of flipping your visual field if required as measured through perceptual adaptation experiments using inversion glasses. This has been demonstrated very drastically in studies, by for instance requiring a participant to wear inversion glasses for a long time. At first they are confused, and unable to orient themselves and do basic tasks, however after enough time the brain can adopt enough to even do activities like riding a bike. This suggests that from the only way you can measure things (i.e. behaviorally) the brain is capable of adapting to an upside down world (some participants even reported that after extended use the world even seemed "right side up"). This is functionally equivalent to the brain being able to process your visual information in either orientation. If it is capable of processing in either orientation, the question of "does my brain flip the image" becomes a pseudo-question and unanswerable.

References

• Taylor, J. G. (1962). The behavioral basis of perception. New Haven: Yale University Pres

• Harris, C.S. (1965) "Perceptual adaptation to inverted, reversed, and displaced vision." Psychological Review 72(6): 419-444. [pdf]]

• Di Paolo, E.A. (2003) "Organismically-inspired robotics: homeostatic adaptation and teleology beyond the closed sensorimotor loop", {Dynamical systems approach to embodiment and sociality: 19-42 [pdf

Answer 2

I think part of what makes this question confusing is the use of expressions like "what the eye sees", "what the brain sees" and "what the frog's eye tells the frog's brain". Nobody sees anything except the experiencing subject. When one stops thinking that the brain (or some visual-system part of the brain) observes the image on the retina, then the question of whether anything is being flipped becomes meaningless.

As regards perceptual adaptation, it is interesting to note that it is not universal. In Sperry's famous frog-eye-inversion experiment, the frogs never adapted.

References

• Roger W. Sperry (1943). Effect of 180 Degree Rotation of the Retinal Field on Visuomotor Coordination. The Journal of Experimental Zoology 92 (3): 263–279

Answer 3

The fact that the image does not appears upside-down has to do with the way visual information is processed in the brain. In his book, Jeff Hawkins argues that the low-level visual features on the retina (being upside down, distorted, and changing rapidly) are lost in the process of forming invariant representation. And it's those representations that we experience consciously.

From On Intelligence (official site, pdf)

The light receptors in your retina are unevenly distributed. They are densely concentrated in the fovea at the center, and get gradually sparser out in the periphery. In contrast, the cells in the cortex are evenly distributed. The result is that the retinal image relayed onto the primary visual area, V1, is highly distorted [and upside-down, if you will]. When your eyes fixate on the nose of a face versus on an eye of the same face, the visual input is very different, as though it is being viewed through a distorting fisheye lens that is jerking violently to and fro. Yet when you see the face, it doesn't appear distorted, and it doesn't appear to be jumping around. Most of the time you aren't even aware that the retinal pattern has changed at all, let alone so dramatically. You just see "face." (Figure 2b shows this effect on a view of a beach landscape.) This is a restatement of the mystery of invariant representation we talked about in chapter 4, on memory. What you "perceive" is not what V1 sees. How does your brain ever know it is looking at the same face, and why don't you know the inputs are changing and distorted?

The process of forming invariant representations is explained in the book, but I won't quote it there because its quite long.

Answer 4

There are many transformations between the light hitting your retina and your perception of the world.

The signals from your retina initially travel through the visual pathways to reach your visual cortex, where visual information is processed. The representation of this visual information in your brain is also shaped by other brain activity representing your other senses (sound, touch, etc) and also internal processes like your current mood or expectations.

By the time all of this information is incorporated into your conscious perception, the orientation of the photons as they hit your retina is gone and all you have is your brain's interpretation of the world around you.

Answer 5

It is not the case that the brain flips the retinal image, nor does it have to, nor are there any images in the brain like there are on the retina. It is just as meaningless to say that the retinal image is upside down in relation to the orientation of our perceptions.

Richard L. Gregory gives a nice explanation:

It is generally accepted that this does not need a special compensation mechanism because retinal images are not seen, as objects are seen […]. A compensating mechanism is not needed as they are not objects of perception but rather one stage of processing lying between objects and vision. […] When the head is tilted, the world remains upright. This extends to standing on one's head, when the retinal image is reversed and yet up and down remain normal. (Richard L. Gregory (2004): Illusions, In: The Oxford Companion to the Mind, 2nd Edition, p. 429).

Another way to grasp this issue is by the difference of physical space and phenomenal space, as Norbert Bischof does:

We thus have to distinguish a physical space, in which our body is located and the brain processes take place, and a space experience which the phenomenal world is embedded in. [… Both] cannot be localized in relation to one another. There is no superordinate coordinate system which they can be jointly fit into. They are […] »incommensurable«, which literally means there is no common scale that could be applied to both. […] (Norbert Bischof (2009): Psychologie, 2nd Edition, pp. 48–49, my translation).

Bischof goes on to state that the retinal image and the consciously experienced phenomenon can not possibly be integrated within one and the same space in the first place, and thus don't stand in any spatial relation whatsoever.