If you look at it, it doesn't seem to make sense. However, you have to be aware that on a flat plane the Quadratic Sphere gradient looks 2-dimensional, but it is in fact a 3-dimensional gradient which goes in all directions in 3D space.
The brightest part, the value 1, is at the center of a sphere - not a circle. From there it decreases to 0. It's not the displacement that affects the color, it's the texture itself.
The Displacement node tries to map 2-dimensional textures into height. So if you now take the Quadratic Sphere texture at a certain level - at XYZ = 0 for example - the textures values there get added and the gradient goes from 0 on the outside to 2 in the center.
This means, the displacement will use a height of 2 at the center - the peak you can see. However, the gradient which gives the color is still a Quadratic Sphere. So the brightest part with a value of 2 is at the center of the peak's bottom. From there it gets darker down to 0 and wherever the mesh meets this gradient in 3D space, this is the color it shows.
Imagine it like this, the left one is your result, the planes to the right like reference images when you're modeling something from blueprints for front, right and top view. As you can see by the dotted yellow lines, this is where your mesh lies in the 3-dimensional gradient. That's why not even the brightest parts on the outside have a value of 2, not even 1.

By the way, if you want the color to represent what the Displacement node uses as height, you have to turn the 3-dimensional gradient into a 2-dimensional gradient: simply set the Z scale in the Mapping node to 0.

Or in the "blueprint" view:
