Unless you know all the stars in the diagram suffer the same extinction, then you should de-redden them individually.
If you do know they have the same extinction (e.g. they are all nearby stars and it is zero; or they are all stars well above the Galactic plane in a similar direction; or stars in a cluster at the same distance) then you can plot a diagram using the observed magnitudes. In the latter scenario of a cluster, this diagram can be used to estimate the reddening of the cluster.
If you want to compare with models, to estimate temperatures or masses, then you must de-redden (or redden the model!)