They are potassium bitartrate crystals (source).
The crystals form because the potassium bitartrate is not very soluble.
Since solubility is a function of temperature, when wine is chilled the solution can become saturated, causing the precipitate to form in the bottle. Stabilizers are often added to prevent this.
In your case, the precipitate is on the cork, and so what I think has happened is that some wine soaked into the cork, perhaps while the bottle was moved. Then later, the bottle was stood upright or on its side where the wine could not reach the cork. As the wine evaporated back into the bottle, the local concentration of potassium bitartrate increased until it reached the saturation point, at which point the crystals grew. Since the crystals take a very long time to dissolve once formed, they will persist even if wine reaches the cork again.
The good news is, they are harmless (especially good news given that you tasted them - please don't make tasting strange crystals a habit!). They are made of the same substance that cream of tartar, which is a common baking ingredient, is made of.