If you want to cancel out glue having natural width 3pt, stretch component 2pt and shrink component 1pt, you have to emit glue in the amount of
-3pt plus -2pt minus -1pt
Stretch and shrink components are unrelated and TeX will use only one of them at a time (or neither). If a line or page has to be stretched, only the stretch components are taken into account and similarly if the line or page has to be shrinked.
If Knuth had chosen to express rubber lengths as
<natural width> stretchable <stretch component> shrinkable <shrink component>
your doubt would probably not have arisen.
Another view on the point you make. Specifying 3pt plus 2pt minus 1pt does not mean that any space from 2pt to 5pt can be used.
It means instead that, when shrinking is necessary, the final space will be between 2pt and 3pt; when stretching, the optimal space will be between 3pt and 5pt, but it can become larger at the expense of line/page badness.
If you say
\documentclass{article}
\begin{document}
\rule{100pt}{0.4pt}
\makebox[100pt][s]{a\hspace{3pt plus 2pt minus 1pt}z}
\end{document}
the output would be

so the glue has been stretched much more than 2pt; however the warning
Underfull \hbox (badness 10000) detected at line 9
\OT1/cmr/m/n/10 a z
will be issued. To the contrary,
\documentclass[draft]{article}
\usepackage{calc}
\begin{document}
\rule{\widthof{az}+1pt}{0.4pt}
\makebox[\widthof{az}+1pt][s]{a\hspace{3pt plus 2pt minus 1pt}z}
\end{document}
would output

with an overfull box that cannot shrink to the required size, because the glue will never go below 2pt.
Again, only the stretch component or the shrink component will be used.