No, it does not work that way. Only clashing options do exclude each other. See this example
\documentclass[10pt]{article}
\usepackage[nofoot]{geometry}
\usepackage{fontspec}
\setmainfont[Ligatures={TeX}, SmallCapsFeatures={%
LetterSpace=7,Letters=SmallCaps},%
Kerning=Uppercase,%
ItalicFeatures={Contextuals=Swash,Kerning=Uppercase,Style=Swash}]{Adobe Caslon Pro}
\begin{document}
\noindent
Test out 123\\ {\addfontfeature{Ligatures=Rare} test in first ct 123\\ {\addfontfeature{Numbers=OldStyle} test in second ct st 123} test in first ct} \\ test out 123
\end{document}
Which yields:

This is because the Ligature and Numbers are separate features: they don't clash.
But look at this example, where I redefine the Numbers feature
\documentclass[10pt]{article}
\usepackage[nofoot]{geometry}
\usepackage{fontspec}
\setmainfont[Ligatures={TeX}, SmallCapsFeatures={%
LetterSpace=7,Letters=SmallCaps},%
Kerning=Uppercase,%
ItalicFeatures={Contextuals=Swash,Kerning=Uppercase,Style=Swash}]{Adobe Caslon Pro}
\begin{document}
\noindent
Test out 123\\
{
\addfontfeature{Numbers=OldStyle} test in first ct 123\\
{
\addfontfeature{Numbers=Lining} test in second ct st 123
} \\
test in first ct 123} \\
test out 123
\end{document}
It will behave just as you said:

Since it's so well written I recommend you take a look at the fontspec documentation.
EDIT
I understand that you may not have the latest version/have some problems with your installation, since it works fine for me.
Since I thought you posted an example about colour, for completeness sake I did the same:
\documentclass[10pt]{article}
\usepackage[nofoot]{geometry}
\usepackage{fontspec}
\setmainfont[Ligatures={TeX}, SmallCapsFeatures={%
LetterSpace=7,Letters=SmallCaps},%
Kerning=Uppercase,%
ItalicFeatures={Contextuals=Swash,Kerning=Uppercase,Style=Swash}]{Adobe Caslon Pro}
\begin{document}
\noindent
Test out 123\\
{
\addfontfeature{Numbers=OldStyle} test in first ct 123\\
{
\addfontfeature{Color=FF0000} test in second ct st 123
} \\
test in first ct 123} \\ test out 123
\end{document}
Yielding
