Usually \rmdefault should expand to a string representing the font name; with your definition it expands to \familyname, whose expansion is not a string: for example, with \newfontfamily\familyname{Hoefler Text}, the expansion of \familyname is \protect\familyname (with a space at the end, as usual with "robusted" commands. The expansion of \familyname is
\fontencoding {EU1}\fontfamily {HoeflerText(0)}\selectfont
The expansion of \rmdefault after \setmainfont{Hoefler Text} is
HoeflerText(0)
So they are quite different.
Also using \rmfamily instead of \rmdefault does not give equivalent results: first of all, \rmfamily should be a robust command; its expansion results in
\protect\rmfamily and then in
\not@math@alphabet\rmfamily\mathrm\fontfamily\rmdefault\selectfont
So, again, also \DeclareRobustCommand{\rmfamily}{\familyname} won't work the same.
\def\rmfamily\familyname? I modified the question. – Yan Zhou Aug 30 '11 at 19:54\fontspec_set_family:Nnncommand. – Will Robertson Aug 31 '11 at 01:11