When you do
\if\nm{dots}\else node[dot](\nm){}\fi
what happens is that TeX expands \nm and compares the two unexpandable tokens it finds; in the first cycle \nm is na1, so what happens is
\if na1{dots}\else node[dot](\nm){}\fi
and, since n and a are different, the "else" branch is followed. The same happens when \nm is dots: d and o are different.
What you need is a different test:
\begin{tikzpicture}[dot/.style={fill,circle,inner sep=1.5pt}]
\def\test{dots}
\foreach \x/\nm/\lbl in {0/na1/$\lnot a_1$, 1/na2/$\lnot a_2$, 2/na3/$\lnot a_3$, 3/dots/$\cdots$, 4/nan/$\lnot a_n$}
\draw (\x,0) \ifx\nm\test\else node[dot](\nm){}\fi node[left=2pt]{\lbl};
\end{tikzpicture}
so TeX will compare the meaning of \nm with the meaning of \test
Another test that can work and doesn't require defining \test is
\ifnum\pdfstrcmp{\nm}{dots}=0 \else node[dot](\nm){}\fi
It has a limitation: you need pdflatex for it to work. If you plan using the code with XeLaTeX or LuaLaTeX, then say in the preamble
\usepackage{pdftexcmds}
\makeatletter
\let\unistrcmp\pdf@strcmp
\makeatother
and use \unistrcmp instead of \pdfstrcmp.
The reason is that XeTeX provides the command under the name \strcmp and LuaTeX doesn't provide it at all. The package defines \pdf@strcmp to be \pdfstrcmp if the engine is pdfTeX, \strcmp if the engine is XeTeX, and defines a Lua function in case the engine is LuaTeX. But the name has @ in it, so we provide an @-free version.
\test)? – jtbandes Feb 26 '12 at 23:14