It does work for the outer one, but in the outer one you have nothing that is affected by it. Everything in your picture is defined in terms of nodes and nodes are generally exempt from non-translation transformations. So the scale=2 is not applied to either node2 or tikzpic, and then since the arrow is defined in terms of those nodes it isn't affected either.
To force a non-translation transformation to have an effect on a node you can use the transform shape key on that node. To force it on all, use `every node/.append style={transform shape}:
\documentclass{article}
%\url{https://tex.stackexchange.com/q/86730/86}
\usepackage{tikz}
\usetikzlibrary{positioning}
\begin{document}
\begin{tikzpicture}
\node[draw](node2){Node 2};
\node[below=of node2](tikzpic){%
\begin{tikzpicture}[scale=2]
\draw [step=0.25cm,lightgray,very thin] (0,0) grid (2.5,1.5);
\end{tikzpicture}
};
\draw[->,ultra thick] (node2) -- (tikzpic);
\end{tikzpicture}
%
\begin{tikzpicture}[scale=2]
\node[rectangle,draw](node2){Node 2};
\node[transform shape,below=of node2](tikzpic){%
\begin{tikzpicture}
\draw [step=0.25cm,lightgray,very thin] (0,0) grid (2.5,1.5);
\end{tikzpicture}
};
\draw[->,ultra thick] (node2) -- (tikzpic);
\end{tikzpicture}
\end{document}
Produces:

Nesting pictures is generally a bad idea - TikZ is not set up with the expectation that tikzpictures are nested and so there is no particular control over "leakage" of settings from one picture to the other (inner to outer or outer to inner). It is better to form your picture as a single picture, using scopes to separate content and matrices to position content. In this particular situation it is very simple to recreate this as a single diagram - which leads me to suspect that it is an example crafted to show the problem you were encountering (which is a very good thing to do!). So while I'd be happy to try to recreate it as a single picture for you it may be that your real example is more complicated and doing this one wouldn't be all that illuminating. If you would like help on reformulating a nested picture as a single one (assuming I've convinced you!), feel free to ask a new question.
(There are quite a number of different posts here dealing with issues of nesting tikz pictures. It is possible so long as you are aware of what you are doing and know what can go wrong. An example of what can go wrong is at Problem with overlay when a tikzpicture is inside another tikzpicture, and there are plenty of linked questions to see other things.)
tikzpictureenvironment inside another one? – Sigur Dec 12 '12 at 15:18