Add anchor=base to your nodes:

\documentclass[varwidth]{standalone}
\usepackage{tikz}
\usetikzlibrary{arrows}
\newcommand*{\ident}[1]{\texttt{\small #1}}
\tikzstyle{refines} = [->, >=open triangle 45]
\newcommand{\refi}[3]{$\begin{tikzpicture}[baseline]%
\node[anchor=base] (A) {#1};%
\node[anchor=base] (B) [right of=A, node distance=4em] {#3};%
\draw[refines] (A) -- node[midway,above=-2pt] {#2} (B);%
\end{tikzpicture}$}
\begin{document}
this is a test \refi{\ident{1}}{\scriptsize text}{\ident{2}}\ident{2} that continues here
this is a test \refi{1}{\scriptsize text}{1}\ident{2} that continues here
\end{document}
This ensures that the baseline of the text in the node sits at level 0.
However, there are two problems: firstly, as @rainer says, depth and height of material in the nodes can make the arrow stop being horizontal; secondly, your code does nothing in the direction of providing an exensible arrow that stretches with the material above the arrow.
The first problem could be used by using the nice tikz-cd package, see e.g. https://tex.stackexchange.com/a/113449/15925. But that won't solve the second one. So I think it is easier let tikz take care of only the arrow and the text above it, and leave the rest to ordinary maths placement:

\documentclass{article}
\usepackage{tikz,mathtools}
\usetikzlibrary{arrows}
\newcommand*{\ident}[1]{\texttt{\small #1}}
\tikzstyle{refines} = [->, >=open triangle 45]
\newsavebox{\mytempbox}
\newcommand{\refi}[3]{%
\sbox{\mytempbox}{\hbox{\( \scriptstyle\mkern5mu#2\mkern17mu \)}}
\( #1
\tikz[baseline=-0.5ex]{\draw[refines] (0,0) --
node[midway,above=-0.3ex]{\usebox\mytempbox} (\wd\mytempbox,0);}
#3 \)}
\begin{document}
This is a test \refi{\ident{1}}{\text{text}}{\ident{2}}\ident{2} that continues here.
This is a test \refi{1}{\text{longer text}}{1} that continues here.
A further test \refi{j_2}{\alpha-x e^{p/q}}{a^2} with more text.
\end{document}
The technique here is to save the material to place above the arrow in a box (I written this assuming it is math mode material). That box is then used to specify the length of the arrow and its contents are placed above the arrow. I have added sufficient space around the text so that overlap with the rather large arrow head does not occur.
\begin{tikzpicture}[baseline,every node/.append style={text depth=0}]to make this work correctly with left/right operands that have descenders. – rainer May 27 '13 at 11:25