I followed the example here and got this:
\documentclass[a4paper]{article}
\usepackage{amsmath,array,booktabs}
\usepackage{tikz}
\usetikzlibrary{calc}
\def\startCirc#1{\tikz[remember picture,overlay]\path node[inner sep=0, anchor=south] (st) {\textbf{#1}} coordinate (start) at (st.center);}%
\def\endCirc#1{\tikz[remember picture,overlay]\path node[inner sep=0, anchor=south] (en) {\textbf{#1}} coordinate (end) at (en.center);%
\begin{tikzpicture}[overlay, remember picture]%
\path (start);%
\pgfgetlastxy{\startx}{\starty}%
\path (end);%
\pgfgetlastxy{\endx}{\endy}%
\pgfmathsetlengthmacro{\xdiff}{\endx-\startx}%
\pgfmathsetlengthmacro{\ydiff}{\endy-\starty}%
\pgfmathtruncatemacro{\xdifft}{\xdiff}%
\pgfmathsetmacro{\xdiffFixed}{ifthenelse(equal(\xdifft,0),1,\xdiff)}%
\pgfmathsetmacro{\angle}{ifthenelse(equal(\xdiffFixed,1),90,atan(\ydiff/\xdiffFixed))}%
\pgfmathsetlengthmacro{\xydiff}{sqrt(abs(\xdiff^2) + abs(\ydiff^2))}%
\path node[draw,rectangle, rounded corners=2mm, dashed, rotate=\angle, minimum width=\xydiff+4ex, minimum height=2.5ex] at ($(start)!.5!(end)$) {};%
\end{tikzpicture}%
}
\begin{document}
\begin{tabular}{
>{$}l<{$}
cc|cc|cc
}
\toprule
\{ \phi(\cdot) , p(\cdot) \}^\dagger & 1768 & 1768 & 765 & 765 & 986 & 986 \\
\{ [\phi(t+h) , p(h)]_3 \} & 1720 & 1720 & \startCirc{7}6\endCirc{7} & 766 & 956 & 956 \\
\{ \phi(\cdot) , p(h_2) \}^\dagger & 1720 & 1720 & 754 & 754 & 959 & 959 \\
\bottomrule
\end{tabular}
\end{document}
The desired output is the second one, color is not important. I think the current approach maybe too "complicated" than necessary?

