You can simply put \large or \huge or whatever you like before \insertframenumber.
This code adds the frame number (with \insertframenumber) a slash, and the total frame number (with \inserttotalframenumber) to the already defined navigation symbols template. It also tells beamer to use the font defined as footline and the foreground (fg) color also defined as footline. This comes a horizontal space of 1em after the existing navigation symbols. The following code might help clarify. Note I've changed the color to match the title, and made the font \huge.
\documentclass{beamer}
\addtobeamertemplate{navigation symbols}{}{%
\usebeamerfont{footline}%
\usebeamercolor[fg]{title}%
\hspace{5em}%
\huge\insertframenumber/\inserttotalframenumber
}
\begin{document}
\begin{frame}
\frametitle{Frametitle}
Text
\end{frame}
\end{document}
