I assume you're using LuaTeX. If so, one can set up a macro called \Setxheight and use it with fontspec's Scale option. Note that the macro uses Lua's division routine instead of TeX's own built-in routine since the scaling factor will likely not be integer-valued.

% !TEX TS-program = lualatex
\documentclass{article}
\usepackage{fontspec}
%% x-height of font is stored in \fontdimen5
\newcommand\Setxheight[1]{\directlua{%
tex.sprint(#1/(\the\numexpr\dimexpr
\the\fontdimen5\font\relax\relax/65536))}}
\begin{document}
\setmainfont{EB Garamond}
\the\fontdimen5\font % show value of x-height
\setmainfont[Scale=\Setxheight{4.78}]{EB Garamond} % reload the font
\the\fontdimen5\font % show value of x-height
\end{document}
Edit: XeLaTeX version
% !TEX TS-program = xelatex
\documentclass{article}
\usepackage{fontspec}
%% x-height of font is stored in \fontdimen5
\ExplSyntaxOn
\newcommand\Setxheight[1]{
\fp_eval:n{(#1/(\the\numexpr\dimexpr
\the\fontdimen5\font\relax\relax/65536))}}
\ExplSyntaxOff
\begin{document}
\setmainfont{EB Garamond}
\the\fontdimen5\font % show value of x-height
\setmainfont[Scale=\Setxheight{4.78}]{EB Garamond} % reload the font
\the\fontdimen5\font % show value of x-height
\end{document}
fontspecprovides theScale=<some positive scalar>option. However, from the description you've given so far I can'te tell if this option might be useful for you. – Mico Aug 12 '15 at 19:39fontspecbut for the font size of the document? – Manuel Aug 12 '15 at 20:38xare not always identical. E.g., forEB Garamond, the x-height is4.0ptwhereas the height of the (roman/upright) letterxis4.05pt. Thus, measuring the heights of uppercase and lowercase letters may not give you the font's caps height and x-height, respectively. – Mico Aug 12 '15 at 21:31