Proofs of the identity of the OP have already been given in answers.
However, it might be interesting so see the calculation which led me to the result which I didn't know in advance.
Part 1: Transformation of integral into a series
Substituting $x\to z-\sqrt{z^2-1}$ the integral becomes
$$i = \int_{1}^\infty \{z\}( \frac{z}{\sqrt{z^2-1}}-1)\,dx\tag{1}$$
Splitting the integral into the intervals $(k,k+1)$, $k=1,2,3,...$ we get
$$i= i_{s} := \lim_{n\to\infty} i_{s}(n) \tag{2a}$$
$$i_{s}(n):= \sum_{k=1}^n a(k)\tag{2b}$$
Letting $z = k + \xi$ we have $\{z\} = \xi$ and the summands become
$$a(k):=\int_0^1 \xi \left(\frac{k+\xi }{\sqrt{(k+\xi )^2-1}}-1\right)\,d\xi
\\=\frac{1}{2} \left(k \sqrt{k^2-1}-(k+1) \sqrt{(k+1)^2-1}\right)+\frac{1}{2} \left(\log \left(k+\sqrt{(k+1)^2-1}+1\right)-\log \left(\sqrt{k^2-1}+k\right)\right)+(\sqrt{(k+1)^2-1}-\frac{1}{2})\tag{3} $$
Summing up from $k=1$ to $k=n$ the first two brackets telescope and one sum is left:
$$i_{s}(n) = p(n) + g(n)\tag{4a}$$
where
$$p(n)=\frac{1}{2} \left(-(n+1)\sqrt{(n+1)^2-1} -n+\log \left(n+\sqrt{n (n+2)}+1\right)\right)\tag{4b}$$
$$g(n) = \sum _{k=2}^{n+1} \sqrt{k^2-1}\tag{5}$$
where in $g$ we have omitted the summand with $k=1$ without altering the sum.
Part 2: asymptotics of $g(n)$
This is the tough part. In order to perform the limit (2a) we need the asymptotic behaviour of the terms in (4). We have to focus on $g(n)$ since the asymptotics of the other terms is simple to obtain.
Writing
$$\sqrt{k^2-1} = k \sqrt{1-\frac{1}{k^2}} = \sum _{m=0}^{\infty } (-1)^m \binom {\frac{1}{2}}{m}\frac{1}{k^{2 m-1}}\tag{6}$$
Performing the $k$-sum, according to $\sum _{k=2}^{n+1} 1/k^{2 m-1}=-1+H_{n+1}^{(2 m-1)}$ we get
$$g(n) = \sum _{m=0}^{\infty } (-1)^m \binom {\frac{1}{2}}{m}(-1+H_{n+1}^{(2 m-1)})\tag{7}$$
Notice that (7) is an exact formula. Now we can take the asymptotic limit (with respect to n) under the $m$-sum using the well-known asymptotics of $H_{n}^{(k)}$ leading to
$$H_{n+1}^{(2 m-1)}-1 \simeq h_0 +h_1 + h_2 \tag{8a}$$
$$h_0=-1, h_1=-\frac{1}{6} m n^{-2 m}+\frac{n^{-2 m}}{12}+\frac{1}{2} n^{1-2 m}-\frac{n^{2-2 m}}{2 m-2}, h_2 = \zeta (2 m-1)\tag{8b}$$
We now insert this into (7) and proceed carefully with the terms and the index $m$. We collect the contributions in $g_i(n)$.
The m-sum over $h_0$ gives $g_0 = 0$.
For the sum $(h_1+h_2)$ we consider first the two summands $m=0$ and $m=1$ separately
$$g_{1}(n) =\lim_{m\to 0}{ (-1)^m \binom {\frac{1}{2}}{m}(h_1+h_2 )}= \frac{n^2}{2}+\frac{n}{2}$$
$$g_{2}(n) =\lim_{m\to 1}{ (-1)^m \binom {\frac{1}{2}}{m}(h_1+h_2 )}= \frac{1}{24 n^2}-\frac{1}{4 n}-\frac{\log (n)}{2}-\frac{\gamma }{2}$$
Notice that for $m\to 1$ we had to keep the sum $(h_1+h_2)$ so that the pole in $\zeta$ at $m=1$ can compete with the other term with a pole.
The rest of the $m$-sum from $m=2$ is considered separately for $h_1$ and $h_2$.
For $h_1$ we could do the complete sum (Mathematica gives a lengthy expression including a hypergeometric function) but up to order $1/n^3$ we need only the term with $m=2$. This gives
$$g_{3}(n) =\lim_{m\to 2}{ (-1)^m \binom {\frac{1}{2}}{m}(h_1 )}= \frac{1}{16 n^2}-\frac{1}{16 n^3}$$
For $h_2$ the complete sum reads
$$g_{4}(n) =\sum_{m=2}^\infty (-1)^m \binom {\frac{1}{2}}{m}\zeta({2m-1})$$
Making use of the well-known integral representation of the $\zeta$-function
$$\zeta (2 m-1)=\int_0^{\infty } \frac{t^{2 m-2}}{\left(e^t-1\right) \Gamma (2 m-1)} \, dt\tag{9}$$
we can do the sum under the integral for which Mathematica gives the result
$$\sum _{m=2}^{\infty } \frac{(-1)^m \binom{\frac{1}{2}}{m} t^{2 m-2}}{\left(e^t-1\right) \Gamma (2 m-1)}= \frac{t-2 I_1(t)}{2 \left(e^t-1\right) t}\tag{10}$$
This gives $g_4(n)$ = $c_{g}$.
Hence we find for the asymptotic behaviour of $g(n)$
$$g_a(n) = g_1+g_2+g_3+g_4 \\
= c_{g}-\frac{1}{16 n^3}+\frac{n^2}{2}+\frac{5}{48 n^2}+\frac{n}{2}-\frac{1}{4 n}-\frac{\log (n)}{2}-\frac{\gamma }{2}\tag{11}$$
Part 3: harvest and final result
To obtain the complete asymptotic Expression according to (4) we need the asymptotics of $p(n)$ which is, however, easily calculated with the result
$$p_a(n) = -\frac{n^2}{2}+\frac{3}{16 n^2}-\frac{3n}{2}+\frac{\log (n)}{2}+\frac{3}{4}+\frac{\log (2)}{2}\tag{12}$$
Adding $p_a(n)$ and $g_a(n)$ the leading terms and the $\log$-terms cancel. Finally, taking the $\lim_{n\to\infty}$ gives
$$i_{s} = c_{g}-\frac{\gamma }{2}+\frac{3}{4}+\frac{\log (2)}{2}\tag{13}$$
which is the result of the OP.
Remark: the simplicity of the final expression surprised me: just a simple fraction, $\log(2)$, and $\gamma$, but at least $c_g$ is a non-trivial quantity which most probably is a new constant.