I'll assume that by "moving freely", you mean that the apices of the small triangles are uniformly distributed over the area in which they can lie such that the small triangles are contained in the large triangle.
Let's introduce Cartesian coordinates with the origin at the apex of the big triangle and the other two vertices at $(1,0)$ and $(0,1)$ and characterise the two small triangles by the coordinates of their apices, $x_1,y_1$ and $x_2,y_2$.
There are two possible configurations: the one you've drawn, where without loss of generality $x_2\gt x_1$ and $y_2\gt y_1$, and another one where without loss of generality $x_2\gt x_1$ and $y_1\gt y_2$.
In the first case, the small triangles overlap in a fraction $\left(\frac12-(x_2-x_1)-(y_2-y_1)\right)^2$ of the big triangle.
In the second case, again without loss of generality, let $x_2-x_1\gt y_1-y_2$. Then the small triangles overlap in a fraction $\left(\frac12-(x_2-x_1)\right)^2$ of the big triangle.
With appropriate symmetry factors of $2$ in the first case and $4$ in the second case, this yields a mean overlap of
\begin{align}
\left(\frac18\right)^{-2}\left(2\int_0^\frac12\mathrm dx_2\int_0^{\frac12-x_2}\mathrm dy_2\int_0^{x_2}\mathrm dx_1\int_0^{y_2}\mathrm dy_1\left(\frac12-(x_2-x_1)-(y_2-y_1)\right)^2\\
+4\int_0^\frac12\mathrm dx_2\int_0^{\frac12-x_2}\mathrm dy_2\int_0^{x_2}\mathrm dx_1\int_{y_2}^{y_2+x_2-x_1}\mathrm dy_1\left(\frac12-(x_2-x_1)\right)^2
\right)\;,
\end{align}
where $\frac18$ is the area in which the apices of the small triangles can lie.
The integrals are readily evaluated in Sage, with the result $\frac1{10}$ confirming your simulations and the answer of a certain other.
