Suppose $u^*$ is the minimiser of $f$ on $U$.
Let $g(u) = \sup \{ h(u) | h \text{ is convex}, h \le f \}$. Note that $u \mapsto f(u^*)$ is convex and majorised by $f$ hence $g(u) \ge f(u^*)$ for all $u \in U$.
Suppose $u' \in U$ with $u' \neq u$ is such that $g(u') = f(u^*)$.
For $r>0$ (and small enough so that $U$ is not completely contained in $B(u^*, r)$), define $m(r) = \min\{ f(u) | u \in U \setminus B(u^*, r) \}$. Note that $m(r) >f(u^*)$ for those $r>0$ for which $m$ is defined.
Excuse the giant picture, attempts to scale failed, but having the picture helps understand the construction.

Let $d = u'-u$ and $u'' \in \operatorname{argmax}_{u \in U} \langle d, u \rangle$. Define the affine function $a(u) = f(u^*)+\alpha \langle d, u -{1 \over 2}(u^*+u')\rangle $ and $\alpha$ is chosen such that
$a(u'') = m({1 \over 2}\|u'-u^*\|))$.
Note that if $u\in U$ is such that $\langle d, u -{1 \over 2}(u^*+u')\rangle \ge 0$ then $u \in U \setminus B(u^*, r)$, where
$r = {1 \over 2}\|u'-u^*\|$ and so $f(u) \ge m(r) > f(u^*)$. Hence
$a$ is majorised by $f$ and since $g(u') \ge a(u') > f(u^*)$ we obtain a contradiction.
Hence no such $u'$ exists.