You must assume that $\mu$ is finite. Otherwise, this may not be true, as pointed out in the comments above. Here is another counterexample to show that $\sigma$-finiteness is not enough: let $X=[0,1]$ with the usual topology and let $\mu(A)$ be the number of rational points in $A$. Since $X$ is the union of a countable number of singletons $\{p\}$, $p\in[0,1]\cap\Bbb Q$, each of which has measure $1$, together with $[0,1]\setminus\Bbb Q$, which has measure $0$, $\mu$ is $\sigma$-finite. However, $\mu(\{0\})=1$, but any open set containing $\{0\}$ must have infinite measure.
Given that $\mu$ is finite, the proof is straightforward. Define the set of Borel sets which are well-approximable by open sets outside and closed sets inside,
\begin{eqnarray*}
{\cal S}&:=&\{S\mid S \text{ is Borel, and for all } \epsilon>0,\text{ there exist}\\
&& \ \ \ \ \ \ \ \ \text{open } G\supseteq S \text{ and closed } F\subseteq S \text{ with } \mu(G\setminus F)<\epsilon\}.
\end{eqnarray*}
If $\cal S$ contains all closed sets and is a $\sigma$-algebra, then it will contain all Borel sets.
To show that $\cal S$ contains all closed sets, if $F$ is closed, write
$$G_n:=\{x\in X\mid d(x,F)<\frac 1n\}, \qquad n=1, 2, 3, \dots$$
Each $G_n$ is open, $G_1\supseteq G_2\supseteq \cdots$, and since $F$ is closed,
$$
\bigcap_n G_n=\{x\in X\mid d(x,F)=0\}=F.
$$
So, $\mu(G_n)$ must converge to $\mu(F)$. This shows that every closed set is in $\cal S$.
Since the complement of an open set is closed and vice-versa, $\cal S$ is closed under complementation. The only remaining step is to prove that it is closed under infinite unions. Now if $\epsilon>0$ and $S$ is the union of $S_1$, $S_2$, $S_3$, $\dots$, where each $S_i\in\cal S$, then by hypothesis there are $F_i\subseteq S_i\subseteq G_i$ with $\mu(G_i\setminus F_i)<\epsilon/2^{i+1}$ ($F_i$ closed, $G_i$ open, $i=1$, $2$, $\dots$) To get an open set $G$ containing $S$ you can simply set $$G:=\bigcup_i G_i.$$ To get a closed set
$F$ contained in $S$ which approximates $S$, set $$\hat F_j:=\bigcup_{i\le j} F_i,\ \ \hat F:=\bigcup_i F_i.$$
$\hat F$ approximates $S$ from inside, but it might not be closed. However, since $\hat F_1\subseteq \hat F_2\subseteq \cdots$ and $\bigcup_i \hat F_i=\hat F$,
there must be some $j$ such that
$$
\mu(\hat F_j) > \mu(\hat F) - \frac{\epsilon}{2}.
$$
Now, $\hat F_j \subseteq S$ is closed, $G\supseteq S$ is open, and
$$
\mu(G\setminus \hat F_j)=\mu(G\setminus \hat F)+\mu(\hat F \setminus {\hat F}_j)\le\left( \sum_i \mu(G_i\setminus F_i)\right)
+\frac\epsilon 2< \epsilon.
$$
This completes the proof that $\cal S$ contains all Borel sets.
Now, to show the first equality in the question,
$$ \mu(A)=\inf\{\mu(O) \mid A\subseteq O, O \text{ open}\},\qquad (*)$$
let $A$ be an arbitrary Borel set and let $I_A$ be the right-hand side of $(*)$. If $A\subseteq O$, then $\mu(O)\ge \mu(A)$, so the infimum must also be at least as large as $\mu(A)$: $I_A\ge \mu(A)$. Fix some $\epsilon>0$. Since $A\in\cal S$, there are sets $F$ and $G$ with $F$ closed, $G$ open, $F\subseteq A\subseteq G$, and $\mu(G\setminus F)<\epsilon$. Since
$$\mu(G)=\mu(G\setminus A)+\mu(A)<\mu(A)+\epsilon,$$
this shows that $I_A<\mu(A)+\epsilon$. Letting $\epsilon\to 0$, $\mu(A)=I_A$. This proves $(*)$. Since $X$ is assumed to be compact, you can change "compact" to "closed" in the second equality in the question. It can then be proved in the same way as the first equality.
