I'm sure there's a simpler answer to this question, but this was the best I could come up with.
One can modify the Vitali set construction in $[0,1]$ such that the resulting nonmeasurable set $V$ has outer measure $1$ and inner measure $0$. This problem is discussed at length here. Then put
$$A = \bigcup_{n\in\mathbb{Z}} (V+n),$$
which is clearly nonmeasurable.
We claim the inner measure $0$ property is retained. Indeed, given $F$ compact with $F\subset A$, we have $(F\cap [n,n+1])-n\subset V$, so we find $\mu(F\cap[n,n+1]) = 0$ and thus $\mu(F)=0$. This proves $\mu_*(A\cap E)\leq\mu_*(A) = 0$ for any measurable set $E$.
To prove the outer measure property, we first assume the measurable set $E$ is bounded, so there is positive integer $N$ large enough that $E\subset [-N,N]$. Also put $A_N: = A\cap [-N,N]$, which is easily seen to have outer measure $2N$. Applying Carathéodory criterion gives
$$\mu^*(A_N) = \mu^*(E\cap A_N)+\mu^*(E^c\cap A_N).$$
Since $E^c\cap A_N\subset [-N,N]\setminus E$, we have
$$\mu^*(E^c\cap A_N)\leq \mu^*([-N,N]\setminus E) = 2N-\mu(E).$$
Thus our inequality becomes $2N\leq \mu^*(E\cap A_N)+2N-\mu(E)$. Rearranging,
$$\mu(E) \leq \mu^*(E\cap A_N)\leq \mu^*(E) =\mu(E),$$
so $\mu(E)= \mu^*(E\cap A_N) = \mu^*(E\cap A)$ in this case.
In the general case, we use inner regularity of measure and continuity of outer measure from below. Measurability of $E$ implies $E=F\cup Z$, where $F$ is $F_\sigma$ and $Z$ is measure $0$. It is sufficient to prove $\mu^*(F\cap A) = \mu(E)$ (exercise). Now, there is a sequence of compact sets $F_n\nearrow F$ such that $\mu(F_n)\nearrow\mu(F)=\mu(E)$ as $n\to\infty$. Each $F_n$ satisfies $\mu(F_n) = \mu^*(F_n\cap A)$. Yet $F_n\cap A\nearrow F\cap A$, so
$$\mu^*(F\cap A) = \lim \mu^*(F_n\cap A) = \lim \mu(F_n) = \mu(E).$$
This completes the proof.