Suppose $\alpha\notin g(\mathbb R\setminus\{\beta\})$.
Let $\epsilon>0$ be arbitrary. Then there exists a $\delta>0$ such that $0<|x-\alpha|<\delta\Rightarrow|f(x)-L|<\epsilon$.
Furthermore, there exists a $\eta>0$ such that $0<|y-\beta|<\eta\Rightarrow|g(y)-\alpha|<\delta$. Since $g(y)\ne\alpha$ by our hypothesis, we have $0<|g(y)-\alpha|<\delta$. Then, by my statement above, this implies that $|f(g(y))-L|<\epsilon$, which shows that $\lim_{y\to\beta}f(g(y))=L$.
P.S.: You can get away with assuming that for a small enough $\epsilon>0$ there are no $x\in\mathbb R\setminus\{\beta\}$ with $|x-\beta|<\epsilon$ such that $\alpha=g(x)$.
When this is not the case, there is a sequence of points $y_n\in\mathbb R\setminus\{\beta\}$ with $y_n\to\beta \ (n\to\infty)$ such that $g(y_n)=\alpha$. Then, $\lim_{n\to\infty}f(g(y_n))=f(\alpha)$, so that $\lim_{y\to\beta}f(g(y))=f(\alpha)$ if this limit exists.