Firstly, I must be pedantic and say that the collection of all open sets in $\mathbb R$ is not an algebra (it is not closed under complementation).
Now that the pedantry is over with, let's get to business. Consider the algebra $\mathcal A$ of all finite unions of half-open intervals in $\mathbb R$ of the form $[a,b)$ for some $a<b$ (along with the associated rays $(-\infty, b)$ and $[a,\infty)$). I claim the $\sigma$-algebra this generates is the Borel $\sigma$-algebra on $\mathbb R$ (this isn't too difficult to see). Now let $\mathcal A_\sigma$ denote the collection of all countable unions of elements of $\mathcal A$. This contains all open sets, as each open set is a countable union of open intervals, and open intervals are countable unions of half-open intervals. But I claim that the closed set
$$K=\{0\}\cup\{\frac{1}{n}:n\in\mathbb N\}$$
is not in $\mathcal A_\sigma$. To see this, suppose $K=\cup_{n\in\mathbb N}E_n$ where $E_n\in\mathcal A$. Then $0\in E_k$ for some $k$, hence there is some $\varepsilon>0$ such that $[0,\varepsilon)\subset E_k\subset\cup_{n\in\mathbb N}E_n=K$, which is clearly false.
To obtain a $\sigma$-algebra, once you add all countable unions you have to add the complements of all countable unions. Then do this again. And again. And again. And once you've done this countably many times, take the union over everything. Then start over. And once you've done this uncountably many times, you finally have a $\sigma$-algebra. For a better account of this, see the notes at the end of chapter 2 of Folland's Real Analysis.