As Kevin P. Barry comments, this is exactly to rule out unwanted objects. For example, consider the following definition of the set $E$ of even integers:
$E$ is the smallest set containing $0$ and closed under the maps $x\mapsto x+2$ and $x\mapsto x-2$.
The "smallest set" condition there is crucial: if we omit it, there are lots of sets satisfying the definition (e.g. $\mathbb{R}$ itself, $E\cup\{\pi+2k: k\in \mathbb{Z}\}$, and so forth). Another famous example of a definition of this form is one of the standard definitions of the natural numbers in set theory:
The set of finite ordinals is the smallest set containing $\emptyset$ and closed under the map $x\mapsto x\cup\{x\}$.
These show up all over the place in mathematics. (A useful term here is "inductive definition" or "recursive definition.") Generally we have the following useful heuristic:
Most of the time, "the smallest set $X$ such that [stuff]" is exactly the intersection of all the sets such that [stuff].
In fact, this is usually how we prove that a smallest such set exists - that's something we can't ignore! (E.g. "The smallest infinite set of natural numbers" isn't a thing.)
Incidentally, I've said a bit more about this in the specific context of formal languages (as in your OP) here.