What follows is more of a comment on the formula cited by
@MarkusScheuer, which we can actually prove. Observe that when $a=0$
we get
$$[t^n] (a+bt+ct^2)^n = [t^n] t^n (b+ct)^n = b^n$$
so we may suppose that $a\ne 0.$
We start from
$$[t^n] (a+bt+ct^2)^n =
\frac{1}{2\pi i}
\int_{|t|=\epsilon}
\frac{1}{t^{n+1}} (a+bt+ct^2)^n
\; dt.$$
We put
$$w = \frac{t}{a+bt+ct^2} = \frac{1}{a} t + \cdots$$
The series tells us that the circle $|t|=\epsilon$ is mapped to a
closed circle in $w$ (one turn) of dominant radius $\epsilon/a$ plus
lower order fluctuations, which we may deform to a circle $|w|=\gamma$
inside said contour, where the map is also analytic in $w.$ This means
that from the two branches
$$t = \frac{1-bw\pm \sqrt{(b^2-4ac)w^2-2bw+1}}{2cw}$$
we must choose the branch
$$t = \frac{1-bw-\sqrt{(b^2-4ac)w^2-2bw+1}}{2cw}
= aw + \cdots$$
(the other one has a singularity at the origin). This converges
(distance to the nearest singularity) either with radius $1/2/b$ when
$b^2-4ac=0$ or the distance from the origin to whichever of
$$\frac{-b\pm 2\sqrt{ac}}{4ac-b^2}$$
is closer. We take $\gamma$ so that $|w|=\gamma$ is inside this
region. With
$$dt = \frac{(a+bt+ct^2)^2}{a-ct^2} \; dw
= \frac{t^2/w^2}{a-ct^2} \; dw.$$
we thus finally obtain the integral
$$\frac{1}{2\pi i}
\int_{|w|=\gamma}
\frac{1}{w^n} \frac{1}{t} \frac{t^2/w^2}{a-ct^2}
\; dw
= \frac{1}{2\pi i}
\int_{|w|=\gamma}
\frac{1}{w^{n+1}} \frac{t/w}{a-ct^2}
\; dw.$$
We claim that
$$\frac{t/w}{a-ct^2} = \frac{1}{1-bw-2ctw}.$$
This is equivalent to
$$t(1-bw-2ctw) = w (a-ct^2)$$
or
$$t = w (a + bt + 2ct^2 - ct^2)
= w \times t / w$$
which holds by inspection. Now
$$\frac{1}{1-bw-2ctw}
= \frac{1}{\sqrt{(b^2-4ac)w^2-2bw+1}}$$
and we have shown that
$$\bbox[5px,border:2px solid #00A000]{
[t^n] (a+bt+ct^2)^n
= [w^n] \frac{1}{\sqrt{(b^2-4ac)w^2-2bw+1}}.}$$