Sum, evaluates the sum $\sum_{k=0}^{o}{\lambda^k\,u(k)}$
With[{o = 3},
Sum[λ^k u[k], {k, o}]
]
λ u[1] + λ^2 u[2] + λ^3 u[3]
Series, generates a power series expansion for $F(\sum_{k=0}^{o}{\lambda^k\,u(k)})$ about the point $0$ to order $(\lambda-0)^o$.
With[{o = 3},
Series[F[Sum[λ^k u[k], {k, o}]], {λ, 0, o}]
]

CoefficientList, gives a list of coefficients of powers of $\lambda$ in the previos expression, starting with power 0.
ExpandAll, expands out all products and integer powers in any part of the expression in the argument.
With[{o = 3},
CoefficientList[
ExpandAll[
Series[F[Sum[λ^k u[k], {k, o}]], {λ, 0,
o}]], λ]
]

Follow the links for the documentation.