Yes it is true under a suitable choice of the topology used to define the integral of an operator-valued function.
First of all we have to define the integral of a function that is operator-valued $I\ni t \mapsto A_t$, where all operators $A_t: D\to H$ have a common domain $D$ in the Hilbert space $H$. If the integral were a sum, the following property would be very obvious.
$$\left\langle y,\sum_i A_{t_i} \Delta t_i ~ x\right\rangle = \sum_i \langle y, A_{t_i} ~ x\rangle \Delta t_i \quad \forall x\in D, y\in H\:.$$
Independently of the notion of integral we want to use, we expect that this property be valid also for integrals since they are the limit of sums. More precisely, the minimal requirement on the used topology in computing the limit is that it is compatible with the continuity of the scalar product, so when passing from sums to integrals, taking a suitable limit of Riemann sums, we have
$$\left\langle y,\int_I A_t dt ~ x\right\rangle = \int_I \langle y, A_t ~ x\rangle dt \quad \forall x\in D, y\in H\:.$$
Our first goal is therefore to define a, possibly unique, operator $\int_I A_tdt$ that satisfies the identity above.
The easiest case is that of a map $$I \ni t \mapsto A_t$$
where $I$ is a real interval and the operators $A_t$ have a common (sub)domain $D$ and satisfy the following requirement. For every $x\in D$ the map $I\ni t \mapsto A_tx$ is continuous in the Hilbert space topology (in other words, $I\ni t \mapsto A_t$ is strongly continuous on its natural domain).
As a consequence, assuming $I$ compact, there is a constant $C_x< +\infty$ such that
$||A_t x||\leq C_x$ for all $t\in I$.
The typical situation is $A_t = e^{itB}$ for some selfadjoint operator $B$, and in that case $D$ is the whole Hilbert space $H$ and $C_x= ||x||$ and we can also drop the compactness of $I=\mathbb R$. However also $A_t:= e^{itB}B^n$ is another, less trivial, case with $D:=Dom(B^n)$ and $C_x= ||B^nx||$.
In the said hypotheses, the map
$$H \ni y \mapsto f(y)_x:= \int_I \langle y, A_t x\rangle dt $$
is well defined, antilinear and continuous, because
$$|f(y)_x| \leq ||y|| C_x \int_I dt\:.$$
Riesz' lemma implies that there is a unique vector denoted by
$$\int_I A_t dt ~ x\in H$$
such that
$$f(y)_x = \left\langle y,\int_I A_t dt ~ x\right\rangle $$
At this point, using uniqueness of the said vector and $x$-linearity of the map $f(y)_x$, it is not difficult to see that the map
$$D\ni x \mapsto\int_I A_t dt ~ x \tag{1}$$
is linear as well.
In summary, there exists a unique operator indicated as in (1) such that
$$\left\langle y,\int_I A_t dt ~ x\right\rangle = \int_I \langle y, A_t ~ x\rangle dt \quad \forall x\in D, y\in H\:.$$
Since we have established a good definition of integral of operator valued functions, we are in a position to answer the raised question. I recall that a sequence of operators $A_{n}$ all defined on the common domain $D\subset H$ is said to weakly converge to $A: D \to H$ if
$$\langle y, A_n x\rangle \to \langle y, Ax\rangle \quad \mbox{as $n\to +\infty$}\quad \forall x\in D, y\in H$$
This seminormed topology is the one for instance used in the LSZ formalism in QFT and it plays a crucial role in physics. It is one of the seven relevant topologies (see comments below).
PROPOSITION. Suppose that the above hypotheses to define the operator $\int_I A_t dt :D \to H$ are valid with $I:= [t_0,t_1]$ and take $t\in (t_0,t_1)$. Then in weak sense it holds
$$\frac{d}{dt} \int_{t_0}^t A_s ds = A_t\:.$$
PROOF.
The thesis is equivalent to
$$\lim_{h\to 0} \frac{1}{h}\left(\int_{t_0}^{t+h}\langle y, A_s x\rangle ds -\int_{t_0}^{t}\langle y, A_s x\rangle ds \right) = \langle y, A_t ~ x \rangle\quad \forall x\in D, y\in H$$
This identity is true because it is equivalent to
$$\frac{d}{dt}\int_{t_0}^t \langle y,A_s x\rangle ds = \langle y,A_t x\rangle$$
which is trivially true because the map
$$I \ni t \mapsto \langle y,A_t x\rangle$$
is continuous and thus one may apply the Fundamental Theorem of Calculus.
QED
ADDENDUM. In case $I\ni t\to A_t$ is such that all $A_t:H\to H$ are bounded and there is a common bound of the norms $||A_t||< C < +\infty$ for all $t\in I$, then everything works with the only hypothesis that $I\ni t \mapsto A_t$ is weakly continuous. The proof is a trivial re-adaptation of that above.
