The total power will be constant as you adjust the resolution bandwidth (RBW), but the power that is indicated with the marker on the spectrum analyzer at any given frequency in band will go down as RBW is reduced, when measuring signals that have occupied bandwidth greater than the resolution bandwidth. This is simply because the marker is indicating how much power is within that resolution bandwidth of what is effectively a bandpass filter to represent that frequency measurement, so if your reduce the bandwidth, you are reducing the amount of power that can get through that filter.
To estimate the total power in the modulated signal, given it has a flat response over a total frequency range $BW$ Hz with a marker indicating the power level is $P$ dBm, use:
$$P_{tot} = P_{marker}-10\log_{10}(RBW)+ 10\log_{10}(BW) $$
$$= P_{marker}+10\log_{10}(BW/RBW)$$
Where all powers given are dB quantities (such as dBm). The result of the first subtraction is a normalized power spectral density in units of dBm/Hz. The final addition of $10\log_{10}(BW)$ is the log approach of multiplying the power spectral density P/Hz by the total bandwidth in Hz resulting in total power P in dB units as dBm.
So for the OP’s screenshots we see (visually) a power level around -50 dBm for the 10 Hz resolution bandwidth, which is a power spectral density of $-50 -10\log_{10}(10) = -60$ dBm/Hz. The total bandwidth appears to be 18 MHz and thus the total power is estimated to be $-60 + 10\log_{10}(18E6) = -17.5$ dBm. In the other screen shot the resolution bandwidth was increased to 10KHz, which is a $10\log_{10}(1000)= 30 $ dB increase in bandwidth, so the marker power should have risen to -20 dBm (something is astray there, typically such measurements are within a few dB of the actual channel power).
The resolution bandwidth is effectively (or equivalently) describing a narrow bandpass filter that sweeps across your signal, and at each instant the power that gets through that filter is presented on the vertical axis of the display, while the center frequency of that filter is presented on the horizontal axis (as further detailed in a block diagram at this post). Thus if we increase the resolution bandwidth, more power gets through and the power displayed in dBm will go up. Using the formula above, we will see that the total power calculated will remain constant.
See page 6-8 of this app note (App Note 1303, old HP) for further details on additional corrections up to 2.5 dB when using the log power or lin power measurements from a spectrum analyzer in order to get "true-rms" power measurements assuming noise floors as limited by Additive White Gaussian Noise (AWGN). Most modern analyzers include a channel power measurement in a given bandwidth utility that already make these compensations, which I would recommend using.