Electromagnetic waves emitted by the magnet carry away momentum and energy, as per Planck's relations
$$E=\hbar\omega, \mathbf{p}=\hbar\mathbf{k},$$
which means that there is a radiative back-action force on the magnetic moments emitting the radiation. Under the action of this force the magnetic moment might either
- flip (or otherwise change its direction - depending on the type of magnetism and whether we discuss these moments classically or in quantum language)
- transfer force/moment to the crystal (i.e., the bulk of the magnet)
The first effect would mean the randomization of the direction of magnetic moments, i.e., gradual loss of magnetization. The second effect would mean slowing the rotation of the magnet as a whole. Which of the two dominates depends obviously on how well the moments are anchored in their positions. For typical ferromagnets slowing is more likely, but there are probably "borderline" materials where interesting effects could be observed. Note also that the magnitude of the effect depends on many factors: how fast the magnet rotates, the geometry of the system, the size of the magnet (since flipping a few moments among billions would produce very insignificant changes), etc.