If I render your file I get no problems with the mask being misaligned:


As for the rough edges by the armpit, those are part of the key from the green screen, they are not part of mask you are using as garbage matte.
The keying needs to be refined, so that you get better separation.

You might want to try blender.2.92 so that you don't have to convert from straight to premultiplied.
Read: How to put animated background to keyed video?
I suspect the issue could related to the kind of video compression, or how the image is being decoded by blender. Sometimes, depending on the codec used, blender fails to read the correct frame, and reads one frame ahead or one frame late.
Try prefetching the video, or rendering the video to an image sequence first, and use that image sequence to generate the key.
Please report back if the issue persists
See: Render is duplicating random frames when loaded into 3D tracker