In short, there are two separate misunderstandings in your reasoning: first, thermal physics doesn't apply to single particles, and is only applicable in the limit of a large number of particles. Second, when massless objects like photons are involved, the definition of the "center-of-mass frame" is different, because you need to use special relativity to account for these objects.
One of the assumptions of thermal physics is that there are a large number of particles present. Hence, thermal physics simply doesn't apply when you're talking about a single system consisting of a single photon bouncing around in a room. This means, in particular, that the conclusion from thermal physics that the pressure on each wall should be equal at all times doesn't hold, because thermal physics doesn't apply. With a single photon bouncing off of walls, the pressure on each wall will be a series of approximate delta functions corresponding to each impact of the photon.
What thermal physics does tell you, in these cases, is what would happen in one of the following two equivalent situations:
- You have a large number of rooms, each of which has a randomly-oriented photon in it, and you want to predict what the average pressure on each wall is; or
- You have a single room, but you don't know anything about the orientation of the photon, and you want the expected value of the pressure on each wall.
In those cases, the conclusion from thermal physics applies: averaging over the behavior of many randomly-oriented photons will give you approximately equal pressure on each wall. Likewise, not knowing anything about the photon will lead to the conclusion that it's as likely to hit one wall as any other at any moment in time. But again, this is because you are considering what will happen in the limit of a large number of particles (in the latter case, it's because you are randomly picking a candidate from a pool of a large number of particles). But if you have a single system with a single photon whose trajectory you already know, then thermal physics doesn't apply.
Moving on, when you're setting up your "center-of-mass" frame, you have to be very careful about what definition for the frame you're using. In Newtonian mechanics, the definition of the center of mass is a weighted average of the positions of all objects in the system, where the weights are the objects' masses. Since, in Newtonian mechanics, all particles that carry momentum also have mass (from $p=mv$), the center of mass takes into account all of the momentum present in the system. As such, in the absence of external forces, the center of mass does not accelerate. But a photon internal to the system has zero mass, and therefore its momentum doesn't factor into the center of mass at all, but it has the ability to transfer this momentum to something that does have mass, which means it has the ability to alter the center of mass's motion without external forces being present. Clearly, we need a different definition for "center-of-mass frame" in relativity.
It turns out that the proper definition of the "center-of-mass frame" in relativity is the frame in which the sum of the momenta of all objects is zero (this is why it is sometimes distinguished as the "center-of-momentum" frame). Note that this definition is equivalent to the Newtonian one in the non-relativistic case, in the absence of massless objects (in fact, proving this is assigned as an exercise in some introductory mechanics textbooks), but it also takes the momentum of the photon into account. In the case of a photon inside some walls, applying the proper definition of the center-of-mass frame will give you a frame where both the photon and the walls are moving at just the right speed to cancel out each other's momentum. In this frame, the center of momentum is completely stationary, both before and after the photon collides with the walls, though both the photon and the walls will accelerate as this happens. Once the photon is absorbed by the walls, the walls will become stationary in the center-of-momentum (aka center-of-mass) frame.