I'm trying to pull the phone wire in my new house, and replace it with Cat6 cable for Ethernet. Upon removing the faceplate for the dual cable (coax) and phone jack, I found what I believe is this gang box:

Obviously I can't see the bracket for the drywall atop it, so I can't be completely sure this is what I've got, but the blue box looks almost exactly like mine...
I've already unscrewed the screw until it (apparently) came out of the bracket, in an effort to remove the box so I could see into the wall at how the phone line is run. Unfortunately, I can't pull it out -- seems that it can't slide off the bracket? Double-unfortunately, the screw seems to have come out of the bracket, and I can't put the box back in the wall, either.
Can I remove this box without putting a big hole in my wall to get at the bracket? I was hoping to avoid that (literal) mess for this little project, and just pull the phone wire and run the Ethernet cable in its place. Is my only option cutting into the drywall anyway?