1

I am currently trying to translate a 3D object in my Blender scene to a region of interest in the respective 2D render. I have found these solutions 1, 2, and 3 that make use of the world_to_camera_view function within the bpy_extras.object_utils package. However, in my application, the region of interest is always offset slightly: 2D render mimicking depth camera

The ROI (denoted by the blue square) should encompass the left hand. However, while it appears to be the right size, its position is offset.

Here is my script.


def camera_view_bounds_2d():
    scene = bpy.context.scene
    camera_object = scene.camera

#focal object for region of interest, selected in UI
mesh_object = bpy.data.objects[scene.Bounding_Box]

x_pix_co = []
y_pix_co = []

verts = (vert.co for vert in mesh_object.data.vertices)
for co in verts:
    world_co = mesh_object.matrix_world @ co
    co_2d = world_to_camera_view(scene, camera_object, world_co)

    render_scale = scene.render.resolution_percentage / 100
    render_size = (
        int(scene.render.resolution_x * render_scale),
        int(scene.render.resolution_y * render_scale),
    )

    x_pix_co.append(round(co_2d.x * render_size[0]))
    y_pix_co.append(round(co_2d.y * render_size[1]))

min_x = min(x_pix_co)
max_x = max(x_pix_co)
min_y = min(y_pix_co)
max_y = max(y_pix_co)

print(f"the bounding box is ({min_x},{min_y}), ({max_x}, {max_y})")

It's worth noting that the hand is animated, and a track_to constraint has been applied to it to ensure that it is always in the view of the camera.

GB-DEV
  • 21
  • 2
  • Recently answered something roundabouts similar here https://blender.stackexchange.com/a/214597/15543 in that the camera is being shifted to frame ROI around OOI & to test created a 2D frame object as a visual. Can confirm the results (pre multiplication by pixels) are same. Make sure you have same X & Y pixel aspect and ii) how are you positioning the ROI on the image? My guess is that the unknown here is the issue, as what is "scene.bounding_Box" ... is it for example a sphere copy loc constrained to the hands vertex groups or a hand bone? – batFINGER Mar 20 '21 at 21:37
  • @batFINGER Thanks for the link, it looks like I might be able to modify your code to suit my needs. The X and Y aspect ratio (or height and width) are taken directly from the render image shape. The ROI is positioned with cv2 in a separate script using the values returned by the print statement: print(f"the bounding box is ({min_x},{min_y}), ({max_x}, {max_y})") – GB-DEV Mar 23 '21 at 22:07
  • @batFINGER "scene.bounding_Box" is the hand mesh that I would like to calculate the bounding box for. I have seperate empty object with a copy location constraint applied a bone in the hand mesh. This is what the camera is tracked to instead of the hand mesh itself because that origin is static. – GB-DEV Mar 23 '21 at 22:30

0 Answers0