12

I have a scene with a many objects and one camera.

I would like to select the objects that the camera sees, even partially.

Is there a way to do that in Python that accounts for the objects bounds?

Note, I know that KD trees and Octrees are used for the lookup. There is mathutils.kdtree but the example only suggests how to query objects close to a point. I'm not familiar with the math for combining camera frustrum and binary trees lookup.

ideasman42
  • 47,387
  • 10
  • 141
  • 223
nantille
  • 819
  • 6
  • 21
  • You might render the scene with shadeless materials and identify the visible objects by their color. In the next blender version there will be a new module which allows offscreen rendering. – pink vertex Jan 19 '16 at 14:05
  • yes there may be a way and some decent workarounds, but it's not clear how you produce the mesh for splines: bevel / extrude? by another spline as taper? by mesh deformed by spline? - Here I see an issue, cause you may need to convert to mesh temporary anyway. After all you wanna see if the Resulting Mesh is in view and not the Spline line. So how are the splines/meshes or lod defined? (for mesh I have sort of a solution, but I'm not sure how to deal with splines) – o.g. Jan 20 '16 at 12:57
  • I'm thinking about doing multiple kdtree searches in space along the vector that follows where the camera points to. First I could have a small hidden sphere (not an object, a math one) that follows the camera and everything colliding with it is meshed. That way nearby objects are meshed and then with several lookups in space a proper intervals, I find other objects in the camera field of view. – nantille Jan 20 '16 at 15:12
  • You could use a kdtree if you have many many objects and so you filter a big part of them at first. Then perform the refined check only on a smaller list (as in example I gave). - For large amounts vs speed it makes sense – o.g. Jan 21 '16 at 10:56
  • The only advantage of using a tree structure here would be if you have a static scene and want to move the camera around it. - In that case you could build a tree and re-use it for lookups. – ideasman42 Jan 21 '16 at 21:43
  • http://blender.stackexchange.com/questions/27937/what-are-the-formulas-to-define-the-cameras-view-cone might be useful to determine what is in-camera. – Mutant Bob Jan 21 '16 at 22:07
  • ideasman42, editing is fine but you could have left what I wrote about curves somewhere in a comment to answer o.g.'s question and yes I do have a static scene in this case, hence the possibility of using a tree. But I'd rather not limit the answers to this case. – nantille Jan 22 '16 at 08:05

3 Answers3

8

There are a few different ways to handle this, you could...

  • project into 2d space, then detect which objects are in the 2d frame.
  • calculate the camera bounds in 3d and detect which objects are inside it.

Here is some sample code that demonstrates the second method. It uses a set of planes (4 or 5 for orthographic cameras) and finds all objects that have any part of their bounding boxes within the planes:

Note, this isn't all that elegant, we could for example have a single function that intersects 2 sets of planes - one for the camera - another for the bound-box. However this is at least working and can give you some starting point - others may like to improve or post a method that uses projection.

def camera_as_planes(scene, obj):
    """
    Return planes in world-space which represent the camera view bounds.
    """
    from mathutils.geometry import normal

camera = obj.data
# normalize to ignore camera scale
matrix = obj.matrix_world.normalized()
frame = [matrix @ v for v in camera.view_frame(scene=scene)]
origin = matrix.to_translation()

planes = []
from mathutils import Vector
is_persp = (camera.type != 'ORTHO')
for i in range(4):
    # find the 3rd point to define the planes direction
    if is_persp:
        frame_other = origin
    else:
        frame_other = frame[i] + matrix.col[2].xyz

    n = normal(frame_other, frame[i - 1], frame[i])
    d = -n.dot(frame_other)
    planes.append((n, d))

if not is_persp:
    # add a 5th plane to ignore objects behind the view
    n = normal(frame[0], frame[1], frame[2])
    d = -n.dot(origin)
    planes.append((n, d))

return planes





def side_of_plane(p, v):
    return p[0].dot(v) + p[1]


def is_segment_in_planes(p1, p2, planes):
    dp = p2 - p1


p1_fac = 0.0
p2_fac = 1.0

for p in planes:
    div = dp.dot(p[0])
    if div != 0.0:
        t = -side_of_plane(p, p1)
        if div > 0.0:
            # clip p1 lower bounds
            if t >= div:
                return False
            if t > 0.0:
                fac = (t / div)
                p1_fac = max(fac, p1_fac)
                if p1_fac > p2_fac:
                    return False
        elif div < 0.0:
            # clip p2 upper bounds
            if t > 0.0:
                return False
            if t > div:
                fac = (t / div)
                p2_fac = min(fac, p2_fac)
                if p1_fac > p2_fac:
                    return False

## If we want the points
# p1_clip = p1.lerp(p2, p1_fac)
# p2_clip = p1.lerp(p2, p2_fac)        
return True





def point_in_object(obj, pt):
    xs = [v[0] for v in obj.bound_box]
    ys = [v[1] for v in obj.bound_box]
    zs = [v[2] for v in obj.bound_box]
    pt = obj.matrix_world.inverted() @ pt
    return (min(xs) <= pt.x <= max(xs) and
            min(ys) <= pt.y <= max(ys) and
            min(zs) <= pt.z <= max(zs))


def object_in_planes(obj, planes):
    from mathutils import Vector


matrix = obj.matrix_world
box = [matrix @ Vector(v) for v in obj.bound_box]
for v in box:
    if all(side_of_plane(p, v) &gt; 0.0 for p in planes):
        # one point was in all planes
        return True

# possible one of our edges intersects
edges = ((0, 1), (0, 3), (0, 4), (1, 2),
         (1, 5), (2, 3), (2, 6), (3, 7),
         (4, 5), (4, 7), (5, 6), (6, 7))
if any(is_segment_in_planes(box[e[0]], box[e[1]], planes)
       for e in edges):
    return True


return False





def objects_in_planes(objects, planes, origin):
    """
    Return all objects which are inside (even partially) all planes.
    """
    return [obj for obj in objects
            if point_in_object(obj, origin) or
               object_in_planes(obj, planes)]


def select_objects_in_camera():
    from bpy import context
    scene = context.scene
    origin = scene.camera.matrix_world.to_translation()
    planes = camera_as_planes(scene, scene.camera)
    objects_in_view = objects_in_planes(scene.objects, planes, origin)


for obj in objects_in_view:
    obj.select_set(True)




if name == "main":
    select_objects_in_camera()
batFINGER
  • 84,216
  • 10
  • 108
  • 233
ideasman42
  • 47,387
  • 10
  • 141
  • 223
  • 2 questions: 1. isn't this what basically bpy_extras.object_utils.world_to_camera_view is giving as a factor (0-1 inside screen)? and 2. the bbox should be a bit larger (with a tolerance or so) so that it considers the result with bevel radius? or is that already there ? – o.g. Jan 21 '16 at 11:04
  • its just a different way to test - using planes instead of projection, wouldn't say its the same though. 2) As for padding the bound-box, curve bevel is taken into account, but perhaps render subdivision levels change the bound-box slightly - so you may want to do this, it depends on your exact use case - YMMV.
    • – ideasman42 Jan 21 '16 at 11:43
  • ok, they are not the same and it may be better to clearly check by planes. As for padding, op stated that "he will not mesh the curves" unless in view, so we can assume the bbox goes only for the naked spline. - Thus the need for padding bbox or camera planes (probably simpler) to compensate for a "future" mesh, slightly larger than the initila bbox. – o.g. Jan 21 '16 at 12:06
  • very cool answers both of you, I'll try all this between today and tomorrow, thank you ! – nantille Jan 21 '16 at 13:24
  • There was an error in clip_segment_in_planes, now fixed. – ideasman42 Jan 23 '16 at 01:27
  • @ideasman42 how can I modify your code, so it only selects an Object when it is for at least for example 20% in the camera view? I couldn't figure it out... – Moritz Hinderer Mar 01 '18 at 16:54
  • What do you mean by "20% in the camera view" ? – ideasman42 Mar 02 '18 at 00:29
  • @ideasman42 Worked well. Although for Blender 2.8 I had to change '*' to '@' for multiplying Matrix and Vector types. – satishgoda Jan 10 '20 at 21:00
  • Updated the multiplication for 2.8. – ideasman42 Jul 10 '20 at 01:16
  • I updated the script to better handle intersections more accurately outside the frustrum: https://blender.stackexchange.com/questions/104989/how-to-not-select-objects-behind-main-camera-with-script-in-python/242346#242346 – S. Magnusson Nov 04 '21 at 02:50