ray_cast function not able to select all the vertices in camera view

Question

The aim of the below code using ray_cast function (thanks to @lemon) is to find all the visible vertices present in the camera view (previous post's Link). It does find some of the vertices of a mesh in the camera view, however, some of the vertices are missed as shown in Figure 1 and 2. But, for my work, I need all the vertices as shown in figure 3 and 4.

import bpy
from mathutils import Vector
from mathutils.bvhtree import BVHTree
from bpy_extras.object_utils import world_to_camera_view

# Create a BVH tree and return bvh and vertices in world coordinates 
def BVHTreeAndVerticesInWorldFromObj( obj ):
    mWorld = obj.matrix_world
    vertsInWorld = [mWorld * v.co for v in obj.data.vertices]

    bvh = BVHTree.FromPolygons( vertsInWorld, [p.vertices for p in obj.data.polygons] )

    return bvh, vertsInWorld

# Deselect mesh polygons and vertices
def DeselectEdgesAndPolygons( obj ):
    for p in obj.data.polygons:
        p.select = False
    for e in obj.data.edges:
        e.select = False

# Get context elements: scene, camera and mesh
scene = bpy.context.scene
cam = bpy.data.objects['Camera']
obj = bpy.data.objects['Cube']

# Threshold to test if ray cast corresponds to the original vertex
limit = 0.0001

# Deselect mesh elements
DeselectEdgesAndPolygons( obj )

# In world coordinates, get a bvh tree and vertices
bvh, vertices = BVHTreeAndVerticesInWorldFromObj( obj )

print( '-------------------' )

for i, v in enumerate( vertices ):
    # Get the 2D projection of the vertex
    co2D = world_to_camera_view( scene, cam, v )

    # By default, deselect it
    obj.data.vertices[i].select = False

    # If inside the camera view
    if 0.0 <= co2D.x <= 1.0 and 0.0 <= co2D.y <= 1.0: 
        # Try a ray cast, in order to test the vertex visibility from the camera
        location, normal, index, distance = bvh.ray_cast( cam.location, (v - cam.location).normalized() )
        # If the ray hits something and if this hit is close to the vertex, we assume this is the vertex
        if location and (v - location).length < limit:
            obj.data.vertices[i].select = True

del bvh

I've modified the above code a bit just to test the behaviour of normal scene.ray_cast function (not the bvh.ray_cast). However, the results are same.

import bpy
from mathutils import Vector
from bpy_extras.object_utils import world_to_camera_view

# Deselect mesh polygons and vertices
def DeselectEdgesAndPolygons( obj ):
    for p in obj.data.polygons:
        p.select = False
    for e in obj.data.edges:
        e.select = False

# Get context elements: scene, camera and mesh
scene = bpy.context.scene
cam = bpy.data.objects['Camera']
obj = bpy.data.objects['Cube']

# Threshold to test if ray cast corresponds to the original vertex
limit = 0.0001

# Deselect mesh elements
DeselectEdgesAndPolygons( obj )

# In world coordinates, get vertices
mWorld = obj.matrix_world
vertices = [mWorld * v.co for v in obj.data.vertices]

print( '-------------------' )

for i, v in enumerate( vertices ):
    # Get the 2D projection of the vertex
    co2D = world_to_camera_view( scene, cam, v )

    # By default, deselect it
    obj.data.vertices[i].select = False

    # If inside the camera view
    if 0.0 <= co2D.x <= 1.0 and 0.0 <= co2D.y <= 1.0: 
        # Try a ray cast, in order to test the vertex visibility from the camera
        ray= obj.ray_cast( cam.location, (v - cam.location).normalized() )
        # If the ray hits something and if this hit is close to the vertex, we assume this is the vertex
        if ray[0] and (v - ray[1]).length < limit:
            obj.data.vertices[i].select = True

Hence, I'm wondering whether there is some error with the ray_cast functions or something I'm doing wrong.

zippy · Answer 1 · 2018-06-28T02:35:44.247

4

Thank you for the excellent suggestions on this problem.

I was also having similar accuracy trouble w/ BVH hit testing, using Animation Nodes.

Based on your suggestions, I discovered much better results by raycasting the path from the vertex back to the camera, then testing whether the ray hit anything in the BVH. If nothing was hit, it's a clear path to the camera.

HTH

Demo file notes: I'm using an empty called Viewer as a proxy for the camera.

Also, Epsilon set to 0 for speed, but on raycast I've got min distance set to 0.01 units so it doesn't get a false positive hit just adjacent to the origin vertex.

edited Jun 28 '18 at 02:35

answered Jun 27 '18 at 05:56

zippy

588
5
11

Interesting approach! Mind sharing screenshots of the node setup or a demo file via https://blend-exchange.giantcowfilms.com/ ? – aliasguru Jun 27 '18 at 06:42
Node layout added and .blend uploaded. Hope it's clear enough. – zippy Jun 28 '18 at 02:36

score 4 · Answer 2 · edited Nov 24 '20 at 23:32

The modified code!:

import bpy
from mathutils import Vector
from bpy_extras.object_utils import world_to_camera_view
Deselect mesh polygons and vertices
def DeselectEdgesAndPolygons( obj ):
    for p in obj.data.polygons:
        p.select = False
    for e in obj.data.edges:
        e.select = False
Get context elements: scene, camera and mesh
scene = bpy.context.scene
cam = bpy.data.objects['Camera']
obj = bpy.data.objects['Cube']
Threshold to test if ray cast corresponds to the original vertex
limit = 0.1
Deselect mesh elements
DeselectEdgesAndPolygons( obj )
In world coordinates, get a bvh tree and vertices
mWorld = obj.matrix_world
vertices = [mWorld @ v.co for v in obj.data.vertices]
print( '-------------------' )
for i, v in enumerate( vertices ):
    # Get the 2D projection of the vertex
    co2D = world_to_camera_view( scene, cam, v )
bpy.ops.mesh.primitive_cube_add(location=(v))
bpy.ops.transform.resize(value=(0.01, 0.01, 0.01))

# By default, deselect it
obj.data.vertices[i].select = False

# If inside the camera view
if 0.0 &lt;= co2D.x &lt;= 1.0 and 0.0 &lt;= co2D.y &lt;= 1.0 and co2D.z &gt;0: 
    # Try a ray cast, in order to test the vertex visibility from the camera
    location= scene.ray_cast(bpy.context.window.view_layer, cam.location, (v - cam.location).normalized() )
    # If the ray hits something and if this hit is close to the vertex, we assume this is the vertex
    if location[0] and (v - location[1]).length &lt; limit:
        obj.data.vertices[i].select = True

score 2 · Accepted Answer · answered Aug 11 '17 at 07:16

2

Has the same problems with raycasts... Seems they are not so reliable on contours. After some digging i just started to shoot several raycasts to small cube around vertex, if any of rays hit the face with this vertex -> vertex is visible

answered Aug 11 '17 at 07:16

IPv6

323
1
10

1

Excellent suggestion and it worked. – Debaditya Aug 11 '17 at 10:09

ray_cast function not able to select all the vertices in camera view

3 Answers3

Deselect mesh polygons and vertices

Get context elements: scene, camera and mesh

Threshold to test if ray cast corresponds to the original vertex

Deselect mesh elements

In world coordinates, get a bvh tree and vertices

Linked