1

I am trying to render depth maps and surface Normals using Cycles and store the results in OpenEXR format using nodes. I make a Blender class and set up the nodes as follow using Blender's Python API:

#blenderClass.py
import importlib

class Blender(object):
    def __init__(self):
        self.bpy = importlib.import_module("bpy")
        self.scene = self.bpy.context.scene
        self.scene.render.use_sequencer = False
        self.scene.display_settings.display_device = 'sRGB'
        self.scene.view_settings.view_transform = 'Raw'
        self.scene.sequencer_colorspace_settings.name = 'Raw'
        self.scene.render.engine = 'CYCLES'
        self.scene.use_nodes = True
        self.setupRenderNodes()

    def setupRenderNodes(self):

        for node in self.scene.node_tree.nodes:
            self.scene.node_tree.nodes.remove(node)
        renderNode = self.scene.node_tree.nodes.new('CompositorNodeRLayers')

        # Depth map
        self.depthOutputNode = self.scene.node_tree.nodes.new('CompositorNodeOutputFile')
        self.depthOutputNode.format.file_format = 'OPEN_EXR'
        self.depthOutputNode.format.color_depth = '32'
        self.depthOutputNode.format.color_mode = 'RGB'
        self.depthOutputNode.format.exr_codec = 'ZIP'
        self.depthOutputNode.base_path = 'somePath/'
        self.depthOutputNode.file_slots[0].path = 'fileNameDepth#'

        # Link
        self.scene.node_tree.links.new(renderNode.outputs[2], self.depthOutputNode.inputs[0])

    def render(self, objPath):
        self.bpy.ops.import_scene.obj(filepath=objPath)
        self.bpy.ops.render.render(write_still=True)

Then I call the rendering function as follow:

#main.py
from multiprocessing import Process
import blenderClass import Blender
blender = Blender()

objPaths = ['obj1.obj', 'obj2.obj', 'obj3.obj', 'obj4.obj']

procList = []
for path in objPaths:
    proc = Process(target=blender.render, kwargs={'objPath': path})
    procList.append(proc)
    proc.start()

for job in procList:
    job.join()

The problem is when I'm trying to store the renderings in EXR format and use multiprocessing package to call bpy.ops.render.render(write_still=True) Blender freezes and the results are not saved on disk. However, if I change OPEN_EXR to PNG and change the color_depth to '16' everything works and I get the rendering results on disk. Also, if I call blender.render() without using multiprocessing everything works as expected. Note that I do not change any of my rendering engine settings like tile size, resolution etc. Does anyone know why Blender freezes and the renderings are not on disk when trying to store them as EXR and use the multiprocessing package?

I'm not sure if this has something to do with the problem that I'm facing but I have compiled Blender manually and import it as a module in the Python installed on my machine. Here are the CMake settings I used to compile Blender with versions 2.78, 2.79 and 2.79b from source. They all have the same issue:

cmake blender \
    -DCMAKE_INSTALL_PREFIX=/usr/lib/python3/dist-packages \
    -DWITH_INSTALL_PORTABLE=OFF \
    -DWITH_PYTHON_INSTALL=OFF \
    -DWITH_PYTHON_MODULE=ON \
    -DPYTHON_SITE_PACKAGES=/usr/lib/python3/dist-packages \
    -DPYTHON_VERSION=3.5 \
    -DWITH_OPENAL=ON \
    -DWITH_CODEC_AVI=ON \
    -DWITH_MOD_OCEANSIM=ON \
    -DWITH_CODEC_FFMPEG=ON \
    -DWITH_SYSTEM_GLEW=ON \
    -DWITH_FFTW3=ON \
    -DWITH_OPENCOLORIO=ON \
    -DWITH_GAMEENGINE=OFF \
    -DWITH_PLAYER=OFF
Amir
  • 3,074
  • 2
  • 29
  • 55
  • 3
    Best report this as a bug. as blender 2.79b cannot save multi layer openexr files using the file output node even when just do it manually. To my knowledge it should be able to do this. A normal save of the multilayer open exr file during render does work. – J. Bakker Mar 29 '18 at 06:21
  • @J.Bakker Thank you. Here's the link to the bug report. – Amir Mar 29 '18 at 13:59
  • @J.Bakker I forgot to mention that the same thing happens when I want to only save depth maps in OPEN_EXR format as well. Did you also experience the same issue? Another question: Is it common for Blender developers not to respond to the reported bugs? How long usually does it take for them to take care of the bug? – Amir Mar 30 '18 at 22:08
  • 1
    Have you tested this manually on your local gui? renderNode.outputs[3] is the render layers normal output, this socket is not visible by default, the output node will not get input from it if it is not visible, therefore no file save. That would mean ensuring the render layers normal option gets enabled. – sambler Mar 31 '18 at 05:10
  • 2
    No worries and be patient. Currently there are the banking holidays. Also devs need some R&R. Next week BI is moving to another location and the code quest will start. So there are events that might influence respond times. – J. Bakker Mar 31 '18 at 08:03
  • @sambler Yes I tested it and things work as normal when using the GUI. I already enable the surface Normal pass, but forgot to post the code here. I'll do it now. – Amir Mar 31 '18 at 14:05
  • @J.Bakker I see. Thanks for the information. Yes I saw a video about the new building yesterday. – Amir Mar 31 '18 at 15:07
  • @J.Bakker I updated my question. It turns out that multiprocessing is causing the freeze up :/ Any ideas on this? – Amir Apr 02 '18 at 15:46
  • @sambler I would like to get your comments on the updated question. A Blender developer posted some comments on the bug report I created. He/she thinks that it's possible that either Blender or OpenEXR have some issues with forking. Do you have any ideas on how one I make sure whether this is a Blender or OpenEXR issue? – Amir Apr 03 '18 at 04:42
  • Using multiprocessing my first thought is threading clashes. I think your issue is from trying to run multiple renders in one instance, one render task will use all cpus so don't expect any benefit from that. To do simultaneous renders on the one machine I would use subprocess.run to spawn new blender instances that won't interact with each other. This and this may inspire other solutions. – sambler Apr 03 '18 at 17:27
  • @sambler I do multiple renders in one instance, sequentially. Basically, the flow is as follow: I use multiprocessing.Process to call a rendering function. I give the rendering function a list of camera positions. The function will do a rendering for each camera position and store the result on disk. Would that make you think differently? – Amir Apr 03 '18 at 20:21
  • @sambler I run multiple processes for different render settings (some with Cycles, some Internal). Each of these processes import their own bpy module and they do not have any conflicts with each other in terms of sharing Blender stuff. – Amir Apr 03 '18 at 20:27
  • The example you show, you create one instance of your Blender class and start four renders using the same instance which all use the same import bpy for all four simultaneous renders. This would be the source of your issue. I can use a python script to run background instances of blender with subprocess that each use a file output node to save to exr and have no problems. Using your multiprocess example I get one frame before freezing. – sambler Apr 05 '18 at 16:36
  • @sambler Thanks for your efforts. I also had decided to somehow use subprocess but I didn't think long enough on how I should do that as I was busy with some other things recently. I do not do four simultaneous renders. I have a four loop in the render() function that does that. Do you think that should work as well? Or should I definitely do import bpy each time I call the sub-process? – Amir Apr 05 '18 at 16:45
  • Your for path... loop starts a background process for each obj file, each starts to render once started, then the script waits for them to finish in the join loop. Have a look at this example I ran it to do four simultaneous animation renders to exr. – sambler Apr 05 '18 at 17:36

0 Answers0