bgl.Buffer() to bytes

Question

I want to transfer an image that has been rendered in Blender via a socket. I have an implementation that does work already, but with a framerate of 0.5fps.

currentImage stores an image that was rendered.

currentImage = obj["RenderToTexture"].source.image
imageSize = obj["RenderToTexture"].source.size
imageData = np.array(currentImage, dtype=np.uint8)

imageSize = (imageSize[1], imageSize[0])

if(clientSocket!=None):
    try:
        height, width = imageSize
        data = imageData.tobytes()
        transmitData = struct.pack("L", len(data))
        transmitData = transmitData + struct.pack("H", height)
        transmitData = transmitData + struct.pack("H", width)
        transmitData = transmitData + data
        clientSocket.sendall(transmitData)
    except socket_error as e:
        print("Error sending data: ", e.strerror)
        bge.logic.endGame()

The following line slows everything down:

imageData = np.array(currentImage, dtype=np.uint8)

Is there another way of sending the content of currentImage using a socket? Blender tells me, that this is a bgl.Buffer() object.

Why are you packing into a numpy array? A bgl buffer is quite similar to a numpy array. You can convert it into a python list using currentImage.to_list() No idea if that would be faster, but it's something to test — sdfgeoff, Jul 01 '17 at 19:08
It may pay to do some feasibility on the bandwidth required. If you're rendering at 1920x1080, with one byte per channel and four channels - that's something like 8Mb per frame. Can your socket handle 0.5Gbps? (that's gigabytes, so 4 gigabits per second). If you're staying local to the PC, that's probably OK. If you're over a network, even a LAN will likely have issues. — sdfgeoff, Jul 01 '17 at 19:11
Hi sdfgeoff, thanks for your comments. I tried the to_list() methods on currentImage, but then the log outputs "AttributeError: 'bgl.Buffer' object has no attribute 'to_list'". The images will only have a resolution of 800x600 and it'll go to localhost. It will get processed by another application that runs locally. — corporeality_carl, Jul 01 '17 at 19:57
The example code on the buffer page works for me in-game on blender 2.78c If it doesn't, I believe list(bgl_buffer) should also work. — sdfgeoff, Jul 01 '17 at 22:01

score 5 · Accepted Answer · answered Jul 03 '17 at 19:16

I actually solved the problem. I have now >30fps. The solution was the following:

Change

imageData = np.array(currentImage, dtype=np.uint8)

to

imageData = np.asarray(currentImage, dtype=np.uint8)

will let numpy try to get the values using the buffer protocol of numpy (maybe np.array will also do this, didn't check). For more informations about the buffer protocol: https://docs.python.org/3/c-api/buffer.html

Then I extended the source code of Blender by implementing the buffer protocol in the bgl.Buffer code. Now, the numpy call will access the buffer of the given buffer object.

The following lines will need to be inserted into the bgl.c source file:

static int itemsize_by_buffer_type(int buffer_type)
{
    if (buffer_type == GL_BYTE) return sizeof(GLbyte);
    if (buffer_type == GL_SHORT) return sizeof(GLshort);
    if (buffer_type == GL_INT) return sizeof(GLint);
    if (buffer_type == GL_FLOAT) return sizeof(GLfloat);
    return -1;  /* should never happen */
}

static const char *bp_format_from_buffer_type(int type)
{
    if (type == GL_BYTE) return "b";
    if (type == GL_SHORT) return "h";
    if (type == GL_INT) return "i";
    if (type == GL_FLOAT) return "f";
    return NULL;
}

static int BPy_Buffer_getbuffer(Buffer *self, Py_buffer *view, int flags)
{
    void* buffer = self->buf.asvoid;
    int itemsize = itemsize_by_buffer_type(self->type);
    // Number of entries in the buffer
    const unsigned long n = *self->dimensions;
    unsigned long length = itemsize * n;

if (PyBuffer_FillInfo(view, (PyObject *)self, buffer, length, false,     flags) == -1) {
        return -1;
    }

    view->itemsize = itemsize;
    view->format = (char*)bp_format_from_buffer_type(self->type);

    Py_ssize_t *shape = MEM_mallocN(sizeof(Py_ssize_t), __func__);
    shape[0] = n;
    view->shape = shape;

    return 0;
}

static void BPy_Buffer_releasebuffer(Buffer *UNUSED(self), Py_buffer *view)
{
    MEM_freeN(view->shape);
}

static PyBufferProcs BPy_Buffer_Buffer = {
    (getbufferproc)BPy_Buffer_getbuffer,
    (releasebufferproc)BPy_Buffer_releasebuffer,
};

Last, but not least, you need to change the following line:

/* Functions to access object as input/output buffer */
NULL,         /* PyBufferProcs *tp_as_buffer; */

into

/* Functions to access object as input/output buffer */
&BPy_Buffer_Buffer,         /* PyBufferProcs *tp_as_buffer; */

I also created a patch for that and I'll hope that this will make its way into a future version of Blender: https://developer.blender.org/D2734

bgl.Buffer() to bytes

1 Answers1

Linked