Visualize rotational data from IMU in real-time

Question

I'm looking for workflows to rotate an object in Blender in real-time with a stream of quaternion data from an inertial measurement unit (IMU). The quaternion data can be provided in whatever way or form is neccessary, so it will suit a proposed Blender workflow.

Answer 1

I learned from the comments and references here that the quaternion data could be provided to Blender via an UDP socket. Then a modal operator could be used to manipulate the properties of the object, which has to be written in Python, using the Blender Python API.

Unfortunately the approach given in Object Controlling using Arduino, which is using a modal operator, did not work for me. I adapted the script to process the rotational data but my version of the script failed because it tried to open the socket multiple times.

Later I came across a very nice blog post by Michał Ciesielski about using sockets in Blender. Instead of a modal operator he is using an application timer. I adapted his code a little to suit my needs.

Here's how I ran the code (on Linux) to make the little cube rotating, which sits there when you start a new project:

1. Create or open the Blender script in the 'Scripting' workspace.

    import bpy
    import socket
   HOST = '127.0.0.1'
    PORT = 65432
   s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    s.bind((HOST, PORT))
   data = None
    bpy.data.objects['Cube'].rotation_mode = 'QUATERNION'
   def handle_data():
        interval = 0.33
        try:
            data, addr = s.recvfrom(128)
            if data.decode('utf-8') == 'quit':
                interval = None
                s.close()
            else:
                bpy.data.objects['Cube'].rotation_quaternion = list(map(float, data.decode('utf-8')[1:-1].split(',')))
        except Exception as e:
            print(e)
    return interval
   
   
   bpy.app.timers.register(handle_data)
   

2. Run the Blender script by pressing the 'Run script' button (triangle).

3. Run the client script from the command line. This script simulates the quaternion data which would come from the IMU.

    #!/usr/bin/env python3
    import time
    import socket
    import numpy as np
   HOST = "127.0.0.1"
    PORT = 65432
   sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    sock.connect((HOST, PORT))
   try:
        while True:
            time.sleep(0.33)
            random_quaternion = list(np.random.uniform(-1, 1, size=(1, 4))[0])
            data = str(random_quaternion)
            sock.send(data.encode())
            print(data)
    except KeyboardInterrupt:
        # Stop with Ctrl-C
        sock.send(str('quit').encode())
   

4. Now you should see the little cube rotating. If you don't, check the error messages. To see the error message that the Blender script creates, start Blender from the command line and redo the above steps.

When I used an actual IMU I noticed that the latency, between rotating the IMU and the cube rotating on the screen, was rather big. I wonder why?! Will the latency shrink when I run Blender on more powerful hardware? Or is the approach to use an application handler very slow?

I'd like to leave the question open and would like to accept an answer that gives a better overview of all the possible workflows to get external (real-time) data into Blender to manipulate objects in real-time.