Using Blender as a multi-axis script editor

Preface.
After writing a few single-axis scripts and wanting to try making my own devices (I’m still gathering money for a 3D printer, as the one I got for free has only 100x100x100 mm print area) it was interesting to try writing multi-axis scripts for interactive masturbators. As for me, just knowing how these scripts play and work on devices can give me a better understanding of how to make them better. Although a great tool like OFS can create multi-axis scripts, I decided to do something alternative for several reasons:

  • I definitely don’t like the recording format of real multi-axis scripts (lots of motion and rotation scripts).
  • I doubt that it’s mathematically correct to use Euler angles (it’s hard to use interpolation, what order to use when rotating - the result will be different, the presence of the “hinge lock”).
  • I didn’t like the toolkit that OFS has (maybe I’m just clumsy).
  • There must have been other reasons - can’t remember why :slight_smile:

I’ve been using Blender as a program for parametric drawing (similar to AutoCAD, SolidWorks) and for modeling 3D models for printing. Luckily this program can also do a lot of things with video and of course I couldn’t help but try to use it as a script editor.

Introductions.
I’m making scripts only for VR video so far and prepare flat video for scripting with ffmpeg command line:
ffmpeg -i video.mp4 -vf "crop=in_w/2:in_h:in_w:in_h, v360=input=hequirect:output=flat:pitch=-40:h_fov=100:v_fov=100:w=1024:h=1024, fps=30" output.mp4
Where:

  • ffmpeg - the utility itself (the full path may look like “C:\ffmpeg\bin\ffmpeg.exe”)
  • video.mp4 - incoming video file in VR format, for example the full path is “D:\VR\VRHush_From_The_Vault_Dani_Daniels_Oculus_HQ_3D_LR_180.mp4”
  • crop=in_w/2:in_h:in_w:in_h - trim the video, leave only the left half.
  • v360=input=hequirect:output=flat:pitch=-40:h_fov=100:v_fov=100:w=1024:h=1024 - converting video from the half of equirectangular projection with rotation by -40 degrees, angles 100x100 degrees and output resolution 1024x1024 pixels
  • fps=30 - number of frames per second
  • output.mp4 - outputs the flat video to a file, for example the full path “C:\Temp\VRHush_From_The_Vault_Dani_Daniels_Flat_30.mp4”.

Usually it’s the flat file that I then work with in OFS and create a per-frame script for complex scenes and for simple ones I use MTFG with manual correction of each movement.

Create a multi-axis script.
In Blender you can import our flat video as a sequence of frames:

  • add a Empty image
  • in the Property area select Object Data Properties to open Image
  • in the file selection window select our video made for OFS
  • look through the duration of our video in the file properties and specify how many frames we will import using the formula (minutes*60 + seconds)*frame count
  • in the Timeline area we also specify the desired number of frames:
  • switch to the Y-axis view by clicking on the necessary icon or by pressing the “Numpad 1” key
  • rotate our set of pictures by an angle of 90 degrees on the X axis and press the “Space” key to watch the video:
  • choose the units of measure we will use for positioning (I’ve set cm)
  • let’s create our Fleshlight, which will be in 3D emulate the movements of a real masturbator
  • I put the length of 20 cm, the discrimeter of 5 cm (which roughly corresponds to a full-size) and 5 vertices at the base (to understand where its sides), immediately put his beginning was in the center of coordinates (Z 10)
  • set the point for calculating movements and rotations to the origin of coordinates (where the 3D cursor is):
  • to import our script (which is a JSON file), I made a simple script, for this you can switch one of the areas to the text editor
  • insert the python code and run it:
import json
import bpy

funscriptName = "C:\\Temp\\Dani Daniels\\VRHush_From_The_Vault_Dani_Daniels.funscript"
scene = bpy.context.scene
obj = bpy.data.objects["Cylinder"]
obj_loc = obj.location

with open(funscriptName, "r") as funscriptFile:
    funscriptJson = json.load(funscriptFile)
    for action in funscriptJson['actions']:
        obj.location = (0.0, 0.0, float(action['pos']/1000))
        obj.keyframe_insert(data_path="location", frame=round((action['at']/1000)*30))

The resulting script in step 2 is only Z-axis motions. At a minimum, we need to add X and Y positions for each keyframe, and add rotations around the XYZ axes. And while adding motions and rotations around ready-made points is easier than writing scripts of 2 motions and 3 rotations from “0”, it’s time consuming and time-consuming to do 2-3 times as much as writing one motion axis as usual.
Very often rotations around Z do not correspond to keyframes (start earlier and end later), so such points need to be created separately.

Install python libraries in Blender.
To work correctly, you need to install some python libraries that Blender doesn’t have.
For example, installing the library for working with Xinput gamepads on Windows.

import subprocess
import sys
import os
 
# path to python.exe
python_exe = os.path.join(sys.prefix, 'bin', 'python.exe')
target = os.path.join(sys.prefix, 'lib', 'site-packages')
 
# upgrade pip
subprocess.call([python_exe, "-m", "ensurepip"])
subprocess.call([python_exe, "-m", "pip", "install", "--upgrade", "pip"])
 
# install required packages
subprocess.call([python_exe, '-m', 'pip', 'install', 'XInput-Python', '-t', target])

print("DONE")

Gamification.
For each keyframe with the Z-axis you need to set 5 different values.

After spending a few hours on the script, the obvious idea was to use the gamepad’s analog stick and trigger axes for movements and rotations. No sooner said than done, and here is the script:

import bpy
import XInput
import mathutils
import time
import random

obj = bpy.data.objects["fleshlight"]
reference = bpy.data.objects["reference"]
camera = bpy.data.objects["camera"]
area = next(area for area in bpy.context.screen.areas if area.type == 'VIEW_3D')

# button & axis mapping
prev_keyframe_btn = 'A'
next_keyframe_btn = 'B'
prev_frame_btn = 'X'
next_frame_bnt = 'Y'
anim_play_btn = 'START'
anim_cancel_btn = 'BACK'
multi_view_btn = 'RIGHT_THUMB'
Z_rotation_btn = 'LEFT_THUMB'
left_location_btn = 'DPAD_LEFT'
right_location_btn = 'DPAD_RIGHT'
forward_location_btn = 'DPAD_UP'
backward_location_btn = 'DPAD_DOWN'
save_rotation_btn = 'LEFT_SHOULDER'
save_location_btn = 'RIGHT_SHOULDER'
thumb_location = 0
thumb_rotation = 1

camera.hide_set(True)

prev_keyframe, next_keyframe, prev_frame, next_frame = False, False, False, False
save_location, save_rotation, anim_play, anim_cancel, enable_anim_play = False, False, False, False, False
forward_location, backward_location, left_location, right_location = False, False, False, False
multi_view, enable_multi_view, Z_rotation, enable_Z_rotation = False, False, False, False

class ModalTimerOperator(bpy.types.Operator):
    """Operator which runs itself from a timer"""
    bl_idname = "wm.modal_timer_operator"
    bl_label = "Modal Timer Operator"

    _timer = None
    
    def modal(self, context, event):
        global prev_keyframe, next_keyframe, prev_frame, next_frame
        global save_location, save_rotation, anim_play, anim_cancel, enable_anim_play
        global forward_location, backward_location, left_location, right_location
        global multi_view, enable_multi_view, Z_rotation, enable_Z_rotation
        if event.type in {'RIGHTMOUSE', 'ESC'}:
            self.cancel(context)
            return {'CANCELLED'}
        obj_pos_x, obj_pos_y, obj_rot_euler_x, obj_rot_euler_y, obj_rot_euler_z = 0.0, 0.0, 0.0, 0.0, 0.0
        if event.type == 'TIMER':
            state = XInput.get_state(0)
            if XInput.get_button_values(state)[anim_play_btn] and not anim_play and enable_anim_play:
                bpy.ops.screen.animation_cancel(restore_frame=False)
                enable_anim_play = False
                anim_play = True
            if XInput.get_button_values(state)[anim_play_btn] and not anim_play and not enable_anim_play:
                bpy.ops.screen.animation_play(reverse=False, sync=False)
                enable_anim_play = True
                anim_play = True
            if not XInput.get_button_values(state)[anim_play_btn] and anim_play: 
                anim_play = False
            if XInput.get_button_values(state)[anim_cancel_btn] and not anim_cancel and enable_anim_play:
                bpy.ops.screen.animation_cancel()
                enable_anim_play = False
                anim_cancel = True
            if not XInput.get_button_values(state)[anim_cancel_btn] and anim_cancel: 
                anim_cancel = False
            if XInput.get_button_values(state)[prev_keyframe_btn] and not prev_keyframe:
                bpy.ops.screen.keyframe_jump(next=False)
                prev_keyframe = True
            if not XInput.get_button_values(state)[prev_keyframe_btn] and prev_keyframe: 
                prev_keyframe = False
            if XInput.get_button_values(state)[next_keyframe_btn] and not next_keyframe:
                bpy.ops.screen.keyframe_jump(next=True)
                next_keyframe = True
            if not XInput.get_button_values(state)[next_keyframe_btn] and next_keyframe:
                next_keyframe = False
            if XInput.get_button_values(state)[prev_frame_btn] and not prev_frame:
                bpy.ops.screen.frame_offset(delta=-1)
                prev_frame = True
            if not XInput.get_button_values(state)[prev_frame_btn] and prev_frame:
                prev_frame = False
            if XInput.get_button_values(state)[next_frame_bnt] and not next_frame:
                bpy.ops.screen.frame_offset(delta=1)
                next_frame = True
            if not XInput.get_button_values(state)[next_frame_bnt] and next_frame:
                next_frame = False
            if XInput.get_button_values(state)[multi_view_btn] and not multi_view and not enable_multi_view:
                area.spaces[0].region_3d.view_perspective = 'CAMERA'
                reference.hide_set(True)
                camera.hide_set(False)
                enable_multi_view = True
                multi_view = True
            if XInput.get_button_values(state)[multi_view_btn] and not multi_view and enable_multi_view:
                area.spaces[0].region_3d.view_perspective = 'ORTHO'
                reference.hide_set(False)
                camera.hide_set(True)
                enable_multi_view = False 
                multi_view = True
            if not XInput.get_button_values(state)[multi_view_btn] and multi_view: 
                multi_view = False
            if XInput.get_button_values(state)[Z_rotation_btn] and not Z_rotation and enable_Z_rotation:
                enable_Z_rotation = False
                Z_rotation = True
            if XInput.get_button_values(state)[Z_rotation_btn] and not Z_rotation and not enable_Z_rotation:
                enable_Z_rotation = True
                Z_rotation = True
            if not XInput.get_button_values(state)[Z_rotation_btn] and Z_rotation: 
                Z_rotation = False
            if XInput.get_button_values(state)[forward_location_btn] and not forward_location:
                obj.location[1] += 0.005
                forward_location = True
            if not XInput.get_button_values(state)[forward_location_btn] and forward_location:
                forward_location = False
            if XInput.get_button_values(state)[backward_location_btn] and not backward_location:
                obj.location[1] -= 0.005
                backward_location = True
            if not XInput.get_button_values(state)[backward_location_btn] and backward_location:
                backward_location = False
            if XInput.get_button_values(state)[left_location_btn] and not left_location:
                obj.location[0] -= 0.005
                left_location = True
            if not XInput.get_button_values(state)[left_location_btn] and left_location:
                left_location = False
            if XInput.get_button_values(state)[right_location_btn] and not right_location:
                obj.location[0] += 0.005
                right_location = True
            if not XInput.get_button_values(state)[right_location_btn] and right_location:
                right_location = False
            if XInput.get_thumb_values(state)[thumb_location] and not enable_Z_rotation:
                obj_pos_x = XInput.get_thumb_values(state)[thumb_location][0] * obj.location[2]
                obj_pos_y = XInput.get_thumb_values(state)[thumb_location][1] * obj.location[2]
                if obj_pos_x != 0 or obj_pos_y != 0:
                    obj.location[0] = obj_pos_x
                    obj.location[1] = obj_pos_y
            if XInput.get_thumb_values(state)[thumb_location] and enable_Z_rotation:
                obj_rot_euler_z = XInput.get_thumb_values(state)[thumb_location][0] * 0.8
                #obj_rot_euler_y = XInput.get_thumb_values(state)[1][0] * 0.8
                if obj_rot_euler_z != 0:
                    obj.rotation_euler[2] = obj_rot_euler_z
            if XInput.get_thumb_values(state)[thumb_rotation]:
                obj_rot_euler_x = -XInput.get_thumb_values(state)[thumb_rotation][1] * 0.8
                obj_rot_euler_y = XInput.get_thumb_values(state)[thumb_rotation][0] * 0.8
                if obj_rot_euler_x != 0 or obj_rot_euler_y != 0:
                    obj.rotation_euler[0] = obj_rot_euler_x
                    obj.rotation_euler[1] = obj_rot_euler_y
            if XInput.get_button_values(state)[save_rotation_btn] and not save_rotation:
                if not enable_Z_rotation:
                    quat = obj.rotation_euler.to_quaternion()
                    vect = obj.location.copy()
                    vect[0], vect[1] = 0.0, 0.0
                    vect.rotate(quat)
                    obj.location[0] = vect[0] + random.uniform(-0.25, 0.25)*0.01
                    obj.location[1] = vect[1] + random.uniform(-0.25, 0.25)*0.01
                    obj.keyframe_insert(data_path="rotation_euler")
                    obj.keyframe_insert(data_path="location")
                    bpy.ops.screen.keyframe_jump(next=True)
                    save_rotation = True
                else: 
                    quat = obj.rotation_euler.to_quaternion()
                    vect = obj.location.copy()
                    vect[0], vect[1] = 0.0, 0.0
                    vect.rotate(quat)
                    obj.location[0] = vect[0] + random.uniform(-0.25, 0.25)*0.01
                    obj.location[1] = vect[1] + random.uniform(-0.25, 0.25)*0.01
                    obj.keyframe_insert(data_path="location")
                    bpy.ops.screen.keyframe_jump(next=True)
                    save_rotation = True
            if not XInput.get_button_values(state)[save_rotation_btn] and save_rotation:
                save_rotation = False
            if XInput.get_button_values(state)[save_location_btn] and not save_location:
                if not enable_Z_rotation:
                    obj.keyframe_insert(data_path="location")
                    bpy.ops.screen.keyframe_jump(next=True)
                    save_location = True
                else: 
                    obj.keyframe_insert(data_path="rotation_euler")
                    obj.keyframe_insert(data_path="location")
                    bpy.ops.screen.keyframe_jump(next=True)
                    save_location = True
            if not XInput.get_button_values(state)[save_location_btn] and save_location:
                save_location = False
        return {'PASS_THROUGH'}

    def execute(self, context):
        wm = context.window_manager
        self._timer = wm.event_timer_add(1/30, window=context.window)
        wm.modal_handler_add(self)
        return {'RUNNING_MODAL'}

    def cancel(self, context):
        wm = context.window_manager
        wm.event_timer_remove(self._timer)
        
def get_override(area_type, region_type):
    for area in bpy.context.screen.areas: 
        if area.type == area_type:             
            for region in area.regions:                 
                if region.type == region_type:                    
                    override = {'area': area, 'region': region} 
                    return override
    #error message if the area or region wasn't found
    raise RuntimeError("Wasn't able to find", region_type," in area ", area_type,
                        "\n Make sure it's open while executing script.")


def menu_func(self, context):
    self.layout.operator(ModalTimerOperator.bl_idname, text=ModalTimerOperator.bl_label)


def register():
    bpy.utils.register_class(ModalTimerOperator)
    bpy.types.VIEW3D_MT_view.append(menu_func)


def unregister():
    bpy.utils.unregister_class(ModalTimerOperator)
    bpy.types.VIEW3D_MT_view.remove(menu_func)


if __name__ == "__main__":
    register()
    bpy.ops.wm.modal_timer_operator()

(I’ll be finishing up)
I put moves on the left stick, rotations on the right stick, Z rotations on the trigger, keyframe navigation on the ABXY buttons, and a write script on the bumpers.
It looks interesting and the script creation speed has increased:
ezgif.com-gif-maker (6)

Script export.
The code itself

import json
import bpy
import math

funscriptName = "VRHush_From_The_Vault_Dani_Daniels"
folder = "C:\\Temp\\"

funscript_orig = folder + funscriptName + ".mono.funscript"

fs_pitch = folder + funscriptName + ".pitch" + ".funscript"
fs_roll = folder + funscriptName + ".roll" + ".funscript"
fs_yaw = folder + funscriptName + ".twist" + ".funscript"
fs_x = folder + funscriptName + ".sway" + ".funscript"
fs_y= folder + funscriptName + ".surge" + ".funscript"
fs_z= folder + funscriptName + ".funscript"

obj = bpy.data.objects["fleshlight"]

with open(funscript_orig, "r") as funscriptFile:
    funscriptJson = json.load(funscriptFile)
    fs_orig_inv = funscriptJson['inverted']
    fs_orig_metadata = funscriptJson['metadata']
    fs_orig_range = funscriptJson['range']
    fs_orig_version = funscriptJson['version']

obj_pitch = {"actions":[], "inverted": fs_orig_inv, "metadata": fs_orig_metadata, "range":fs_orig_range, "version":fs_orig_version}
obj_roll = {"actions":[], "inverted": fs_orig_inv, "metadata": fs_orig_metadata, "range":fs_orig_range, "version":fs_orig_version}
obj_yaw = {"actions":[], "inverted": fs_orig_inv, "metadata": fs_orig_metadata, "range":fs_orig_range, "version":fs_orig_version}
obj_x = {"actions":[], "inverted": fs_orig_inv, "metadata": fs_orig_metadata, "range":fs_orig_range, "version":fs_orig_version}
obj_y = {"actions":[], "inverted": fs_orig_inv, "metadata": fs_orig_metadata, "range":fs_orig_range, "version":fs_orig_version}
obj_z = {"actions":[], "inverted": fs_orig_inv, "metadata": fs_orig_metadata, "range":fs_orig_range, "version":fs_orig_version}
#obj_yaw = {"at": , "pos":}
#obj_x = {"at": , "pos":}
#obj_y = {"at": , "pos":}

cur_frame = 0
frames = obj.animation_data.action.fcurves[2]
for kf in frames.keyframe_points:
    frame = int(kf.co[0])
    print(frame)
    bpy.context.scene.frame_set(frame)
    obj_pitch['actions'].append({"at":int((frame/30)*1000), 
    "pos":round(math.degrees(obj.rotation_euler[0])*1.11)+50})
    obj_roll['actions'].append({"at":int((frame/30)*1000), 
    "pos":round(math.degrees(obj.rotation_euler[1])*1.11)+50})
    obj_yaw['actions'].append({"at":int((frame/30)*1000), 
    "pos":round(math.degrees(obj.rotation_euler[2])*1.11)+50})
    obj_x['actions'].append({"at":int((frame/30)*1000), "pos":round(obj.location[0]*1000)+50})
    obj_y['actions'].append({"at":int((frame/30)*1000), "pos":round(obj.location[1]*1000)+50})
    obj_z['actions'].append({"at":int((frame/30)*1000), "pos":round(obj.location[2]*1000)})
        
with open(fs_pitch, 'w') as fp:
    json.dump(obj_pitch, fp, separators=(',', ':'))

with open(fs_roll, 'w') as fp:
    json.dump(obj_roll, fp, separators=(',', ':'))
    
with open(fs_yaw, 'w') as fp:
    json.dump(obj_yaw, fp, separators=(',', ':'))
    
with open(fs_x, 'w') as fp:
    json.dump(obj_x, fp, separators=(',', ':'))
    
with open(fs_y, 'w') as fp:
    json.dump(obj_y, fp, separators=(',', ':'))
    
with open(fs_z, 'w') as fp:
    json.dump(obj_z, fp, separators=(',', ':'))

Then, when I have free time, I need to put everything in order and make an addon for Blebder.

To be continued…

12 Likes

Pretty awesome. I could see blender being useful for 2d as well. Simply because things tracked move in 3 dimensions when we’re trying to estimate movement in one axis… can be a bit difficult with no visual aid.

1 Like

It’s hard to appreciate multi-axis movements - in Blender it seems easier to me.

In theory, I’m thinking of adding the ability to generate a point cloud from 3D video in Blender. But a little later :wink:

3 Likes

Thanks for the infos. Don’t you think this fits more in the #howto category?

You may be right. Although it’s a manual now, I hope it transforms into something more.

1 Like

Updated the topic.

this is really interesting when multiaxis toys become the norm. its a while since i last used blender maybe i should dl it again

1 Like

The global problem with devices is that they do not make stationary, but mobile, and a 6-axis device would be very difficult to make mobile. So it turned out no devices, no scripts, and vice versa.

So I did everything.
The link has both the video file, the blender file, the python scripts that I used and the scripts that came out in the end.
I don’t know what to do with the resulting data - I don’t have anything to test it on yet.

1 Like

Small update.

Already wrote several multi-axis scripts - sharing my experiences
Tried the following gamepads:

  1. Xbox 360 - a timeless classic (one of the most comfortable for gaming in my opinion).
    Problems:
  • low sticks - hard to position and maintain positions in exact +/- 5 degrees in terms of rotation and position.
  • short triggers - it’s hard to make Z-axis rotation with +/- 5 degrees accuracy.
  • no paddles - it’s convenient to hang on them to switch between frames.
    Advantages:
  • comfortable and after many hours your hands do not get tired, it was him who wrote the script with Danni Daniels completely.
  1. Xiaomi Gamepad - well, why not, especially here it is.
    Problems:
  • dancing with tambourine around Windows to get it to work correctly.
  • no paddles - they are convenient to switch between frames.
    Advantages:
  • the sticks are taller and the triggers are more authentic. than the xbox 360
  • handy though I haven’t written much - I tried it for a couple of hours and moved on to the next member.
  1. Steam Controller - seemed like a winner and I wrote two multi-axis scripts with it!
    Problems:
  • very tired hands because of the uncomfortable grip for me - after a couple of hours straight to the point of pain.
  • very short triggers - i couldn’t make turns by z, i had to do it manually.
  • the crackling of the touchpad is annoying, and there is no tactile contact when it is switched off.
    Advantages:
  • there are paddles - it’s handy to hang on them to switch between frames.
  • I can do a lot with Blender via Steam (it can be done for any gamepad :wink: )
  • there’s a gyroscope - I haven’t figured out how to use it.
  1. Xbox Elite Series 2 - thought it would be worse than the 360. but one script for 42 minutes is written by him and I’m writing the second one now.
    Problems:
  • didn’t want to be friends with my bluetooth.
  • heavier than the previous ones - but maybe that’s an advantage, it feels different in the hands
    Advantages:
  • there are paddles. and even 4 (!) - it’s convenient to hang on them to switch between frames and something else
  • high stick for rotation - accuracy now +/- 1 degree.
  • quite long trigger - accuracy +/- 3-5 degrees.
  1. Nintendo Wii Remote + Nunchack - can’t say anything yet, haven’t tried it

Added how to install python libraries, fixed the export script.

Added gamepad button bindings, updated the script (the script has 3D video output, I’ll add a description later).

I guess I found the perfect controller for multi-axis scripting - HTC Vive Controller. Two base stations and a controller connected to a PC using Steam Controller reciever.
Yes, I lost a week while I was setting this all up - the speed of scripting has already increased significantly. I’m still doing everything in 3 passes (XY rotations, XY moves, Z rotations) I’m getting used to it. I think the next script will be written in two passes (XY rotations + XY moves, Z rotation) and once I get used to it I will write it in one pass ;).
I am satisfied.

1 Like