On Script Interpolation / Curved Strokes

General
, , , , ,
17

Obviously it depends on how the device handles thing, if the device receives commands ahead in time (the handy does as far as i know), then it can do these calculations runtime as it knows when they become relevant. Bluetooth obviously wont be capable of doing this. But any device that can cache info and sync the timer and has a fast enough processor in it can do these things.

Note that calculating a curvature doesnt realy need that much info. For audio they only need double the Hz to make the audio resolve properly with quite some complexity. Sure, scripting does require a bit diffirent handling (audio doesnt require the point to be at its peak for example), but its not like it will need an entire script here, probably using just 5 points (2 ahead and 2 behind of the point you want to calculate) it can already interpolate things accurately on the fly (the 2 behind are there to consider the actual current position and adjust things if needed). And that means just 4 points ahead is technicaly already enough.

But the main issue is in its details, making everything smooth isnt always desired. Quick movement changes in otherwise slow paced action might end up distorted, while the uninterpolated script might have done this more accurately. Sure, this can be handled by using nodes at proper locations (a sharp peak can be created), but guiding the device here by simply stating it must be sharp saves some guessing effort (which still is calculation time).

So these things are very well capable to be done within the device. But again, its mostly about standardizing this, since when every device does it diffirent, it makes scripting unreliable.

1 Like
18

Well, quick movement changes require high power and cause high forces and wear. So the motor control will inherently smoothen the curve. The knowledge of how to approach a point (with high speed, continuing in the same direction afterwards, or stopping in order to reverse direction) can only help the motor control to make the motion more accurate.

Sure, some few points ahead would be sufficient, but this requires changes in protocols that rely on the time a command / point is being sent. Just some short latency compensation does not help, as it won’t always cover the next few points.

2 Likes
19

When Raser1 was developing Joy-Funplayer one big item was the ability to program with a mouse “on the fly” which creates “raw script data” with a large file. It is very smooth with all the points it creates.

There are several interpolation methods. For an example look at the Ver 4 of this player and you will see the choice of Linear, raw, cubic, or cosine based interpolations. You can test what each does with different funscripts.

1 Like
20

Thanks for the input, I think I should definitely look into that software more.

I have made one script with an incredible amount of action points before. It was generated with the motion tracking funscript generator. So I believe it’s possible to create silky smooth curves by adding more points.

However what I’m concerned about is the compatibility of these scripts. I’ve ran the script on the Handy (via wifi) and it went like a seizure. If devices like the Launch couldn’t handle the action-rich script we posted on this site, I doubt they can handle anything beyond…

21

When raser1 was talking about developing a new script format (like 2 years ago) I suggested adding tangent information to script points. That way you could use tangent handles in scripter software to create really accurate motions without having to add multiple points. I think the consensus was that it would be unnecessary and it would make scripting longer. It would also not work on most devices.

MFP wont mind if the script is already interpolation, it will just unnecessarily do some math but it will work fine.

Yea I suggested that a long time ago. The player would calculate tangents using multiple points on its side and then it would just send next point and next tangent for interpolation on OSR.

Next thing that I would add would be a command buffer in the firmware, this would require the firmware communication to be two way. OSR would send a request for a chunk of next commands to the player each time it is about to finish the last chunk, for example 1-2 seconds.
This way there would be no stuttering at all caused by player/OSR trying to synchronize the next script command, currently the player has to send a move command and time it just right to send the next one when it finishes on the firmware. Inaccuracies in this timing can cause micro stutters.

1 Like
22

Personally, I think this stuff all comes up because robotics is really hard. If the devices cost 10x as much and we had top-of-the-line actuators and everything, then we’d be far closer to ‘the stroker moves exactly where the script says’. But in reality, there’s lots of slop, latency, inaccuracy, etc. The firmware is designed to do its best to create smooth, continuous motions around the hardware’s limitations, but that design is based on what ‘normal’ scripts look like. That is, a script is kind of an abstracted guide to what the stroker should do, rather than an explicit ‘you should be at this exact position at this exact time’.

So when we add tons of extra actions to try to ‘override’ what the firmware does to smooth things out, it just gets confused and we get jerky, inaccurate motions. I think the best thing to do is to keep scripts simple and accurate, and let the firmware do its job. It also means that our scripts are more likely to work with more kinds of devices, including devices that haven’t been invented yet :smiley:

1 Like
23

I don’t think bidirectional communication is needed for this. It should be sufficient to transmit some commands ahead, a number enough to cover transmit delays but known to not exceed the devices buffer.

Synchronization can be done by adding two types of timestamps:

  • one timestamp per command, when to execute this
  • one timestamp per packet (each packet containing one or several commands to reduce overhead) giving the current time the packet is sent.

This way, the device firmware can synchronize it’s internal clock to the current time, ignoring or averaging out timestamps from delayed packets, and knows exactly when to execute the next command. Some additional “flush buffer” command could help when skipping or switching between scripts.

24

I think that’s a good mentality to have when making scripts. Thanks for your input.

In the case for the Handy, it doesn’t have a smoothing algorithm in its firmware (it feels “blunt” when changing direction). However is well capable of reflecting those manually interpolated points. I’ll keep adding them for that reason, but also makes a simple script for wider compatibility.

There’s perhaps some misunderstanding that needs to be cleared up, though. The interpolation I was talking about is not an overall optimization to the entire script (because smooth has many advantage, I guess?). It was done to emphasize the character of this particular motion, which ease-in and ease-out at both ends of a stroke.

image
image775×127 24.6 KB

Here’s another example, in which the strokes has a specific character to be dealt with:


In this scenario, you’ll want all upward strokes to ease-out (starting fast, then decelerates), whilst all downwards strokes to be fast and powerful, doesn’t slow down until it bounces off the floor.

Should we make a “simple” script that looked like this, and leave the rest to the firmware -
image

The firmware, which interpolates all actions based on a singular algorithm, will, probably, turn it into something like this. Which is not the correct dynamic for strong thrusts shown in the video.
image

(oh geez that’s a horribe choice of scene… Just imagine it’s a hot porn star smashing their hips on you okay???)

So in the end, even if the firmware is capable of interpolate between actions, it needs the script’s command to select the type of function appropriate to the scene. Scripters will be involved in the process, until the machine learns how to watch porn.

25

Another example where manual smoothing is needed is when you want the pitch and roll to make a circular rotation like grinding. You wil need sine waves for pitch and roll, 90 degrees out of sync. The result with simple funscript without manual interpolation is a jerky, not round, motion. Each interpolation algorithm provided in the firmware or software will make the motion different.

2 Likes
26

Oh yeah totally - I more meant that like, adding too much detail might add information that works against the way the motor controllers and stuff are designed to work. So in your example with the banana-beatdown, I’d definitely script it in the simpler way.

The more detailed one looks much better in the simulator, but the simulator is able to respond instantly and perfectly. The actual motor has weight and momentum and friction and stuff.

idk maybe people have had different experiences to me, but whenever I’ve tried to add those ‘smoothing’ actions into my funscript, I’ve noticed the Handy kind of stuttering at that point as the motor attempts to slow down the slider to match the more detailed script movements.

If my Handy wasn’t busted, I’d make a bunch of test scripts and record them with my phone camera’s slow-motion mode to see what the optimal rate is :thinking:

1 Like
27

That high power is obviously something that the device can compensate for if the script requires excessive values. Again, if the device does the calculations of those curves, this can just be part of that. It can even use some extra logic to detect when a curve can no longer be followed and the max speed is desired anyway. No scriptplayer can does this reliably on the fly as everything is always delayed by the connection.

So a protocol change would indeed be needed, but do not thing that this is excessively difficult either (only its initial step would be). Firmware can be designed to handle this. For example, lets say the handy normaly expects full file uploads. Firmware now changed it so it can accept an emtpy script (but instead contains metadata), this metadata then can give feedback on which features are usable, its limitations etc. Or just ask for a certain version of a file. There is a massive amount of freedom as there is no true standard.

The hardest step is to make sure that unupdated devices can give the feedback it doesnt work reliably, so the program can fall back to an older version automaticly. Once the protocol includes a firmware version/script version that it can handle, programs can already act on that.

Its not like USB where due to global and excessive usage standards are a lot more strict. The current devices are still mostly driven by reading a file and loading it internaly. And this same behaviour makes it very much possible to add in extra handling.

28

Well I would hope that future devices will have better motors the launch, and better api than the handy.
I wouldnt make everything for the lowest common denominator device because we will be stuck with super simplified funscripts forever. Current scripts lack any acceleration and are just triangle waves.

Adding many points to achieve a smooth curve is currently just a solution to funscript limitation.
Thats why I suggested using tangents instead, because then you can send positions and tangents to the device and it can perform smooth moves, instead of having to send small linear steps. Kinda like spline moves (G5) in gcode.
Only problem is that all devices would have to support this kind of moves, but scripter software could save two versions of the script, one with tangents, and one where the splines are sampled to linear steps.

1 Like
29

I think you’re overestimating the differences between that initial script and the smoothed version. The character of the strokes are going to feel the same. It’s still a relatively slow rise followed by a comparatively rapid drop. You’re not going to notice the subtle differences beyond possibly feeling a slightly less jarring change in direction at the top.

30

If the example is bad, I hope the idea is consistent

The interpolation I was talking about is not an overall optimization to the entire script - #24

making everything smooth isnt always desired. Quick movement changes in otherwise slow paced action might end up distorted - #17

smoothing actions can often be undesired and break immersion. - #4

31

Maybe you cant tell on the handy or launch, but personally on OSR the more defined the script it the better.
The script just has more character and is more interesting by having varying acceleration than just mindless up-down strokes which I could generate without a script.

MFP with OSR doesnt care about how many points the script has. I have some old scripts that were iirc done in JFS that have 333 commands per second and are like 5-8MB. Works perfectly fine.

2 Likes
32

I think tangents are a great idea. It could be made backwards-compatible by adding the tangents as a separate array in the funscript, so devices that don’t support custom tangents would just ignore that section of the file. It would also give even current-gen devices some idea of the ‘intent’ of the curve and would probably eliminate the jerkiness problem.

1 Like
33

No need for a separate array, script actions could be extended like so:

{
  "at": 1000
  "pos": 100
  "inTangent": 0.984721
  "outTangent": 0.124609
}

Every player/scripter/device should just discard any extra properties it doesnt understand.

1 Like
34

Oh true that would be more convenient in lots of ways.
So long as they were called in and out since JSON property names hugely inflate filesizes :stuck_out_tongue:

35

The point of the person I replied to was that algorithmic smoothing changed the nature of the stroke, reducing the desired effect. I replied that I don’t think it does.

I use an OSR2 with spline smoothing because it’s better, and it doesn’t alter the nature of the stroke (e.g. slow upstroke followed by fast downstroke).

Near as I can tell, your reply doesn’t address either position.

36

Not exactly what I was describing… or was trying to describe, though. :grin:

I knew my horrible examples have caused many misunderstandings. Let me make a remedy real quick.