On Script Interpolation / Curved Strokes

[Yoooi]

When raser1 was talking about developing a new script format (like 2 years ago) I suggested adding tangent information to script points. That way you could use tangent handles in scripter software to create really accurate motions without having to add multiple points. I think the consensus was that it would be unnecessary and it would make scripting longer. It would also not work on most devices.

@Falafel

Conflict with script players / device firmware that interpolates between actions themselves.

• For instance MultiFunPlayer’s built-in interpolation feature @Yoooi

MFP wont mind if the script is already interpolation, it will just unnecessarily do some math but it will work fine.

@hellrider

Right, protocols like T-Code would have to be extended for device based interpolation.

Yea I suggested that a long time ago. The player would calculate tangents using multiple points on its side and then it would just send next point and next tangent for interpolation on OSR.

Next thing that I would add would be a command buffer in the firmware, this would require the firmware communication to be two way. OSR would send a request for a chunk of next commands to the player each time it is about to finish the last chunk, for example 1-2 seconds.
This way there would be no stuttering at all caused by player/OSR trying to synchronize the next script command, currently the player has to send a move command and time it just right to send the next one when it finishes on the firmware. Inaccuracies in this timing can cause micro stutters.

[defucilis]

Personally, I think this stuff all comes up because robotics is really hard. If the devices cost 10x as much and we had top-of-the-line actuators and everything, then we’d be far closer to ‘the stroker moves exactly where the script says’. But in reality, there’s lots of slop, latency, inaccuracy, etc. The firmware is designed to do its best to create smooth, continuous motions around the hardware’s limitations, but that design is based on what ‘normal’ scripts look like. That is, a script is kind of an abstracted guide to what the stroker should do, rather than an explicit ‘you should be at this exact position at this exact time’.

So when we add tons of extra actions to try to ‘override’ what the firmware does to smooth things out, it just gets confused and we get jerky, inaccurate motions. I think the best thing to do is to keep scripts simple and accurate, and let the firmware do its job. It also means that our scripts are more likely to work with more kinds of devices, including devices that haven’t been invented yet

[hellrider]

@Yoooi

Next thing that I would add would be a command buffer in the firmware, this would require the firmware communication to be two way. OSR would send a request for a chunk of next commands to the player each time it is about to finish the last chunk, for example 1-2 seconds.
This way there would be no stuttering at all caused by player/OSR trying to synchronize the next script command, currently the player has to send a move command and time it just right to send the next one when it finishes on the firmware. Inaccuracies in this timing can cause micro stutters.

I don’t think bidirectional communication is needed for this. It should be sufficient to transmit some commands ahead, a number enough to cover transmit delays but known to not exceed the devices buffer.

Synchronization can be done by adding two types of timestamps:

• one timestamp per command, when to execute this
• one timestamp per packet (each packet containing one or several commands to reduce overhead) giving the current time the packet is sent.

This way, the device firmware can synchronize it’s internal clock to the current time, ignoring or averaging out timestamps from delayed packets, and knows exactly when to execute the next command. Some additional “flush buffer” command could help when skipping or switching between scripts.

[Falafel]

I think that’s a good mentality to have when making scripts. Thanks for your input.

In the case for the Handy, it doesn’t have a smoothing algorithm in its firmware (it feels “blunt” when changing direction). However is well capable of reflecting those manually interpolated points. I’ll keep adding them for that reason, but also makes a simple script for wider compatibility.

@defucilis

The firmware is designed to do its best to create smooth, continuous motions around the hardware’s limitations, but that design is based on what ‘normal’ scripts look like.

There’s perhaps some misunderstanding that needs to be cleared up, though. The interpolation I was talking about is not an overall optimization to the entire script (because smooth has many advantage, I guess?). It was done to emphasize the character of this particular motion, which ease-in and ease-out at both ends of a stroke.

Here’s another example, in which the strokes has a specific character to be dealt with:

In this scenario, you’ll want all upward strokes to ease-out (starting fast, then decelerates), whilst all downwards strokes to be fast and powerful, doesn’t slow down until it bounces off the floor.

Should we make a “simple” script that looked like this, and leave the rest to the firmware -

The firmware, which interpolates all actions based on a singular algorithm, will, probably, turn it into something like this. Which is not the correct dynamic for strong thrusts shown in the video.

(oh geez that’s a horribe choice of scene… Just imagine it’s a hot porn star smashing their hips on you okay???)

So in the end, even if the firmware is capable of interpolate between actions, it needs the script’s command to select the type of function appropriate to the scene. Scripters will be involved in the process, until the machine learns how to watch porn.

[hosenguy]

Another example where manual smoothing is needed is when you want the pitch and roll to make a circular rotation like grinding. You wil need sine waves for pitch and roll, 90 degrees out of sync. The result with simple funscript without manual interpolation is a jerky, not round, motion. Each interpolation algorithm provided in the firmware or software will make the motion different.

[defucilis]

@Falafel

There’s perhaps some misunderstanding that needs to be cleared up, though. The interpolation I was talking about is not an overall optimization to the entire script (because smooth has many advantage, I guess?). It was done to emphasize the character of this particular motion, which ease-in and ease-out at both ends of a stroke.

Oh yeah totally - I more meant that like, adding too much detail might add information that works against the way the motor controllers and stuff are designed to work. So in your example with the banana-beatdown, I’d definitely script it in the simpler way.

The more detailed one looks much better in the simulator, but the simulator is able to respond instantly and perfectly. The actual motor has weight and momentum and friction and stuff.

idk maybe people have had different experiences to me, but whenever I’ve tried to add those ‘smoothing’ actions into my funscript, I’ve noticed the Handy kind of stuttering at that point as the motor attempts to slow down the slider to match the more detailed script movements.

If my Handy wasn’t busted, I’d make a bunch of test scripts and record them with my phone camera’s slow-motion mode to see what the optimal rate is

[SomeoneRandom]

That high power is obviously something that the device can compensate for if the script requires excessive values. Again, if the device does the calculations of those curves, this can just be part of that. It can even use some extra logic to detect when a curve can no longer be followed and the max speed is desired anyway. No scriptplayer can does this reliably on the fly as everything is always delayed by the connection.

So a protocol change would indeed be needed, but do not thing that this is excessively difficult either (only its initial step would be). Firmware can be designed to handle this. For example, lets say the handy normaly expects full file uploads. Firmware now changed it so it can accept an emtpy script (but instead contains metadata), this metadata then can give feedback on which features are usable, its limitations etc. Or just ask for a certain version of a file. There is a massive amount of freedom as there is no true standard.

The hardest step is to make sure that unupdated devices can give the feedback it doesnt work reliably, so the program can fall back to an older version automaticly. Once the protocol includes a firmware version/script version that it can handle, programs can already act on that.

Its not like USB where due to global and excessive usage standards are a lot more strict. The current devices are still mostly driven by reading a file and loading it internaly. And this same behaviour makes it very much possible to add in extra handling.

[Yoooi]

@defucilis

So when we add tons of extra actions to try to ‘override’ what the firmware does to smooth things out, it just gets confused and we get jerky, inaccurate motions. I think the best thing to do is to keep scripts simple and accurate, and let the firmware do its job. It also means that our scripts are more likely to work with more kinds of devices, including devices that haven’t been invented yet

Well I would hope that future devices will have better motors the launch, and better api than the handy.
I wouldnt make everything for the lowest common denominator device because we will be stuck with super simplified funscripts forever. Current scripts lack any acceleration and are just triangle waves.

@defucilis

Oh yeah totally - I more meant that like, adding too much detail might add information that works against the way the motor controllers and stuff are designed to work. So in your example with the banana-beatdown, I’d definitely script it in the simpler way.

Adding many points to achieve a smooth curve is currently just a solution to funscript limitation.
Thats why I suggested using tangents instead, because then you can send positions and tangents to the device and it can perform smooth moves, instead of having to send small linear steps. Kinda like spline moves (G5) in gcode.
Only problem is that all devices would have to support this kind of moves, but scripter software could save two versions of the script, one with tangents, and one where the splines are sampled to linear steps.

[Thanny]

I think you’re overestimating the differences between that initial script and the smoothed version. The character of the strokes are going to feel the same. It’s still a relatively slow rise followed by a comparatively rapid drop. You’re not going to notice the subtle differences beyond possibly feeling a slightly less jarring change in direction at the top.

[Falafel]

If the example is bad, I hope the idea is consistent

The interpolation I was talking about is not an overall optimization to the entire script - #24

making everything smooth isnt always desired. Quick movement changes in otherwise slow paced action might end up distorted - #17

smoothing actions can often be undesired and break immersion. - #4

[Yoooi]

@Thanny

I think you’re overestimating the differences between that initial script and the smoothed version. The character of the strokes are going to feel the same. It’s still a relatively slow rise followed by a comparatively rapid drop. You’re not going to notice the subtle differences beyond possibly feeling a slightly less jarring change in direction at the top.

Maybe you cant tell on the handy or launch, but personally on OSR the more defined the script it the better.
The script just has more character and is more interesting by having varying acceleration than just mindless up-down strokes which I could generate without a script.

MFP with OSR doesnt care about how many points the script has. I have some old scripts that were iirc done in JFS that have 333 commands per second and are like 5-8MB. Works perfectly fine.

[defucilis]

I think tangents are a great idea. It could be made backwards-compatible by adding the tangents as a separate array in the funscript, so devices that don’t support custom tangents would just ignore that section of the file. It would also give even current-gen devices some idea of the ‘intent’ of the curve and would probably eliminate the jerkiness problem.

[Yoooi]

No need for a separate array, script actions could be extended like so:

{
  "at": 1000
  "pos": 100
  "inTangent": 0.984721
  "outTangent": 0.124609
}

Every player/scripter/device should just discard any extra properties it doesnt understand.

[defucilis]

Oh true that would be more convenient in lots of ways.
So long as they were called in and out since JSON property names hugely inflate filesizes

[Thanny]

The point of the person I replied to was that algorithmic smoothing changed the nature of the stroke, reducing the desired effect. I replied that I don’t think it does.

I use an OSR2 with spline smoothing because it’s better, and it doesn’t alter the nature of the stroke (e.g. slow upstroke followed by fast downstroke).

Near as I can tell, your reply doesn’t address either position.

[Falafel]

@Thanny

slow upstroke followed by fast downstroke

Not exactly what I was describing… or was trying to describe, though.

I knew my horrible examples have caused many misunderstandings. Let me make a remedy real quick.

[Falafel]

To sum things up:

1. The interpolation we are talking about is not an overall smoothing algorithm you apply to the entire script. But rather, a command sent to the software / firmware that indicates the curve used for the upcoming stroke.

2. For general smoothing, spline would do a decent job. But for building curved strokes, there exists more than a single method. When scripting videos, these different curves can be used to match the dynamic of strokes represented in the video, and could largely enhance the experience if used correctly and artistically.

Here’s some scripts by @99DM, see how different curves are used strategically to match motions in the video.

These are scripted for the Handy under wifi connection, and the nuances are already noticeable. For a stroker like OSR2/SR6, no way it will feel indifferent to triangular waves.

With what we have now, the curves are manually plotted out by adding in extra actions. They are incompatible with devices that uses Bluetooth connection such as Launch / A10 Piston and will feel horribly off. This spawns a script compatibility issue. As a result, 99DM had to make a “simple” version of the same script consisting triangular waves only.

Therefore I see the need for a more elegant solution to these curved strokes. Not only does that allows fallback compatibility (so we don’t have to keep duplicates of scripts), and it could potentially be more efficient, with greater tolerance on transmission frequency. Maybe BLE is quite capable afterall and companies will have no reason to bin it?

[Yoooi]

@defucilis

So long as they were called in and out since JSON property names hugely inflate filesizes

Eh, why does it matter that the script will be 100kb bigger. But I guess you could also use m0/m1, and it will be more mathy.

@Thanny

The point of the person I replied to was that algorithmic smoothing changed the nature of the stroke, reducing the desired effect. I replied that I don’t think it does.

I use an OSR2 with spline smoothing because it’s better, and it doesn’t alter the nature of the stroke (e.g. slow upstroke followed by fast downstroke).

Depending on the script layout and the interpolation type used yes it can change the stroke:

Unless you are looking at it at super basic level, then yes, an up stroke will be followed by a down stroke.

@Falafel

Therefore I see the need for a more elegant solution to these curved strokes. Not only does that allows fallback compatibility (so we don’t have to keep duplicates of scripts), and it could potentially be more efficient, with greater tolerance on transmission frequency.

Yea IMO the tangent data added to each script point is the solution.
So we just need:

• Decide on the format, like property names, which spline to use, maybe the script could specify the spline type?, maybe each point could use different spline type? etc.
• New TCode version with spline moves
• New OSR/SR6 firmware which supports spline moves
• OFS needs to rework the whole script editor to support in/out tangent handles at each point
• Script players need to add support for it
• Script creators need to use it, and it will probably increase script time by 50%-100% since you cant go frame by frame

On that last point, maybe OFS could have a spline point fit option, that way you could convert old scripts to spline ones, and you could script frame by frame and then convert it to spline after.

Another solution is to just script very detailed scripts with a lot of points and ignore anything that uses BLE.

@Falafel

Maybe BLE is quite capable afterall and companies will have no reason to bin it?

Pls no.

Even with spline support scripts might have to add points close together to get the desired curve.
BLE is useless for anything real time related. At minimum it needs some full power modern bluetooth version made for high frequency data transfer. But ideally it should be wifi or usb.

[Falafel]

@Yoooi

Even with spline support scripts might have to add points close together to get the desired curve.
BLE is useless for anything real time related. At minimum it needs some full power modern bluetooth version made for high frequency data transfer. But ideally it should be wifi or usb.

Right. I thought about this after making the previois post.

In the example script there also exists other small actions such as vibrations at the bottom. Even if BLE devices can handle curve movements on their end, these vibrations will still be messed up. (I don’t think their manufacturers will issue a firmware update just for the curves either)

---

There’s another possible solution, if people would not like to go through the hassles of implementing tangents (and most importantly, start using it).

@quickfix

In case your device is not able to handle it, the player software could reduce the number of points if needed.

Like quickfix said - to have some sort of simplify function built in script players. So any complex scripts can be converted into simple ones with peaks only, thus run smoothly on BLE devices.

OFS already has a simplify (Ramer-Douglas-Peucker algorithm) function built in. However it’s not very accessible for an end user who’s not familiar to scripting.

@defucilis Do you think it would be plausible to have such a feature in Funscript.io as well?

[Husky]

Just a friendly reminder here: you don’t have to add those points manually. You can use My Tools extension for OFS to make it semi-automatic. Here is the result that can be adjusted as you wish:

It’s pretty simple, I guess someone could create lua script to make it one step process, but it’s still faster than adding points manually for longer parts of the script:

1. Select desired part of the script
2. Add actions - 3
3. Select middle x2
4. Delete
5. Select the same part of the script
6. Select middle
7. Select top
8. Scale - STEP by STEP:
   Start: 40% (you can adjust it to your taste, make sure it’s the same as end value)
   End: 40%
   Step: 5
   Scale from 100: selected
   Min travel: 0
9. Select the same part of the script
10. Select middle
11. Select bottom
12. Scale - STEP by STEP:
    Start: 40% (you can adjust it to your taste, make sure it’s the same as end value)
    End: 40%
    Step: 5
    Scale from 0: selected
    Min travel: 0

I guess it’s not ideal and probably won’t work in some cases, but you could see if it helps a bit at least.