How to use FunscriptToolBox MotionVectors Plugin in OpenFunscripter

[brubba]

i have an idea to make this tool more accurate. What if we start to play the whole video, and in the same time start an OBS session, and always zoom to the moving parts?
So basically in the recorded video which would be made with OBS, only the actual moving would be recorded, and later used by the script. Like if there are 3 actors in the scene, and 1 or 2 actors are “working”, the 3rd actor who is not important/not moving, could be cut out by OBS. At the end of a session like this, the whole funscript would be much more accurate. It is just an idea, i have to try it.

[brubba]

hint: my above idea works, but Bandicam is a better tool for that instead of OBS. Screen area crop is better with Bandicam.

[WIZARD]

Love the program, a interesting alternative to the tracker… works well so far! I had a few questions:

1. Are there any recommendations as to speed up the “creating a motion vectors optimized video” process for VR videos?
2. Is there any way to use more GPU vs CPU for these optimization processes?
3. Is there any difference in speed when using VRLeft vs. VRMosaic?
4. Also, is there any video compression going on when the optimized video is being created? This process is done twice?
   I am already using a lower quality video to reduce the optimization time (ex: 1080p from 8k). Any advice would be appreciated!

Thanks in advance!

[Zalunda]

You’re already using the biggest reduction with 1080p instead of 2048, which is 4 times less stuff to encode. VRLeft doesn’t have the 2 smaller views. I thought those views could be useful for doggy position but I guess you could go without them. This would remove another ~33% (i.e. 2h instead of 3h).

The last part .mvs-visual.mp4 is optional. You could skip it by adding --novisual,
and also --keeppframes, in case you change your mind later on (if not, the first .mp4 will be deleted and you’ll have to recreate it to make .mvs-visual.mp4). You won’t see an image if you are using the UI but it’s not a big deal. But I’m not sure it will save you a lot of time if you create an I-frames version of your video anyway, which work a lot better in OFS.

I haven’t used hardware-enable in ffmpeg because I read that it’s not as good as CPU version. I’m not 100% certain if it would support the needed feature (i.e. -bf 0 -g 10000 to get no B-frames at all, and the least amount of I-frames as possible). For this first release, I wanted to make sure to be as safe as possible.

You can try to play around with hardware compression yourself. The “preparation” is, more or less, simply running those commands (I put the VRLeft filter at 1080 in my example):

ffmpeg    -i ".\VIDEONAME.mp4" ^
		  -filter:v crop=in_w/2:in_h:0:0,scale=-1:1080 ^
		  -bf 0 -g 100000 -c:a copy ^
		  ".\VIDEONAME.mvs-p-frames.mp4"
"pathToFunscriptToolBox\MotionVectorsExtractor\MotionVectorsExtractor.exe" ^
          ".\VIDEONAME.mvs-p-frames.mp4" ^
		  ".\VIDEONAME.mvs"
ffmpeg  -flags2 +export_mvs ^
          -i "VIDEONAME.mp4" ^
		  -filter:v codecview=mv=pf+bf+bb ^
		  -g 1 -c:a copy ^
		  "VIDEONAME.mvs-visual.mp4"

[shamona_heehee]

Would it be possible to be able to specify a timeframe for the reencode instead of reencoding the whole video? That could potentially save a lot of time.

[Zalunda]

Encoding only part of the video might work but he will have to ‘stitch’ together all his script afterward. Using the audio sync verbs might help but there will still be manual stuff to do.

There is another parameter to the preparation that could be set, something like:
--additionalinputffmpeg "-ss 6:50 -to 12:24"

It should work but I never tried something like that.

[Sscott237890]

Thanks for building this. It has really ramped up my scripting.

[Zalunda]

Updated the tool/plugin to 1.2.3. Fixed a bug in the plugin.

@Zalunda

1.2.3

• OFS Plugin: Fixed bug where some actions were not “unvirtualized”. For example, if you generated actions then move ahead without making a change then click on increment ‘top offset’ twice. The first increment would affect all the actions, even the one behind the current time, and the second was only affecting the future action, as it should.
• OFS Plugin: Added better “debug logs”.

[cool69]

The plugin does seem good after playing with it for a while. Motion tracking is pretty accurate overall and its fast to use.

I had some small issue what i messaged to @Zalunda in DM.
I didn’t notice the actions might stay selected after you have tracked some part, and you can remove the “selected” parts if you start tracking next one.

If you move the last point what you have motion tracked, the box should say empty and you are good to move on to next part.

[Zalunda]

Updated the tool/plugin to 1.2.4. Small fixes/changes in the plugin. Nothing major.

I also added a FAQ in the Original Post:

• The original video already contains motion vectors, why do we need to reencode it?
• How do the virtual actions adjusting work exactly?

@Zalunda

1.2.4

• OFS Plugin: Added the possibility to set the default values for the “adjust” settings.
• OFS Plugin: Added the possibility to reset all the “adjust” settings to the default.
• OFS Plugin: Added the possibility to automatically reset some of the “adjust” settings when creating virtual points (i.e. the checked “R” will be reset).
• OFS Plugin: Minimum Percentage can now be lower than 1.
• OFS Plugin: Fixed the channel locking mechanism. You should be able to use the plugin in multiple OFS opened at the same time.

Since the plugin has been updated, make sure to start --FSTB-Installation.bat to update it.

[memeweaver]

Demo worked fine was able to test with the video you provided but when i tried on my own video i got this ‘FunscriptToolbox’ is not recognized as an internal or external command,
operable program or batch file.

[memeweaver]

if i move the funscripttoolbox.exe to the same directory or copy it i get: Unhandled Exception: System.IO.FileNotFoundException: Could not load file or assembly ‘log4net, Version=2.0.15.0, Culture=neutral, PublicKeyToken=669e0ddf0bb1aa2a’ or one of its dependencies. The system cannot find the file specified.
at FunscriptToolbox.Program.Main(String args)
Press any key to continue . . .

[memeweaver]

moved the script to the root folder now i get this Unhandled Exception: System.IO.FileNotFoundException: Could not load file or assembly ‘log4net, Version=2.0.15.0, Culture=neutral, PublicKeyToken=669e0ddf0bb1aa2a’ or one of its dependencies. The system cannot find the file specified.
at FunscriptToolbox.Program.Main(String args)
Press any key to continue . . … I cant get that part working

[Zalunda]

Copying/moving the executable around is not a good idea. All the files need to be in the “structure” extracted from the zip file.

If not, you’ll get “FileNotFoundException” as you had.

Your problem is probably the path in “–FSTB-PrepareVideoForOFS.1.0.bat” file.
Did you move the FunscriptToolbox folder after you run “–FSTB-Installation.bat”?

This part (green box in image below) should be the folder where you extracted the zip file:

[WIZARD]

Hello once again!
Is there any way to generate multiple motionvectors for more than one video at a time? I haven’t tried runnig multiple instances of the .bat - but I can imagine this would push a lot of load on the CPU. If there was someway to do this I would love to hear any ideas. Loving the program so far! Thanks in advance.

[Zalunda]

Starting multiple instances of the .bat file should work but it will probably take more time than generating files in sequence. Ffmpeg is already multi-threaded. So, a single instance should be able to use 100% of the CPU. Running multiple instances would only add CPU and disk contention overhead.

[WIZARD]

Thanks for the reply. I definitely noticed the CPU load after trying to play a video game while a video was being generated lol.

Would you happen to know how one might be able to create a sort of queue to generate mvs-visuals for multiple videos? I was thinking one could put multiple videos in a folder → start the bat file → end .bat after completion (press any button to close window) and repeat the process until there is a .mvs visual generated video for each video in the folder? Would this have to be a program/script, or could this even be done in the .bat parameters that are in the PrepareVideoforOFS folder? Thanks in advance.

[Zalunda]

The batch file already support it. The batch file call the tool with the parameter “*.mp4”, which tell it to process all mp4 file in the folder, in sequence.
You just have to put multiple files in PrepareVideoForOFS folder and start the .bat file.

Small unrelated tip:
When the tool is processing files, if you want to use your computer at ‘full speed’ for a while, it’s possible to “pause” the tool by “selecting text” in the command prompt. It doesn’t pause right away, but it should pause after ~30 seconds. And then, you can unpause by remove the selection with ESC key. This trick work for most of the command line application that output to the console.
Unfortunally, I just notice that it doesn’t work in the default command prompt in Windows 11.

[WIZARD]

Thanks so much for this information, also very interesting tip! Do you use another command line tool? Can it work in something like powershell?

[Unknown21]

Do we have to use the same file in your demo to get this working for other videos? I placed a video i wanted to script into the “prepare for video” folder and converted it. I load the newly created MVS video into OFS and attempt to script (no points), it says no “unable to find .MVS file for this video” even though i see it in the folder…Am i doing something wrong?