Starting multiple instances of the .bat file should work but it will probably take more time than generating files in sequence. Ffmpeg is already multi-threaded. So, a single instance should be able to use 100% of the CPU. Running multiple instances would only add CPU and disk contention overhead.
Thanks for the reply. I definitely noticed the CPU load after trying to play a video game while a video was being generated lol.
Would you happen to know how one might be able to create a sort of queue to generate mvs-visuals for multiple videos? I was thinking one could put multiple videos in a folder → start the bat file → end .bat after completion (press any button to close window) and repeat the process until there is a .mvs visual generated video for each video in the folder? Would this have to be a program/script, or could this even be done in the .bat parameters that are in the PrepareVideoforOFS folder? Thanks in advance.
1 Like
The batch file already support it. The batch file call the tool with the parameter “*.mp4”, which tell it to process all mp4 file in the folder, in sequence.
You just have to put multiple files in PrepareVideoForOFS folder and start the .bat file.
Small unrelated tip:
When the tool is processing files, if you want to use your computer at ‘full speed’ for a while, it’s possible to “pause” the tool by “selecting text” in the command prompt. It doesn’t pause right away, but it should pause after ~30 seconds. And then, you can unpause by remove the selection with ESC key. This trick work for most of the command line application that output to the console.
Unfortunally, I just notice that it doesn’t work in the default command prompt in Windows 11.
1 Like
Thanks so much for this information, also very interesting tip! Do you use another command line tool? Can it work in something like powershell?
Do we have to use the same file in your demo to get this working for other videos? I placed a video i wanted to script into the “prepare for video” folder and converted it. I load the newly created MVS video into OFS and attempt to script (no points), it says no “unable to find .MVS file for this video” even though i see it in the folder…Am i doing something wrong?
The .mvs path is computed from the video path (i.e. .mp4) that you use in OFS (i.e. pick different media).
Can you select the “Show logs” menu:
And then, look for something like this in the logs:
You’ll see the names that are ‘tested’ by the plugin.
More or less, the plugin tries to remove all the “layer of extensions” and replace them with “.mvs”.
So, for example, if the mvs file is named:
VideoName.mvs
VideoName.mp4 => Should work
VideoName.mvs-visual.mp4 => Should work
VideoName.toto.mp4 => Should work
VideoName-OnlyIFrame.mp4 => Will not work! Need to rename VideoName.OnlyIFrame.mp4 or something
In order for the toolbox to generate actions, do i need to have a script with a few actions on it first?
Yes, you need to script a few actions.
1 Like
Thank you for everything! That worked. Do you find that it only scripts so far (like for me, 2 out of 3 minutes) and you have to manually script the rest of the video as well? Sorry for all the questions, this is a game changer and i was struggling with it for a few hours.
If you go to extensions → click the extension → show window → there’s a section for “Actions Generation,” there you will find a duration for how long actions will be generated (I believe default is 120s). Hope that helps!
1 Like
Thank you!!!
1 Like
I get the following error when trying to run --FSTB-Installation.bat
I have full folder permission access.
Anybody know how to fix?
“Adding extension 'FunscriptToolBox.MotionVectors to OpenFunscripter folder ‘\OFS\OFS3_data’.
Creating use case folder ‘FSTB-CreateSubtitles’.
System.UnauthorizedAccessException: Access to the path ‘FSTB-CreateSubtitles’ is denied.”
You need to be able to create a folder in the same directory as the --FSTB-Installation.bat file.
Are you trying to run the .bat file from a network folder or something like that? It might not work in that context.
1 Like
Or if the folder is in “program files” then you need admin access rights. Run the bat-file as administrator from the right click popup menu or use a command prompt running as administrator and run the bat-file from there.
1 Like
That was it. Somehow I extracted it to the OFS main folder in program files. I moved it to it’s own folder and it completed. Thanks.
I made some small refinements to your script to make it a bit more flexible. thank you for providing the base and please enjoy.
$env:Path += ";"+ (Resolve-Path -Path ".\..")
$mp4Files = Get-ChildItem -Path .\* -Include *.mp4 -Exclude *.mvs-visual.mp4
foreach ($file in $mp4Files) {
Write-Host — motionvectors.prepare for $file —
$height=& “$(Join-Path -path (Resolve-Path -Path ".\") -ChildPath "ffmpeg\bin\ffprobe.exe")" -v error -select_streams v:0 -show_entries stream=height -of default=noprint_wrappers=1:nokey=1 $file
FunscriptToolbox.exe motionvectors.prepare --ffmpegfilter=2D --ffmpegfilterHeight=$height $file
}
I wonder how this would compare to optical flow techniques? Compression does motion estimation, yes, but there’s a reason optical flow is a growing (and separate) area of research.
Sick idea and respect for having something to show for it. It’s easy to armchair hypothesize about alternate ways to do this, but a bird in hand is worth two in the bush!
2 Likes
After researching a bit about optical flow, I would assume there’s so much potential for this to be a PhD research project for someone in the field. I’m particularly interested in seeing it applied to complex tracking scenarios, for ex, where the subject/reference is not entirely visible. I suspect larger studios are already utilizing some algos for script generation. However, I’m concerned about the computational cost of using optical flow for longer videos, although I’m sure there are optimization techniques available. I stumbled on this project which seems to showcasing some of the latest developments in optical flow:
I would also imagine, using some sort of optical flow algorithm with border/corner detection would make the quality of the scripts a lot more precise.
Another cool project here:
1 Like
There’s also the Segment Anything model from FB/Meta which could potentially be employed on a frame-by-frame basis as an alternative to optical flow.
It wouldn’t be smart enough to segment individual body parts (unless it was trained from the ground up to do so) but it would probably be able to segment entire bodies and find their centroid or lowest point.
1 Like
Yeah, the segment-anything project is fascinating. There’s tools like the Handy Simple Scripter in-Browser tool that uses Blazenet and Posenet neural network models for hand/pose tracking, but the tracking isn’t ideal. Perhaps combining optical flow with something like the hand, pose, facial models found here could improve tracking (doesn’t seem like it would need training besides for certain body-parts). As mentioned previously, studios such as SLR is probably at the forefront of these developments with their AI work.