Motion Tracking Funscript Generator v0.5.x

Not sure why it was sometimes worse than other times. Maybe OFS calls “update” more often in some context.

you are right. The update function is called several times a second (depending on the gui refresh time), each time trying to open the corresponding file, which leads to a lot of open files (because of the missing close instruction).

Thanks for the bug report. I will create a new release.

2 Likes

Very nice application but i was wondering if it would be possible to generate a funscript in all categories at once such as roll, twist, up and down. Generating one at a time is a bit slow for me.

@amazingman302 most of the code needed for this function is already implemented. The main problem to get this function working is the output assignment to the correct scripts. Currently I use the active script as output for an single categorie. If we have multiple outputs you could possibly control it by the names of the funscript. But this would only work if the users respect the given naming scheme (Maybe we could use the default names (post-fixes) from OFS shortcut functions for this). If someone knows a solution I am open to suggestions to implement the multi axis output.

1 Like

Hey :v:

Thanks for the great program, I have a problem and maybe someone here can help me. I have an Ubuntu operating system and am trying to get the OFS plugin to work.

I have cloned the repo and created a runtime environment via conda as described in the readme. This all worked great and I can also do moition tracking via the GUI.

To use the software productively I am trying to get the OFS plugin to work. I also use the latest OFS version 1.4.4.

After I copied the main.lua script into the folder with the extension, I can see the UI of the plugin in OFS. Unfortunately, the start button does not launch MTFG. Nothing happens when I click on it.

I then took a closer look at the Lua script and found that it references the .exe file and partially uses Windows-specific paths. Under Ubuntu with the conda runtime environment this .exe file does not exist. The same is true for VERSION.txt.

Can someone tell me how I have to adapt the Lua script so that it works the OFS plugin also under Ubuntu?

@linxguy I add linux conda support to the latest main.lua script. I tested the setup with this install script. If something does not work for you with the new lua script, let me know.

Nice, thank you :pray:
The install and build worked great via the script. Also the tracking now works in OFS now :tada:

What influences tracking speed - is it CPU or GPU? I’m using an old i7 2.3GHz Mac Mini from 2012, and I get around 12 FPS for accurate, 30 FPS for fast. I wonder if one of the newer Intel NUC boxes might be handy as I don’t have a PC.

What influences tracking speed - is it CPU or GPU?

There are several factors that influence the tracking speed. But the CPU performance has the strongest influence on the tracking speed. Another factor that should not be missed is the transfer speed of the RAM, since larger amounts of data are processed and copied here.
The GPU is only used to decode the video, whereby special decoders insied the GPU are used for the corresponding video file formats. Here it is sufficient to use a reasonably current GPU to be able to decode all common video formats with hardware acceleration. If you have older Hardware you should use videos with older video formats such as H.264 to get working hardware acceleration. If there is no hardware decoder in the GPU for the given video format, the CPU is used for decoding (slower).

Really interesting, thanks! It’s a very clever use of computer vision.

Nothing of value to add, just a reminder that the creator of this is an absolute hero to the community and if there was a church of funscripts this man would be canonised twice over by now!

4 Likes

When using the new version, I encountered a problem with tracking quick actions. Everything was fine on the old version.
Ver. 0.2.4
image
image

Ver. 0.0.9
image
image
Here are examples of tracking the same area. Please write if anyone knows what could be wrong.

2 Likes

@qweer Thank you for reporting this issue. I have rewrite the post processing part to provide more detection options in the newer versions. In this context i also introduce an filter for the local min max detection which was hard-coded to 3 frames. I changed the default filter length back to 1 to get the same behavior as before. Please try the new version v0.2.5 and let me know if it works again.

4 Likes

I tested the tracking on a few quick episodes, now everything works as before. Thank you for your great contribution to the community and quick fixes.

1 Like

@mchyxnaaiorxfwrivv

Newer to the forum here, but having snooped through your git I can’t help but notice you had some experiments in FunPos in November 2021. Out of curiosity, have you had any further luck tampering in this space?

Curious to know what the roadmap is in your head for what the next new features in MTFG are most likely to be.

@bobsaget

Newer to the forum here, but having snooped through your git I can’t help but notice you had some experiments in FunPos in November 2021. Out of curiosity, have you had any further luck tampering in this space?

My experiments with Deep Learning for the automatic generation of funscripts (the FunPos repository on Github) could not achieve competitive results to the MTFG method so far.

Detailed notes for the technically interested:
I wanted to try the Vision Transformer models anyway. Therefore i experiment with different Video Vision Transformer Models. But the main problem is that Vision Transformer and Video Vision Transformer in particular have much too high hardware requirements to get usable results. We are talking about more than 256GB of RAM! There are now papers which suggest ways to reduce this hardware requirement with some clever techniques. But I have not yet found one which convinces me completely. The experiment with some older ConvLSTM architecture result in an more usable Model (but the results were not as good as the MTFG). The main problem with these models was that they mainly learned to turn motion in the video into a funscript regardless of where the motion takes place.

Before I created MTFG I tried 3D Pose Estimation with detectron2 (I don’t have this code publicly available because it is too messy). One advantage of this model is that you could generate a multi axis output. However, in my experiment I only output the y axis. For some positions it even gave acceptable results. However, it was very slow with 2-3 fps on GPU. Therefore this method is not really an alternative. In addition to improve such a model, you would have to label the 3d human positions in the video for fine tune the model. But this is far too time-consuming for this kind of required labels.

One idea I haven’t tried yet would be to train an R-CNN on the penis and use the output mask for the prediction of funscripts. But for this I would have to label a few images first, which I currently have no motivation to do. Therefore i will postpone this test even further.

Curious to know what the roadmap is in your head for what the next new features in MTFG are most likely to be.

At the Moment I am looking for which technology could replace the tracking. I don’t have a planned roadmap with new features, because this project is currently not very high prioritized for me and the general functions are now implemented. Currently I’m only working on it when I need some time off from my non porn projects.

That’s actually super interesting! Since this is so similar to some stuff I’ve worked on in the past I can’t help but continue pulling at some strings here.

The first thing I looked for in your git was any trace of pose haha

Instead of a 3d model, have you tried using a lightweight 2d one like a tensorflow-lite (tflite) posenet implementation? There’s a limited but doable degree of 3d estimation from 2d skeletons in this very specific case because the skeletons should have the same proportions (same porn actor/ress in video). In short something based on

  • Not optimizing the net on the accuracy of the points, but instead on the accuracy of the angles between pose keypoint vertices (e.g. the angle between woman right hip - left hip angle vs man right hip - left hip)
  • In doing so, we have labelled data pre-generated somewhat for us in the form of twist and roll scripts. Yep they’re affected by personal tastes…etc, but maybe a year or so down the road we will have hopefully sufficient data such that we can probably slap in the views/downloads in the reward function.

That would however move away from a tool used to generate funscripts accurately and instead move towards a tool used to generate funscripts that are simply popular, or enjoyable.

Understandable. If whenever I find myself in a situation where I can peel off work I’d be down to try cobble something together and have you take a look. With full disclosure that my strength at work is usually more on the math side and I consider myself to be quite abysmal at programming, and with full understanding that I’m not expecting you to take your time sanitizing my chicken scratch.

I tried to use MTFG with a 2D video for the first time today. When using it with VR, I got used to rotating the view to get a better alignment with movement. In the flat projection, I kind of missed that feature so I changed the projection to allow a rotation of the 2D image.

@mchyxnaaiorxfwrivv: Thanks for the nice way that you implemented the parameters/keys in the file, I was able to add the new rotation parameters without changing the code.

flat:
  name: '2D Video'
  video_filter: "pad='max(ih,iw):max(ih,iw):max(ih,iw)/2:max(ih,iw)/2',scale=${width}:-1,rotate='(PI/180)*${rotation}'"
  parameter:
    height: 1240
    width: 1240
    rotation: 0
  keys:
    rotation:
      increase: "q"
      decrease: "e"

If anyone wants to try it, it’s possible to change the “projection.yaml” file manually (use the menu "Extensions\Funscript Generator Windows\open directory, then open the file “…\funscript-editor\funscript_editor\config\projection.yaml”).

Hello fellow password generator user. I’m working on something similar. I have one button in the OFS GUI that launches the computer vision application and dumps the result to a file, very similar to your approach.

Once the computer vision is done there are multiple buttons in the OFS gui, “import X”, “import Y”, “Import PCA 1” and a few sliders, the buttons invoke a python script that performs some post-processing on the computer vision data and pastes the result in the active timeline. The csv file isn’t removed automatically so you can do multiple imports from one computer vision run.

I suppose one could also perform the postprocessing in lua, depending on the complexity.

Great idea to add rotating to the 2d video option. I have adopted your code in the default config on GitHub.

1 Like

Thank you for the idea. It is definitely a good basis on which I could start. Maybe i should switch to a json output for better multi axis handling. I might have to come up with something to make your idea active only for multi-axis outputs (e.g. checkbox in OFS) …

In any case a good suggestion. I will definitely take a closer look in the next few weeks.