Funscript AI Generation - VR (& 2D POV now?) - Join the Discord :)

Hi @GoNikky

Just FYI, here is the result of the current version of the algorithm.

2D - non VR related here:

In red, the version of a human made script ( @g90ak , master of osr2+ and legendary upscales), blue, what the current algorithm can output:

Definitely better on 2D POV videos now (no ? what do you think), even if initially designed for VR.

2 Likes

sorry if this is a silly question, but…

where do i put this YOLO model file?
i dont see a models folder anywhere to put it in

Hi @Dankiroth,

Create a “models” folder where FSGenerator.py is an put the models in it.

Join the discord to download the models.

Also, you can now skip the resizing step if your video’resolution is beyond 1440p, it is now done on the fly thanks to @fenderwq.

If you are here for 2D POV, don’t forget to untick the VR SBS box just under the video file selection.

The tool is initially designed for VR, still in the making and comes with bugs.

2D POV is a hit or miss, depending on settings of the scene (see section 4 above with no detection in blue).

But recent algorithm adjustments made it a tad better than initial test here for that type of video : Funscript AI Generation - VR (& 2D POV now?) - Join the Discord :) - #176 by k00gar

A lot more tweaking is needed anyhow.

1 Like

A new update was pushed to GitHub, including the following changes and fixes:

  • Inference performance
    - The on-the-fly resizing option will now default to 1080p as it seems to be a sweet spot for performance/reliability (can be changed in config.py)

  • Funscript pos. estimation
    - Tweaked the use of the visible penis length, used only when no class touching is identified by YOLO, but penetration still considered as ongoing
    - Fixed a bug in the way the visible part of penis was taken into account
    - Tweaked moving averages weights
    - Added a IOU filter logic to discard fake positives of hand touching penis
    - Added a change in the position estimation based on the class

  • Funscript report generation
    - Added a SPOILER_ prefix so the content is blurred when posted on Discord, before clicking

  • Debugging
    - The output video are now compressed with h265, full res (max 1080p), and saved with a SPOILER_ prefix so the content is blurred when posted on Discord, before clicking

  • 2D POV
    - Added a 2D POV mode during video selection
    - Tweaked the actuation speed of the locked penis box (to compensate specific cases, the camera not being still as in VR for instance)

  • Minor UI changes

6 Likes

Wow, that’s much better indeed! Thank you for looking into this. It only seemed to struggle with the non-pov doggy part. Did it not create any output at all there or did you omit it because it didnt work well?

Anyway, I’m very interested to see where this is going. I have a feeling there might be a lot of people interested in a tool that works for non-vr (and even non-pov) videos. Keep up the good work!

@GoNikky thank you, and very valid question.

I checked it out quickly and understood what happened.

First, the algorithm was initially trying to determine cuts in the video to reinitialize the tracker state, but that was counterproductive and generated some signal loss. So I removed it.

Second, in the doggy part, the butt size compared to the frame was beyond a threshold triggering some specific conditions, valid in VR (close up scene), but not optimized for 2D POV, so I tweaked it for in case of 2D POV.

Based on those changes, the algo retrieved way more info to use.

Here is the before (red), after (blue):

And compared to the human-made script (red), the generated one is in blue (currently investigating what happened in section 4):

2 Likes

double ewwwwwww, crypto scam and fucking felon schmusk’s doge of all things xD

1 Like

From the sound of it you have gotten to point of about matching what was it? SLR? I haven’t had the chance to test it but from the graphs and screens I gotta say fantastic work. data set building is a pain in the ass and it sounds like you pushed through it and made it work in training.

I would ask why the weights aren’t on the git and on discord instead though.

1 Like

Hi @hinro and thank you for your kind words :slight_smile:

Let’s be honest: no, it is not matching what this company developed.

I do not have the knowledge, nor the expertise in Computer Vision nor ML to achieve what they did, at a professional scale, and on my free time only.

So by no means am I trying to compete, I am just exploring and trying to give back to the community.

This being said, progress is made every day, trying to tackle unexpected/unwanted behavior of the algorithm wrapping the yolo model detection layer. And as you know, while you close one tap, two others opened in your back and the water is pouring.

As we covered it in the discord, the dataset is lacking of specific cases (it is struggling with latex scenes…), so it would need to be augmented, but as you mentioned it, this is really not the funny part of the job.

I am still struggling with some very specific scenes (grinding moves for instance), where your approach could be leveraged and optimized by focusing on a very specific part of the video, and on a very specific part of the frame (ROI). Happy to discuss with you if we could join efforts somehow.

Regarding the weights, since I had no clue what I was doing in the beginning (and mostly, even now), I first thought that it might be better to not expose them right away on GitHub, and that GitHub was mainly for code, not for actual weights. I initially uploaded an apple version of the weight, and it was seen as a directory, that was a mess, and it wouldn’t be cloned, making a mess :frowning:

Anyway…

Second reason is I was trying to foster engagement, and having people join the Discord actually helped:

  • Not spamming this thread with installation troubleshooting (and I can tell you I spent time helping a couple people with their install), nor with questions on how the tool works, nor suggestions, food for thought, etc.
  • Getting interest from developers, and already some Pull Requests have been merged (thank you!) and more are coming !
  • Posting videos, images, etc. that would have made this thread heavy
  • etc.

I was more envisioning this thread (which looks quite messy now, my bad, sorry guys) as a place to cover new releases, enhancements and such, even if we had kind of a breakthrough in this very thread thanks to people like @Zalunda , @jambavant and @fenderwq with the ffmpeg v360 filter to unwarp the VR frames for instance.

So there is absolutely no hidden agenda with the Discord, no pay wall to get the weights at all.

There is no monetization goal -at all-, but once again, if someone wants to put a coin in the ko-fi piggy bank, that would easen up my wallet’s pain from buying a computer to handle all this development, model training, inferring, etc. haha :sweat_smile:

And a simple thank you, a nice word from newcomers or veterans of this forum simply made my day a couple times already!

This is a very transparent post, like I am in real life, I hope it makes things clear :wink:

Looking forward to having you onboard!

4 Likes

I would say no need to undersell the accomplishments. I’ve tried to do what you did and lost my dang mind. From the screenshots that were shown when it was being promoted they were using yolo and even if you don’t have as many classes as they do I would say getting as far as you have so far is quite the accomplishment. In your spare time you have matched where they were.

As for the latex I could be wrong but I would think that the problem there would be the uniform color not having the usual details that yolo would look for to differentiate parts as it usually would look for. Perhaps treating latex scenes like a green screen and having large areas of voids changed to a different color or a gradient could bring out the details enough for it to work. Assuming of course that the problem is the tracking is not picking up enough details to properly track.

Yeeeeeah it’s always a fun time when you get one bug squashed then finding two more that were hiding :sob: it never ends.

1 Like

Thank you @hinro , I really appreciate.

Regarding the latex scenes:

I will definitely investigate that option, makes total sense to try that out before running to the raw solution of enriching the dataset with more latex scene pictures (which might also have side effects on the detection quality for regular scenes… who knows…).

1 Like

Updates: v0.0.1_25-01-16

  • Funscript pos. estimation - Fixed - Close up did not deactivate the locked-penis-box, interfering with funscript pos, when it should be 100

  • Funscript pos. estimation - Fixed - Doggy not triggering action, under specific circumstances

  • Funscript pos. estimation - Fixed - Major issue in penis-inserted length computation

  • Funscript pos. estimation - Fixed - Cowgirl not defaulting to tracking absolute position of breast when pussy is covered (her red cloth at some point of the video), under specific circumstances

  • Funscript pos. estimation - Reworked - Part of the body parts interaction logic, lessening sudden jerky moves from one position to the other

  • Funscript pos. estimation - Reworked - Discarded hands positions passing over penis location during penetration

  • Application - Added - A proper logging system, writing to both console and FSGenerator.log => please share that file for any debugging purpose along with all other output files

  • Application - Added - A version number, for reference, both in logs and in funscript ‘version’ field

  • Application - Removed - All message dialog boxes that would prevent a batch processing approach as suggested by @jcwicked (thanks again, and PR on hold for now)

These changes have a major impact on the output files!

5 Likes

Updates: v0.0.2_25_01_18

  • Fixed an issue in case of processing more than one video without restarting the app

More awesome stuff coming!

3 Likes

HotFix: v0.0.3

  • Funscript tweaking - Fixed - Deactivating the simplification was raising an error
  • Funscript tweaking - Fixed - The rounding of the position was not deactivated in sync with simplification state
  • Funscript generation - Fixed - Duplicate timings in reference funscript were raising an error

Soon, currently in the making:

  • Major performance improvement in the making (FPS x2 on my Mac for instance, as of now)
  • Better HJ/BJ handling via a 2 pass approach
  • And much more!

Buy Me a Coffee at ko-fi.com

5 Likes

HotFix: v0.0.4_hf01

  • Tweaked the initial P-detection and locking. The logic was recently loosened up to be less restrictive, leading to erroneous early tracking

If you have already processed videos, you can reprocess them in minutes if you kept the matching _rawyolo.json file, you do not need to go through the hassle of a full YOLO detection, this fix only concerns the tracking layer of the logic.

Is the discord link expired that is above?

Hmm, should be working fine. Are you having any trouble joining ?

I think the link is set to not expire.

This one: VR Funscript AI Generation

I get this when I click it
Screenshot 2025-01-25 180559

Be in touch with me in PM, as we keep having new people joining, the link seems to be ok for most of us :frowning: