gonna sound of topic but will this work for “regular” non porn stuff or help with translations I’ve got lot of media that isnt in a language I can speak with no english subtitles
Yes, it could be used for transcribing/translating all types of content but the initial config that’s created when installing the application is ‘optimized’ for VR porn. The SystemPrompt/UserPrompt in the config tells the AI to play the role of a “translator specialized in adult film subtitles” with rules specific to that task. You would need to change the config to fit another domain (while keeping the rule about the format of the JSON).
@Zalunda My understanding is that the Private Poe bot is used to create more accurate subtitles that have additional context, is that correct? In the directions, it mentions a step where we go through each generated .txt file and paste the full prompt to the Poe bot to get back a translation.
Is there a step that I’m missing or was I doing it correctly? I’m pretty lazy so I was considering skipping this step lol.
Process so far is pretty straightforward. Going off of what @TheFauxPro mentioned, I used ChatGPT to create python script to filter only specific translations I wanted. I can try posting the program if anyone’s interested (No clue how to do this, probably just posting the code?).
Original
Reduced the translations to only [mergedvad] and [claude-3-haiku]
Creating a “private poe bot” is used mainly to reduce the size of the requests we must give to the chatbot. So, instead of providing the full ‘instructions’ in each of the prompts, we set the instructions once in the private bot “system instruction”, then we can only give the JSON when “talking” to the chatbot. It’s also useful because there is a limit on the size of a request.
Also, this is not related to the additional context. It’s possible to give more context using a private bot or not. BTW, giving additional context is done on the ‘refining subtitle’ step (i.e. creating .perfect-vad.srt file). That means taking a little bit more time and adding stuff like “{Context:…}” or “{Talker:…}” in the text of subtitles:
Once added, the additional context will then be included in the prompts to help the AI give better translations. Of course, even though it might give better translations, it’s optional.
3 Likes
Added a 3rd tips in OP:
Tips #3 - Fixing WaveForm visualisation in SubtitleEdit (for some video)
When working with audio in SubtitleEdit, you may encounter a problem where the waveform appears noisy, making it difficult to identify the start and end of voice sections, like this:
Upon further inspection in Audacity, I noticed that the audio had “long amplitude signal” throughout the audio:
This low-frequency noise is causing the poor visualization in SubtitleEdit. To fix this, we need to filter out the low-frequency sound in the audio. It should be noted that it probably has no effect, good or bad, for whisper transcription since, I assume, Whisper already ignores those low-frequency signals (until they add ‘Whale’ to the supported list of languages).
Fixing the Issue for All Future Videos
- Open
--FSTB-SubtitleGeneratorConfig.jsonconfig file. - Locate the “AudioExtractor” section.
- Add
-af highpass=f=1000to the “DefaultFfmpegWavParameters” to filter out frequencies below 1000 Hz. You can also addloudnorm=I=-16:TP=-1to normalize the audio (unrelated to the problem):"AudioExtractor": { "DefaultFfmpegWavParameters": "-af \"highpass=f=1000,loudnorm=I=-16:TP=-1\"" }, - Save the config file.
Applying the Fix
- Open the
.perfect-vad.srtfile in SubtitleEdit, which should automatically load the associated video and audio from the.mp4file. - Drag the
\<videoname\>-full-all.wavfile from the\<videoname\>_Backupfolder over the waveform part of SubtitleEdit. It will only replace the waveform/audio. The video can still be played.
You should now see an improved waveform, like this:
This waveform is much easier to work with, as the voice sections are clearly distinguishable from the background noise.
2 Likes
would any of you who are learning to do this be willing to take requests? like we do here for scripts? lots of amazing jav scenes now getting AI scripts on SLR, would be next level to get subtitles for our faves?
2 Likes
FYI, I made a new version:
FunscriptToolbox 1.3.4 release
New Features:
- Added an attribute PreviousTranslationId to AIMessageHandlerJson which allows refining another translation (ex. “claude-3.5-sonnet” and then “claude-3.5-sonnet-refined” in Example.json file).
- Added a new output type “Wav” that can create a transformed wav file (ex. with option “highpass=f=1000”) that’s not used when doing the transcription. Transformed wav file can be used in SubtitleEdit, instead of the original audio.
- Added a new TranscriberToolExternalSrt which can be used in 2 ways:
- Import an existing subtitles file as-is (i.e. when trying to improve an old subtitle file).
- Use an external service like CapCut to transcribe, and then import the transcription result as-is.
- Removed “Whisper” in the transcriber name (“WhisperSingleVADAudio”, “WhisperFullAudio” and “WhisperMergedVADAudio” => “SingleVADAudio”, “FullAudio” and “MergedVADAudio”). All those transcribers can now also be used with an external tool.
- Added a new “AutoSelectSingleChoice” property to WiPSrt. False by default.
- Updated “–FSTB-SubtitleGeneratorConfigExample-1.2.json” with the new options.
Bug fixes:
- Fixed bug when subtitles file contains only a number on a text line (ex. “3”).
- Fixed bug when “WIPSrt\IncludeExtraTranscriptions == false”.
- Fixed/Changed timestamp format in AIRequest files (1940.1 => 32:20.104). With the old format, it was possible to have duplicate entries and, also, it was harder to understand if you wanted to search for the location in the video.
Note: If you have already had a configuration file, you’ll need to replace those string in your .json file:
- “WhisperFullAudio” => “FullAudio”
- “WhisperMergedVADAudio” => “MergedVADAudio”
- “WhisperSingleVADAudio” => “SingleVADAudio”
If not, I suggest starting with the new --FSTB-SubtitleGeneratorConfigExample-1.2.json that is created when running --FSTB-Installation.bat.
1 Like
Thanks so much for putting this together. It’s awesome. Apologies if I missed this somewhere or if it’s overtly obvious, but what’s the best way to add in subtitles it missed manually? Best fix I’ve found is playing the audio into my internal mic with Google Translate but it’s not great at picking up whispers.
If you use SubtitleEdit and have configured Whisper like I wrote, you can add location by hand and then force a whisper transcription for that subtitles:
Brilliant, thanks. I knew there was something but there’s so many options in here.
The grok beta in Poe does a good job with Japanese translation. It doesn’t refuse requests and can correct homophones in Japanese to the appropriate characters.
1 Like
I just did a test with Grok. It’s not bad, but I still prefer Sonnet-3.5+. Grok is not as good at understanding the context of the scene.
With a private bot based on Sonnet-3.5+ (as explained in the OP), I rarely get refused. And if it does, it’s a ‘soft refuse’. I just have to say something like, “Aren’t you an adult translator?”, and it will reply, “Yes, you are right. I’m an adult translator…” and translate it without complaining. To reduce even more the number of times that it does that, I also add this in each prompt: Remember that you are an adult movie translator..
Is there any tool more accurate than Whisper? I tried V3 Turbo, but the results weren’t better than V2. Now I’ve tried improving the audio quality with ffmpeg, and the transcription results are pretty good.
1 Like
I’m having an issue on the Transcribing ‘singlevad’ step, where it seems to endlessly be looping at a “detecting language” step. Any idea what’s going on here?
Its probably a section where it can’t detect any language. Usually it default to english in that case but there might be case where it loop.
You should try specifying the source language in the batch file (–FSTB-CreateSubtitles.1.6.bat):
--sourcelangage Japanese
Thanks! That seems to fix part of the issue, but it still appears to be looping. It’s also flashing text like “50% | 1/2 | 00:00 < 00:00 | 0.58 audio” or different variations of that, while the main text “[PurfviewWhisper][0/1969]” remains the same. It doesn’t seem to be making progress because it always stays at [0/1969]. I appreciate the help!
I’m assuming that it worked for full and mergedvad transcription?
I’m surprised that it would work for those but not for singlevad, unless there is a weird .wav file that is created during the process, maybe an empty file or something.
Can you try running whisper-faster directly and tell me if there is an error or something?
In short, this command, in a command prompt, while replacing PATHTOPURVIEW, PATHTOVIDEO, and DATETIME with value for your machine/context:
"PATHTOPURVIEW\Purfview-Whisper-Faster\whisper-faster.exe" --model Large-V2 --language ja --task transcribe --batch_recursive --print_progress --beep_off --output_format json "PATHTOVIDEO_backup\DATETIME-singlevad-*.wav"
You might want to try the command with DATETIME-mergedvad-all.wav to be sure that everything is OK for that one on the command line.
Looks like it was actually processing, the 0/1969 wasn’t updating so I thought it was was looping. All good here! One thing I noticed for the AI prompts, is it explicitly states the man does not speak. I am using this method to translate a normal JAV scene. Do you have an pre-made verbiage that doesn’t have language regarding ignoring the man’s speech? If not I can just edit myself. Thanks!
Are you on an English OS? Because I’m parsing the output of whisper to show the progression. Right now, I’m looking for “Starting …filename…”. If whisper writes something in french, for example, “Démarrage …filename…”, I won’t be able to update the progression but, like you say, it will still work in the background.
Also, singlevad takes more time than the other type for transcription so I can understand why you thought it was stuck on a 1969 files batch.
As for the VR/Not-VR, if you use sonnet-3.5 or any ‘high quality’ AI, you can simply write it in the context.
Something like this in the first subtitle of the file:
{Context:This is not a POV or VR scene, its a 2D scene. There is 3 peoples in the room that will talk. A woman, who's blah blah, her boyfriend, ...}
You might want to add {Talker:Boyfriend} on each subtitle but, with 1969 subtitles, it might be a serious pain in the ass so you can hope that the AI will be able to pick up who’s talking and keep a ‘coherent’ understanding of the scene.
If you have to change the context information later in the file (i.e. if the setup of the scene has changed or something), always start by repeating that part.
Yes I am using an English OS. It was running in the background so we’re all good.
Thanks for the advice with the context, I will try that, though I’m starting to feel the effort isn’t worth it for the length of the video. I might just stick to shorter VR scenes for now haha.