sorry for the claude slop. I can have a look at it myself and perhaps setup CI to build linux releases.
I only tested cuda once on windows and for me it kept falling to DML until I dropped the dlls next to the executable. It probably requires a separate cuda installation on linux too additionally to the nvidia driver. I don’t know if there’s a plug and play solution.
CPU is not viable it is way too slow for v-jepa
edit: I did push another build this time with a windows dml and a linux cuda build.
You do still need to install the dependencies though. I tried the webgpu route for linux but it’s like 2 times slower than dml and almost 3 times slower than cuda. not worth the maintenance effort.
This is really interesting. i have a small gripe though. whilst trying a vr scene..when i get to the “aim the vr clip” part.. i click on each window and then choose(click and drag) the spot best for the tracker..but if i go through the windows again it seems to just stay in the location where i set it in the last window. is there some way to make it stay in the place i chose or is it just one location for the whole video. GREAT work by the way
no that is the point you are supposed to get it into one window. so sometimes you have to increase the hfov for that.
I could change it so it’s tied to the shot cuts just like auto-crop rects. But I think for most VR scenes it’s not that hard to get it into one window and then cropping stage remains the way to get it framed correctly
hey man, im running into an error running 5.1. i have a 5080 and its failing during auto crop. also i have all my drivers up to date. maybe im missing something. i can message you the error log if that would help?
It works on my machine™️
Try this if that works I can create a new release
# 1. Reproduce in seconds, no video needed — this should fail at node_mha0 today
./goblinscript --bench
# 2. The same bench with the switch — this should pass
ORT_ENABLE_CUDNN_FLASH_ATTENTION=0 ./goblinscript --bench
# 2.5 If it still doesn't work you can run with. which may give some pointers what's going wrong
ORT_ENABLE_ATTENTION_KERNEL_DEBUG_INFO=1 ORT_ENABLE_CUDNN_FLASH_ATTENTION=0 ./goblinscript --bench
# 3. If 2 passes, that is the workaround for a real draft
ORT_ENABLE_CUDNN_FLASH_ATTENTION=0 ./goblinscript "video.mp4"
goblin error.txt (2.1 KB)
this is the error im getting when trying to run videos now on 5.1. like i said before im running a 5080 and all drivers are up to date.
First off, thank you so much for creating and sharing GoblinScript — it’s an amazing tool.
I had an idea and was curious about your thoughts:
Have you ever considered training position-specific heads (or conditioning the model based on detected poses/positions)?
For example, having a classifier goblin identify the scene/position first, and then routing inference through a head trained specifically for that motion pattern.
Do you think specialising by position might increase accuracy, or would the extra complexity/data labelling not be worth the trade-off compared to one general model?
I could try taking an existing pose estimator checkpoint and feeding it’s outputs additionally into the model to see if it enhances the output in any way.
I don’t want to label hundreds of hours of video by hand.
I did not instead I tried using the 5.0 version and it worked perfectly. I did have another question then. Is it possible in VR to set different view points because I can’t always get all of the things happening in one selected view as the scene moves along?
