Funny enough, I attempted to make some personal adjustments to this project last week.
The one that I believe would reduce the barrier of entry is that of the color selection when choosing the blue and green. Instead of having to fidget with the color values, I’d recommend making it like a color picking tool where you just click on the webcam preview where the color is and then being able to adjust the threshold from that to narrow it down as needed.
The other adjustment I couldn’t get to work was making project utilize my phone as the webcam through the DroidCam app but it was having a ton of difficulties with it. I was able to get it to show on Test Webcam after using OBS as a passthru but it wouldn’t work with any of the other tests unfortunately. This is unique for me so not a beneficial adjustment for many others.
The final adjustment I didn’t get to was getting this to work with HereSphere.
Hello. For use phone as webcam i can recommend to use Camo Studio with USB wired connection (less latency then in wireless), but you need to take into account that turned on backlight (or some external Ring Light) with constant web camera focus is mostly required for making it work stable (for correct color selection), also it is very recommended 60fps (for more accurate control) camera at Full HD, also the usb webcam i think will gives less latency between get frame on PC then using phone even by usb connection. Now it mostly working only with modified VLC for support sync in milliseconds (which is very important for accurate work)
I am using cheap web camera similar to “Angetube 1080P 60fps Webcam with Ring Light and Privacy Cover 862Pro”, with Full HD with 60fps with auto fucus (manual focus is also supported by API), but it gives very good results with external Ring Light (but it also has own backlight, as i remember it doesn’t give the same colors stability).
Just checked on Samsun A72 without external light (but with turned on Flash light, manual focus, manual temperature, 720p 30fps (not 1080p 60fps due to high input lag, especially with higher resolution and frame rate)) in Camo Studio (free version) with USB wired connection, also working not so bad, but results are less accurate, most likely, the reason is large delay between the time of the real frame and the time when this frame was received by the program (the connection is not direct, but through the intermediary program Camo Studio on both the phone and PC)
Just talked with AI, using smartphone as web camera is fully not recommended.
The reason is End-to-End Latency (real image time and it’s obtain on PC), then higher this value then less possible any correct control, especially on speed 40+.
For comparison End-to-End Latency:
smartphone USB (wire) connection 120–160 ms
smartphone Wi-Fi (wireless) connection 200–350+ ms Web Camera 1080p 60fps in good case 50–80 (In low light conditions, due to auto exposure, it increases to 100–120 ms)
8.00 version was released.
Updated webcams support and hismith control, now smartphones are also supported with USB connection.
Also hismith control now should be more accurate especially with webcams 1080p 60fps.
Added support for selection color ranges in “Test Webcam” (Ctrl or Shift + mouse selection), but you still will need manually adjust colors.
Highly recomended to adjust “webcam_end_to_end_latency” in settings.xml especially if will use Smartphone even by USB connection.
for good Webcam: 50
for Smartphone by USB connection: 120
End-to-End Latency is difference in real image time and it’s obtain on PC, then higher this value then less possible any correct control, especially on speed 40+.
Known End-to-End Latency:
Smartphone USB (wire) connection 120–160 ms
Smartphone Wi-Fi (wireless) connection 200–350+ ms
Webcam 1080p 60fps in good case 50–80 (In low light conditions, due to auto exposure, it increases to 100–120 ms)
The next statistic was obtain by tests:
Smartphone 1280x720 30fps (for decrease End-to-End Latency) “webcam_end_to_end_latency” set to 120
average_dif_end_pos: 6 deviation_dif_end_pos: 19
average_dif_end_pos: 7 deviation_dif_end_pos: 23
average_dif_end_pos: 12 deviation_dif_end_pos: 25
average_dif_end_pos: 3 deviation_dif_end_pos: 25
Webcam 1920x1080 60fps “webcam_end_to_end_latency” set to 0 (btw, this diff is according by Webcam, but in reality due to End-to-End Latency moves will become about 50-120ms later, so it is highly recommended to set “webcam_end_to_end_latency” as min to 50)
average_dif_end_pos: 6 deviation_dif_end_pos: 16
average_dif_end_pos: 5 deviation_dif_end_pos: 21
average_dif_end_pos: 8 deviation_dif_end_pos: 20
average_dif_end_pos: 11 deviation_dif_end_pos: 21
average_dif_end_pos: 2 deviation_dif_end_pos: 13
Webcam 1920x1080 60fps “webcam_end_to_end_latency” set to 50
9.00 version was released.
Added support to set manual exposure and get webcam end-to-end latency.
IT IS HIGHLY RECOMMENDED TO SET MANUAL WEBCAM FOCUS AND MANUAL WEBCAM EXPOSURE TO ABOUT -7 (BEST VARIANT) OR -6 TO MINIMIZE WEBCAM END-TO-END LATENCY You can set manual webcam focus (for example to 300) and exposure if it support it by using shown hotkeys in “Test Webcam”, also as change manual webcam focus, exposure and fps (for example to 30 or 60) through settings.xml
It is highly recommended to set “webcam_end_to_end_latency” in settings.xml especially if will use Smartphone even by USB connection
End-to-End Latency is difference in real image time and it’s obtain on PC, then higher this value then less possible any correct control, especially on speed 40+.
You can manually measure real “webcam_end_to_end_latency” by press “Get Webcam End-to-End Latency”.
You will need to use monitor in this case, then higher monitor frame rate then more accurate data will be obtained.
Known End-to-End Latency:
Smartphone USB (wire) connection 120–160 ms
Smartphone Wi-Fi (wireless) connection 200–350+ ms
Webcam 1080p 60fps in good case 50–80 (In low light conditions, due to auto exposure, it increases to 100–120 ms)
The next statistic was obtain by tests:
Webcam 1920x1080 60fps “webcam_end_to_end_latency” set to 50
average_dif_end_pos: 6 deviation_dif_end_pos: 18
average_dif_end_pos: 7 deviation_dif_end_pos: 22
average_dif_end_pos: 8 deviation_dif_end_pos: 19
average_dif_end_pos: 8 deviation_dif_end_pos: 21
average_dif_end_pos: 9 deviation_dif_end_pos: 20
average_dif_end_pos: 3 deviation_dif_end_pos: 16
Smartphone 1280x720 30fps (for decrease End-to-End Latency) “webcam_end_to_end_latency” set to 120
10.00 version was released.
Updated algorithm for get hismith position and added Random Video Player to support.
Supported video players
This project now supports:
Highly recommended: VLC special build - which provides high sync with video do to not only providing video time in milliseconds but it also provides real system time related to it and correct video playback rate, you can find it in the latest artifacts in 3rdParty VLC fork project
Random Video Player - is also mostly accurate due to providing video time in milliseconds, but in version 1.64 it has 2 issues and 1 nice to have feature:
incorrect reported video playback rate: always report 1
it doesn’t always provides correct play/pause state: if set to pause and change video position it can start to play video but still report that it is on pause
it doesn’t provides real system time in milliseconds related to video time for make video sync more accurate
Standard VLC - you can also try to use standard VLC but be aware that re-synchronization of Hismith moves and video timeline can be ~500-1000 milliseconds and there isn’t any guaranty that it will work correctly.
If you correctly configured VLC you will can check this in Firefox (in my case Chrome doesn’t open it) by using:
start VLC and open some video in it (you can pause video)
open http://127.0.0.1:8080/requests/status.xml in Firefox or Chrome (it will should ask user and password (in my case empty username field and password: 1234), after provide it will should show VLC statistics data)
If you will use Random Video Player:
You will need to activate ‘Start timecode server’ in Settings → Sync tab
11.00 version was released.
Added Here Sphere video player to support and made fixes for video speed rate support.
Supported video players
This project now supports:
Highly recommended: VLC special build - which provides high sync with video do to not only providing video time in milliseconds but it also provides real system time related to it and correct video playback rate, you can find it in the latest artifacts in 3rdParty VLC fork project
Random Video Player - is also mostly accurate due to providing video time in milliseconds, but in version 1.64 it has 2 issues and 1 nice to have feature:
incorrect reported video playback rate: always report 1
it doesn’t always provides correct play/pause state: if set to pause and change video position it can start to play video but still report that it is on pause
it doesn’t provides real system time in milliseconds related to video time for make video sync more accurate
Here Sphere - VR video player
Standard VLC - you can also try to use standard VLC but be aware that re-synchronization of Hismith moves and video timeline can be ~500-1000 milliseconds and there isn’t any guaranty that it will work correctly.
If you correctly configured VLC you will can check this in Firefox (in my case Chrome doesn’t open it) by using:
start VLC and open some video in it (you can pause video)
open http://127.0.0.1:8080/requests/status.xml in Firefox or Chrome (it will should ask user and password (in my case empty username field and password: 1234), after provide it will should show VLC statistics data)
If you will use Random Video Player:
You will need to activate ‘Start timecode server’ in Settings → Sync tab
