One of the ideas I had for allowing remote operation of an OpenROV (as opposed to controlling it from a tethered laptop) was to use PubNub, a real-time messaging platform.
tl;dr: I got consistent round-trip latencies of 130-160ms, meaning ~75ms latency for issuing remote commands. This was faster and more consistent than relaying commands over a node.js server and socket.io. This will be plenty fast for issuing remote commands to the ROV.
I wrote a sample app with three parts:
- a server to serve pages and accept commands from the cockpit
- a cockpit page to respond to commands sent over the PubNub channel, responds on PubNub, and repost the command to the server
- a remote page with buttons you can click to send commands over the PubNub channel
[Note - no guarantees how long I'll leave the app up, so don't be surprised if you read this in the future and the app is gone]
To test it out:
- open http://openrov-latency-test.herokuapp.com/remote
- open http://openrov-latency-test.herokuapp.com/cockpit in another window (there's nothing to see here, it just needs to be open)
- click buttons to issue commands
- see the latencies for PubNub and SocketIO communication
- (the node server and socket.io connection are a little flaky on the Heroku host - if you don't get a result, try reloading both pages and waiting a few seconds before issuing commands)
These are for round-trips in milliseconds from /remote to /cockpit and back.
Latency (ms) when using a local server:
- Pubnub: 100-170
- Server: 60-70
On the Heroku server:
- Pubnub: 120-160
- Server: 150-190
~150ms for the PubNub roundtrip means ~75ms for the tethered laptop to receive and relay the commands, which should be more than fast enough.
See the code here: https://github.com/pchristensen/pubnub_test. It took about 3 hours to do this. Most of this was for the server - the PubNub part took less than an hour.
Software Boss Level
On the cockpit UI, if we separate the user actions (key presses, button clicks) from the commands sent to the ROV, then the PubNub message handler can bind to the same commands without having to do something janky like triggering UI elements.
The latency for issuing commands is probably much less than whatever latency is possible for relaying video.
In order to let multiple users share this infrastructure without having to program, we can add a button to cockpit to generate a random hash that can be added to the messages. This way different ROVs can share the same channel without their commands clashing.
Since all the communications happen between HTML pages, the server doesn't need to do anything. We can server a static HTML page and static JS file.
I don't know how to make the video relay streaming work. This link has come up before, where they turn a BeagleBone and Logitech HD Pro Webcam C920 into a streaming camera. I think Jim Trezzo said he was going to look at it because he has that model webcam.