Joystick Control (HOTAS) and HUD/OSD

user-interface

#1

Just wondering if there is anyone out there working on programming joystick control for the OpenROV? (Just to clarify, I have *no* knowledge of Arduino or programming, this is all just speculative suggestions!)

It would allow full control of all the systems on the ROV without having to take your hands off the joystick?

Obviously, the joystick itself (taking a Microsoft Sidewinder II as the example) would allow Forward/Reverse, Left/Right Yaw, using the trigger and the large thumb-switch to control vertical Ascent/Descent.

The throttle could be programmed to alter the thruster speed, to allow fine-tuned control or more powerful (in currents/for larger transits).

The hat switch could then be used to control the camera tilt controls. The LEDs could be activated using one of the smaller two thumb buttons.

Finally, an On Screen Display/HUD might be handy, giving you compass headings etc.

Any thoughts?


#2

I am programming a joystick / alternative input control for the OpenROV as a separate project, nothing 'official'. See my post about my prototype for controlling the OpenROV with a 3D mouse, http://openrov.com/forum/topics/controlling-openrov-with-a-3d-mouse

The important take away from that endeavor is that I was happy to be able to inject external inputs into the ROV control stream. My first external control was to use the 3D mouse, the next is a joystick. I have a wireless Logitech joystick (no longer sold), http://www.logitech.com/en-us/support/game-gear/286, but in the past I have used it via the Microsoft DirectX SDK to capture joystick events.

Currently I'm in the process of updating an application to intercept the joystick controls and send them to the ROV web page using web sockets just like I've done with the 3D mouse.

-Marcus


#3

Also, make sure to check out the UI Dominik and Simone have been working on.

Info here: http://openrov.com/profiles/blogs/openrov-cockpit

Github here: https://github.com/codewithpassion/openrov-software-ui


#4


Funny, I have been wondering about the same thing for a while. Studying the OpenROV Cockpit it looks like it is going to be really nice. I think the arrow keys are going to be a great first step, but I have to agree and think in the long-run a joystick is going to be so much nicer.

A while back I was starting to work on a pet-project called HID-Flinger, oddly enought for potential ROV control. The idea is to take common-off-the-shelf USB joysticks and "fling" the output to another protocol (could be anything, but my thought was that UDP over ethernet would be the most useful). The USB joysticks are very cheap (usually $10-$15) and plug into a USB-Host port. They typically have not only the normal x,y,z-axis control you would expect but also have a throttle, hat, fire-button and various other buttons. Typically 10-14 different analog and digital outputs. So the idea is "why not use this thing to control things and not just games". At this price you cannot beat getting a number of proportional analog inputs along with a slew of digital inputs all in one nice package.

Anyway, the trick is you need a USB Host Controller (on any laptop) but the usual intended use is to have applications running on the host to consume the joystick inputs. What we really want here is to be able to send (hence "fling") the output from the joystick to another client. The clinet would typically NOT be on the host machine, but rather some remote machine connected via ethernet or wifi (in other words a network connection).

USB Joysticks and other Human Interface Devices (HIDs) provide their information based on the USB HID class specification.What is we took the raw HID frames from the USB controller, packaged them up and sent them out using a UDP frame and recovering that frame at the client side and acting on the input? I believe a forum member Chris Konstad in his post already alluded to such an idea. His involved more than just relaying control data to the ROV, it also contained PID control of the ROV and feedback and other cool things.

In light of some conversations and drawing on some of the work posted by Bran Sorem in his post about streaming video it seems like this idea is an ideal fit within the direction OpenROV cockpit is taking. I would like to know more about the internal OpenROV architecture, but it seems from this video description it is a very nice, flexible solution. I feel it is also very easily adapted to the way sensor data from the ROV might be handled as well as input into the ROV might be handled.

Exploring this further, perhaps what is needed is a way to take USB-HID messages from the USB controller and package them into a network protocol frame using a Socket.io server on the host side. At the client side (the ROV) a Socket.io client would listen to these frames, parse them and use them as ROV control inputs. The ROV would then have x,y,z-axis among various other switches to control things like lights, robotic arms, whatever. I think this is what you are looking for? Me too!

I put together a diagram based on my very limited (and probably wrong) interpretation of what the existing architecture is like. This is just my way of thinking it through. The yellow boxes are code which would need to be written to make this happen. The overall purpose of the "HID-Flinger" application would be to take USB Joystick data and "fling" it to a client, the OpenROV.

What do you think?

There are probably other possibilities for implementing such a thing as well. I have read recently of Google's Chrome browser supporting the Gamepad API Specification.From what I have read this looks very promising, but might be very limited to only certain gamepads (i.e. Xbox controllers, some Logitech devices, etc.) but maybe that is good enough. I have tried some of the examples posted on http://www.gamepadjs.com/ but havnt had much luck. But this may be a more platform agnostic, web-browser centric way to get joystick data into the OpenROV. Worth some investigation anyway.

Well, that's my $.02 worth.


#5

This is great! I love the diagram. We have been planning to use GamepadAPI* for a while, but other things have taken priority.

As for sending/receiving commands, we'd like it to be as simple to extend as possible. One method is to send commands over Javascript and have them automatically pass to the Arduino. We have a protocol for sending Arduino commands:

name(param...);

Where 'name' is the name of the function (for instance, go(90,90,90); - would be go) and the parameters. It's essentially a limited remote procedure call. We also have a way of sending sensor data from the Arduino:

sensor:value;

This is still a work in progress, but that's the idea for now.

*If you happened to have been digging through the code you may have stumbled across src/lib/static/js/gamepad.js - this was the first test of using the GamepadAPI.


#6

I have aquaired an rov through a collage project, but the control system for it and software for it were not included with it. some of the circuit boards were in made , which leaves it to many unknowns, I am thinking of using the hard wire method and joy sticks with relays , The motors are 48vdc and the theater length is 550' , the rov is tested for little over 400' , by using joysticks and relays it seems the only way i think (my computer skills have alot to be questioned)

any ideas ???? Thank you


#7

Has anybody thought of VR headsets like the new Oculus Rift?


#8

I picked up a Logitech X3D gaming joystick the other day to try it out. Once I managed to link the profiler to a random game, I was able to map the buttons. So far it works great! the Joystick has rotational yaw that works well for the side to side buttons. It seems to be spot on in Chrome and Firefox. I even mapped a refresh browser so when the video hiccups I can restart easy. It has more than enough buttons for expansion so you probably an run everything from the controller. Once my rig is running Ill post pic or vid.

http://www.logitech.com/en-us/support/extreme-3d-pro


#9

Hi @John_309 , I’m a new guy from Italy, I’m new in this amazing OpenROV Community!!! It’s great!!! So, in a couple of week, will be arrive in my city in Italy, our first miniROV v. 2.8; I ask to you if the gamepad Logitech X3D ( http://gaming.logitech.com/en-us/product/extreme-3d-pro-joystick ) is set up with this miniROV v. 2.8 … many thanks for your support!!
Cheers,

Alessio Canalini


#10

The gamepad and joysticks all use the same browser APIs, so if the joystick works on this test page, it should work on the ROV. Please note that you may end up still wanting to remap buttons, which you can do directly in code at the moment, or eventually through a user interface.

http://html5gamepad.com