Controlling OpenROV with a 3D mouse


#1

Hi,

I have a 3D mouse, a SpaceNavigator by 3DConnexion, and like any 3D mouse it allows for six Degrees of Freedom (6DoF) of movement. I believe this type of control is perfect for underwater exploration where you are natively working in a three dimensional environment.

When I get my ROV I want to the 3D mouse to control it so I coded up a working prototype that shows that a 3D mouse can be used to control the ROV through web sockets. I was looking through the existing codebase and it appears something similar has already been done with a gamepad, https://github.com/OpenROV/openrov-software/blob/development/src/static/index.html

Here's how my system works using a 3D mouse:

1. A Windows application is started and is acts as a “3D mouse movement server”. It is responsible for implementing a HTML 5 compliant web socket server and capturing events from the 3D mouse. The application relays the 3D mouse events to listening web socket clients, such as the ROV control web page.

2. The ROV web page, acting as a client, on load creates a web socket connection to the 3D mouse movement server and waits for mouse movement events to arrive.

3. When the page receives a mouse movement event it translates the received data into an appropriate ROV directional control command

4. The web page sends the directional control command to the ROV on a separate web socket connection

Here is a video showing the mouse be moved around, the server capturing the movement data and then sending that data to the web page. The web page then plots the values as a historical timeline graph using the HTML 5 canvas object.

http://www.youtube.com/watch?v=O4Sb6XNgqSk

Implementation details:

I used the 3DConnexion sample application as the starting point because it was already capturing all the 3D mouse events. I then embedded in that sample application an open source web socket server, Alchemy. Then in the existing mouse event handler I simply took the values it was capturing and sent them off to any client connections that are connected.

One big takeaway from this exercise was learning to use web sockets to allow external data to be injected into the ROV control system. This concept is where I think the real power lies in being able to control the ROV intelligently. The web browser is as a thin client and can connect to heavy server processes through web sockets. It’s those external heavy processes that excite me because that really is the hook in point, or the plugin point. I really like the fact that multiple inputs can be listened to by the browser in order to derive a final output command to the ROV.

This 3D Connexion sample app is available for free from their FTP site:

http://www.3dconnexion.com/forum/viewtopic.php?f=19&t=1802&sid=3d0f68bf810d98905caa3bf4f8ac66c6

ftp://ftp-us.3dconnexion.com
user: examples
password: examples
subdir: C#
filename: 3DMouse_WPF.zip

Alchemy web socket server, https://github.com/Olivine-Labs/Alchemy-Websockets

Wikipedia describes 6DoF as:

http://en.wikipedia.org/wiki/Six_degrees_of_freedom

Specifically, the body is free to move forward/backward, up/down, left/right (translation in three perpendicular axes) combined with rotationabout three perpendicular axes, often termed pitch, yaw, and roll.

Not all aspects of 6DoF can be implemented by the OpenROV. The list below describes the movement that are controllable:

  • Moving up and down aka “heaving” (vertical thruster)
  • Moving forward and backwards aka “surging” (horizontal thrusters)
  • Turning left and right, “yaw” (horizontal thrusters)

Degrees of Freedom that are not implemented by OpenROV:

  • “Swaying” - Moving left and right because it requires additional side thruster(s).
  • “Roll” - Tilting side to side because it’s very difficult to accomplish and no real practical reason at this point exists for it
  • “Pitch” - Tilting forward and backward. Not really required because the camera servo gimbal handles this movement.

#2

Hello Marcus, I have a question concerning your interface implementation in C# to communicate with the ROV.

Did you change any source code or settings on the ROV? Or how did you archive that the ROV is connection as a client to your C# socket.io Alchemy server?

Thanks Sören


#3

Marcus,

Congrats on a great software hack! We have a similar device at HQ we use for cadd’ing that I might be able to ‘borrow’ to recreate your work. This also provides an exciting path for other similar cool peripherals that are not natively browser aware to be used. :rocket: