OpenROV software intern interested in ROV computer vision projects

software
#1

Hi!

My name is Gilbert Montague and I am an intern this summer with OpenROV. My undergraduate degree is in Physics but I am no stranger to robotics. I was an intern for eight months at the NASA Kennedy Space Center working on a swarming robotics project and then spent another eight months at the NASA Langley Autonomy Incubator developing software for Small Unmanned Aerial Systems.

I wanted to use this thread to introduce myself to the awesome OpenROV community, and also as a platform to document my progress. Feel free to pop in to contribute, give advice, or just say hello!

I will be mostly involved in the software side of things this summer. Specifically, I will be spending most of my time with firmware, computer vision, and coffee consumption.

Firmware

For the first week, or so, I will be contributing to the firmware of the robots with the primary focus being implementing a new way for the various software modules to share information with each other. The proposed architecture will use Google’s Protocol Buffers.

Protocol buffers are Google’s language-neutral, platform-neutral, extensible mechanism for serializing structured data – think XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages.

Computer Vision

Wouldn’t it be cool if the ROV could visually track an interesting object, such as a fish? Well, this is the first computer vision project I plan to tackle. If you have any other exciting computer vision projects you would like to see implemented, contact me or post in this thread. More on this later.

I am super excited to a part of the OpenROV team and community. Again, please feel free to stop by this thread to say hello and brainstorm.

#2

First off congratulations

It’s a community everyone will have different needs wants and desires so use your own judgment as to your priorities and focus. Don’t get swamped and don’t bite off more than you can chew just do what you think you can achieve

If I was looking a vision systems I can think of a few that could make very useful additions to increase OpenROV’s capabilities for preforming tasks

In no particular order

GoPro integration
The new Trident is going to have a Wi-Fi connection a significant number of people have a GoPro used for various reasons I use them for down facing vision to Collect data for Structure from Motion

As a starting point just a simple picture in picture display of the GoPro image (even a still every say second) would be handy

Ideally some way to collect the data and write to the image exif data the depth, pitch, roll and yawl of the ROV when the image was taken (and in the future location from some of @Jim_Trezzo (and/or others) ROV location work)

Have a look at some of the Commands that can be sent to a GoPro and you might get a few other thoughts as well

Vision Enhancement
Underwater environments can be difficult visually with turbidity and backscatter, loss of light and loss of colour @badevguru has identified an example for low light (and there are many others)

Have a look at
A Survey on Underwater Image Enhancement Techniques and A Review on Underwater image enhancement

Visual Odometry
Basically tracking the OpenROV when near the sea bottom

Simplistically even a Height above the bottom so rather than a “depth lock” (eg keep the unit at this depth) in the fly by wire a “bottom lock” (eg keep the unit this far off the bottom)

@Jim_N may also be able to give a few pointers

All the best with it

Scott

#3

Gil, Gil, Gil…

I’ve said it once, I’ve said it one-thousand times, I’d love to see AVEDac, MBARI, implemented on a ROV.

Here’s a pitch. How about we collaborate on it? Check it out, Old posts of mine on here detail it out. I can also repost the links. I’ve spoken with it’s creator a number of times with another student intern from our Fall semester last year. We had it running off-board, but not real time. We were comparing CAFE and the Neuromorphic library, implemented in AVEDac, from USC. Would be very very cool to modify it and have it running off-board, topside, tracking various marine animals.

I’d love to work with OpenROV on this :slight_smile:

Checkout https://bitbucket.org/mbari/avedac and http://www.mbari.org/wp-content/uploads/2015/10/Bwye.pdf

Let me know what you think. I can email Danille at MBARI do the intro thing, if you all think it’s worth the effort…I personally think it is.

Jim

#4

@Scott_W, you are a true wealth of knowledge sir! Also, we got somewhat delayed in regards to the SfM work, but picking up up again ina a week or so. We just got our new ROV kit up and running…more to come! thanks again for the links on datasets!

Jim

#5

Hi gilbert

I am facing a problem in OpenROV software and I wrote about it here :

I wonder if anyone can help !
thanks

Structure From Motion For Free
#6

@hannahsaud,
You might try to see if you can talk to the BBB using Wetty at http://192.168.254.1:3009 with your Chrome browser.

openrov login: rov, Password: OpenROV

If you can, then maybe one of the OROV software developers can help you recover you system.

Regards,
TCIII AVD