Control Latency Issues


#1

Hi there. I am hoping that the community here can provide me with some pointers:

I have constructed my OpenROV 2.6, and all appears to be working well. However i am currently getting significant control latency and regular 'freeze-ups' that make the device impossible to pilot. This includes the video stopping, with the only way to get it to restart is to either reboot the BB, or restart the cockpit service in the BB.

Basically what i am seeing is that i can operate the motors for only short periods (20 secs) before they simply stop - none of the controls seem to work and the video stops. About 3-5 or more seconds later the motors start responding again (although the video will not start up again). At the point things stop responding, interestingly, if i have the lights on, they will shut off, and i have to cycle them twice to turn them on again - they also appear not to be subject to the same delay as the motors. In addition, there is also typically a 0.5-1 second delay between a button press and the reaction at the ROV (as well as around the same latency on the video). Its a little strange and the best way i can describe it is that its like the BB is just really struggling to keep up.

The weirdest thing is that this effect is exacerbated when the device is in the water (My first thought was this is obviously something to do with water intrusion, but after more investigation, i dont think so). I place the device in the water, drive it for a few seconds and struggle with the control issues i describe, get frustrated, then pull it out, put it on dry land and the problems seem to lessen (although not completely go away) - so i put it back in the water and immediately i have the same control latency issues. Repeat ad infinitum... I have checked and my sealing is good with no water getting in.. Perhaps it could be something at the motor end??

Another thought is to create another BB image and re-flash the arduino.. but i am not sure. I would really appreciate any suggestions you might have to get to the bottom of this.

Thanks in advance

Tim


#2

Hi Tim:

This is an issue that is seen to varying degrees with all the ROVs that we have. It seems to vary with the topside computer being used, and the browser being used, so you might want to switch things up and see if you get different results. For most people Chrome works the best for a browser, but other people have luck with FireFox.

When the video freezes, try refreshing the browser (or is this what you meant by "restart the cockpit service"?). This will normally get things going again.

I know Brian Adams is working on this issue, and has over time made some changes in the way that Cockpit is rendered on the browser with the aim of reducing the latency. In the past, people have been able to mitigate things by playing with the resolution and frame rate of the camera. But I think there is still some underlying mechanism we don't understand, probably involving the BeagleBone Black.

If there are any Linux experts out there who would like to help out with OpenROV, this is probably the most irritating bug in the software right now, and any help that can be provided would be welcome.

-W


#3

Walt,

Thanks for the response - Yes, Hopefully someone (with more expertise & time than me) will be able to shed some light on this. Hopefully soon too!

In response to your question on the video freezing. Unfortunately, refreshing the browser doesn't work (interestingly - neither does restarting the ROV by removing/reapplying USB power), The only way to restore video is to log into the BB and then stop and start the OpenROV daemon manually.

Thanks, Tim


#4

Tim:

See this blog post from today. This might shed some light as to why things are different for you in the water.

-Walt


#5

When running in Air, if you are seeing latency issues on the control there are two things to check:

1) ping tests to the ROV. Make sure your not seeing high latency in the ping times. Worst case should be <45ms.

2) check your browser process. The version of the software you are running will only make use of a single core and if that maxes out... well all bets are off.

-Brian