I've been following the buildout on your blog and I love the videos! As far as the reverse goes, there are two possibilities.
1) The most likely possibility is that the software has a bad configuration. It is highly dependent on the version of the software that is being used. Work arounds include going to the diagnostics screen and then closing it (using the close button), which in some versions re-sends the config values to the Arduino fixing the issue. Power on/off the ROV might also clear the issue, but if the issue is tied to timing of commands around when the ROV starts up, it may not consistently clear the issue). Ultimately, be sure your running the latest version of the software, as there are several fixes for this exact problem.
2) The ESC has lost programing. The ESCs can loose programing is rapidly power-cycled or if the programming buttons are held down.
If issue #2 keeps haunting you, the latest software release also contains code that can automatically program and calibrate Afro ESCs, and such problems can be fixed without having to open up the e-chassis. (Note: It is rare that I have seen the ESCs actually loose programming, so figuring out if there is a wiring issue or something that is rapidly cycling power might also be a good solution).