[Solved] Issues whist trying up update Cockpit on a New 2.6 unit

solved
electronics
software
sos
#1

Hi

I'm Scott from Sydney Australia and this is my first post I am looking of a bit on a hand with updating the software on the unit

I am just finishing off building a new 2.6 unit and I have basically substantially finished its construction (just a few hydrostatic tests prior to its first use). I have had the ROV basically operational on the table top (lights / motors / camera / tilt etc) but have run into an issue when trying to update the system to the latest software version.

From within the Dashboard it appears I have openrov-cockpit ‐ 2.5.0-rc1-18 and am trying to upgrade to V2.5 Release Candidate 3 (plus any other software upgrades)

e44f0daaa5ba6576.JPG987×615

I have followed steps 1 thought to 13 without any difficulties but then during step 14 the Building Firmware Fails and the operation is aborted (see full dump bellow)

I have also tried download the image for flashing to eMMC and booting the BBB from the SD card and it too halted at the same step

Any assistance would be appreciated for how to go forward

Thanks in advance

Scott

Error log

undefinedstaging: build dir is /tmp/tmp.kZcUbVgYN1

staged src in to build folder

src/OpenROV.ino

Searching for Arduino lib version file (version.txt) ... /usr/share/arduino/lib/version.txt

Detecting Arduino software version ... 1.0.5 (1.0.5)

Scanning dependencies of src

Scanning dependencies of arduino

Scanning dependencies of EEPROM

Scanning dependencies of SPI

Scanning dependencies of Wire

src/MPUQuaternion.cpp

src/MinIMU_I2C.cpp

src/MinIMU_Vector.cpp

src/MinIMU9.cpp

src/MinIMU_L3G.cpp

src/CalibrationLaser.cpp

src/Cape.cpp

src/MS5803_14BA.cpp

src/MinIMU_Compass.cpp

src/Thrusters2X1.cpp

src/Motors.cpp

src/Timer.cpp

src/I2Cdev.cpp

src/CalLib.cpp

src/Settings.cpp

src/MPUVector3.cpp

src/Device.cpp

src/Lights.cpp

src/MinIMU9AHRS.cpp

src/inv_mpu.cpp

src/MinIMU_LSM303.cpp

src/FreeMem.cpp

src/CameraMount.cpp

src/MinIMU_Output.cpp

src/Command.cpp

src/Pin.cpp

src/Pilot.cpp

src/MinIMU_matrix.cpp

src/MPU9150Lib.cpp

src/MPU9150.cpp

src/openrov_servo.cpp

src/MinIMU_DCM.cpp

src/inv_mpu_dmp_motion_driver.cpp

src/controllerboard25.cpp

src/controllerboard25.cpp:20:15: error: 'A8' was not declared in this scope

make: *** [.build/uno/src/controllerboard25.o] Error 1

Searching for Board description file (boards.txt) ... /usr/share/arduino/hardware/arduino/boards.txt

Searching for Arduino lib version file (version.txt) ... /usr/share/arduino/lib/version.txt

Detecting Arduino software version ... 1.0.5 (1.0.5)

Searching for Arduino core library ... /usr/share/arduino/hardware/arduino/cores/arduino

Searching for Arduino standard libraries ... /usr/share/arduino/libraries

Searching for Arduino variants directory ... /usr/share/arduino/hardware/arduino/variants

Searching for make ... /usr/share/arduino/hardware/tools/avr/bin/make

Searching for avr-gcc ... /usr/share/arduino/hardware/tools/avr/bin/avr-gcc

Searching for avr-g++ ... /usr/share/arduino/hardware/tools/avr/bin/avr-g++

Searching for avr-ar ... /usr/share/arduino/hardware/tools/avr/bin/avr-ar

Searching for avr-objcopy ... /usr/share/arduino/hardware/tools/avr/bin/avr-objcopy

Make failed with code 2

firmware-build.sh: : Compile of the Arduino image failed.

firmware-installfromsource.sh: Building Firmware Failed! Aborting

#2

The update is failing because something got really confused. This is the same error we see then the code that tries to figure out if you are using a controllerboard or a cape has failed and defaulted to cape (the arduino 328p in the cape does not have an A8 pin).

Have you tried power-cycling everything and trying just the firmware upload again?

#3

Thanks for the reply Brian

I have tried again to upgrade to V2.5 Release Candidate 3 (again via steps 1 through 13) via a SD card

On the first re- attempt after your post it cycled part way through the procedure and then it produced the identical error log as above

After powering down again on the second attempt if 'froze' at the last step 'Searching for avr-objcopy ... /usr/share/arduino/hardware/tools/avr/bin/avr-objcopy' I left the machine to its own devices for over an hour and it had not moved on from here

I again powered down and attempted to update from the SD card this time it said it successfully updated avr-objcopy but then appeared to have finished

I have again powered down removed the SD card and then powered it up and have now not been able to connect though Dashboard http://192.168.254.1/# (I waited again for over 1 hour in case it was updating anything)

Powered up again and this time it connected to Dashboard and Cockpit although the software version is still openrov-cockpit ‐ 2.5.0-rc1-18 and when it is now connected Cockpit see's the camera and is able to power up the LED's but it does not have control over the motors or camera servo anymore

More than a little confused

Any thoughts on a way forward would be appreciated

Thanks in advance

Scott W

#4

Hey Scott,

We have to copies of the SD image in the release. On is meant to run with the SD card installed. The other (with the name flash in it) is intended to upgrade the on board beagle bone flash image and then be removed. Can you clarify which you are using. If you are using the non-flash SD card image, you have to leave the SD card in. Removing it would revert to using the beagle bone flash image which would explain the behavior you mentioned above.

Sorry it has been so confusing. There is one known race condition that cause the update process to seem to crash. I had thought that was rare, but there have been more than several reports, so I will tackle that this week.

-Brian

#5

Hi Brian

Thanks again for the reply

I have tried both methods multiple times (the SD image release and also Booting the BBB with the flash image version) both seem to be falling over at the same step. But yes I have also at one attempt removed the SD card after the process.

I have since gone back and tried Controller board 2.5 Compatible Cockpit Beta 2 which appeared to load with no issues

It appears to be more "stable" and less "erratic" currently (previously differing things would or would not work each time the unit was started up (eg lights or lasers may or may not turn on/off similarly with the motors and actuator). Currently all seems to be somewhat working (everything turns on and off although the motors only run in one direction even after calibration)

I had thought that was rare, but there have been more than several reports, so I will tackle that this week

I am more than happy to wait sitting on the Controller board 2.5 Compatible Cockpit Beta 2 release till this gets ironed out

Scott

#6

Hi

After waiting hoping for a few issues to be ironed out, I have just tried updating a new 2.6 ROV again using the freshly released V2.5 software and my system still will not update.

I have attempted both techniques running OpenROV-2.5-26.img from the SD card as well as flashing OpenROV-flash-2.5-26.img to the eMMC (appeared to have flashed successfully as all 4 BBB LED's stayed solid lit) in both cases after considerable time (say greater than 40 minutes) and numerous attempts I still cannot log onto either Cockpit 192.168.254.1:8080 nor Dashboard 192.168.254.1/# to load

The latest version of the software that I can upload to the unit that then allows an "Upload firmware from SD card to Arduino" to be successfully is OpenROV-2.5-05.img. although this version does not appear to have the Dashboard

I can upload and run from the SD card OpenROV-2.5-25.img but it will not successfully "Upload firmware from SD card to Arduino" as it fails at the previously stated point of the upload, although this version of the software will allow access to the Dashboard

Searching for avr-objcopy ... /usr/share/arduino/hardware/tools/avr/bin/avr-objcopy

Make failed with code 2

firmware-build.sh: : Compile of the Arduino image failed.

firmware-installfromsource.sh: Building Firmware Failed! Aborting

Any help on how to proceed would be appreciated

#7

Hi Scott

I don’t know if this is of any significance but I was having trouble communicating or connecting to the ROV. I found that by swapping the USB ports that I was using on my computer, it suddenly started to work. I have no explanation for this as I thought all USB’s were created equal but it’s worth a try. I have also found that sometimes restarting the computer with everything connected helps. My problems I think are more related to the computer I’m using then the ROV

Dave

#8

Hi Dave

Thanks for the reply, but unfortunately after 3 different USB ports and 3 different restarts of the laptop none of them allowed connection to the ROV using the V2.5 Release OpenROV-2.5-26.img from the SD card.

Therefore, I have gone back to OpenROV-2.5-25.img on the SD card which for my machine cannot successfully "Upload firmware from SD card to Arduino" and seems a bit "patchy" (this reboot isn't running the lights but runs the motors/servo and laser)

Any thoughts or suggestion happily accepted

Scott

#9

Hi again

I had a faulty board which OpenROV replaced. I was also having problems with the download so that is also a possibility for you.

Dave

#10

Thanks for the report Scott. I'll double check the image and get back to you.

-Brian

#11

There is a work around for uploading the firmware if it keeps failing like that:

  1. ssh on to the rov
  2. "sudo /opt/openrov/linux/arduino/firmware-installfromsource.sh"

That will run the process straight from linux, which bypasses a known instability that is fixed in the dev tree, but not in the latest 2.5 release.

-Brian

#12

Hi Brian

Thanks again for the help

The OpenROV-2.5-25.img mounted on the SD card updated successfully using your work around and I could log onto both Cockpit and Dashboard and all seemed good

I then attempted doing similar with OpenROV-2.5-26.img (I also tried the OpenROV-flash-2.5-26.img after it had finished flashing to the eMMC with the same results) and this then failed with the following extracted from Putty

rov@OpenROV:~$ sudo /opt/openrov/linux/arduino/firmware-installfromsource.sh

[sudo] password for rov:

unpacking

staging: build dir is /tmp/tmp.RepSbDnnuV

Traceback (most recent call last):

File "/usr/local/bin/ino", line 3, in <module>

from ino.runner import main

File "/usr/local/lib/python2.7/dist-packages/ino/runner.py", line 23, in <module>

import ino.commands

File "/usr/local/lib/python2.7/dist-packages/ino/commands/__init__.py", line 5, in <module>

from ino.commands.build import Build

File "/usr/local/lib/python2.7/dist-packages/ino/commands/build.py", line 16, in <module>

from ino.environment import Version

File "/usr/local/lib/python2.7/dist-packages/ino/environment.py", line 19, in <module>

from glob2 import glob

ImportError: No module named glob2

firmware-stage.sh: Failed to initialize the Arduino project directory. Aborting

firmware-installfromsource.sh: Staging Firmware Failed! Aborting

rov@OpenROV:~$

I have since gone back to the OpenROV-2.5-25.img mounted on the SD card, which successfully updated using your work around

Hope the Putty dump helps work through any issues

Thanks Again

Scott

#13

As I indicated I have since gone back to the OpenROV-2.5-25.img mounted on the SD card, which successfully updated using your work around

However, when trying to calibrate the motors (Step 85 in the Guide) (after rechecking the programming of the ESC's Step 84). I notice I am still getting strange and inconsistent performance. Basically the motors are just randomly turning on and off by themselves and running in different random directions regardless of the ESC programming of the neutral position and failing to stop on a space bar depression (I can understand strange behaviour prior to programming the ESC's)

Additionally the IMU readings on the screen deviate and freeze (whist the IMU is in a fixed location & approx 400mm from the closest motor) and then I have also noticed that the runtime freezes and then resets see image below (this was actually after about an hour of uptime and I watched it reset a few times at around 40 seconds several times prior to the screen grab).


Any assistance is greatly appreciated

#14

Well good news. I found the issue that essentially killed the latest image right out of the door :-(. One of the libraries we used moved versions which broke the image. We had caught and fixed this in our development branch but missed updating the release candidate. The new image is building right this second and will be posted by the end of the day.

Thanks for everyone that tried getting it up and running. Super sorry for the false start. I'll post back here when the new image is ready for download.

-Brian

#15

Thanks for the report. I'll double check this workflow on the new image once it finishes building.

Tracking here: https://github.com/OpenROV/openrov-software/issues/245

-Brian

#16

I've opened a ticket for the IMU but I don't have any initial opinion right away (https://github.com/OpenROV/openrov-software/issues/246).

As far as the ESCs go. There is a bug in the older version you are running where we are sending too big a signal in the reverse direction and that could be causing the issue you are seeing. If you don't want to wait for the new image, you can manually apply this change to your code on your image: https://github.com/OpenROV/openrov-software/commit/9b78e8452800136305d5c41fd17c41c7bb767d96

#17

Thanks Brian

From my end, it great to know I'm not crazy and doing something stupid, and I do appreciate how much work you guys are doing in updating software and at the end of the day this sort of stuff just happens

#18

Brian

After posting that image above I have also noticed at the very bottom of this right hand column that a couple of different errors come up there error codes were

These sort of changed after each "reboot"/"Run time reset" and were

Firmware write failed -4

Failed to load dmp firmware -3

mpu_init failed with code -1

additionally

Firmware write failed -5

Failed to load dmp firmware -5

mpu_init failed with code -1

additionally

Firmware write failed -5

Failed to load dmp firmware -3

mpu_init failed with code -1

Failed to read compass -6

#19

Hey Scott,

Do you have in IMU installed on our ROV? If not, you can disable the depth sensor and IMU in the AConfig.h file in the arduino code to avoid seeing those messages. If however, you do have an IMU, it would infer total failure to connect in which case I would recommend double checking the I2c wiring.

-Brian

#20

Okay. The firmware upload issue was fixed. Thanks! https://github.com/OpenROV/openrov-software/releases/tag/v2.5.0