How to use stored telemetry data?


#1

I’m curious about how to use or view telemetry data after the dive. When recording a dive, info like depth and direction are displayed, and I assume this is stored. What’s the format, and are there any tools that parse/show this data?


#2

(I’m referring to Trident, and the private beta OpenROV Cockpit on the Google Play store.)

I’ve been able to see the data by turning on Developer mode and then swiping in from the right hand side of the Pilot view. I haven’t been able to track down where the data is being stored, or how to get it off the Android controller.

I’d love to get to this data myself. If it’s timestamped, we could build overlays and plots after the dive.


#3

Looks like it’s in Couchbase.

I’ve started a document at https://github.com/halmueller/trident-hacks to collect useful, but undocumented, tidbits. Contributions and pull requests encouraged!


#4

FWIW, the .h264 file is the raw video stream. It can be converted (without transcoding of the video, simply remuxing) to mp4 with ffmpeg, using a command like this:

ffmpeg -r 30 -i guid.h264 -c copy -r 30 guid.mp4

The file db.forest.1 seems to contain quite a bit of useful info. For example, it contains info on the following parameters, seemingly sampled once per second, in something that looks a little like json structure. Unfortunately these json structures seem to be fairly regularly interrupted with some binary data.

attitude.calc.yaw
attitude.calc.roll
attitude.orientation.w
attitude.orientation.z
attitude.orientation.y
attitude.orientation.x
attitude.calc.pitch
fuelgauge.status.battery_state.percentage_
depth

I have no knowledge of this couchbase file format (if that is what it is), but so far it’s been easier to look at the file contents in a hex editor rather than getting couchbase working. Are you familiar with couchbase from before? Couchbase seems like a mess, they have a perfectly decent looking (but sometimes very slow) website, the installation instructions require manual downloads and installation of packages and refer to the fairly ancient Ubuntu 14.04, and searching online has led me to a bunch of outdated repos in their github pages.

It seems that there’s a github repo that features a cblite tool to query a cblite file, available on https://github.com/couchbase/couchbase-lite-core/. Unfortunately I’m on ubuntu 16.04 and that appears to require an initial building of libc++, perhaps as described here: https://stackoverflow.com/questions/39332406/install-libc-on-ubuntu.

I’m not going to start that right now, but if there are no other options I guess something like this is necessary to access the data.


#5

On macOS, the bundled QuickTime Player can’t read or play the H264 files in …/data. However, I was able to convert one (I only tried one) with Handbrake, which is multiplatform. https://handbrake.fr/downloads.php

I pulled down the Cblite tool last night but haven’t tried to build it yet. I haven’t used Couchbase but it has a decent mindshare in the iOS developer world.


#6

Using CBLitePro from the Mac App Store, I was able to browse the sessions.cblite2 and .cblite2 files. sessions.cblite2 lists the .cblite2 files, along with a timestamp for each one. .cblite2 is indeed the per-dive telemetry file.

Screenshots are at the updated https://github.com/halmueller/trident-hacks. Star or follow that repository to see updates. I’m going to try out the command line tool mentioned above, and then perhaps build a simple Mac tool to extract and visualize the data, but I won’t have time for that until next week. Source for that effort will land in the same Github repo.


#7

Thank you halmueller, your tinkering is very much appreciated.

Looking under the bonnet of Trident is on my list of things to do so I am now even more curious to find out more.


#8

Interesting that you got info out of the cblite DBs. I’ll see about compiling it on Ubuntu when I get the chance, or if there are other tools available.

AFAIK, handbrake simply uses ffmpeg/x264 to transcode/remux the files, I prefer sticking with ffmpeg since I’m fairly familiar with it. I don’t know if it’s the way to go, but ffmpeg can also write text and overlay images on a video, that could be one way to display telemetry data together with the video.


#9

And thank you ketil1, I appreciate your tinkering and commentary on the various files I am likely to encounter when I look under the bonnet.


#10

I don’t know how it works on the Trident, but I’m using the telemetry data from my OpenROV V2.8. On the V2.8, it’s pretty simple: you press the ‘r’ button to start recording the telemetry data. When you’re done, click on the “download” button on the cockpit, and the browser will automatically download a .json file with the gathered data. To visualize it better, you can get the file to a “pretty json” website. The sampling frequency is approximately 1hz. Actually, if someone has any idea as to how to increase the sampling frequency, I’d be very grateful!

I don’t know if that answers the original question, but maybe it can be of some help to someone!


#11

On my Trident,/ Android, at this time, I am using a quick and dirty work around by using the app DU Recorder which is recording the screen on my tablet as I fly it around.
I would love to see a long/latitude display as well.
Hope this helps


#12

That’s an interesting workaround, though I’m really hoping to get at the data itself.

Can anyone from OpenROV chime in? Why go for couchbase with a forestdb backend? Is it an old version of couchbase that defaulted to forestdb, or have you implemented a custom storage backend using forestdb for some reason? Can you hack together a quick tool (Java, c, anything) that could export data from couchbase?

Based on my experience so far asking for help on the couchbase forum, I’m really hoping you would switch to vanilla sqlite in a future version. The data would be so much more accessible. sqlite has a proven record of wide system support and backwards compatibility - very much unlike couchbase, as it turns out.


#13

Ketil got a potentially useful nugget from Jens, one of the CouchDB engineers, over on their forum:

Part of the confusion here, I think, is that you saw the filename suffix .cblite2 and assumed this was created by Couchbase Lite 2. It wasn’t. The 2 in the filename just means it’s the second major version of the file format; it’s used by all shipping versions of Couchbase Lite.

You should be able to open and read the database by using Couchbase Lite 1.4 and enabling the ForestDB storage engine.

I had come to the same (erroneous) conclusion and was puzzled that the cblite2 command line tool wouldn’t read the files. I have a little time now to tinker again and I’m going to try Jens’s suggestion.


#14

Instead of converting the video files with ffmpeg or other tools, they can be viewed directly with VLC using a command like:

vlc -f --demux h264 --h264-fps 30 GUID.h264

The -f simply means to launch in full screen mode, drop it if you like.

This was tested on Ubuntu 16.04 with stock VLC, but I expect it to work fine on other platforms and versions.


#15

I’ve had some luck automating the extraction of telemetry data from the cbase2/db.forest.X files. Feel free to have a look at the code at https://github.com/kefir-/openrov-utils.git. The tool trident-metadata.py takes a -a option, and then it outputs everything from the database to a flat file that looks something like the snippet included below. The export depends on the tool forestdb_dump, and I’m afraid it’s terribly slow.

The example data below is a bit odd since the depth is negative. Perhaps it’s from a short recording where the ROV was never submerged. The format of the data can easily be changed, but I figured something like this should be easy enough to manipulate further for plotting, video overlays, etc.

1530567419.736000000 pressure.fluid_pressure_: 1018.4000244140625
1530567419.737000000 pressure.header_.stamp_.nanosec_: 0
1530567419.737000000 pressure.variance_: 0.0
1530567419.738000000 temp.water.id: ms5837
1530567419.738000000 temp.water.temperature.header_.stamp_.nanosec_: 0
1530567419.738000000 temp.water.temperature.temperature_: 20.84000015258789
1530567419.755000000 imuCallibration.gyro: 3
1530567419.755000000 imuCallibration.mag: 0
1530567420.211000000 attitude.calc.roll: -178.768950154289
1530567420.211000000 attitude.calc.pitch: -0.5944740459352972
1530567420.211000000 attitude.orientation.y: 0.99993896484375
1530567420.688000000 fuelgauge.status.battery_state.capacity_: NaN
1530567420.689000000 fuelgauge.status.battery_state.charge_: 6.894000053405762
1530567420.689000000 fuelgauge.status.battery_state.design_capacity_: NaN
1530567420.689000000 fuelgauge.status.battery_state.header_.stamp_.nanosec_: 0
1530567420.689000000 fuelgauge.status.battery_state.power_supply_health_: 0
1530567420.690000000 fuelgauge.status.battery_state.power_supply_technology_: 0
1530567420.690000000 fuelgauge.status.battery_temperature: 29.94685
1530567420.752000000 imuCallibration.accel: 0
1530567420.752000000 imuCallibration.id: tail
1530567420.752000000 imuCallibration.system: 0
1530567420.755000000 pressure.header_.stamp_.sec_: 0
1530567420.755000000 depth: -0.14530964
1530567420.757000000 temp.water.temperature.header_.stamp_.sec_: 0
1530567420.757000000 temp.water.temperature.variance_: 0.0
1530567421.24000000 internal_pressure.header_.stamp_.sec_: 0
1530567421.27000000 temp.internal.id: mpl
1530567421.27000000 temp.internal.temperature.header_.stamp_.nanosec_: 0
1530567421.27000000 temp.internal.temperature.temperature_: 31.375
1530567421.123000000 i2cstats.hard_reset_count: 1
1530567421.124000000 i2cstats.latch_failure_count: 0
1530567421.124000000 i2cstats.recovery_count: 0
1530567421.223000000 attitude.calc.roll: -178.768950154289
1530567421.223000000 attitude.calc.pitch: -0.5944740459352972
1530567421.223000000 attitude.orientation.y: 0.99993896484375
1530567421.690000000 fuelgauge.status.average_current: -0.881
1530567421.690000000 fuelgauge.status.battery_state.cell_voltage_: []
1530567421.690000000 fuelgauge.status.battery_state.current_: -0.8690000176429749
1530567421.691000000 fuelgauge.status.battery_state.header_.stamp_.sec_: 0
1530567421.691000000 fuelgauge.status.battery_state.percentage_: 0.8299999833106995
1530567421.691000000 fuelgauge.status.battery_state.power_supply_status_: 0
1530567421.691000000 fuelgauge.status.battery_state.present_: false
1530567421.691000000 fuelgauge.status.battery_state.voltage_: 11.57699966430664
1530567421.691000000 fuelgauge.status.average_power: 6.469
1530567421.748000000 imuCallibration.gyro: 3
1530567421.748000000 imuCallibration.mag: 0
1530567421.769000000 pressure.fluid_pressure_: 1022.4000244140625
1530567421.770000000 pressure.header_.stamp_.sec_: 0
1530567421.770000000 depth: -0.08310659
1530567421.777000000 temp.water.temperature.header_.stamp_.sec_: 0
1530567421.777000000 temp.water.temperature.variance_: 0.0
1530567422.37000000 internal_pressure.fluid_pressure_: 245.37750244140625
1530567422.37000000 internal_pressure.header_.stamp_.nanosec_: 0
1530567422.37000000 internal_pressure.variance_: 0.0
1530567422.38000000 temp.internal.temperature.header_.stamp_.sec_: 0
1530567422.38000000 temp.internal.temperature.temperature_: 31.375
1530567422.234000000 attitude.calc.roll: -178.768950154289
1530567422.234000000 attitude.calc.pitch: -0.5944740459352972
1530567422.235000000 attitude.orientation.y: 0.99993896484375

#16

Here are the first few graphs I’ve plotted from the telemetry data, based on three dives I did today. All three show water temperature and depth together, and the output seems to make sense, even. :slight_smile:


#17

If you didn’t see my note in the other thread: used your scripts on a Mac and everything worked perfectly. I also was able to use your work to create an Xcode project to build a basic forestdb application, precursor to some nicer Mac viz tools. Thanks!


#18

Nice! Glad it can be of use, it’d be a shame to have all this data and not use it for anything. :slightly_smiling_face: I managed to dramatically speed up the processing as well, so while it’s not fast by any means, it is not as horribly slow as it was. It’s only necessary to run this once per dive, so I don’t see a great need for further optimization.


#19

I’ve figured out a workable method to make video overlay with telemetry data, and I’ve made a bare-bones proof-of-concept. The values seem to make sense, though the temperature seems to lag a bit. Here’s a sample from a recent dive:

I’m hoping to replace pitch, roll and yaw with small images that can be rotated according to the values, I think that’ll be easier to interpret than showing raw numbers.

For Windows users, an app like DashWare may be a good option to create an overlay like this once the data is available. DashWare doesn’t work for me on Ubuntu Linux, and it presumably won’t work for Mac users either.


#20

While not nearly as polished as Ketil’s work, I’d like to share a work-in-progress for Mac-based animation of telemetry. This work uses Ketil’s data extraction scripts, plus a Mac project of mine. The pitch/roll/yaw axes on this sample are wrong, and I’ll sort that out in a day or three.

The Mac project is available on Github at https://github.com/halmueller/trident-can-opener. Build it yourself. If you’re a Mac user and would like to be able to download a finished executable, say something in the comments.