Investigating WebRTC MCU solution for tele-robotics applications


#1

@Henry_Stewart and I have been working on standing up the backend for a webrtc based video conferencing solution. Our goal is to tailor it for robot applications over time.

WebRTC is a painfully immature technology, but it is moving fast and it is the only game in town with native built in browser support which is key for our OpenROV software stack.

Most of the examples you see for webRTC based examples are setting up peer-2-peer connections which as super fast, but also require a direct connection between each participant in the conference. If you have 5 people on, that is 5X the bandwidth needed at each end point.

We want to leverage this technology in the field where bandwidth is still likely to be a premium. So we need a solution that only requires a bi-directional connection from the ROV’s laptop in to the cloud. The server that sits in the cloud is called a MCU (mulitipoint control unit).

Henry is currently working through trying to install a working version of on of the MCU solutions on the market. I’m sharing in hopes that you will both find the investigation interesting, and if you have any familiarity with the products, might lend a hand.

The products we are looking at so far:


#2

Reviewing Jitsi:

Given that we are optimizing for one outbound feed begin repeated this could work. It is a shame it does not support building a composite stream that would be needed for efficient bi-directional video.

Unlike expensive dedicated hardware videobridges, Jitsi Videobridge does not mix the video channels into a composite video stream. It only relays the received video flows to all call participants.


#3

@Henry_Stewart, as far as “Do we need transcoding”. I don’t think that is super important for our use case. Due to our use of WebRTC all of the video is either h.264 or VP8. Both with will be supported natively in both mobile and desktop browsers.

As far as I can tell, the difference between the MCU and SFU (selective forwarding unit) is that the SFU is simply routing the video by cloning the feed and sending it to many. This is the minimum capability we need, thus my prior comment on Jitsi.


#4

“There are many ways to skin a cat” as “they” say. Our goal is to broadcast a WebRTC screen sharing stream to as many people as possible with as low latency as possible. Implementing this seems straight forward:

  • initialize a WebRTC video/audio/screen/tab (media) stream from the pilot’s ROV
  • send pilot’s ROV media stream to a central server that can route media to many (hundreds)
  • other clients connect to routed central server to receive the pilot’s media stream

Here’s an update of where things are at


#5

Kurento:

Kurento is an open source WebRTC media server that supports broadcasting, transcoding, recording, & custom plugin features. Kurento Media Server (KMS) acts as a one to many “dispatcher” to amplify a WebRTC stream to many clients.

Currently we have a KMS running on an AWS EC2 m3.medium instance. The current blocker is installing “InstallCert.java” under Java 7 for the KMS server as it does not run SSL out of the box – the only apparent way to add SSL based on their documentation is with “InstallCert.java” file:

http://builds.kurento.org/dev/latest/docs/faq.html

Compiling with

javac InstallCert.java

& and running it with the options “localhost:8080” or “localhost:8433” like

java InstallCert localhost:8080

throws a javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?...Could not obtain server certificate chain exception.

As of writing this post, Kurento’s latest code pushes have broken their tutorials, so it’s been a challenge to integrate this with our tele-robotic video feed. I will be in contact with their support staff to see if we should continue with Kurento.


#6

OPENTOK

OpenTok is a semi-open source WebRTC platform. OpenTok is backed by multi-national Telefonica. OpenTok’s backend supports “relayed” & “routed” WebRTC traffic. Routed connections are what we are after b/c it supports WebRTC broadcasting.

I used OpenTok’s javascript client library “OpenTok.js” to support screen sharing. Following the directions here and
here I created a Chrome extension, and loaded the packed extension onto the Chrome developer webstore here. After loading the extension onto my browser, I registered the extension with the command

OT.registerScreenSharingExtension('chrome', 'alocckonefdmbfllfgeonlemhkgkmbji')

When I checked the OT.checkScreenSharingCapability(callback) I noticed that the OT JS library does not recognize my plugin as installed. My plugin is properly registered (which I’ve verified with the JS lib), but the JS library cannot recognize my extension as actually installed in my browser. When checking the console, the OT JS lib throws that ‘1500’ error. This error isn’t even 1 of the errors listed in the their documentation regarding screen sharing. I’m currently in contact with their technical support to figure out the cause of this issue. I’ll update here with progress.


#7

Hmm. I wonder if the cert ssl issue is due to trying to access the self signed cert service from a computer, in this case the same computer, where the self signed cert has not been added to a trusted cert store yet.

http://www.java-samples.com/showtutorial.php?tutorialid=210


#8

Hello,

My name is Brent and I’m a developer evangelist at Twilio. I believe we met when I was out to get our OpenROV repaired back in January. At the time, I remember saying we were working on some things that may be of extreme interest but I could not discuss. Those things are directly relevant to this thread.

We announced in May our Video product which is built on (and makes easier the use of) WebRTC. I was wondering if we could discuss your progress so far with WebRTC and see if there’s a way we can get the video stream from the OpenROV available as the video source in a Twilio Video conversation. Our infrastructure would probably be a perfect fit for whatever backend you are attempting to build. I can get you beta access and would be more than willing to work with you. We would love to have something demo-ready by a developer conference we are attending in August if possible.

Any chance you’d want to work with me on this,

Thanks,
Brent