Doorphone Audio Widget using jssip, WebRTC and asterisk

Hi there!

I am trying to port my simple existing SIP doorphone audio HTML/JS page using jssip and WebRTC to a HABPanel widget.

This code was inspired by this repo: GitHub - tommyjlong/doorvivint-card: Home Assistant Video Doorbell Card for Vivint Doorbell

Just a big notice if somebody wants to try this out!
The whole thing really only works if you use HTTP over TLS (=HTTPS) for the webpages!
Plain HTTP won’t work. You will not be able to get the required audio media streams if the webpage is loaded with plain HTTP.

So what I have done is to provide valid LetsEncryp certificates everywhere. Also for the asterisk websocket connection (wss://…).
To ease things I am using LetsEncrypt with DNS01 challenge and a DynDNS provider which supports this challenge (https://dynv6.com/).
This way I am able to get a certificate for a public domain and use it just locally on my LAN only.
I.e. that the public domain names resolve to private IP addresses only here in my LAN.

Currently I am able to use my HTML/JS page using the template widget and by adding the following snippet:

<div>
    <iframe allow="camera;microphone" style=" width: 100%;height: 500px;" src="https://myhost.mydomain.dynv6.net:8089/static/index.html">      
    </iframe>
</div>

@ysc
Can you give me a starting point on how to “convert” this to a HABPanel widget?
The main problem is that the jssip instance has to live in the background to keep the SIP websocket connection in the background open if you want to act upon the SIP ringing.

The HTML page looks like this:

<!DOCTYPE html>
<html>
<head>

    <meta charset="utf-8">
    <meta name="description" content="Test">
    <meta name="viewport" content="width=device-width, user-scalable=yes, initial-scale=1, maximum-scale=1">
    <meta name="mobile-web-app-capable" content="yes">
    <meta id="theme-color" name="theme-color" content="#ffffff">

    <base target="_blank">

    <link rel="stylesheet" href="mystyle.css">

    <title>JSSIP audio</title>

</head>

<body>

<div id="container">

    <audio id="gum-local" autoplay></audio>

    <div class='button'>
            <button raised id='btn-make-call'> Call the Doorbell </button>
            <button style='display:none' raised id='btn-accept-call'> Accept call </button>
            <button style='display:none' raised id='btn-reject-call'> Reject call </button>
            <button style='display:none' raised id='btn-end-call'> Terminate call </button>
    </div>

    <div>
        <span id="errorMsg"></span>
    </div>

</div>

<script src="js/jssip-3.7.0.js"></script>
<script src="js/main.js"></script>

</body>
</html>

The JS part:

'use strict';

// Put variables in global scope to make them available to the browser console.
const remoteAudio = document.querySelector('audio');

// Create our JsSIP instance and run it:

var socket = new JsSIP.WebSocketInterface('wss://myhost.mydomain.dynv6.net:8089/ws');
var configuration = {
  sockets  : [ socket ],
  uri      : 'sip:199@myhost.mydomain.dynv6.net',
  authorization_user: "199",
  password : '199@mypass'
};

var ua = new JsSIP.UA(configuration);

ua.start();
//Register callbacks to tell us SIP Registration events
ua.on("registered", () => console.log('SIPPhone Registered with SIP Server'));
ua.on("unregistered", () => console.log('SIPPhone Unregistered with SIP Server'));
ua.on("registrationFailed", () => console.log('SIPPhone Failed Registeration with SIP Server'));

ua.on('newRTCSession', function(data) {
  console.log('News RTC session');
  let session = data.session; 

  if (session.direction === "incoming") {
	console.log('Session - Incoming call from ' + session.remote_identity );	  
	
	let acceptCallBtn = document.getElementById('btn-accept-call');
	let rejectCallBtn = document.getElementById('btn-reject-call');
	let endCallBtn = document.getElementById('btn-end-call');
	let makeCallBtn = document.getElementById('btn-make-call');	  

	makeCallBtn.style.display = 'none';
	acceptCallBtn.style.display = 'inline-flex';
	rejectCallBtn.style.display = 'inline-flex';	  
	  
	//Register for various incoming call session events
	session.on("accepted", () => {
		console.log('Incoming - call accepted');
		acceptCallBtn.style.display = 'none';
		rejectCallBtn.style.display = 'none';
		endCallBtn.style.display = 'inline-flex';
	});	  
	
	session.on("confirmed", () => console.log('call confirmed'));
	session.on("ended", () => {console.log('call ended')});
	session.on("failed", () =>{console.log('call failed')});
	session.on("peerconnection", () => {
	  session.connection.addEventListener("track", (e) => {
		  console.log('adding audio track')
		  // set remote audio stream (to listen to remote audio)
		  // remoteAudio is <audio> element on page
		  remoteAudio.srcObject = e.streams[0];
		  remoteAudio.play();
	  })
	});
	
	acceptCallBtn.addEventListener('click', () => {
		let callOptions = { mediaConstraints: { audio: true, video: false }//, mediaStream: window.stream
		};
			
		session.answer(callOptions);
	});
	endCallBtn.addEventListener('click', () => session.terminate());
	rejectCallBtn.addEventListener('click', () => {
		session.answer(callOptions);
		setTimeout(() => {
			session.terminate();
		}, 1000);
	});

	acceptCallBtn.style.display = 'inline-flex';
	rejectCallBtn.style.display = 'inline-flex';	
  }
  
  if (session.direction === "outgoing") {
	console.log('Session - Outgoing Call Event')

	let endCallBtn = document.getElementById('btn-end-call');
	let makeCallBtn = document.getElementById('btn-make-call');

	makeCallBtn.style.display = 'none';
	endCallBtn.style.display = 'inline-flex';
	endCallBtn.addEventListener('click', () => session.terminate());

	//Register for various call session events:
	session.on('progress', function(e) { 
		console.log('Outgoing - call is in progress');
	});
	session.on('failed', function(e) {
		console.log('Outgoing - call failed with cause: '+ e.cause);
		if (e.cause === JsSIP.C.causes.SIP_FAILURE_CODE) {
			console.log('  Called party may not be reachable');
		};
		location.reload();
	});
	session.on('confirmed', function(e) {
		console.log('Outgoing - call confirmed');
	});
	session.on('ended', function(e) {
		console.log('Outgoing - call ended with cause: '+ e.cause);
		location.reload(); 
	});
	
	//Note: peerconnection never fires for outoing, but I'll leave it here anyway.
	session.on('peerconnection', () => console.log('Outgoing - Peer Connection'));

	//Note: 'connection' is the RTCPeerConnection instance - set after calling ua.call().
	//    From this, use a WebRTC API for registering event handlers.
	session.connection.addEventListener("track", (e) => { 
	    console.log('add outgoing audio track');
	    remoteAudio.srcObject = e.streams[0];
	    remoteAudio.play();
	});
	
	//Handle Browser not allowing access to mic and speaker
	session.on('getusermediafailed', function(DOMError) {
		console.log('Get User Media Failed Call Event ' + DOMError )
	});
  }
});

let MakeCallBtn = document.getElementById('btn-make-call');
MakeCallBtn.addEventListener('click', () => {
    console.log('Making Call...');
    let callOptions = { mediaConstraints: { audio: true, video: false },// mediaStream: window.stream
	};
	ua.call('sip:**620@myhost.mydomain.dynv6.net', callOptions);
});

Check out Dynamic Background - #3 by ysc or Parsing JSON to a web page - #4 by ysc among others.
Basically you’d want to define an AngularJS controller with your logic in a separate JS file that you put in the html configuration folder, lazy-load it in your template with oh-lazy-load and then reference it using ng-controller.

You may also replace the event listeners that you attach to buttons like acceptCallBtn.addEventListener('click', ... with regular functions on the controller’s scope and simply call them with ng-click directives on the buttons.

@ysc Thanks for the quick reply.

I have read about oh-lazy-load. However, I am unsure about the lifecycle of the those two objects:

var socket = new JsSIP.WebSocketInterface()
...
var ua = new JsSIP.UA(configuration);
ua.start();

When exactly will the AngularJS controller start to live and when will it end?
If I switch to another dashboard, will it die?

I thought that I would have to use a service worker for background SIP part…

The lazy loader will instantiate the controller when the dashboard is displayed.
If you want to clean up your objects after navigating away your best chance is to listen for the scope’s $destroy event.

Or if you don’t want to clean them up and keep them around, put them in $rootScope (quick & dirty) and retrieve them back if they exist.

The less quick & dirty version is to create a service and inject it in your controller (example: https://stackoverflow.com/a/16876785 but there are other ways to do it, for instance HABPanel uses $inject) and note the difference between service, factory and provider (dependency injection - AngularJS: Service vs provider vs factory - Stack Overflow).

I managed to get a first working version:

Template for the template widget:

<div oc-lazy-load="'/static/doorphone.controller.js'" style="position: absolute; top: 0; bottom: 0; left: 0; right: 0;">
  <div ng-controller="DoorPhoneController">
    <div style='overflow:auto; padding: 16px; text-align: center;'>
            <button style='color:black;' raised id='btn-make-call'> Call the Doorbell </button>
            <button style='color:black;display:none;' raised id='btn-accept-call'> Accept call </button>
            <button style='color:black;display:none;' raised id='btn-reject-call'> Reject call </button>
            <button style='color:black;display:none;' raised id='btn-end-call'> Terminate call </button>
    </div>
</div>


Then I have put the file doorphone.controller.js to the openhab html folder in the config folder with the following content:


(function() {
    'use strict';
	
	angular
		.module('app.widgets')
		.controller('DoorPhoneController', DoorPhoneController);

		DoorPhoneController.$inject = ['$rootScope', '$scope', 'OHService'];

		function DoorPhoneController($rootScope, $scope, OHService) {

			$rootScope.loadData = function () {
				let script = document.createElement('script');
				script.type = 'text/javascript';
				script.src = '/static/jssip-3.7.4.min.js';
				document.getElementsByTagName('head')[0].appendChild(script);
				script.onload = () => {
					console.log('Script loaded');
					
					let remoteAudio = document.createElement('audio');
					
					var socket = new JsSIP.WebSocketInterface('wss://myhost.mydomain.dynv6.net:8089/ws');
					var configuration = {
					  sockets  : [ socket ],
					  uri      : 'sip:199@myhost.mydomain.dynv6.net',
					  authorization_user: "199",
					  password : '199@mypassword'
					};

					var ua = new JsSIP.UA(configuration);

					ua.start();
					//Register callbacks to tell us SIP Registration events
					ua.on("registered", () => console.log('SIPPhone Registered with SIP Server'));
					ua.on("unregistered", () => console.log('SIPPhone Unregistered with SIP Server'));
					ua.on("registrationFailed", () => console.log('SIPPhone Failed Registeration with SIP Server'));
					
					ua.on('newRTCSession', function(data) {
					  console.log('News RTC session');
					  let session = data.session; 

					  if (session.direction === "incoming") {
						console.log('Session - Incoming call from ' + session.remote_identity );	  
						
						let acceptCallBtn = document.getElementById('btn-accept-call');
						let rejectCallBtn = document.getElementById('btn-reject-call');
						let endCallBtn = document.getElementById('btn-end-call');
						let makeCallBtn = document.getElementById('btn-make-call');	  

						makeCallBtn.style.display = 'none';
						acceptCallBtn.style.display = 'inline-flex';
						rejectCallBtn.style.display = 'inline-flex';	  
						  
						//Register for various incoming call session events
						session.on("accepted", () => {
							console.log('Incoming - call accepted');
							acceptCallBtn.style.display = 'none';
							rejectCallBtn.style.display = 'none';
							endCallBtn.style.display = 'inline-flex';
						});	  
						
						session.on("confirmed", () => console.log('call confirmed'));
						session.on("ended", () => {console.log('call ended')});
						session.on("failed", () =>{console.log('call failed')});
						session.on("peerconnection", () => {
						  session.connection.addEventListener("track", (e) => {
							  console.log('adding audio track')
							  // set remote audio stream (to listen to remote audio)
							  // remoteAudio is <audio> element on page
							  remoteAudio.srcObject = e.streams[0];
							  remoteAudio.play();
						  })
						});
						
						acceptCallBtn.addEventListener('click', () => {
							let callOptions = { mediaConstraints: { audio: true, video: false }//, mediaStream: window.stream
							};
								
							session.answer(callOptions);
						});
						endCallBtn.addEventListener('click', () => session.terminate());
						rejectCallBtn.addEventListener('click', () => {
							session.answer(callOptions);
							setTimeout(() => {
								session.terminate();
							}, 1000);
						});

						acceptCallBtn.style.display = 'inline-flex';
						rejectCallBtn.style.display = 'inline-flex';	
					  }
					  
					  if (session.direction === "outgoing") {
						console.log('Session - Outgoing Call Event')

						let endCallBtn = document.getElementById('btn-end-call');
						let makeCallBtn = document.getElementById('btn-make-call');

						makeCallBtn.style.display = 'none';
						endCallBtn.style.display = 'inline-flex';
						endCallBtn.addEventListener('click', () => session.terminate());

						//Register for various call session events:
						session.on('progress', function(e) { 
							console.log('Outgoing - call is in progress');
						});
						session.on('failed', function(e) {
							console.log('Outgoing - call failed with cause: '+ e.cause);
							if (e.cause === JsSIP.C.causes.SIP_FAILURE_CODE) {
								console.log('  Called party may not be reachable');
							};
							//location.reload();
						});
						session.on('confirmed', function(e) {
							console.log('Outgoing - call confirmed');
						});
						session.on('ended', function(e) {
							console.log('Outgoing - call ended with cause: '+ e.cause);
							//location.reload(); 
							makeCallBtn.style.display = 'inline-flex';
							endCallBtn.style.display = 'none';
						});
						
						//Note: peerconnection never fires for outoing, but I'll leave it here anyway.
						session.on('peerconnection', () => console.log('Outgoing - Peer Connection'));

						//Note: 'connection' is the RTCPeerConnection instance - set after calling ua.call().
						//    From this, use a WebRTC API for registering event handlers.
						session.connection.addEventListener("track", (e) => { 
							console.log('add outgoing audio track');
							remoteAudio.srcObject = e.streams[0];
							remoteAudio.play();
						});
						
						//Handle Browser not allowing access to mic and speaker
						session.on('getusermediafailed', function(DOMError) {
							console.log('Get User Media Failed Call Event ' + DOMError )
						});
					  }
					});

					let MakeCallBtn = document.getElementById('btn-make-call');
					MakeCallBtn.addEventListener('click', () => {
						console.log('Making Call...');
						let callOptions = { mediaConstraints: { audio: true, video: false },// mediaStream: window.stream
						};
						ua.call('sip:**620@myhost.mydomain.dynv6.net', callOptions);
					});
					
				};
			};

			// fetch the first page
			$rootScope.loadData();
		}
})();

Of course you have to put the jssip-x.y.z.js file also into the html folder.

Things that have to be solved and improved:

  • better handle error conditions, e.g. if a call is busy do not change the buttons as there is nothing to terminate
  • make sure to handle life cycle of the JsSIP objects and the Audio-Element within the HAPPanel app properly: it would be nice to have the call ongoing while navigating to a new dashboard. Or at least put a dialog that informs the user about the fact the cal is being terminated if he navigates away from the current dashboard.

I have dropped Asterisk and created a basic webrtc-gw which can be used in front of a Fritzbox from AVM.
I dropped Asterisk because I did not want to fiddle around with extensions and dialplans. All of this is already available from the simple PBX in the Fritzbox.
So it just needed two things: conversion from SIP-over-websockets to normal SIP (UDP) and a media bridge. All of this is realised by the combination of kamailio and rtpengine.

You can find the project here: GitHub - nanosonde/webrtc-gw: A WebRTC-GW for Fritzbox based on Kamailio and rtpengine
I have it successfully running as a docker container on my Synology NAS (DS918+).