OH MQTT binding to "The Things Network" (TTN)

Hi,

I am working on LoRa and am trying to integrate LoRa “things” (using TTN network) in OH. Therefore I installed MQTT on my OH (1.8.3). Result is that in the log file it says that MQTT client is generated, MQTT binding and event bus binding is done successfully.

I have a couple of issues:

1> Is there a way to see the raw incoming messages? Have changed the logback.xml and included DEBUG logging for binding.mqtt and io.transport.mqtt - however cannot see message details in the logs.

2> When e.g. the message only contains temperature I believe the following must be done: Create item and define the transformation/regex that must be done to extract the temperature information from the MQTT message - agree?

Number LoRa_Temp “LoRa temp [%s]” {mqtt="<[TTN:70B3D57ED0000285/devices/0000000031D2E38E/up:state:transform / regex]"}

3> The TTN broker sends message which comtain quite a bit of info. On Node-Red I can easily decode them and am wondering what the best way is to do this on OH. In the end I want to use the information in the payload (base64 encoded). Below an example:

{“payload”:“W3sibGF0IjoiNTIuMTgxODU0MCJ9LCB7ImxuZyI6IjUuOTM5OTk5MSJ9LHsiYWx0IjoiIDQ1LjcifSx7ImNycyI6IjEzNC43NiJ9LHsia21oIjoiIDAuMzEifV0=”,“port”:1,“counter”:106,“dev_eui”:“0000000031D2E38E”,“metadata”:[{“frequency”:868.1,“datarate”:“SF7BW125”,“codingrate”:“4/5”,“gateway_timestamp”:1773600928,“channel”:0,“server_time”:“2016-06-30T09:12:08.952785832Z”,“rssi”:-49,“lsnr”:9,“rfchain”:0,“crc”:1,“modulation”:“LORA”,“gateway_eui”:“5CCF7FFFFF1B7205”,“altitude”:0,“longitude”:0,“latitude”:0}]}

Thanks,
Jeroen

1> I don’t know. I figured switching to DEBUG would do it. You can try TRACE and see if that logs them.

2> That looks right in concept.

3> The message appears to be JSON. I would probably create a JavaScript transform to extract the value. I don’t know if there is a base64 decoder built into the JavaScript engine OH uses for the transformation or not. If not you can create a rule to decode it and use new String(java.util.Base64::getDecoder.decode(Item.state.toString)) to convert it back to a String.

Thanks Rich.

1> All MQTT message arrive to OH now (use DEBUG logging)
2> Using REGEX to filter out message which I do not need (based on the dev_eui)
3> In Node-Red I decoded the LoRa messages via JS as well. However I need to get used to the OH way :slight_smile:

So I can trigger the javascript and return a test value to OH.

Number LoRa_Temp “LoRa temp [%s]” {mqtt="<[TTN:70B3D57ED0000285/devices/0000000031D2E38E/up:state:JS(lora.js):.*0000000031D2E38E.*]"}

However I do not know how to access the data which is delivered by the MQTT message inside the JS. Have not been able to find a good example so hopefully somebody here can push me in the right direction. Thanks

When using a JavaScript transform the incoming value is placed in the input variable I think. However, keep in mind that I don’t think you have have both a REGEX and a JS Transform at the same time. I could be wrong on that of course.

Search the forum for “JavaScript Transform” and you should see several examples.

1 Like

Thanks Rich, that was exactly the info I was looking for. So yesterday I was able to finish the draft working solution. For those also working with LoRa TTN and OH I added the draft code below.



‘use strict’;

// Jeroen - 30-6-2016
// This javascript is called from Openhab in which an Item is defined
// MQTT connection to LoRa TTN broker example:
// String LoRa_Temp “LoRa temp [%s °C]” {mqtt="<[TTN:70B3D57ED0000285/devices/0000000031D2E38E/up:state:JS(lora.js):.0000000031D2E38E.]"}
//
// REGEX is used for filtering only (no data manipulation) / in above example to only process dev_eui 0000000031D2E38E
// Openhab is passing the MQTT information to the javascript in the variable “input” which is used (see below).
// The JS file (lora.js) to be placed in /etc/openhab/configurations/transform directory

// Basic base64 decoder function used to decode the actual LoRa payload
// Decoder using the LoRa AppSKey is not implemented (yet) - on my ToDo list

function decodeBase64(s) {
var e={},i,b=0,c,x,l=0,a,r=’’,w=String.fromCharCode,L=s.length;
var A=“ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/”;
for(i=0;i<64;i++){e[A.charAt(i)]=i;}
for(x=0;x<L;x++){
c=e[s.charAt(x)];b=(b<<6)+c;l+=6;
while(l>=8){((a=(b>>>(l-=8))&0xff)||(x<(L-2)))&&(r+=w(a));}
}
return r;
};

function lora_parse() {
var msg = JSON.parse(input);
var sensordataraw = decodeBase64(msg.payload);
var sensordata = JSON.parse(sensordataraw);

// Generic information provided by the LoRa / TTN network
var counter = msg.counter;
var deveui = msg.dev_eui;
// Generic meta info
var gweui = msg.metadata[0].gateway_eui;
var gwTimestamp = msg.metadata[0].gateway_timestamp;
var srvTime = msg.metadata[0].server_time;          
var rssi = msg.metadata[0].rssi;
  
// Information hidden in the actual payload (sensor specific) / below an example of GPS module info
// You can change it to your specific sensor info
var devinfo1 = sensordata[0].lat;
var devinfo2 = sensordata[1].lng;
var devinfo3 = sensordata[2].alt; 
var devinfo4 = sensordata[3].crs;
var devinfo5 = sensordata[4].kmh;
// The value you actual want to return to OpenHab 
return devinfo1;

}

lora_parse(); // Main: Call the function lora_parse() - as the return statement must be inside a function

1 Like

Hi @Jeroen

I’m trying to set up the MQTT to the TTN network but get some issues.
How did you configure this?
I can connect with mosquitto directly to get the data from the OpenHab server so all openings and passwords should be just fine.

I created this in the: “/etc/openhab2/services/mqtt.cfg”

ttn.url=tcp://eu.thethings.network:1883
ttn.allowLongerClientIds=true
ttn.user=MYUSER
ttn.pwd=ttn-account-v2.MYPASSWORD
ttn.qos=0

I just get this output in the logs:

2018-02-13 14:21:35.924 [ERROR] [t.mqtt.internal.MqttBrokerConnection] - Error starting consumer
org.eclipse.paho.client.mqttv3.MqttException: MqttException
at org.eclipse.paho.client.mqttv3.MqttClient.subscribe(MqttClient.java:329) [208:org.openhab.io.transport.mqtt:1.11.0]
at org.eclipse.paho.client.mqttv3.MqttClient.subscribe(MqttClient.java:315) [208:org.openhab.io.transport.mqtt:1.11.0]
at org.openhab.io.transport.mqtt.internal.MqttBrokerConnection.startConsumer(MqttBrokerConnection.java:469) [208:org.openhab.io.transport.mqtt:1.11.0]
at org.openhab.io.transport.mqtt.internal.MqttBrokerConnection.addConsumer(MqttBrokerConnection.java:453) [208:org.openhab.io.transport.mqtt:1.11.0]
at org.openhab.io.transport.mqtt.MqttService.registerMessageConsumer(MqttService.java:173) [208:org.openhab.io.transport.mqtt:1.11.0]
at org.openhab.binding.mqtt.internal.MqttGenericBindingProvider.processBindingConfiguration(MqttGenericBindingProvider.java:63) [206:org.openhab.binding.mqtt:1.11.0]
at org.openhab.core.binding.internal.BindingConfigReaderDelegate.processBindingConfiguration(BindingConfigReaderDelegate.java:49) [207:org.openhab.core.compat1x:2.2.0]
at org.eclipse.smarthome.model.item.internal.GenericItemProvider.internalDispatchBindings(GenericItemProvider.java:341) [135:org.eclipse.smarthome.model.item:0.10.0.b1]
at org.eclipse.smarthome.model.item.internal.GenericItemProvider.internalDispatchBindings(GenericItemProvider.java:310) [135:org.eclipse.smarthome.model.item:0.10.0.b1]
at org.eclipse.smarthome.model.item.internal.GenericItemProvider.processBindingConfigsFromModel(GenericItemProvider.java:195) [135:org.eclipse.smarthome.model.item:0.10.0.b1]
at org.eclipse.smarthome.model.item.internal.GenericItemProvider.modelChanged(GenericItemProvider.java:377) [135:org.eclipse.smarthome.model.item:0.10.0.b1]
at org.eclipse.smarthome.model.core.internal.ModelRepositoryImpl.notifyListeners(ModelRepositoryImpl.java:314) [134:org.eclipse.smarthome.model.core:0.10.0.b1]
at org.eclipse.smarthome.model.core.internal.ModelRepositoryImpl.addOrRefreshModel(ModelRepositoryImpl.java:143) [134:org.eclipse.smarthome.model.core:0.10.0.b1]
at org.eclipse.smarthome.model.core.internal.folder.FolderObserver.checkFile(FolderObserver.java:247) [134:org.eclipse.smarthome.model.core:0.10.0.b1]
at org.eclipse.smarthome.model.core.internal.folder.FolderObserver.processWatchEvent(FolderObserver.java:311) [134:org.eclipse.smarthome.model.core:0.10.0.b1]
at org.eclipse.smarthome.core.service.WatchQueueReader.run(WatchQueueReader.java:209) [109:org.eclipse.smarthome.core:0.10.0.b1]
at java.lang.Thread.run(Thread.java:748) [?:?]

Hi Michael,
try to setup /etc/openhab2/services/mqtt.cfg in this way:

ttn.url=tcp://eu.thethings.network:1883
ttn.user=Your Application ID (i.e. my-test)
ttn.pwd=Your Application Access Key (i.e. ttn-account-v2.I7…)
ttn.qos=0
ttn.retain=false
ttn.async=true

In .items add a line like this one
Number LoRa_Temp “LoRa temp [%.1f]” {mqtt="<[ttn:YourApplicationID/devices/YourDeviceID/up/YourDecodedData:state:default]"}
i.e.
Number LoRa_Temp “LoRa temp [%.1f]” {mqtt="<[ttn:my-test/devices/my-test-device/up/temperature:state:default]"}

in .sitemaps add a new row
Text item=LoRa_Temp

Gianluigi

Thanks for your replay…

Closer but something is really strange…

2018-02-27 10:14:31.212 [ERROR] [org.apache.felix.configadmin ] - Cannot use configuration org.openhab.mqtt for [org.openhab.core.scriptengine.action.ActionService, org.osgi.service.cm.ManagedService,
id=312, bundle=210/mvn:org.openhab.action/org.openhab.action.mqtt/1.11.0]: No visibility to configuration bound to mvn:org.openhab.io/org.openhab.io.transport.mqtt/1.11.0
2018-02-27 10:21:13.010 [INFO ] [el.core.internal.ModelRepositoryImpl] - Refreshing model 'lora.items’
2018-02-27 10:21:13.736 [ERROR] [t.mqtt.internal.MqttBrokerConnection] - Error starting consumer
org.eclipse.paho.client.mqttv3.MqttException: MqttException

I will try to connect a ESP8266 to see if I can get the MQTT from this working.

If you doesn’t define any item in the lora.items file, did you get any error in the log file ?
If there are no error in the log, could you post your working mosquitto command line and the related lora item definitions row in lora.items ?
Gianluigi

Hi @gigi130358

I may be a loot closer now…
I don’t get any issues in the logfile, I may parse the file wrong since my items are not updated with any values.

/etc/openhab2/services/mqtt.cft

ttn.url=tcp://eu.thethings.network:1883
ttn.user=starman
ttn.pwd=ttn-account-v2.MYPASSWORD
ttn.qos=0
ttn.retain=false
ttn.async=true

/etc/openhab2/items/lora.items

Number LoRa_Temp "LoRa temp [%.1f]" {mqtt="<[ttn:starman/devices/a81758fffe034049/up/externalTemperature:state:default]"}
Number LoRa_VDD "LoRa temp [%.1f]" {mqtt="<[ttn:starman/devices/a81758fffe034049/up/vdd:state:default]"}
String LoRa_Payload "Payload [%s]" {mqtt="<[ttn:starman/devices/a81758fffe034049/up/payload:state:default]"}

/etc/openhab2/sitemap/lora.sitemap

sitemap rooms label="Lora" {
    Frame {
        Text label="Device1" icon="temperature" {
            Default item=LoRa_Temp label="LoRa_Temp"
            Default item=LoRa_VDD label="LoRa_VDD"
            Default item=LoRa_Payload label="LoRa_Payload"
        }
    }
}

BASH with mosquitto_sub:

mosquitto_sub -h eu.thethings.network -t 'starman/devices/+/up' -u 'starman' -P 'ttn-account-v2.MYPASSWORD' -v

starman/devices/a81758fffe034049/up {“app_id”:“starman”,“dev_id”:“a81758fffe034049”,“hardware_serial”:“A81758FFFE034049”,“port”:5,“counter”:8,“confirmed”:true,“payload_raw”:“AQDyAhID//89Bw3zDADoDQAUAA+SAg==”,“payload_fields”:{“digital”:0,“externalTemperature”:23.2,“humidity”:18,“pressure”:1020.418,“temperature”:24.2,“vdd”:3571,“x”:-1,“y”:-1,“z”:61},“metadata”:{“time”:“2018-03-26T09:07:11.446393094Z”,“frequency”:868.5,“modulation”:“LORA”,“data_rate”:“SF8BW125”,“airtime”:143872000,“coding_rate”:“4/5”,“gateways”:[{“gtw_id”:“eui-46584258c00001d0”,“timestamp”:854241132,“time”:"",“channel”:2,“rssi”:-33,“snr”:7.8,“rf_chain”:1}]}}

If you didn’t already activate it, I suggest to enable the mqtt log as reported in https://community.openhab.org/t/mqtt-on-oh2-help-needed/20747/2:
You can enable DEBUG level of logging for this binding from the Karaf Console (ssh openhab@localhost -p 8101 with password habopen) by:
log:set DEBUG org.openhab.binding.mqtt

I also suggest to use only one item (i.e. Number LoRa_Temp “LoRa temp [%.1f]” {mqtt="<[ttn:starman/devices/a81758fffe034049/up/externalTemperature:state:default]"} )
and take a look in openhab.log to see if there is any info related to MQTT.

@gigi130358

Well that didn’t give me more understanding :slight_smile:
I uninstalled/installed the MQTT “binding-mqtt1 - 1.11.0” from papperUI

Debug enabled:

2018-03-26 14:57:02.922 [DEBUG] [org.openhab.binding.mqtt            ] - BundleEvent INSTALLED - org.openhab.binding.mqtt
2018-03-26 14:57:03.125 [DEBUG] [org.openhab.binding.mqtt            ] - BundleEvent RESOLVED - org.openhab.binding.mqtt
2018-03-26 14:57:03.140 [DEBUG] [org.openhab.binding.mqtt            ] - BundleEvent STARTING - org.openhab.binding.mqtt
2018-03-26 14:57:03.146 [DEBUG] [.binding.mqtt.internal.MqttActivator] - MQTT binding has been started.
2018-03-26 14:57:03.236 [DEBUG] [org.openhab.binding.mqtt            ] - ServiceEvent REGISTERED - {org.osgi.service.event.EventHandler}={event.topics=openhab/*, component.name=org.openhab.binding.mqtt.MqttItemBinding, component.id=220, service.id=359, service.bundleid=245, service.scope=bundle} - org.openhab.binding.mqtt
2018-03-26 14:57:03.285 [DEBUG] [org.openhab.binding.mqtt            ] - ServiceEvent REGISTERED - {org.openhab.model.item.binding.BindingConfigReader, org.openhab.binding.mqtt.MqttBindingProvider}={component.name=org.openhab.binding.mqtt.genericbindingprovider, component.id=221, service.id=358, service.bundleid=245, service.scope=bundle} - org.openhab.binding.mqtt
2018-03-26 14:57:03.306 [DEBUG] [binding.mqtt.internal.MqttItemConfig] - Loaded MQTT config for item 'LoRa_Temp' : 1 subscribers, 0 publishers
2018-03-26 14:57:03.314 [DEBUG] [binding.mqtt.internal.MqttItemConfig] - Loaded MQTT config for item 'LoRa_VDD' : 1 subscribers, 0 publishers
2018-03-26 14:57:03.318 [DEBUG] [binding.mqtt.internal.MqttItemConfig] - Loaded MQTT config for item 'LoRa_Payload' : 1 subscribers, 0 publishers
2018-03-26 14:57:03.353 [DEBUG] [ng.mqtt.internal.MqttEventBusBinding] - MQTT: Activating event bus binding.
2018-03-26 14:57:03.359 [DEBUG] [org.openhab.binding.mqtt            ] - ServiceEvent REGISTERED - {org.osgi.service.event.EventHandler, org.osgi.service.cm.ManagedService}={event.topics=openhab/*, service.pid=org.openhab.mqtt-eventbus, component.name=org.openhab.binding.mqtt.eventbus, component.id=222, service.id=361, service.bundleid=245, service.scope=bundle} - org.openhab.binding.mqtt
2018-03-26 14:57:03.368 [DEBUG] [org.openhab.binding.mqtt            ] - BundleEvent STARTED - org.openhab.binding.mqtt

This is my log:

2018-03-26 20:34:52.164 [INFO ] [penhab.io.transport.mqtt.MqttService] - MQTT Service initialization completed.
2018-03-26 20:34:52.200 [INFO ] [t.mqtt.internal.MqttBrokerConnection] - Starting MQTT broker connection 'ttn'
2018-03-26 20:34:58.359 [DEBUG] [org.openhab.binding.mqtt            ] - BundleEvent STARTING - org.openhab.binding.mqtt
2018-03-26 20:34:58.370 [DEBUG] [.binding.mqtt.internal.MqttActivator] - MQTT binding has been started.
2018-03-26 20:34:58.374 [DEBUG] [org.openhab.binding.mqtt            ] - BundleEvent STARTED - org.openhab.binding.mqtt
2018-03-26 20:34:58.468 [DEBUG] [org.openhab.binding.mqtt            ] - ServiceEvent REGISTERED - {org.openhab.model.item.binding.BindingConfigReader, org.openhab.binding.mqtt.MqttBindingProvider}={component.name=org.openhab.binding.mqtt.genericbindingprovider, component.id=221, service.id=344, service.bundleid=234, service.scope=bundle} - org.openhab.binding.mqtt
2018-03-26 20:35:02.052 [DEBUG] [binding.mqtt.internal.MqttItemConfig] - Loaded MQTT config for item 'LoRa_Temp' : 1 subscribers, 0 publishers
2018-03-26 20:35:02.212 [DEBUG] [binding.mqtt.internal.MqttItemConfig] - Loaded MQTT config for item 'LoRa_Light' : 1 subscribers, 0 publishers
2018-03-26 20:35:02.269 [DEBUG] [binding.mqtt.internal.MqttItemConfig] - Loaded MQTT config for item 'LoRa_Bat' : 1 subscribers, 0 publishers
2018-03-26 20:35:02.334 [DEBUG] [org.openhab.binding.mqtt            ] - ServiceEvent REGISTERED - {org.osgi.service.event.EventHandler}={event.topics=openhab/*, component.name=org.openhab.binding.mqtt.MqttItemBinding, component.id=220, service.id=346, service.bundleid=234, service.scope=bundle} - org.openhab.binding.mqtt
2018-03-26 20:35:02.396 [DEBUG] [org.openhab.binding.mqtt            ] - ServiceEvent REGISTERED - {org.osgi.service.event.EventHandler, org.osgi.service.cm.ManagedService}={event.topics=openhab/*, service.pid=org.openhab.mqtt-eventbus, component.name=org.openhab.binding.mqtt.eventbus, component.id=222, service.id=347, service.bundleid=234, service.scope=bundle} - org.openhab.binding.mqtt
2018-03-26 20:35:02.490 [DEBUG] [ng.mqtt.internal.MqttEventBusBinding] - MQTT: Activating event bus binding.

There are some differences, for example your log doesn’t have
“BundleEvent STARTED - org.openhab.binding.mqtt”

Looking for installed version with “openhab> feature:list | grep MQ” I get

esh-tp-paho                                 │ 0.10.0.b1        │          │ Uninstalled │ distro-2.2.0            │ MQTT v3 Client
openhab-transport-mqtt                      │ 2.2.0            │          │ Uninstalled │ distro-2.2.0            │ MQTT Transport
openhab-action-mqtt                         │ 1.11.0           │          │ Uninstalled │ openhab-addons-2.2.0    │ MQTT Action
openhab-binding-mqtt1                       │ 1.11.0           │ x        │ Started     │ openhab-addons-2.2.0    │ MQTT Binding
openhab-binding-mqttitude1                  │ 1.11.0           │          │ Uninstalled │ openhab-addons-2.2.0    │ OwnTracks (formerly MQTTitude) Binding
openhab-persistence-mqtt                    │ 1.11.0           │          │ Uninstalled │ openhab-addons-2.2.0    │ MQTT Persistence

Did you restart openhab after mqtt installation ?
Take a look also at the last post in MQTT on OH2 - help needed

@gigi130358

I can’t thank you enough for all your efforts.
After a clean installation of the whole OpenHAB I experienced the same issues, the logfile showed me this when trying to enable the MQTT from the PapperUI:

2018-03-27 11:20:33.008 [ERROR] [core.karaf.internal.FeatureInstaller] - Failed installing 'openhab-binding-mqtt1': Error resolving artifact org.openhab.addons:openhab-addons-external:cfg:mqtt:1.11.0: [Could not transfer artifact org.openhab.addons:openhab-addons-external:cfg:mqtt:1.11.0 from/to openhab (https://dl.bintray.com/openhab/mvn/online-repo/2.2/): Read timed out]

I swaped to the snapshot build of OpenHAB instead and re-installed the MQTT plugin from the papperUI.

Finaly success! Everything work as expected…

Have a good day! And again a big THANKS!

//Michael Stjerna

Happy for this :wink:
Gianluigi

By the way. Did you parse out the Metadata from TTN?
I want to store the MetaData and GW information to OpenHAB

{       "app_id":"starman",
        "dev_id":"a81758fffe034049",
        "hardware_serial":"A81758FFFE034049",
        "port":5,
        "counter":5,
        "payload_raw":"AQD1AgwDAwQ9Bw3uDADnDQAUAA+k5g==",
        "payload_fields":{
                "digital":0,
                "externalTemperature":23.1,
                "humidity":12,
                "pressure":1025.254,
                "temperature":24.5,
                "vdd":3566,
                "x":3,
                "y":4,
                "z":61
        },
        "metadata":{
                "time":"2018-03-27T13:28:44.857209839Z",
                "frequency":867.1,
                "modulation":"LORA",
                "data_rate":"SF7BW125",
                "airtime":77056000,
                "coding_rate":"4/5",
                "gateways":[{
                        "gtw_id":"eui-46584258c00001d0",
                        "timestamp":4163345947,
                        "time":"",
                        "channel":3,
                        "rssi":-37,
                        "snr":10.5,
                        "rf_chain":0}]
        }
}

:slight_smile:

No, I didn’t.
I get only the functional data related to Temperature, Lux and of course the battery voltage that at this stage is the more interesting parameter to monitor in order to evaluate how long a lora node could live.

Hello,

Maybe someone have example how to configure mqtt client with TLS?

From Command prompt i use this string:
mosquitto_sub -h eu.thethings.network -t ‘#’ -u ‘my_user’ -P ‘ttn-account-v2.mypassword’ -v --cafile mqtt-ca.pem -p 8883

Maybe i can get help how to configure .cfg file?

Ed