ZigBee network map when using only the ZigBee Binding

Tags: #<Tag:0x00007f2fb580ba38>

Hi fellow openhabians,
In this post I would like to share a method I am using to create a ZigBee network map while using the ZigBee Binding. As far as I know, the capabiity to create ZigBee network map exists when using Zigbee2MQTT but not when using only the ZigBee Binding, which is my case.

The problem:
Create, update and display in the Sitemap a ZigBee network map when using only the ZigBee Binding.

Input:
Two inputs can be used with similar results

  1. The Things file (epoch–org.eclipse.smarthome.core.thing.Thing.json) from /var/lib/openhab2/jsondb/backup . I am using the backup file just to be on the safe side and not to work on the actual file used by the system. This is the input that I will use further down in the script.
  2. Get all the Things information via the REST API e.g http://openhab_ip/rest/things. In that case substitute .value.properties.xxx with .properties.xxx in the examples bellow (e.g. use .properties.zigbee_routes instead of .value.properties.zigbee_routes)

Tools needed:

  1. jq : Used to parse the json file used as input and get the information needed
  2. sed : Used for clean-up and formating of the input
  3. Graphviz (the dot command) : Used to create the graph

Execution:
I have created the following script which is called periodically from crontab and creates a png image with the ZigBee network map. The script creates three input files. One with zigbee nodes information, one with routing information and one with the zigbee neighbors. The three files are then concatenated into one and a dot file is created. The dot file is then used to create the network map image which is stored in /etc/openhab2/html/ so that it can be displayed as an item in the Sitemap. The “strict digraph” graphing method is used to avoid duplicated links from being shown in the map.

Script

#   
# !/bin/bash
#

dir=/home/openhabian/zigbee_map

#copy Things backup file to working directory
cp `find /var/lib/openhab2/jsondb/backup -name *org.eclipse.smarthome.core.thing.Thing.json | tail -1` $dir/Thing.json

#extract zigbee nodes information, the entry for the coordinator with id 0 is added manually at the end of the command
jq '.[] | select(.value.properties.modelId != null) | .value.properties.zigbee_networkaddress + ",[shape=record,label=^{" + .value.properties.modelId + "|" + .value.properties.zigbee_networkaddress +"|"+ .value.properties.zigbee_logicaltype +"}^]"' $dir/Thing.json | sed -e 's/ /./g' -e 's/"//g' -e 's/,/ /g' -e 's/\./_/g' -e 's/\///g' -e 's/\^/"/g' -e '1s/^/0 [shape=record label="{CC2531|0|COORDINATOR}"]\n/' > $dir/zigbee_things

#extract zigbee nodes routing
jq '.[] | select(.value.properties.zigbee_networkaddress != null) |  select (.value.properties.zigbee_routes != "[]") | .value.properties.zigbee_networkaddress, .value.properties.zigbee_routes' $dir/Thing.json | sed 's/},{/\n/g' | grep -v INACTIVE | grep -v UNDERWAY | sed -e 's/"//g' -e 's/\\//g' -e 's/\[{//g' -e 's/}]//g' -e 's/next_hop/-> /g' -e 's/destination/ -> /g' -e 's/://g' -e 's/://g' -e 's/,//g' -e 's/stateACTIVE/ ;/g' | awk '/^[0-9]/{val=$1;print;next} NF{sub(/^ +/,"");print val,$0;next} 1'| grep ";" > $dir/zigbee_routes

#extract zigbee nodes neighbors
jq '.[] | select(.value.properties.zigbee_networkaddress != null) |  select (.value.properties.zigbee_neighbors != "[]") | .value.properties.zigbee_networkaddress, .value.properties.zigbee_neighbors' $dir/Thing.json | sed -e 's/},{/\n/g' -e 's/"//g' -e 's/\\//g' -e 's/\[{//g' -e 's/}]//g'| cut -d , -f2 | sed 's/address:/ -> /g' | awk '/^[0-9]/{val=$1;print;next} NF{sub(/^ +/,"");print val,$0;next} 1'| sed 's/$/ ;/g' | grep ">" > $dir/zigbee_neighbors

#create concatenation file and add date and time as label
echo "label=\"`date +%d-%m-%Y\ %H:%M`\";" > $dir/zigbee_map.temp

#concatenate all inputs
cat $dir/zigbee_things >> $dir/zigbee_map.temp
cat $dir/zigbee_routes >> $dir/zigbee_map.temp
cat $dir/zigbee_neighbors >> $dir/zigbee_map.temp

#add beggining and end of dot file
sed -e '1s/^/strict digraph G {\n/' -e '$ a }' $dir/zigbee_map.temp > $dir/zigbee_map.dot

#graph map
dot -Tpng $dir/zigbee_map.dot -o $dir/zigbee_map.png

#copy map file to html server location
cp $dir/zigbee_map.png /etc/openhab2/html/zigbee_map.png
    
exit

Item

Group    Zigbee_Network    "Zigbee Network Map"    <zoom>

Sitemap

Group item=Zigbee_Network label="Zigbee Network Map" icon="zoom" {
          Image url="http://<openhab_ip>/static/zigbee_map.png"
        }
4 Likes

That’s great! When I’m off the business trip back at home I need to implement this. Do you maybe already have a picture?

@NoneWhereTo Here is an example ZigBee network map image

1 Like

Hi Nikolaos,

I tried your script, but it fails in the command line. Somehow the copy cp operations does not work. I get these results:

: not foundngmap.sh: 4: zigbeebindingmap.sh:
: not foundngmap.sh: 6: zigbeebindingmap.sh:
cp: cannot create regular file '/home/openhabian/zigbee_map'$'\r''/Thing.json'$'\r': No such file or directory
: not foundngmap.sh: 9: zigbeebindingmap.sh:
: Directory nonexistent: zigbeebindingmap.sh: cannot create /home/openhabian/zigbee_map
/Thing.json: No such file or directorypenhabian/zigbee_map
: not foundngmap.sh: 12: zigbeebindingmap.sh:
: Directory nonexistent: zigbeebindingmap.sh: cannot create /home/openhabian/zigbee_map
/Thing.json: No such file or directorypenhabian/zigbee_map
: not foundngmap.sh: 15: zigbeebindingmap.sh:
: Directory nonexistent: zigbeebindingmap.sh: cannot create /home/openhabian/zigbee_map
/Thing.json: No such file or directorypenhabian/zigbee_map
: not foundngmap.sh: 18: zigbeebindingmap.sh:
: Directory nonexistent: zigbeebindingmap.sh: cannot create /home/openhabian/zigbee_map
: not foundngmap.sh: 21: zigbeebindingmap.sh:
: Directory nonexistent: zigbeebindingmap.sh: cannot create /home/openhabian/zigbee_map
: Directory nonexistent: zigbeebindingmap.sh: cannot create /home/openhabian/zigbee_map
: Directory nonexistent: zigbeebindingmap.sh: cannot create /home/openhabian/zigbee_map
: not foundngmap.sh: 26: zigbeebindingmap.sh:
: Directory nonexistent: zigbeebindingmap.sh: cannot create /home/openhabian/zigbee_map
: not foundngmap.sh: 29: zigbeebindingmap.sh:
/zigbee_map.dot't open /home/openhabian/zigbee_map
: not foundngmap.sh: 32: zigbeebindingmap.sh:
cp: cannot stat '/home/openhabian/zigbee_map'$'\r''/zigbee_map.png': No such file or directory
: not foundngmap.sh: 35: zigbeebindingmap.sh:

The directory zigbee_map is there as I created it manually. Also the json files are in the original location, though they are starting with an epoch timestamp but this is handled by the * operator. Do you know what’s wrong with the cp line? (I copied your script without modification).

Christian

Update 1:
Maybe permission issue. I’ll look into it and get back :slight_smile:

Update 2:
Changed the directory to html where openhab should have rw access to, but still the same errors come up :frowning:

Would you consider raising an issue at the openhab2-addon for the zigbee binding pointing to your work?

Would be nice, if the zigbee binding would provide such an network overview at least in the admin UI(s). Probably ping @chris for that.
He did something quite similar for Zwave in the HABmin UI…

Maybe he could even provide this as a channel (channel-type image) of the coordinator thing?

The ZigBee binding does produce this information - I think that is what is being used in this script?

The UI needs to take the information that the ZigBee binding creates, and turn this into the map. I did do this a few years back in PaperUI for a commercial user of the binding and in theory I could probably publish this if I can find the code now :slight_smile:

2 Likes

Hi @NoneWhereTo ,
It does look like a permission error. I suggest to check the permissions on the /home/openhabian/zigbee_map dir. Could it be possible that you might have created the script directory with one user and running the script with another (e.g. one with openhab user and the other with openhabian user)?
So, I would sugget to check the owner and permissions of the script directory and that the user running the script has access to the script directory and the jsondb/backup directory. Also, I would suggest to try running the commands of the script one by one in a terminal to find which is the first part that it is not working (e.g. just do a plain copy of one org.eclipse.smarthome.core.thing.Thing.json file in the script directory and rename it as Things.json and then try the rest of the commands in the script).

BR,
Nikos

Hi @chris and @curlyel,
Indeed the information I use as input is produced by the ZigBee binding and it would be great if such a network map could be incorporated in the binding so that it could be displayed in the BasicUI or any other place.
I am not familiar with binding development so I have to ask, would it be possible to re-use what I have done in the binding or it has to be written from scratch? If the former is the case then it would be great and I would be happy to follow @curlyel sugestion.

BR,
Nikos

Just for info, this is what I did for the customer a few years back -:

Clicking on a node shows the node properties. The graph is updated as information is updated, and nodes can be dragged around (they have a kind of gravity function to align them).

PaperUI theming is the Eclipse SmartHome theme.

If I can find the source to this I will see if I can make it available - someone might want to take it and add it to PaperUI (or the next-gen PaperUI).


3 Likes