Beginner Question: Ideas on how to implement my proposed setup using Pi's

Hello,
My experience in programing languages is very minimal but im willing to learn and listen! I come from an era back in the qbasic days where we could control devices using parallel ports on old machines with very basic scripts. I have been tinkering with GPIO setups on a few different raspberry pi’s and i have been trying to build a distributed model of home automation room by room. IE the garage would be controlled by one raspberry the living room being controlled by another… so on so forth. However i would like to have a central server to manage all the commands to these pis and allow them to work together with one common user interface. Open Hab seems to be the way to do that but im finding the whole setup process very confusing and am not sure what path to take at this point. Any suggestions or ideas would be great i appreciate everyone help. I will answer as many questions as the group has to help this along as well.

This kind of distributed computing is a very complex task for beginner. I suggest you make your room RPis as stupid as possible - just translating network commands to relays and back from sensors to network. And your main RPi would control everything. This is easiest path.

Thanks for the input…Any idea what add-ons i would use to accomplish this with OH and how that would work?

This architecture is very similar to mine. I have three Pis scattered through the house with various things wired up to the GPIO pins. I use OH as the central control and MQTT for communications between the HUB and the satellite Pis. It works very well for me.

My Python code is hosted here:

It currently supports simple ON/OFF and Toggle commands for actuators as well as reporting on the status of the pins for sensors.

My specific architecture includes but is not limited to:

Manticore (Pi):

  • RFM69HW gateway which translates sensor data wirelessly reported over RFM69HW (915 MHz) to MQTT
  • Bluetooth device detection

Hydra (Pi):

  • Reed sensors wired to GPIO reported over MQTT
  • Bluetooth device detection

Cerberos (Pi):

  • IP camera
  • Reed sensors wired to GPIO reported over MQTT
  • Relay wired to GPIO to simulate garage door opener for remote control
  • Bluetooth device detection (both my script and reelyActive)

Chimera (Old laptop, main server):

  • openHAB 2 in a Docker container
  • Mosquitto MQTT broker running in a Docker container
  • InfluxDB for OH persistence and charting, running in a Docker container
  • Grafana for charting of OH persistence data, running in a Docker container
  • Zoneminder for CCTV control and management, running in a Docker container
  • sensorReporter, same script as deployed to the Pis
    • Bluetooth device detection
    • Execute shell scripts based on receipt of messages from OH over MQTT (because I run OH 2 in a Docker container the Exec binding doesn’t help much)
    • Dash button press detection

But note that given your experience this will be a pretty big undertaking so be sure to start small and gradually build up your system. Experiment with my script, or at least look at it to see an example in code of something very similar. Experiment with the openHAB Demo config to get a feel for how OH itself works with Items and Rules and Sitemaps and such. Next edit the Demo config in little ways or create a new simple config and use the Demo as a reference and do simple stuff like a single light or interact with a web API like getting weather from Yahoo.

There is a lot already built for you out there but it is going to take a lot of learning and work on your part.

Hello,

Thanks for your great information and skills in setting this up. I have a few more questions. Can you explain the relationship between MQTT and OH ? I guess im looking for a little more guidance to push me in the right direction. So once OH is installed you have to install these modules and then can you configure them through the gui web ? Do you point them to the raspberry pi’s by ip? I guess i just wish there was some sort of guide to get things started. I am willing to start small but i just need to get a better understanding of how to set this up. I have a ubuntu server VM spun up with OH installed still trying to get the config to work and then ill start with 1 pi that only needs to use GPIO pins and relays to control two garage doors and garage lights and two magnetic sensors to run inputs and display whether the doors are open or not. If we can start there that would be great and i think once i get the hang of how this all works ill be able to figure the rest out i just feel a bit overwhelmed. but i appreciate everyones help.

MQTT is a messaging protocol developed by the oil industry and IBM for their SCADA systems to communicate. It provides a hub and spoke communications pattern with a central Broker at the hub and clients as the spokes. To send a message a client connects to the Broker and publishes a message to a “topic” which you can think of as like a mail bin in the office. To receive a message a client subscribes to a “topic” and it will have any messages published to that topic delivered to them. More than one client can publish to and/or subscribe to a topic.

In this situation OH is a client as are your Raspberry Pis. But OH needs know nothing about the other clients. It only needs to know how to communicate with the Broker and what topics to subscribe to and/or publish to. This makes setup and configuration very easy for your remote devices.

There are many brokers to choose from though I think Mosquitto is the most popularly used one around here.

I’m afraid that OH 2 is not yet to the point where you can do everything through the GUI. You will be spending a lot of time in text files. This is whether or not you choose to go the MQTT route or not. The GUI support is not yet to the point where you can do everything, and legacy bindings like MQTT are not supported in the GUI at all.

The order I would do this is:

  • Get the Pis working with my script or some other script to verify you can control things through the GPIO pins locally.
  • Install an MQTT broker and augment/configure the Pi script to talk on MQTT (I used the Paho library in my script). Test this by sending/receiving messages using the command line test tools like mosquitto_sub and mosquitto_pub.
  • Install OH and configure the MQTT Binding. Watch the logs to make sure MQTT connects
  • Create your Items to control/represent the remote devices and bind them to the topic they publish/subscribe to
  • Put the Items on the sitemap so you can see them and interact with them through a GUI.

All of this is very doable even for non-technical types who are willing to try and learn. However, if this seems too overwhelming you might consider a more off-the-shelf approach with zwave or one of the other commercial standards. They are ultimately far less work and have a much lower learning curve.

Hello,

Thank you again for all your responses and help with this. Should i get stuck along the way i will be sure to ask here as you are very helpful. I am non technical in programing but i actually work in IT im a networking engineer haha… Routers switches and fw’s and routing protocols make more sense to me then programing so its just something Ive never really dove into. IM very excited to figure this all out and hope that i can put it all together and make it work. Ill let you know if i have more questions as i progress.

Alright so im starting this setup and i just want
to be clear that im headed in the right direction here. I am installing
your following scripts to the pi. default.ini mqttconn.py
rpiGPIOActuator.py and rpiGPIOSensor.py also the execactuator.py.

Then i have to install the mosqito mqtt on my server side as the mqtt
broker. At this point i should be able to point the mqttcon.py to the
broker using connection 1 with a host: IP where the broker is. My first
question at this time where do i set the pins that are going to be
acctuated and or sensors that will be posting. Is that in the INI file?
and do i have to make modifications to the other files in order to make
my setup work? Lets star there and i will move on from there. Thank you
again so much for your help.

PS I already have raspbian setup with GPIO and python installed as well.

You need the main files too:

  • signalProc.py
  • sensorReporter.py

default.ini is an example file. you will want to write your own using default.ini as a template. The install.sh script in the config folder has you save the ini file as sensorReporter.ini (or link to a hostname.ini). That is what is expected by the start scripts in the config folder.

All configuration takes place in the .ini file. You need a user, password, host, port, a topic to listen to for commands sent to sensorReporter, and a LWT topic and message which gets published when sensorReporter disconnects from the Broker.

The .ini file. Each pin will have its own sensor or acutator section.

You should not have to edit any of the .py files. Everything is configured through the .ini file.

Hey so good news… I had a buddy give me a hand figuring this all out and we have the following set up.

We have the pi set up with your scripts and talking with the MQTT broker. We also have HAB talking with the MQTT broker. However there is one issue we ran into. For some reason we were able to get the pi to subscribe to the topic on the broker using a username and password set up however when we tried to to this with OH we keep getting an authentication error from the broker. We have both looked over the user and pass a million times and everything seems to be in order but it still keeps throwing this error. Both the Open hab installation and the broker are on the same server. Things seem to be going along alright so far and i really appreciate all your help. Ill keep you posted on how we progress.

The only things I can think of are:

  • make sure the port is correct on the URL
  • make sure the clientId is unique for each client

Good luck!

Hello,

We have been working on this for a while now but we seem to have hit a snag. We are having issues updating a status sensor with an MQTT message inside openhab. We see the OH server is connecting to the broker but it does not seem to respond to the messages.
Here is an example of a manual command we are trying to run to update the sensor item called door1

mosquitto_pub -h 10.5.5.100 -p 8883 -t ‘openhab/garage/door1’ -m '1’
Do we have the syntax wrong here?

Here is the section of the items file pertaining to door1

/Garage Doors/

String
Door_G_bayright “Right Door Switch [MAP(switch.map):%s]”
(Doors, gG)
{mqtt="<[mqttbroker:openhab/garage/door1:state:default]"}

When we run the above command from a pi for testing we do not see the item update on the site map like its supposed to. Any thoughts?

Step 1: Use mosquitto_sub to verify the message is being published.

Step 2: Are there errors in the log? Especially when debugging you should be watching the log files. This will tell you whether the message was received but not processed correctly or not received at all.

The code looks correct by eye.

Hello,

Thanks again. I have verified that i am indeed posting to the topic. using the testing tool however my gut tells me that open hab is not actually subscribing to the topic and hence not updating the status of the right door switch to 1 when posted. I am unsure which logs i should be looking at in regards to open hab to see weather or not its subscribing with the broker. just so i make myself clear the way i have this set up i have a site map item called right door switch and im expecting it to update with whatever the value is that is posted to the topic.
Thanks again
SteveT

Assuming it is a standard apt-get installation on Linux the log files are located in /var/log/openhab.

The openhab.log will have errors and other information. You should be seeing lines along the line of

MQTT Service initialization completed.
Starting MQTT Broker connection 'broker name' 

when OH first starts up showing that OH is successfully connecting to the broker.

In events.log you should be seeing lines along the lines of:

Item 'Door_G_bayright` received update 1

when you publish the message.

The problem with relying only on the sitemap for feedback is that there is so much that goes on between the sender of the message and the sitemap that can go wrong that when the sitemap doesn’t update you have no clue where the error occurred. It could be the sitemap is simply failing to update (a problem not too long ago which I think was fixed, maybe not) to the binding is not starting or connecting to the broker.

Thanks again… The log file in the location you specified is empty so this makes me think that its either not logging? or is logging in a different locations. Is that something that would be set in the config file? Just an FYI i do see the MQTT service initialization and i see it starting the mqtt broker connection “home” which is the name i change the alias to yesterday and updated the items file to reflect. I am digging through the event.log now to see if i can see that item being used.
This is what open hab reports when it starts up about the mqtt broker
2016-11-10 08:31:01.823 [INFO ] [.io.transport.mqtt.MqttService] - MQTT Service
initialization completed.
2016-11-10 08:31:01.823 [INFO ] [o.i.t.m.i.MqttBrokerConnection] - Starting MQTT
broker connection ‘home’

Thanks,
SteveT

so upon review of the items file… I enacted the update to the mqtt topic several times and then checked the event logs and i do not see anything referring to the Door_G_bayright in fact there is very little in this log … probably because i dont have anything else really going on right now till i get this working. So running mosquitto_sub openhab/garage/door1’ i can clearly see my updates coming through as 1 or open or closed or whatever i append to the end of the mosquitto_pub command from the raspberry. So the topic and broker is working. Also i can see that open hab appears to be connecting to the broker by this line. When i watch the Misquitto connections come in after starting misquitto i can see
1478786810: New client connected from 10.5.5.100 as root.1478786782661 (c1, k60). This i believe is open hab
as well as
New client connected from 10.5.5.100 as mosqsub/3606-HomeAutoma (c1, k60). This is the mosquittosub to view the topic entrys.
and this
1478787015: New client connected from 10.5.2.200 as mosqpub/31511-raspberry (c1, k60).
1478787015: Client mosqpub/31511-raspberry disconnected.
This is the pi connecting and updating the topic and then disconnecting… I hope that makes sense
Thanks

Any other thoughts ?

How did you install openHAB?

I installed openhab by downloading the tar from the openhab website and then extracting it and following the instructions form the openhab getting started site. all my files for open hab are in the /opt/openhab folder.

OK. If you are running on a Debian based Linux I cannot recommend strongly enough to do an apt-get install instead. It solves so many problems and it gives us on the forums a baseline of assumptions we can make, like the locations of files.

Since you did a manual install, EVERYTHING is in /opt/openhab, including the logs. So there should be a logs folder somewhere in there containing openhab.log and events.log files. These are the files you need to look in.