MQTT not talking to OH2

lbarnard · October 21, 2017, 8:08am

Hi guys,

Thanks for looking at this post as I know there are 100’s of similar posts.

So I have set up OH2, MQTT, and the ESP device. I have followed almost every tutorial out there (this is my third day trying to figure it out).

MQTT talks to the ESP perfect via MQTT.fx and I can change the state of my LED with no issus.

My RaspberryPi IP is 192.168.1.44

Also do you normally have to reboot after changes made to any file?

So here is my code, please point out if anything is missing or I have made a stupid error!!

/etc/openhab2/services/mqtt.cfg:

mqtt:broker.url=tcp://192.168.1.44:1883
#mqtt:broker.clientId=openhabian
#mqtt:broker.user=
#mqtt:broker.pwd=
#mqtt:broker.qos=1
#mqtt:broker.retain=true
#mqtt:broker.async=false

I have commented out the rest apart from URL as I saw that in a tutorial.

.

/var/log/mosquitto/mosquitto.log:

1508572673: Client openhabian disconnected.
1508572673: New client connected from 192.168.1.44 as openhabian (c1, k60).
1508572683: New connection from 192.168.1.44 on port 1883.
1508572683: Client openhabian already connected, closing old connection.
1508572683: Client openhabian disconnected.
1508572683: New client connected from 192.168.1.44 as openhabian (c1, k60).
1508572693: New connection from 192.168.1.44 on port 1883.

.

/etc/mosquitto/mosquitto.conf:

# Place your local configuration in /etc/mosquitto/conf.d/
#
# A full description of the configuration file is at
# /usr/share/doc/mosquitto/examples/mosquitto.conf.example

pid_file /var/run/mosquitto.pid

persistence true
persistence_location /var/lib/mosquitto/

log_dest file /var/log/mosquitto/mosquitto.log

include_dir /etc/mosquitto/conf.d

.

/etc/openhab2/items/home.items:
As you will notice im using MK Smart House as my guide line.

Switch MKSprinklerSystemZone1 "Sprinklers Zone 1" <water> {mqtt=">[broker:MK-SmartHouse/utilities/MK-SprinklerSystem1:command:ON:Z1ON],>[broker:MK-SmartHouse/utilities/MK-SprinklerSystem1:command:OFF:Z1OFF]"}
Switch MKSprinklerSystemZone2 "Sprinklers Zone 2" <water> {mqtt=">[broker:MK-SmartHouse/utilities/MK-SprinklerSystem1:command:ON:Z2ON],>[broker:MK-SmartHouse/utilities/MK-SprinklerSystem1:command:OFF:Z2OFF]"}
Switch MKSprinklerSystemZone3 "Sprinklers Zone 3" <water> {mqtt=">[broker:MK-SmartHouse/utilities/MK-SprinklerSystem1:command:ON:Z3ON],>[broker:MK-SmartHouse/utilities/MK-SprinklerSystem1:command:OFF:Z3OFF]"}

.

/etc/openhab2/sitemaps/home.sitemap:

...
        Frame label="Outside"
        {
                Switch item=MKSprinklerSystemZone1
                Switch item=MKSprinklerSystemZone2
                Switch item=MKSprinklerSystemZone3
                Switch item=SprinklerCycle
                Switch item=SprinklerSystemSchedule

        }
...

And this is the status of the MQTT:


â mosquitto.service - Mosquitto MQTT Broker
   Loaded: loaded (/lib/systemd/system/mosquitto.service; enabled)
   Active: active (running) since Sat 2017-10-21 15:20:02 AWST; 40min ago
     Docs: man:mosquitto(8)
           https://mosquitto.org/
 Main PID: 438 (mosquitto)
   CGroup: /system.slice/mosquitto.service
           ââ438 /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf

Dim · October 21, 2017, 8:20am

Try to set a clientId in your mqtt.cfg and check the contents of your /var/lib/openhab2/config/org/openhab/mqtt.config for stale entries.

broker.url=tcp://192.168.1.44:1883
broker.clientId=something_new

(no need to use mqtt: in front of everyline in your /etc/openhab2/services/mqtt.cfg)

OH 2.1 or 2.2?

lbarnard · October 21, 2017, 8:40am

Hi Dim,

Thank you for your speedy response!

I have changed both files to match:

Before

broker.async="false"
broker.clientId="openhabian"
broker.pwd=""
broker.retain="true"
broker.url="tcp://192.168.1.44:1883"
broker.user=""
mymqtt.clientId="openhabian"
mymqtt.qos="1"
mymqtt.url="tcp://192.168.1.44:1883"
service.pid="org.openhab.mqtt"

After:

broker.async="false"
broker.clientId="oh2"
broker.pwd=""
broker.retain="true"
broker.url="tcp://192.168.1.44:1883"
broker.user=""
mymqtt.clientId="openhabian"
mymqtt.qos="1"
mymqtt.url="tcp://192.168.1.44:1883"
service.pid="org.openhab.mqtt"

Also:

broker.url=tcp://192.168.1.44:1883
broker.clientId=oh2

openHAB 2.1.0-1 (Release Build)

I have rebooted the RasPi & ESP, but still not luck using OH2 to control the ESP

Cheers

lbarnard · October 21, 2017, 8:48am

Also this is in the log:


1508575530: Socket error on client MK-SmartHouseDevice1, disconnecting.
1508575530: New connection from 192.168.1.33 on port 1883.
1508575530: New client connected from 192.168.1.33 as MK-SmartHouseDevice1 (c1, k60).
1508575620: Client MK-SmartHouseDevice1 has exceeded timeout, disconnecting.

192.168.1.33 is the IP address and the Device name of the ESP so it must be getting something? But its timing out, is there a reason for that?

I also took out the following from /var/lib/openhab2/config/org/openhab/mqtt.config:

mymqtt.clientId="openhabian"
mymqtt.qos="1"
mymqtt.url="tcp://192.168.1.44:1883"

Dim · October 21, 2017, 8:54am

There are stale entries stored in your /var/lib/openhab2/config/org/openhab/mqtt.config coming from previous trials.

Currently, you are initiating 2 parallel connections from your OH2.1 installation to the broker. This will create a conflict.

Follow these steps to troubleshoot:

stop OH2 service (systemctl stop openhab2)
delete file /var/lib/openhab2/config/org/openhab/mqtt.config
Fix your /etc/openhab2/services/mqtt.cfg

broker.url=tcp://192.168.1.44:1883
broker.clientId=oh2

Update your OH2 item configs to use the correct broker name (broker or whatever you like) that you set in the mqtt.cfg
Start OH2 and monitor logs (systemctl start openhab2). OH2 will recreate the mqtt.config file (in var) from the contents of the mqtt.cfg (in etc)

Make sure that each client is using a unique ID to connect to the broker.

Dim · October 21, 2017, 9:00am

there could be several reasons… from config errors to network stability to ESP code issues
try to stabilize the OH2 -> MQTT Broker connection first and then try to troubleshoot the remote ESP client connection after.

For starters, I would change the clientId on the ESP from MK-SmartHouseDevice1 to something else that doesn’t include the minus (-) character (e.g. ESP001). I don’t think that this is the root-cause but you never know

Another thing to try is:

listener 1883 0.0.0.0

in your /etc/mosquitto/mosquitto.conf

lbarnard · October 21, 2017, 9:18am

Okay…so I followed your steps fine, and I started again and it has created a new /var/lib/openhab2/config/org/openhab/mqtt.config file! Which now contains:

broker.clientId="oh2"
broker.url="tcp://192.168.1.44:1883"
service.pid="org.openhab.mqtt"

So I guess that is correct…right?

I have also reflashed the ESP ClientID to ESP001 as you suggest, thanks for pointing that out!

So I also restart the RasPi, but still no luck!

How do I go about doing the listener? Just insert that into the mosquitto.conf file, and monitor from MQTT.fx, or the log?

This is the log:


1508577152: mosquitto version 1.4.14 terminating
1508577184: mosquitto version 1.4.14 (build date Mon, 10 Jul 2017 23:48:43 +0100) starting
1508577184: Config loaded from /etc/mosquitto/mosquitto.conf.
1508577184: Opening ipv4 listen socket on port 1883.
1508577184: Opening ipv6 listen socket on port 1883.
1508577197: New connection from 192.168.1.33 on port 1883.
1508577197: New client connected from 192.168.1.33 as ESP001 (c1, k60).
1508577225: New connection from 192.168.1.34 on port 1883.
1508577225: New client connected from 192.168.1.34 as d3f4e8c8f515484ca73664022b42ab23 (c1, k60).
1508577273: New connection from 192.168.1.44 on port 1883.
1508577273: New client connected from 192.168.1.44 as oh2 (c1, k60).
1508577307: Client ESP001 has exceeded timeout, disconnecting.
1508577307: Socket error on client ESP001, disconnecting.
1508577307: New connection from 192.168.1.33 on port 1883.
1508577307: New client connected from 192.168.1.33 as ESP001 (c1, k60).
1508577397: Client ESP001 has exceeded timeout, disconnecting.
1508577397: Socket error on client ESP001, disconnecting.
1508577397: New connection from 192.168.1.33 on port 1883.
1508577397: New client connected from 192.168.1.33 as ESP001 (c1, k60).
1508577487: Client ESP001 has exceeded timeout, disconnecting.
1508577487: Socket error on client ESP001, disconnecting.
1508577487: New connection from 192.168.1.33 on port 1883.
1508577487: New client connected from 192.168.1.33 as ESP001 (c1, k60).
1508577577: Client ESP001 has exceeded timeout, disconnecting.

192.168.1.34 is the PC that I’m on

Thank you for your continuing support!!

Dim · October 21, 2017, 9:34am

yeap. that looks good and in the log I see that the connection from OH2 to the Broker is stable (no reconnects etc) for the 192.168.1.44 IP.

Now you need to find out that is going wrong with your ESP client… it does not hold a stable connection to the Broker… IP 192.168.1.33 gets disconnected due to timeouts…

Check your ESP code. Are you using WiFi.mode(WIFI_STA); before Wifi.connect(....); ?
which MQTT library are you using?

You could try a sample basic MQTT code for your ESP to eliminate possible hardware issues.

lbarnard · October 21, 2017, 9:41am

Here is the code that I am flashing to the ESP. And WiFi.mode(WIFI_STA) is in there mid way down.

Where would i find a basic MQTT code? Do you mean just to turn on and off an LED?

I have flashed a wifi connect test to the ESP and it seems stable?


#include <ESP8266WiFi.h>
#include <MQTTClient.h>
#include <ESP8266WebServer.h>
#include <ESP8266mDNS.h>
#include <ESP8266HTTPUpdateServer.h>


const char* ssid = "*****";
const char* password = "*****"; 

/* Web Updater Settings */
// Host Name of Device
const char* host = "MK-SprinklerSystem1";

// Path to access firmware update page (Not Neccessary to change)
const char* update_path = "/firmware";

// Username to access the web update page
const char* update_username = "admin";

// Password to access the web update page
const char* update_password = "password";

/* MQTT Settings */
// Topic which listens for commands
char* subscribeTopic = "MK-SmartHouse/utilities/MK-SprinklerSystem1"; 

//MQTT Server IP Address
const char* server = "192.168.1.44";

//Unique device ID 
const char* mqttDeviceID = "ESP001"; 



int channel1 = 14;
int channel2 = 12;
int channel3 = 13;

//webserver 
ESP8266WebServer httpServer(80);
ESP8266HTTPUpdateServer httpUpdater;

//MQTT
WiFiClient net;
MQTTClient client;

unsigned long lastMillis = 0;

//Connect to WiFI and MQTT
void connect();

//Setup pins, wifi, webserver and MQTT
void setup() 
{
  // set pin modes
  pinMode(channel1, OUTPUT);
  digitalWrite(channel1, LOW);

  pinMode(channel2, OUTPUT);
  digitalWrite(channel2, LOW);

  pinMode(channel3, OUTPUT);
  digitalWrite(channel3, LOW);

 WiFi.mode(WIFI_STA);    //<------this is here??
  
  WiFi.begin(ssid, password);
  client.begin(server, net);
  client.onMessage(messageReceived);

  connect();

  MDNS.begin(host);

  httpUpdater.setup(&httpServer, update_path, update_username, update_password);
  httpServer.begin();

  MDNS.addService("http", "tcp", 80);
}

//Connect to wifi and MQTT
void connect() 
{
  while (WiFi.status() != WL_CONNECTED) 
  {
    delay(1000);
  }

  while (!client.connect(mqttDeviceID)) 
  {
    delay(1000);
  }

  client.subscribe(subscribeTopic);
}

void loop() 
{
  // MQTT Loop
  client.loop();
  delay(10);

  // Make sure device is connected
  if(!client.connected()) 
  {
    connect();
  }

  httpServer.handleClient();

}

// Change the state of a relay based on the MQTT Message
void messageReceived(String &topic, String &payload) 
{
    String msgString = payload;

  if (msgString == "Z1ON")
  {
    digitalWrite(channel1, HIGH);
    delay(250);
  }
  else if (msgString == "Z1OFF")
  {
    digitalWrite(channel1, LOW);
    delay(250);
  }
  else if (msgString == "Z2ON")
  {
    digitalWrite(channel2, HIGH);
    delay(250);
  }
  else if (msgString == "Z2OFF")
  {
    digitalWrite(channel2, LOW);
    delay(250);
  }
  else if (msgString == "Z3ON")
  {
    digitalWrite(channel3, HIGH);
    delay(250);
  }
  else if (msgString == "Z3OFF")
  {
    digitalWrite(channel3, LOW);
    delay(250);
  }
}

Dim · October 21, 2017, 9:44am

ESP code looks good… (checking again now)

just add the listener config line in your /etc/mosquitto/mosquitto.conf and restart the mosquitto service.
Make sure that the websockets protocol is not enabled on your Mosquitto 1.4.14 listener.

At the bottom of your mosquitto.conf file:

[...]
# Networking
listener 1883 0.0.0.0
protocol mqtt

lbarnard · October 21, 2017, 9:53am

Phew! Thats good.

I also just ran a hello world network test on the ESP and that came back fine, and was stable.

I’ve added the code and restarted the mosquitto service.

Log shows this:


1508579351: New connection from 192.168.1.33 on port 1883.
1508579351: New client connected from 192.168.1.33 as ESP001 (c1, k60).
1508579441: Client ESP001 has exceeded timeout, disconnecting.

As I changed the ESP to ESP001, I dont need to change that anywhere else do I?

There is also TeamViewer if you want to take a look for yourself? Ha Ha, as i’m so confused why this isnt working!

Dim · October 21, 2017, 10:04am

1508579351 = GMT: Saturday, October 21, 2017 9:49:11 AM
1508579441 = GMT: Saturday, October 21, 2017 9:50:41 AM
(https://www.epochconverter.com/)

So… 90 seconds after the initial connection, the ESP client times out…

Try to see if there is any kind of keepalive option that you can set.
I can’t pin point the issue… it seems to be related to the ESP code that does not keep the connection alive.

lbarnard · October 21, 2017, 10:06am

Okay, ill look into that now, this is the ESP im using, but ill try a few things out.

Is it normal that the MQTT.fx continues to work?

Again thanks for your help, and ill get back to you soon.

ESP-8266 Dev Board Layout

Dim · October 21, 2017, 10:09am

yeah. the MQTT.fx is just a software client for the MQTT Server.

You have 3 clients on the server (the MQTT Broker):
OH2
MQTT.fx
ESP

The first 2 seem to be keeping a stable connection while the 3rd times out every 90 seconds (I saw this also in your previous logs)

lbarnard · October 21, 2017, 11:56am

Still trying to find a solution to this, have you got any hints?

Cheers

lbarnard · October 26, 2017, 7:30am

Hey, im still trying to figure out what is wrong here.

I open (see screenshot) and when i push the button the openhabian is able to see it working so that makes me think its still a config error! but when i change the switches on the openhab web page, i dont get any results!

Please help ha ha!

Cheers

Dim · October 27, 2017, 2:46pm

post you items configs to check them (post also your current mqtt.cfg)

What do you mean “when i push the button” ? which button, where?

lbarnard · October 29, 2017, 6:16am

Hi mate,

Hope the screenshots below help!

I mean when i push the attached button to the ESP, please see photo.

Dim · October 29, 2017, 9:41am

so: the ESP works ok and sends the messages to the broker…

now, post your item config to check it.
your mqtt.cfg looks ok (you should remove the mqtt: in front but it should work like this also)

(if you still have the items configured like in the first post: change them to inbound configurations <… you currently have them outbound)

lbarnard · October 29, 2017, 2:52pm

Hi thanks again for your reply.

Okay great, i have removed the mqtt: part.

Is this the file you mean? as you can see i have tried a few different ways.

and how to i change them to inbound?

.