Mqttany + mcp23017

Add-ons Bindings
, ,
1

Hello, as i am trying to set up MCP23017 on OH3 and not able to do that with MCP23017 binding, i try to do it with MqttAny.py. the problem is that i guess somthing is wrong with my configuration. In The logs it seems that connection is successfull to the mcp but i dont get any events to the gpios. and nothing is mentioned in the log that any of the pins where allocated.

this is my config:
mosquitto_sub -v -h 127.0.0.1 -p 1883 -t ‘#’

mqtt:
host: ‘127.0.0.1’
port: 1883

i2c:
mcp23017-1:
device: ‘mcp23017’
bus: 1
address: 0x20
mcp23017:
batch-01:
pin: [01,02,03,04,05,06,07,08,09,10,11,12,13,14,15,16]
name: ‘{pin_id}’
direction: ‘output’
initial state: OFF

mcp23017-2:
device: ‘mcp23017’
bus: 1
address: 0x21
mcp23017:
batch-01:
pin: [01,02,03,04,05,06,07,08,09,10,11,12,13,14,15,16]
name: ‘{pin_id}’
direction: ‘output’
initial state: OFF

mcp23017-3:
device: ‘mcp23017’
bus: 1
address: 0x22
mcp23017:
batch-01:
pin: [01,02,03,04,05,06,07,08,09,10,11,12,13,14,15,16]
name: ‘{pin_id}’
direction: ‘output’
initial state: OFF
mcp23017-5:
polling interval: 1
device: ‘mcp23017’
bus: 1
address: 0x24
mcp23017:
batch-01:
pin: [01,02,03,04,05,06,07,08,09,10,11,12,13,14,15,16]
name: ‘{pin_id}’
direction: ‘input’
resistor: ‘pullup’
initial state: OFF

mcp23017-6:
polling interval: 1
device: ‘mcp23017’
bus: 1
address: 0x25

mcp23017:
  batch-01:
    pin: [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16]
    name: '{pin_id}'
    direction: 'input'
    resistor: 'pullup'
    initial state: OFF

mcp23017-7:
polling interval: 1
device: ‘mcp23017’
bus: 1
address: 0x26
mcp23017:
pin-a1:
pin: 01
name: ‘{pin_id}’
direction: ‘input’
resistor: ‘pullup’
initial state: OFF

and this is the log:
2021-09-15 11:00:45,081 [INFO] [core ] [core ] MQTTany 0.14.3 starting
2021-09-15 11:00:45,215 [INFO] [core ] [core.gpio ] Detected board: RASPBERRY_PI_3B
2021-09-15 11:00:45,217 [DEBUG] [core ] [core.gpio ] Detected GPIO character device
2021-09-15 11:00:45,219 [DEBUG] [core ] [core.gpio ] Detected sysfs GPIO interface
2021-09-15 11:00:45,221 [DEBUG] [core ] [config ] Loading config file
2021-09-15 11:00:45,402 [DEBUG] [core ] [core ] Loading module ‘mqtt’
2021-09-15 11:00:45,561 [DEBUG] [core ] [core ] Module ‘mqtt’ is a communication module
2021-09-15 11:00:45,564 [DEBUG] [core ] [mqtt ] Parsing config
2021-09-15 11:00:45,566 [DEBUG] [core ] [mqtt ] Config loaded successfully
2021-09-15 11:00:45,568 [DEBUG] [core ] [core ] Module ‘mqtt’ loaded successfully
2021-09-15 11:00:45,570 [DEBUG] [core ] [bus ] Module ‘mqtt’ added as a receiver
2021-09-15 11:00:45,574 [DEBUG] [core ] [bus ] Module ‘mqtt’ added as a transmitter
2021-09-15 11:00:45,576 [DEBUG] [core ] [core ] Loading module ‘i2c’
2021-09-15 11:00:45,605 [DEBUG] [core ] [core ] Module ‘i2c’ is an interface module
2021-09-15 11:00:45,615 [DEBUG] [core ] [i2c ] Parsing config
2021-09-15 11:00:45,620 [DEBUG] [core ] [i2c ] Config loaded
2021-09-15 11:00:45,622 [DEBUG] [core ] [i2c ] Configuring MCP23017 ‘mcp23017-1’ at address 0x20 on bus ‘/dev/i2c-1’
2021-09-15 11:00:45,624 [DEBUG] [core ] [i2c ] Configuring MCP23017 ‘mcp23017-2’ at address 0x21 on bus ‘/dev/i2c-1’
2021-09-15 11:00:45,627 [DEBUG] [core ] [i2c ] Configuring MCP23017 ‘mcp23017-3’ at address 0x22 on bus ‘/dev/i2c-1’
2021-09-15 11:00:45,629 [DEBUG] [core ] [i2c ] Configuring MCP23017 ‘mcp23017-5’ at address 0x24 on bus ‘/dev/i2c-1’
2021-09-15 11:00:45,632 [DEBUG] [core ] [i2c ] Configuring MCP23017 ‘mcp23017-6’ at address 0x25 on bus ‘/dev/i2c-1’
2021-09-15 11:00:45,634 [DEBUG] [core ] [i2c ] Configuring MCP23017 ‘mcp23017-7’ at address 0x26 on bus ‘/dev/i2c-1’
2021-09-15 11:00:45,636 [DEBUG] [core ] [core ] Module ‘i2c’ loaded successfully
2021-09-15 11:00:45,638 [DEBUG] [core ] [bus ] Module ‘i2c’ added as a publisher
2021-09-15 11:00:45,642 [DEBUG] [core ] [bus ] Module ‘i2c’ added as a subscriber
2021-09-15 11:00:45,649 [INFO] [core ] [core ] Module ‘mqtt’ started successfully
2021-09-15 11:00:45,655 [DEBUG] [mqtt ] [mqtt ] Creating MQTT client
2021-09-15 11:00:45,658 [INFO] [core ] [core ] Module ‘i2c’ started successfully
2021-09-15 11:00:45,662 [DEBUG] [mqtt ] [mqtt ] Attaching callbacks
2021-09-15 11:00:45,665 [DEBUG] [core ] [bus ] Starting Message Bus Receive thread
2021-09-15 11:00:45,665 [DEBUG] [i2c ] [i2c ] Opening I2C bus streams
2021-09-15 11:00:45,666 [DEBUG] [mqtt ] [mqtt ] Queuing connect event
2021-09-15 11:00:45,670 [DEBUG] [core ] [bus ] Starting Message Bus Transmit thread
2021-09-15 11:00:45,670 [DEBUG] [mqtt ] [mqtt ] Starting MQTT client thread
2021-09-15 11:00:45,671 [DEBUG] [i2c ] [i2c.mcp23017 ] Writing initial values to registers of MCP23017 ‘mcp23017-1’ at address 0x20 on bus ‘/dev/i2c-1’
2021-09-15 11:00:45,678 [DEBUG] [i2c ] [i2c ] Successfully setup MCP23017 ‘mcp23017-1’
2021-09-15 11:00:45,681 [DEBUG] [i2c ] [i2c.mcp23017 ] Writing initial values to registers of MCP23017 ‘mcp23017-2’ at address 0x21 on bus ‘/dev/i2c-1’
2021-09-15 11:00:45,686 [DEBUG] [i2c ] [i2c ] Successfully setup MCP23017 ‘mcp23017-2’
2021-09-15 11:00:45,689 [DEBUG] [i2c ] [i2c.mcp23017 ] Writing initial values to registers of MCP23017 ‘mcp23017-3’ at address 0x22 on bus ‘/dev/i2c-1’
2021-09-15 11:00:45,691 [INFO] [mqtt ] [mqtt ] Connected to broker ‘127.0.0.1:1883’
2021-09-15 11:00:45,693 [DEBUG] [mqtt ] [mqtt ] Resuming previous session
2021-09-15 11:00:45,694 [DEBUG] [i2c ] [i2c ] Successfully setup MCP23017 ‘mcp23017-3’
2021-09-15 11:00:45,696 [DEBUG] [mqtt ] [mqtt ] Heartbeat
2021-09-15 11:00:45,697 [DEBUG] [i2c ] [i2c.mcp23017 ] Writing initial values to registers of MCP23017 ‘mcp23017-5’ at address 0x24 on bus ‘/dev/i2c-1’
2021-09-15 11:00:45,703 [DEBUG] [i2c ] [i2c ] Successfully setup MCP23017 ‘mcp23017-5’
2021-09-15 11:00:45,706 [DEBUG] [i2c ] [i2c.mcp23017 ] Writing initial values to registers of MCP23017 ‘mcp23017-6’ at address 0x25 on bus ‘/dev/i2c-1’
2021-09-15 11:00:45,712 [DEBUG] [i2c ] [i2c ] Successfully setup MCP23017 ‘mcp23017-6’
2021-09-15 11:00:45,714 [DEBUG] [i2c ] [i2c.mcp23017 ] Writing initial values to registers of MCP23017 ‘mcp23017-7’ at address 0x26 on bus ‘/dev/i2c-1’
2021-09-15 11:00:45,719 [DEBUG] [i2c ] [i2c ] Successfully setup MCP23017 ‘mcp23017-7’
2021-09-15 11:00:45,722 [DEBUG] [i2c ] [i2c ] Starting polling timer with interval of 60s

2

So you are not using the openHAB MCP23017 binding.

You have installed MQTTany Successfully with 6 MCP23017?

Can you connect to the MQTT broker with a different client like MQTT Explorer and see messages from MQTTany?

3

Yes, i set up 6 mcp’s and disabled them in OH binding and jus try to run on MqttAny.

if i try to subscribe to topics
mosquitto_sub -v -h 127.0.0.1 -p 1883 -t ‘#’

i get:
openhabian/lwt Online
openhabian/mcp23017-1/address 0x20
openhabian/mcp23017-2/address 0x21
openhabian/mcp23017-3/address 0x22
openhabian/mcp23017-5/address 0x24
openhabian/mcp23017-6/address 0x25
openhabian/mcp23017-7/address 0x26

i expect that there should be more topics associated to Pins…

4

Hi @Audrius_S, looks like they were successfully configured and initialized. Farther down the log when the I2C binding starts you can see the devices are setup and configuration registers are written for each device.

Continuing your question from the PM you also sent me about this: interrupts for MCP230xx devices are planned but not currently implemented in MQTTany unfortunately. I am not sure when I will get this added, hopefully a more aggressive polling frequency will be sufficient for your application in the mean time.

Some other suggestions:
Code fences, your configuration and log are barely readable as they are.

```yaml
MQTTany config files are written in YAML
Use language specific fences like this
```

```text
Generic code like logs can be put in
plain text fences like this
```

Posting a question and then 3 hours later sending a PM is a good way to annoy someone. We’re all volunteers here, be patient, we’ve got lives and work and our own projects as well.

5

When you press a button connected to MCP23017 do you see a new topic show up?

6

Thank you for the answer, and sorry for the impatience :slight_smile:
Would it be possible to have polling interval in milliseconds instead of seconds? because 1sec in my case is a bit too much…i use this mcp approach in my house for turning on the lights with wall switches, so 1 sec is way too much…
I could do the changing in the code myself, just dont want to have it in mind when doing updates :slight_smile:

7

You could change the code to support fractional seconds for an interval, yes. Assuming that your Pi is running the I2C bus at 100kbps you should be able to approach 300 reads per second of all 8 devices, though practically that may not work and would spam your MQTT broker.

If you know any Python it might even be easier to implement the interrupt detection in MQTTany. The concept is fairly simple and interrupt detection I already implemented for GPIO pins.

If you want to try fast polling, change int to float on line 62 of mqttany/modules/i2c/common.py. This should allow you to enter something like 0.25 for a polling interval of 250ms.

8

yes, yesterday i did some tests with float and it works, but as you mentioned the problem then gets to the mqtt broker side…
i have checked the pi4j mcp23017 implementation and their solution is based on checking the interrupt register and sending callbacks just to those pins that had those interrupts… this approach on mqttany would save tons of requests to the broker…
i am more a .net guy so python is not my best friend, but having some spear time i will try to play with it.

this is the snippet from pi4j:

                         int pinInterruptA = device.read(REGISTER_INTF_A);
                        // validate that there is at least one interrupt active on port A
                        if (pinInterruptA > 0) {
                            // read the current pin states on port A
                            int pinInterruptState = device.read(REGISTER_GPIO_A);

                            // loop over the available pins on port B
                            for (Pin pin : MCP23017Pin.ALL_A_PINS) {
                                int pinAddressA = pin.getAddress() - GPIO_A_OFFSET;

                                // is there an interrupt flag on this pin?
                                //if ((pinInterruptA & pinAddressA) > 0) {
                                    // System.out.println("INTERRUPT ON PIN [" + pin.getName() + "]");
                                    evaluatePinForChangeA(pin, pinInterruptState);
                                //}
                            }
                        }

as i understand, this part of code in mqttany should be improved:


 if self._setup:
            self._log.debug("Polling all pins")
            self.read_gpio()
            for pin in self._pins:
                if pin is not None:
                    pin.publish_state()

other aproach could be, that each pin would store in memory laststate value and send message to the broker just if lastvalue and currrent value does not match. so this part of code could be modified:

 def publish_state(self) -> None:
        state = self.state_log
        common.publish_queue.put_nowait(
            PublishMessage(path=self._path, content=TEXT_STATE[state])
        )
9

Great information! I think setting up an interrupt trigger on a GPIO pin that triggers a function is the way to go. Said function would read the interrupt register on the device in question and trigger a publish for each pin that changed.

10

i did a quick fix and it worked (for now):
def publish_state(self) → None:
state = self.state_log
if(self._last_state !=TEXT_STATE[state]):
self._last_state= TEXT_STATE[state]
common.publish_queue.put_nowait(
PublishMessage(path=self._path, content=TEXT_STATE[state])
)
so mqtt broker is no more flooded with messages.

But i got one of a million situations, that my raspbery pi i2c interface died and it caused some weird behaviour. these are the errors i got in the logs:
Sep 19 11:15:33 openhabian python3[679]: 2021-09-19 11:15:33,716 [ERROR] [i2c.mcp23017 ] I2C error on bus ‘/dev/i2c-1’: [Errno 5] Input/output error
Sep 19 11:15:33 openhabian python3[679]: 2021-09-19 11:15:33,748 [WARN] [i2c.mcp23017 ] Failed to read from MCP23017 ‘mcp23017-3’ at address 0x22 on I2C bus ‘/dev/i2c-1’
Sep 19 11:27:12 openhabian python3[679]: 2021-09-19 11:27:12,177 [ERROR] [i2c.mcp23017 ] I2C error on bus ‘/dev/i2c-1’: [Errno 110] Connection timed out
Sep 19 11:27:12 openhabian python3[679]: 2021-09-19 11:27:12,181 [WARN] [i2c.mcp23017 ] Failed to read from MCP23017 ‘mcp23017-1’ at address 0x20 on I2C bus ‘/dev/i2c-1’
Sep 19 11:27:12 openhabian python3[679]: 2021-09-19 11:27:12,238 [ERROR] [i2c.mcp23017 ] I2C error on bus ‘/dev/i2c-1’: [Errno 5] Input/output error
Sep 19 11:27:12 openhabian python3[679]: 2021-09-19 11:27:12,240 [WARN] [i2c.mcp23017 ] Failed to read from MCP23017 ‘mcp23017-1’ at address 0x20 on I2C bus ‘/dev/i2c-1’

and what happened next, is that it tiggered my input pins and caused some outputs to turn on…
first idea that came to my mind - some bug in the code of mqttany, but was not able to identify any…

what i understand has happend in this situation is that somehow while reading gpios ,a zero value was returned and that was interpreted as a normal trigger of the input pin.

the only thing, that comes to my mind is that this part of the code, would put some protection if same situation appear again:

int pinInterruptA = device.read(REGISTER_INTF_A);

if instead of just checking the registry of gpios, mqttany would validate that interrupt was really trigered it would be like a double check that this was not some kind of fake event.

11

Your previous state check looks like a good interim solution. Is that working for you?

I don’t really understand the rest of your post though. Are you testing code to catch the interrupt signal when getting these errors? They look like MQTTany was no longer able to access /dev/i2c-0.
You go on to talk about getting a zero response, from what?
Reading the interrupt register and publishing only those pins with a flag set would be the only way I would do interrupts for these devices.

12

  1. Yes, the previuos state check works like a charm. Its runing now in production in my house and this change took away the load from the mqtt broker. So that is a good news :slight_smile:

  2. No, id did not try to do anything with the chip interrupts as the above described solution seemed to do the job for me.

  3. i will try to explain my story :slight_smile:
    I have a running solution on rasberry that has 48 inputs (wall switches) and 48 outputs (lamps, doors, lawn water, etc.). Each input control ref. output. (input #1 control output #1 and so on).
    Two days ago i started this new solution with MQTTany running in my house, and put away my old solution that was based on mcp23017 OH binding. It was running successfully for about 24 hours with no problem at all and all of the sudden all of my outputs switched ON (lamps, my garage opened, etc).
    i checked the logs and found thoses errors:
    "I2C error on bus ‘/dev/i2c-1’: [Errno 5] Input/output error"
    then i restarted my Rasberry and got another errors for all of my MCPs:
    No ack from MCP23017 ‘mcp23017-1’ at address 0x20 on I2C bus '/dev/i2c-1’
    Failed to setup MCP23017 'mcp23017-1’

so basicly somehow physical i2c interface died on my raspberry, that caused those errors described above. i did couple more restarts but that did not help. So then i put my SD card to a different raspberry and it now works again with no problem at all.

The only problem that i am woried now - how to prevent this happening again, or saying in other words, how to be ready for this situation if it happens again. how to make sure that my outputs are not switched on if some other problem related to i2c appear.

i checked your code expecting to find a bug, but i did not find any. So what comes to my mind, that before the death of the i2c it somehow returned closed state for the inputs that was interpreted as normal interrupt and the state was sent to the mqtt broker.

So my proposal for this solution would be to “ALWAYS check” the INTF registry before reading the input value from GPIO. That would put a double validation on top of the inputs.
The problem is that doing that in python is a bit too difficult for me :slight_smile:

13

Ok now I understand everything, thank you for the extra details.

That is quite a strange thing to have happened. I believe you are correct about it being a hardware problem (or at least not an MQTTany problem) as MCP230xx support has been very reliable since the last release.

Did you try running i2cdetect on the Pi that was giving errors? Have you changed the default bus speed? What interval are you running the polling at in MQTTany?

Do you have any errors in the log when the issue happened or does it show the pins being set?

Your proposal is a good idea but only for your use case with fast polling. In theory it might be faster than comparing to the previous state like you are doing in your patch. For what polling is designed for though, it doesn’t make sense to do this. Interrupts should publish pin states instantly when they change, and polling should periodically publish all pin states.

14

In the evening i will try to run that i2decect on that broken RPi to check what response it returns. Just wanted to mention that with this particular rpi i already faced some weird behaviour earlier, even with mcp23017 binding, so it was left for a while on the shelf and just for this new update with mqttany i tried to use it again (this rpi was running for 3-4 years almost without downtime until that unwanted behaviuor started). But my point is that when similar situation happened on OH binding, i did not get that side effect that all of my inputs turned ON. and i am pretty sure that this did not happen just because in ri4j lib there is an additional check that interrupt really happened.
because in my case a value of zero (0x00) returned in this part of the ( data = self._bus.read_word_data(self._address, register)) would cause the state change of all the input pins for given MCP. So additional step, to check the INTF would be like a preventions.

15

Ok so you’re saying that your input MCP devices read incorrect values (all the same) that cause OH to turn on all the pins of the output MCP devices? Meaning the input devices were giving errors and the output ones were still working?

16

outputs where responding to the commands if i triggered them via openhab basicUI. So commands could be send to the mcps but responses did not come back.

i checked the logs one more time and what caught my eyes the errors in MQTTany started at 2021-09-19 11:15:33,716 and stopped at 2021-09-19 11:28:11,402

2021-09-19 11:28:10,260 [WARN] [i2c.mcp23017            ] Failed to read from MCP23017 'mcp23017-1' at address 0x20 on I2C bus '/dev/i2c-1'
2021-09-19 11:28:10,266 [WARN] [i2c.mcp23017            ] Failed to read from MCP23017 'mcp23017-1' at address 0x20 on I2C bus '/dev/i2c-1'
2021-09-19 11:28:10,304 [ERROR] [i2c.mcp23017            ] I2C error on bus '/dev/i2c-1': [Errno 5] Input/output error
2021-09-19 11:28:10,305 [WARN] [i2c.mcp23017            ] Failed to read from MCP23017 'mcp23017-1' at address 0x20 on I2C bus '/dev/i2c-1'
2021-09-19 11:28:11,377 [ERROR] [i2c.mcp23017            ] I2C error on bus '/dev/i2c-1': [Errno 110] Connection timed out
2021-09-19 11:28:11,377 [ERROR] [i2c.mcp23017            ] I2C error on bus '/dev/i2c-1': [Errno 5] Input/output error
2021-09-19 11:28:11,383 [WARN] [i2c.mcp23017            ] Failed to read from MCP23017 'mcp23017-1' at address 0x20 on I2C bus '/dev/i2c-1'
2021-09-19 11:28:11,402 [WARN] [i2c.mcp23017            ] Failed to read from MCP23017 'mcp23017-2' at address 0x21 on I2C bus '/dev/i2c-1'

and in openhab logs i got all those inputs state change at 2021-09-19 11:28:11.541 straight away after the last error was received…

it seems that those mcps somehow got restarted or something like that and from that moment wall switches did not respond to the commands and outputs could be triggered just via UI.

would you consider adding a INTF register check before reading input values?

17

Interesting, after the last error do you see MQTTany publish the states to the broker that caused all the outputs to turn on?

Again, checking for interrupts when polling is not expected behaviour so I will not add it there. Polling is meant to periodically read all pin states, the default is set to 60 seconds. I made MQTTany to use with my OH setup and polling allows me to make sure OH items are in the same state as my hardware in case à device or system restarted or OH missed a message or something. Interrupts are meant to give you instantaneous updates for pins that change.

Maybe this weekend I will work on it and get you to do some testing if you’re interested.

18

Yes, state changes was received on the OH things that are bound to mqtt topics. couple of examples from the log:

2021-09-19 11:28:11.915 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'contact35' changed from CLOSED to OPEN
2021-09-19 11:28:11.919 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'contact34' changed from CLOSED to OPEN
2021-09-19 11:28:11.922 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'contact37' changed from CLOSED to OPEN
2021-09-19 11:28:11.924 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'contact36' changed from CLOSED to OPEN
2021-09-19 11:28:11.938 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'contact33' changed from CLOSED to OPEN
2021-09-19 11:28:11.941 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'contact39' changed from CLOSED to OPEN
2021-09-19 11:28:11.943 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'contact38' changed from CLOSED to OPEN
2021-09-19 11:28:11.945 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'contact40' changed from CLOSED to OPEN

about polling i agree and disagree with you :slight_smile: if polling is ment just to check the current state, when yes - interrupt registry could be skipped. but if polling is done to get interrupts, then interrupt registry reading is a must to my opinion.

the full flow of the interrupt (without polling) is done via INTA/B pins. You connect those pins to the input pins of RPi, add a callback on them and just on the event to this callback you check the INTF, determine which pins caused that trigger and when read the value. Similar flow can be done with polling without using INTA/B pins the way ri4j has done.

19

I did not know that ri4j is doing it that way, but I think that’s the wrong way to do it. I’ll be using the interrupt pins to trigger an event so we only use the I2C bus when needed.

20

interrupt pins is a correct way to do, but for people who want to move from OH mcp23017 binding to mqttany it will be a new challenge as additional wiring has to be done. sometimes it can be even impossible to use those interrupt pins.
would be nice if mqttany had 3 different aproach to get statuses:
regular polling (current one) / interrupt polling (the one that i described) / full interrupt (event based)