Get Data from ESP8266 via webserver or mqtt?

Setup, Configuration and Use Migration
1

Hi there,
I am searching for several hours now, but can’t find the right clue, or example for my problem.
I am super new in openhab, just got it running a few weeks back and connected some devices and created some pages.
But, I am rather expierienced with Arduinos and therefore have some lying around.
For this reason I wanted to build a weather station based on a NodeMCU (ESP8266). So far I created a webserver from with the ESP8266, which shows me the temperature and humidity (will be extended later on with wind speed, pressure, etc.).

Now My Problem is, how do I get this data in my openhab?
So far I added the http binding on my openhab.
I created a Thing with the Basis-URL and it says it is online.

Thing_created
Thing_created945×779 16.9 KB

I also created a channel in the thing and the code is as follows:

UID: http:url:------------- (here is the normal UID,I just deleted it for the post)
label: HTTP_GET_Test
thingTypeUID: http:url
configuration:
authMode: BASIC
ignoreSSLErrors: true
baseURL: http://192.168.178.56/
delay: 0
stateMethod: GET
refresh: 3
commandMethod: GET
timeout: 3000
bufferSize: 2048
location: Büro
channels:

  • id: temperature
    channelTypeUID: http:number
    label: temperature
    description: null
    configuration:
    mode: READONLY
    unit: °C

I have now expierence with get commands and don’t really know where to start, or which commands to put where. I asked Bing copilot and it said I should create a http.cfg file in the openhab services folder. I also read something about regex, but I can’t reall put this together. What I am missing, is a simple example on how to get a data point from a website in openhab.

In my sourcecode for the webserver I have these Ids:

/head>
body>
h2>ESP8266 DHT Server
p>
i class=“fas fa-thermometer-half” style=“color:#059e8a;”>
span class=“dht-labels”>Temperature
span id=“temperature”>23.40
sup class=“units”>°C
/p>
p>
i class=“fas fa-tint” style=“color:#00add6;”>
span class=“dht-labels”>Humidity
span id=“humidity”>49.00
sup class=“units”>%
/p>
/body>

Is there someone who can help me out, or show me a link where it is described in detail and within the openhabian UI.

Thank you

2

The easiest would be to add MQTT to the ESP and have that publish the sensor readings over MQTT. OH then subscribes through the MQTT binding. That approach is kind of the standard approach for ESPs integration with home automation hubs (MQTT is one of the main transport protocols used in this problem space).

But since you already have the ESP serving up a web page the HTTP binding is the next best choice.

When posting configs, please click the code tab and paste the full text you find there in code fences.

```
code goes here
```

Screen shots are not terribly useful and without code fences the text gets formatted in unhelpful ways. This works great for stuff like HTML too as it preserves the < >.

For this use case you wouldn’t use the command properties at all. Each Channel represents two way communication. Device → OH is handled with the “state” properties and OH → Device is handled by the “command” properties. You don’t have a case where OH is telling the ESP to do something (e.g. turn on a light) so you have no commands at all. You just want OH to periodically pull and parse the web page hosted by the ESP.

Pretty much all of the AI chatbots are worse than useless when it comes to OH configs.

The HTTP binding is one of the hardest to work with because it’s one of the lower level bindings.

At a high level:

  • ignore all the command stuff, you’re not going to be using them

  • in your Channel config the two most important fields will be:

    • the “State URL Extension” which I presume you don’t need here. This is where you would add the last part of the URL to get to the web page where the status data can be retrieved and it’s useful in cases where the URL for the state is different from the URL for commands, or the Thing needs to hit several URLs across multiple Channels.

    • the second filed is the “State Transformation”. This is where you put the stuff that parses and extracts the piece of data you need for that Channel. In this case I’d expect either REGEX (REGEX:.*temperature>(.*)</span>.*) or, assuming XHTML XPath could be used (this doesn’t look like XHTML but it’s hard to tell without code fences).

Transformations are add-ons that must be installed from the Add-on Store.

Here is a relatively simple HTTP example that pulls information from the National Weather Service:

UID: http:url:nws
label: National Weather Service Forecast
thingTypeUID: http:url
configuration:
  authMode: BASIC
  ignoreSSLErrors: false
  baseURL: https://api.weather.gov/gridpoints/{Station Name}/{gridpoints,gridpoints}/forecast
  delay: 0
  stateMethod: GET
  refresh: 6000
  commandMethod: GET
  contentType: application/json
  timeout: 3000
  bufferSize: 2048
channels:
  - id: Updated
    channelTypeUID: http:datetime
    label: Updated
    description: ""
    configuration:
      stateTransformation: JSONPATH:$.properties.updated

I’m just showing the one Channel. the full Thing can be found at NWS Weather API - #2 by rlkoshak. You can see that it;s all the same end point so all I really need to configure is the stateTransformation to extract that one piece of data from the data pulled down. Note that it’s JSON formatted so I use the JSONPATH transform to extract the DateTime when the forecast was last updated (in this case).

Another simple one that has both a command and state config on the Channel:

UID: http:url:adguard
label: AdGuard
thingTypeUID: http:url
configuration:
  headers:
    - ""
  ignoreSSLErrors: true
  stateMethod: GET
  refresh: 30
  commandMethod: POST
  timeout: 3000
  authMode: BASIC_PREEMPTIVE
  baseURL: https://{adguard}/control
  password: password
  delay: 0
  contentType: application/json
  bufferSize: 2048
  username: username
location: Global
channels:
  - id: protection_status
    channelTypeUID: http:switch
    label: Protection Status
    description: ""
    configuration:
      mode: READWRITE
      onValue: "true"
      commandTransformation: 'JINJA:{"protection_enabled": {{value}}}'
      offValue: "false"
      stateExtension: /status
      commandExtension: /dns_config
      stateTransformation: JSONPATH:$.protection_enabled

This interacts with the AdGuard API to keep track of whether ad blocking is enabled and to be able to turn ad blocking on and off through OH.

Note that except for the JINJA transform stuff in { } has been redacted and you would need to replace them with your values to have a working Thing.

1 Like
3

Another approach you could take is to integrate your project with the ESPHome framework and use the ESPHome binding to integrate it with openHAB (disclaimer: I’m the binding author)

I understand that some of the DIY pleasure might be lost by moving to ESPHome, but IMHO offloading some of the sensor gruntwork just leaves time for a more fancy weather station project :slight_smile:

Cheers

2 Likes
4

Thank you both for your comments, especially rlkoshak for your detailed one. I will look into this and also try to figure out how to post code properly.
It seems I was on the right track, just didn’t really know how to get the final things straight.

I will also look into the ESP home binding.

Either way, when I solved my problem I will let you know.

5

7 posts were split to a new topic: MQTT Discussion

7

I don’t know how complex the mqtt binding on the arduino side will be. The webserver approach for me seems elegant, because I can debug it from the arduino side quite well, because I just have to look at the browser if it works :sweat_smile:.
But I will see what’s the best way to do it, I hope.

8

MQTT is usually really simple to set up and use. It can be as simple as:

  1. Import the library (Paho, HiveMQ, etc.)
  2. One or two lines of code to connect to the broker
  3. One line of code to publish a message to a topic (one line of code per sensor, or just one line of code if you post all the sensors in one message)

The advantage of MQTT or something like the ESPHome native protocol over HTTP is it’s push instead of pull. While that doesn’t matter much for something like a temperature sensor, it’s not great for something like a motion sensor.

It’s also way more energy efficient for the microcontroller to remain asleep, wake up, push out a bunch of sensor readings, then go back to sleep than it is to stay always online connected to WiFi hosting a web page that gets hit every X minutes.

But if you don’t have a sensor where you need timely reporting and are mains powered these concerns don’t matter.

The “best” way is always going to depend on your specific requirements. But MQTT is very popular and used for a lot of good reasons.

11

Yeah, only sending data when there is an actual change seems like a better idea.

12

So, I added the MQTT Binding and now I want to create a bridge.
There it asks me for a broker Hostname/IP and I have a Error:COMM.
I tried openhabian and openhabianpi, but it did not work. For openhabian it says Connection refused…

So apart from the binding I need the mosquitto broker, is that right? and I can install this broker by getting right on the raspi itself and after startup:

sudo apt install mosquitto
sudo systemctl start mosquitto
sudo systemctl daemon-reload
sudo systemctl enable mosquitto
13

Without any additional setting, there will be no user and no password to use the broker, so leave these parameters empty.

14

So, I got the broker running by installing mosquitto, and I have my arduino messaging via mqtt. Now I have to figure out how this message gets into the openhab.

15

That’s really easy, as you already know the topic and the payload :slight_smile:

  • install mqtt addon
  • create a mqtt bridge (i.e. the link to mosquitto) and set host to the ip of mosquitto (or use localhost as mosquitto is at the same machine)
  • create a generic mqtt thing for your esp device and set it to use the mqtt bridge
  • add a channel to the generic mqtt thing and set the stateTopic to the topic, which is used to publish your data. The type of channel depends on the type of data
  • if your data is plain number or text, you’re done with the “hard” part and all what is left is to link the channel to an apropriate Item. If you put your data to a special format, let’s say JSON, you will need to use an additional addon like JSONPath to disassemble the data to its discrete values.
16

First welcome to the fun

I see you have decided to go the MQTT route.

It is important to familiarise yourself with the openHAB terminology MQTT Things and Channels - Bindings | openHAB

This is what a text config can look like

image
image1293×274 20.5 KB

You should use the Main UI to configure however they use the same terms to set things up.

So first add a broker bridge that I see you are having trouble with.

I see that you may be using openHABian and then tried to install not through the openHABian Menu

Mosquitto may need to be configured to allow openHAB to connect as the newest version may need authentication to make is safer to use.

I would recommend setting it up to allow any connection on the loopback connection 127.0.0.1
Then point openHAB to this IP and get conncetion as the first step.

Then ADD a thing and use the working broker to connect
then add a channel to your think like number temperature stateTopic=“your 8266 publish topic”

17

Yeah,
I tried it again today and it worked:
My Code was as follows:

UID: mqtt:topic:xxxxxxxxxxxxx
label: ESP8266
thingTypeUID: mqtt:topic
configuration: {}
bridgeUID: mqtt:broker:xxxxxxxx
location: Arbeitszimmer
channels:
  - id: esp2866_temperature
    channelTypeUID: mqtt:number
    label: Temperature
    description: ""
    configuration:
      stateTopic: esp/dht/temperature

And here is the temperature reading shown:

image
image991×387 13.3 KB

Now I have my first simple Example and can work with that.

Again Thanks for your help. Last hint I needed was from @Udo_Hartmann “…link the channel to an appropriate item…”