Extracting values from webpage

Setup, Configuration and Use Beginners
1

I would like to extract a couple of values from a webpage and include them in my openHAB UI but I have no idea where to start. The webpage in question is that shown on logging in to my Ecomon Energy Monitor and I want to get the values for Generating and Exporting. There are two variables in the webpage that hold these values which I presume are extracted from the Ecomon via a java script which I can’t see. The manufactures provide an API which extracts logged data but not instantaneous values. Any suggestions would be appreciated.

2

You could try to “get” the web page with a http-request in a string and trim away all unwanted data.

3

@lesterb

Try http1 binding with regex transformation.

Examples: here and here or here…

Or search the forum.

4

The login page and the page with the values I want are in the attached files.loginpage.txt (622 Bytes) valuespage.txt (103.7 KB)

The two values I want are skw and expd but I assume regex is only going to get the text skw and expd not the actual values that are displayed on the webpage.

Opening http://192.168.1.34/login.html requires a password, how do I automate that process to get from the login window to the next one with the values displayed?

5

Please use code fences.

6

Those are placeholders. Real data is fetched by javascript embedded in the page, from different URL, using AJAX techniques - but probably with JSON really. Then the data gets “edited” into the visible webpage for your browser to render.

That means you cannot just scrape string data off the webpage, because it is not there unitil the embedded scripts run.
If you read the scripts you should be able to find the URL to fetch the data,probably as JSON.

1 Like
7

Thank you rosska you have pointed me in the right direction. I found the .json file and sending
192.168.1.34/ecostatus.json returns the data shown in the attached file. I can retrieve this and use regex to extract the values I want.
It looks as though I can run this without having to be logged in which solves another problem.
ecostatus.txt (1.8 KB)

8

I’ve been working on other things for a while and am just getting back to taking this further.
So I’ve found that 192.168.1.34/ecostatus.json displays the data I want and a lot more that I don’t. I thought of using regex to extract the data I want but now that I do more reading it looks like I should use the HTTP binding and use JSONPATH to extract the data.
Can someone confirm this is the way to go?