How can I retrieve data from a not well formed html webpage with the http-binding?

I just tried to replace the first line with nothing and the second line as full line.

But this is not working

AC40corrected.replace("<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\"","")

If I append the replace after the first two replaces it works. Is it not possible to replace text in variable?

The String with the replaced text gets returned. The original String remains unchanged. You would need to use:

AC40corrected = AC40corrected.replace("<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\"","")

I’ve written it now in a very long oneliner.

But I still get this version error!

Thank for the tip with the variable. I should have seen that, but I was concentrated with escaping the double quotes :slight_smile:

The missing “>” has to be a different place and perhabs nothing to do with the version.

Here comes the complete error log

==> /var/log/openhab2/openhab.log <==
2017-03-21 22:04:14.978 [ERROR] [t.internal.XsltTransformationService] - transformation throws exception
javax.xml.transform.TransformerException: javax.xml.transform.TransformerException: com.sun.org.apache.xml.internal.utils.WrappedRuntimeException: The declaration for the entity "HTML.Version" must end with '>'.
        at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:749)[:1.8.0_112]
        at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:351)[:1.8.0_112]
        at org.eclipse.smarthome.transform.xslt.internal.XsltTransformationService.transform(XsltTransformationService.java:82)
        at org.eclipse.smarthome.core.transform.actions.Transformation.transform(Transformation.java:43)
        at sun.reflect.GeneratedMethodAccessor61.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:1.8.0_112]
        at java.lang.reflect.Method.invoke(Method.java:498)[:1.8.0_112]
        at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter.invokeOperation(XbaseInterpreter.java:1085)[145:org.eclipse.xtext.xbase:2.9.2.v20160428-1452]
        at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter.invokeOperation(XbaseInterpreter.java:1060)[145:org.eclipse.xtext.xbase:2.9.2.v20160428-1452]
        at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter._invokeFeature(XbaseInterpreter.java:1046)[145:org.eclipse.xtext.xbase:2.9.2.v20160428-1452]
        at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter.invokeFeature(XbaseInterpreter.java:991)[145:org.eclipse.xtext.xbase:2.9.2.v20160428-1452]
        at org.eclipse.smarthome.model.script.interpreter.ScriptInterpreter.invokeFeature(ScriptInterpreter.java:114)[129:org.eclipse.smarthome.model.script:0.9.0.b4]
        at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter._doEvaluate(XbaseInterpreter.java:901)[145:org.eclipse.xtext.xbase:2.9.2.v20160428-1452]
        at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter._doEvaluate(XbaseInterpreter.java:864)[145:org.eclipse.xtext.xbase:2.9.2.v20160428-1452]
        at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter.doEvaluate(XbaseInterpreter.java:223)[145:org.eclipse.xtext.xbase:2.9.2.v20160428-1452]
        at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter.internalEvaluate(XbaseInterpreter.java:203)[145:org.eclipse.xtext.xbase:2.9.2.v20160428-1452]
        at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter._doEvaluate(XbaseInterpreter.java:826)[145:org.eclipse.xtext.xbase:2.9.2.v20160428-1452]
        at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter.doEvaluate(XbaseInterpreter.java:263)[145:org.eclipse.xtext.xbase:2.9.2.v20160428-1452]
        at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter.internalEvaluate(XbaseInterpreter.java:203)[145:org.eclipse.xtext.xbase:2.9.2.v20160428-1452]
        at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter._doEvaluate(XbaseInterpreter.java:446)[145:org.eclipse.xtext.xbase:2.9.2.v20160428-1452]
        at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter.doEvaluate(XbaseInterpreter.java:227)[145:org.eclipse.xtext.xbase:2.9.2.v20160428-1452]
        at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter.internalEvaluate(XbaseInterpreter.java:203)[145:org.eclipse.xtext.xbase:2.9.2.v20160428-1452]
        at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter.evaluate(XbaseInterpreter.java:189)[145:org.eclipse.xtext.xbase:2.9.2.v20160428-1452]
        at org.eclipse.smarthome.model.script.runtime.internal.engine.ScriptImpl.execute(ScriptImpl.java:77)[130:org.eclipse.smarthome.model.script.runtime:0.9.0.b4]
        at org.eclipse.smarthome.model.script.engine.ScriptExecutionThread.run(ScriptExecutionThread.java:42)[129:org.eclipse.smarthome.model.script:0.9.0.b4]
Caused by: javax.xml.transform.TransformerException: com.sun.org.apache.xml.internal.utils.WrappedRuntimeException: The declaration for the entity "HTML.Version" must end with '>'.
        at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.getDOM(TransformerImpl.java:578)[:1.8.0_112]
        at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:739)[:1.8.0_112]
        ... 24 more
Caused by: com.sun.org.apache.xml.internal.utils.WrappedRuntimeException: The declaration for the entity "HTML.Version" must end with '>'.
        at com.sun.org.apache.xalan.internal.xsltc.dom.XSLTCDTMManager.getDTM(XSLTCDTMManager.java:427)[:1.8.0_112]
        at com.sun.org.apache.xalan.internal.xsltc.dom.XSLTCDTMManager.getDTM(XSLTCDTMManager.java:215)[:1.8.0_112]
        at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.getDOM(TransformerImpl.java:556)[:1.8.0_112]
        ... 25 more
2017-03-21 22:04:15.015 [ERROR] [ore.transform.actions.Transformation] - Error executing the transformation 'XSLT': transformation throws exception
2017-03-21 22:04:15.021 [ERROR] [.script.engine.ScriptExecutionThread] - Rule 'Process AC40': The argument 'command' must not be null or empty.

An tis is the prepared xml for the transforming.

2017-03-21 22:05:59.862 [INFO ] [.smarthome.model.script.Process AC40] - Beginn Auswertung
2017-03-21 22:05:59.885 [INFO ] [.smarthome.model.script.Process AC40] -
 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">

<html>
<head>

<title> Address List </title>
<style type="text/css">
body { background-color:#C0C0C0; color:#0000FF; }
</style>
</head>

<body>

<h2>Liste der ausgew&auml;hlten FG-Adressen:</h2>
----------------------------------------------------------------------------------------------------
<br/><pre><b>TA300A00B91       </b>Dachgeschosstemperatur              Analogeingang mit Grenzwert
<br/>21-03-17  22:59:08            <a href="fga/TA300A00B91     " target="_blank">22.76119 </a>
<br/></pre>
----------------------------------------------------------------------------------------------------
<br/><pre><b>TL300E01B01       </b>Aussenluftlufttemperatur            Analogeingang mit Grenzwert
<br/>21-03-17  22:59:08            <a href="fga/TL300E01B01     " target="_blank">6.592045 </a>
<br/></pre>
----------------------------------------------------------------------------------------------------
<br/><pre><b>TL300E02B01       </b>Zulufttemperatur                    Analogeingang mit Grenzwert
<br/>21-03-17  22:59:08            <a href="fga/TL300E02B01     " target="_blank">21.39304 </a>
<br/></pre>
----------------------------------------------------------------------------------------------------
<br/><pre><b>TL300E51B01       </b>Raumablufttemperatur                Analogeingang mit Grenzwert
<br/>21-03-17  22:59:08            <a href="fga/TL300E51B01     " target="_blank">21.89055 </a>
<br/></pre>
----------------------------------------------------------------------------------------------------
<br/><pre><b>TL300G51B01       </b>Fortlufttemperatur                  Analogeingang mit Grenzwert
<br/>21-03-17  22:59:08            <a href="fga/TL300G51B01     " target="_blank">16.54229 </a>
<br/></pre>
</body>
</html>
2017-03-21 22:05:59.891 [INFO ] [.smarthome.model.script.Process AC40] - Ende Auswertung


Try eliminating that first line entirely. I don’t think XSLT requires a DOCTYPE to work.

Done

No the error log has changed

2017-03-21 22:13:01.093 [ERROR] [t.internal.XsltTransformationService] - transformation throws exception
javax.xml.transform.TransformerException: javax.xml.transform.TransformerException: com.sun.org.apache.xml.internal.utils.WrappedRuntimeException: The entity "auml" was referenced, but not declared.
        at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:749)[:1.8.0_112]
        at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:351)[:1.8.0_112]
        at org.eclipse.smarthome.transform.xslt.internal.XsltTransformationService.transform(XsltTransformationService.java:82)
        at org.eclipse.smarthome.core.transform.actions.Transformation.transform(Transformation.java:43)
        at sun.reflect.GeneratedMethodAccessor61.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:1.8.0_112]
        at java.lang.reflect.Method.invoke(Method.java:498)[:1.8.0_112]
        at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter.invokeOperation(XbaseInterpreter.java:1085)[145:org.eclipse.xtext.xbase:2.9.2.v20160428-1452]
        at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter.invokeOperation(XbaseInterpreter.java:1060)[145:org.eclipse.xtext.xbase:2.9.2.v20160428-1452]
        at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter._invokeFeature(XbaseInterpreter.java:1046)[145:org.eclipse.xtext.xbase:2.9.2.v20160428-1452]
        at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter.invokeFeature(XbaseInterpreter.java:991)[145:org.eclipse.xtext.xbase:2.9.2.v20160428-1452]
        at org.eclipse.smarthome.model.script.interpreter.ScriptInterpreter.invokeFeature(ScriptInterpreter.java:114)[129:org.eclipse.smarthome.model.script:0.9.0.b4]
        at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter._doEvaluate(XbaseInterpreter.java:901)[145:org.eclipse.xtext.xbase:2.9.2.v20160428-1452]
        at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter._doEvaluate(XbaseInterpreter.java:864)[145:org.eclipse.xtext.xbase:2.9.2.v20160428-1452]
        at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter.doEvaluate(XbaseInterpreter.java:223)[145:org.eclipse.xtext.xbase:2.9.2.v20160428-1452]
        at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter.internalEvaluate(XbaseInterpreter.java:203)[145:org.eclipse.xtext.xbase:2.9.2.v20160428-1452]
        at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter._doEvaluate(XbaseInterpreter.java:826)[145:org.eclipse.xtext.xbase:2.9.2.v20160428-1452]
        at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter.doEvaluate(XbaseInterpreter.java:263)[145:org.eclipse.xtext.xbase:2.9.2.v20160428-1452]
        at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter.internalEvaluate(XbaseInterpreter.java:203)[145:org.eclipse.xtext.xbase:2.9.2.v20160428-1452]
        at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter._doEvaluate(XbaseInterpreter.java:446)[145:org.eclipse.xtext.xbase:2.9.2.v20160428-1452]
        at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter.doEvaluate(XbaseInterpreter.java:227)[145:org.eclipse.xtext.xbase:2.9.2.v20160428-1452]
        at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter.internalEvaluate(XbaseInterpreter.java:203)[145:org.eclipse.xtext.xbase:2.9.2.v20160428-1452]
        at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter.evaluate(XbaseInterpreter.java:189)[145:org.eclipse.xtext.xbase:2.9.2.v20160428-1452]
        at org.eclipse.smarthome.model.script.runtime.internal.engine.ScriptImpl.execute(ScriptImpl.java:77)[130:org.eclipse.smarthome.model.script.runtime:0.9.0.b4]
        at org.eclipse.smarthome.model.script.engine.ScriptExecutionThread.run(ScriptExecutionThread.java:42)[129:org.eclipse.smarthome.model.script:0.9.0.b4]
Caused by: javax.xml.transform.TransformerException: com.sun.org.apache.xml.internal.utils.WrappedRuntimeException: The entity "auml" was referenced, but not declared.
        at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.getDOM(TransformerImpl.java:578)[:1.8.0_112]
        at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:739)[:1.8.0_112]
        ... 24 more
Caused by: com.sun.org.apache.xml.internal.utils.WrappedRuntimeException: The entity "auml" was referenced, but not declared.
        at com.sun.org.apache.xalan.internal.xsltc.dom.XSLTCDTMManager.getDTM(XSLTCDTMManager.java:427)[:1.8.0_112]
        at com.sun.org.apache.xalan.internal.xsltc.dom.XSLTCDTMManager.getDTM(XSLTCDTMManager.java:215)[:1.8.0_112]
        at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.getDOM(TransformerImpl.java:556)[:1.8.0_112]
        ... 25 more
2017-03-21 22:13:01.129 [ERROR] [ore.transform.actions.Transformation] - Error executing the transformation 'XSLT': transformation throws exception
2017-03-21 22:13:01.136 [ERROR] [.script.engine.ScriptExecutionThread] - Rule 'Process AC40': The argument 'command' must not be null or empty.


I eleminated this line as well

<h2>Liste der ausgew&auml;hlten FG-Adressen:</h2>

No I’am missing the error logs. :grinning:

But what I was trying to do? Ah I wanted to transfer some important values to an item.

This still fails

2017-03-21 22:18:02.420 [ERROR] [.script.engine.ScriptExecutionThread] - Rule 'Process AC40': The argument 'command' must not be null or empty.


It doesn’t like this part of the header string.

That is the HTML escaped character for the ä.

XML does not include everything that HMTL does when it comes to escaped characters and such so it is likely this is not part of standard XML.

Try replace("&auml;", "ä").

Look for other places where there might be an HMTL escaped character and replace those as well.

You missed my answer

Print out the result of the transform and make sure you are getting something.

It looks preaty good

2017-03-21 22:28:04.449 [INFO ] [.smarthome.model.script.Process AC40] - Ende Auswertung
2017-03-21 22:28:04.520 [INFO ] [.smarthome.model.script.Process AC40] - 21.64179
2017-03-21 22:28:04.525 [ERROR] [.script.engine.ScriptExecutionThread] - Rule 'Process AC40': The argument 'command' must not be null or empty.


this is the last line to transfer the value to the item

TL300E02B01.sendCommand (AC40_TL300E02B01)

the line has the following error

The method or field TL300E02B01 is undefined

Do I have to change the dot in the number to an comma?

The error is saying that the Item TL300E02B01 doesn’t exist. Make sure you don’t have a typo in on that Item as well.

The dot is fine. You would probably have problems if you used the comma.

Make sure there is no leading or trailing spaces in the String returned from the transform.

May be I have a trailing space.

How can I strip the string? Is there a command like .strip ?

.trim

Here is my item-file

String AC40_Raw { http="<[AC40CacheRaw:60000:REGEX((.*))]" }

Number TL300E02B01  "Zulufttemperatur Lueftung [%.1f °C]"   <temperature>

followed by the rule file

rule "Process AC40"
when
    Item AC40_Raw received update
then
	logInfo("Process AC40", "Beginn Auswertung")
//    val AC40corrected = AC40_Raw.state.toString.replace("<br>", "<br/>").replace("<META HTTP-EQUIV=\"Refresh\" CONTENT=\"30\">", "").replace("<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\"","").replace("\"http://www.w3.org/TR/html4/loose.dtd\">", "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtd\">")
    val AC40corrected = AC40_Raw.state.toString.replace("<br>", "<br/>")
    AC40corrected = AC40corrected.replace("<META HTTP-EQUIV=\"Refresh\" CONTENT=\"30\">", "")
    AC40corrected = AC40corrected.replace("<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\"","")
    AC40corrected = AC40corrected.replace("\"http://www.w3.org/TR/html4/loose.dtd\">", "")
    AC40corrected = AC40corrected.replace("<h2>Liste der ausgew&auml;hlten FG-Adressen:</h2>","")
    logInfo("Process AC40", AC40corrected)
    logInfo("Process AC40", "Ende Auswertung")
    val AC40_TL300E02B01 = transform("XSLT", "AC40_TL300E02B01.xsl", AC40corrected)
    logInfo("Process AC40", AC40_TL300E02B01)

    // Process and post the tags and values to the appropriate Items
 
    TL300E02B01.sendCommand (AC40_TL300E02B01)

end

the rule file is not error free

Here it is

Your are right. Indeed the http binding is working.

But don’t tell me that it was easy

I never said it was easy. Dealing with raw data from websites and other devices is hard even for experienced programmers. The fact that you are dealing with a not well-formed HTML that you wanted to use XSLT on made it even more of difficult.

It would have been easier had you taken my hint and used Regex instead.

What version of Designer are you using? Is it 0.9? If so that is your problem. There is a major problem in the 0.9 version that is causing it to no recognize Items. You should use the 0.8 version.

Thats a good question. I downloaded both, but in the running Designer I can’t see on which Version I’am, but I supose I’am a lucky man and startet immediately with the newest version to have less trouble, Only with my openhabian I was a little bit to conservative. (Thats also not true, I realy was one of the first users)

On Thursday I will try to get the Modbus Binding via USB-RS485 to get to run.

What do you think about that?

Thanks a lot for your patience and help.

I don’t know much about this binding. Good luck!

Hello I have to go on with this thread.

I now have the problem, that I have Setpoints and Switches I would like to change.

So I have to change my items and rule file.

I would like to keep my

String AC40_Raw { http="<[AC40CacheRaw:60000:REGEX((.*))]" }

but instead of selecting everything I would like to do the necessary changes which in the moment are done in the rule file

when
    Item AC40_Raw received update
then
//	logInfo("Process AC40", "Beginn Auswertung")
    val AC40corrected = AC40_Raw.state.toString.replace("<br>", "<br/>")
    AC40corrected = AC40corrected.replace("<META HTTP-EQUIV=\"Refresh\" CONTENT=\"30\">", "")
    AC40corrected = AC40corrected.replace("<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\"","")
    AC40corrected = AC40corrected.replace("\"http://www.w3.org/TR/html4/loose.dtd\">", "")
    AC40corrected = AC40corrected.replace("<h2>Liste der ausgew&auml;hlten FG-Adressen:</h2>","")

directly.

The sendcommands in my rule file

    TA300A00B91.sendCommand (transform("XSLT", "AC40_TA300A00B91.xsl", AC40corrected).trim)
    TL300E01B01.sendCommand (transform("XSLT", "AC40_TL300E01B01.xsl", AC40corrected).trim)
    TL300E02B01.sendCommand (transform("XSLT", "AC40_TL300E02B01.xsl", AC40corrected).trim)

should be placed in the items file directly

Number TA300A00B91{ http="<[AC40_Raw:10000:XSLT(AC40_TA300A00B91.xsl)]" }

And later on I would like to define the http post

Number TA300A00B91{ http="<[AC40_Raw:10000:XSLT(AC40_TA300A00B91.xsl)] >[CHANGED:POST:http:////192.168.1.200/cgi-fga/fga/fgacmd/TA300A00B91?pw=0123456789&amp;nv=18]" }

The final “18” should become replaced by the changed value.

What do I have to change to get in running?

Is it possible to get a multiple selection with a regex?

A regex can select multiple things but I don’t think OH is able to take advantage of it.

See the HTTP binding for how to configure the binding to send requesters. One thing I notice is for outgoing you should use > instead of <