REST text encoding

Hi all,

to get the item labels I would like the REST API, but I´m not able to display special characters in my web page correctly. Is my suspicion correct that the REST API does not use UTF-8/Unicode for the data transmission?

If yes - do you know what encoding/code page is used and or if this can be configured?

My request headers:

thanks for any hint & kind regards,
Patrik

I tried this under 2.1.0 and didn’t see a problem:

[{"link":"http://localhost:8080/rest/items/Ruby_Diamond","state":"NULL","type
":"String","name":"Ruby_Diamond","label":"ÐÆ","tags":[],"groupNames":[]}]

Did you verify your items file is correctly encoded?

If you still see an issue, you could try adding the charset to the rest request header:

curl -X GET --header "Accept: application/json; charset=utf-8" --header "Content-type=text/plain; charset=UTF-8" localhost:8080/rest/items/

Thank you for your reply. The files are encoded UTF-8 without BOM. To verify I use the REST UI from OH and jQuery.

Strange - it does not work; also tried to escape the chars => same results:

Number ItemAirQualityOzon "Ozon O₃\u2083 (100 µg/m\u00B3)"

=>

"label": "Ozon O?? (100 �g/m�)",

Hi all,

I did some more research using fiddler to see why I can not get special characters to my web page. Here my findings.

Item

Number ItemTemperaturBastelzimmer "G\u00E4stezimmer [%.1f °C]"

Request

GET /rest/items/ItemTemperaturBastelzimmer HTTP/1.1
Host: haus:8080
Connection: keep-alive
Accept: */*
X-Requested-With: XMLHttpRequest
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36
Content-Type: application/json
Referer: http://haus:8080/static/Visualization/MultiLineChart_1.html?Group=GroupTemperaturZimmer&days=1
Accept-Encoding: gzip, deflate
Accept-Language: de,de-CH;q=0.8,en-US;q=0.6,en;q=0.4
Cookie: Squeezebox-player=00%3A04%3A20%3A28%3A65%3A91

Response

HTTP/1.1 200 OK
Date: Fri, 11 Aug 2017 16:22:15 GMT
Content-Type: application/json
Content-Length: 349
Server: Jetty(9.2.19.v20160908)

{"link":"http://haus:8080/rest/items/ItemTemperaturBastelzimmer","state":"24.1","stateDescription":{"pattern":"%.1f  C","readOnly":false,"options":[]},"type":"Number","name":"ItemTemperaturBastelzimmer","label":"G stezimmer","category":"temperatur","tags":[],"groupNames":["GroupPersistrrd4jHour","GroupTemperaturOGAverage","GroupTemperaturZimmer"]}

Finding

As an “application/json” response I expect UTF-8 encoding; which it actually is - but the character \u00E4 (ä) is sent to the client as \uFFFD (label=G\uFFFDstezimmer). Thus jetty used a replacement character here … basic UI displays the item correctly:

2017-08-11 18_32_50-Raumklima Stufe 1

Any idea what might be wrong?

with kind regards,
Patrik

You should check the encoding of the Basic UI page.

Not sure what I need to do; the page shows UTF-8 in its header, the source displays as

	<span  class="mdl-form__label">
		G&auml;stezimmer 
	</span>

Other code in the page seems to confirm the utf-8 encoding:

<div class="mdl-form__value">♀♂</div>

I do no see any REST request for the item label - thus the label seems to be generated by the server.

Well, now this is interesting. I duplicated your item.

String German "G\u00E4stezimmer"

Now, if I pull the list of items via /rest/items, I get the right text in the right encoding. The umlauted A is represented by 0xC3A4:

000001f0h: 22 6C 61 62 65 6C 22 3A 22 47 C3 A4 73 74 65 7A ; "label":"Gästez
00000200h: 69 6D 6D 65 72 22 2C 22 74 61 67 73 22 3A 5B 5D ; immer","tags":[]

However, if I pull the specific item by itself via /rest/items/German, I get this instead, with the special character represented by 0xE4:

000000a0h: 22 3A 22 47 65 72 6D 61 6E 22 2C 22 6C 61 62 65 ; ":"German","labe
000000b0h: 6C 22 3A 22 47 E4 73 74 65 7A 69 6D 6D 65 72 22 ; l":"Gästezimmer"
1 Like

Thank you - I´m happy that this can be reproduced. Do you know if I should open an issue for this? If so do you know where that belongs to; is https://github.com/openhab/openhab-distro/issues the right place?

with kind regards,
Patrik

Probably should, but I’m not sure distro is the right place.

Looks to me like the REST api lives in ESH.

So unless @Kai wants to provide better knowledge, I’d say start there.

Yes, all stuff about the REST API should be reported at ESH - thanks!

thanks :slight_smile:

Hi all,

just an update; this problem is unfortunately still present in openHAB 2.2:

2017-12-22 17_30_42-Raumklima Stufe A

With kind regards,
Patrik

… use the full list of item REST api call as a workarround; this endpoint works well:

// Get the item label from the list of all items received from 
// openHAB; this is necessary with OH 2.2 due to encoding problems 
// if we query the item directly.
// 
// Required API:
// - http://code.google.com/p/jsonpath/
function GetItemLabelFromJSON(json, itemName) {
    var path = "$.[?(@.name==\"" + itemName + "\")].label";
    return jsonPath(json, path);
}

function GetAllItemInformationJSON() {
    var allItemsUrl = baseURL.concat("rest/items?recursive=false");
    return GetData(allItemsUrl);
}

with kind regards,
Patrik