Replacement of mapdb by jsondb


(Kai Kreuzer) #21

I cannot reproduce this. If I look at my json-serialized LIFX Thing, I see:

      "configuration": {
        "properties": {
          "deviceId": "1",
          "fadetime": 300
        }

and this is correctly passed to the handler.
Note: I created the Thing manually, since I don’t have a bulb at hand right now. You probably used discovery, maybe that makes a difference. Can you check your jsondb/org.eclipse.smarthome.core.thing.Thing.json file?


(Wouter Born) #22

@Kai My issue is fixed when I use build #627! :slight_smile:

With build #624 integers of other bindings were also persisted as floats in the .json file. I used the discovery and inbox to add things. E.g. with the NTP binding I have:

Build #624:

      "configuration": {
        "properties": {
          "timeZone": "Europe/Amsterdam",
          "hostname": "0.pool.ntp.org",
          "refreshInterval": 60.0,
          "refreshNtp": 30.0
        }
      },

Build #627:

      "configuration": {
        "properties": {
          "timeZone": "Europe/Amsterdam",
          "hostname": "0.pool.ntp.org",
          "refreshInterval": 60,
          "refreshNtp": 30
        }
      },

(Kai Kreuzer) #23

I have absolutely no clue why it should behave differently between those two builds, but as it is working, I won’t ask any further…


(Lolodomo) #24

My things and items are defined in config files. Is it mormal to not find them in the new json files ?
The only file that contains data is the one attached to discovery.


(Chris Jackson) #25

Yes - the storage database (be it mapdb or jsondb) stores the data that is managed through the UI. If you define your configuration in the DSL config files, then they can not be managed through the UI.


(Zirum) #26

Hi! Hope i’m not intruding to much in the important announcement section, but are trying to get my system running with the new implementation of json storage.

As far as I have understood so far, it should allow easy backup and edit (as it is plaintext) that are replicated into paperui, right? Does it mean that the old *.items files should be depricated?

As of yet, I have not known of any other way to use OH2 than by configuring items and sitemaps in the classic files, and things thru paper ui. Right now I feel even more unsure where is the right place to create and maintain my things/items/sitemaps, to avoid any further rework. yes, I know, using snapshot and so on, that must be expected. But it seems like this will be the issue for all upgrading any version of OH2, even the betas, right?

Currently, any changes to the items-file in conf folder, will force-update the json-file. Deletions in json-file, does not seem to delete from paperui. So right now, I feel like what I should do, is to add a things-file to my conf folder, and go on the classic way. That way I have full control again my system, while being able to test and review updates rather painlessly. Or would you advise differently (hard to predict future, but best guess works for me)? :slight_smile:

Any inputs much appreciated!


(Ben Clark) #27

I’m no authority in answering your question so someone might correct me in a few places, but to put it simply, there are two ways of adding an object that openHAB can use. They currently have no interaction with each other. For the normal user of openHAB, you shouldn’t notice a difference in mapDB and jsonDB, only that you’re able to see what’s going on “under the hood” now.

  1. Through the paperUI - Everything handled inside the paperUI or through internal services is saved in a database (which is now jsonDB for openHAB), any changes noticed by openHAB will be reflected in this database some how. The change to jsonDB was primarily so that people could edit or debug anything that was automatically set for them, but the usual way to edit anything set this way is to use paperUI or the console.

  2. Using text files in the conf folder - Anything in the conf folder overrides what openHAB has set internally. This means that you can quite easily define an .items, .sitemap file etc. and openHAB will be aware of these items. Because they’re in the conf folder and are editable by you only, you can’t make changes to these through the paperUI. AFAIK, sitemaps are only configurable through this way anyway. For people who are comfortable with text, this is often a much faster way of getting exactly what you want.

Which method you use is entirely up to you, openHAB here is about choice. I use a mixture of both but at the moment am leaning more towards setting things up through paperUI,. I know that a few developers, Kai included have said that they prefer textual configuration.


(Chris Jackson) #28

It’s up to you - personally I use the UI for configuration, and not the items files. The JsonDB storage works the same as the MapDB storage - it just allows you to view/edit/correct them if you need/want to.

No - this is correct. The jsondb files are not read back from disk, other than on startup. So, if you edit them manually, you should restart OH2.

The JsonDB storage is meant to be used mostly for configuration through the UI - on a day-to-day basis at least. If you want to do some mass update, or cut and paste of items etc, and you edit the files directly, then you will need to restart.


( ) #29

@chris Thanks for stating that. In between all the discussions I was silently waiting for that information. This should be emphasized for whoever might indeed want or need to hack around in there.


(Patrick) #30

I have a similar problem. But in jsonDB file the value is stored correctly and when I load configuration in thing handler I get a decimal.

...
"configuration": {
  "properties": {
	...
	"openHabAlias": "home",
	"openHabPort": 8080,
	"zwayServerProtocol": "http",
	...
  }
},
...

In thing handler: getConfig().get(BRIDGE_CONFIG_OPENHAB_PORT) results in 8080.0 and getConfig().get(BRIDGE_CONFIG_OPENHAB_PORT).getClass().getSimpleName() results in BigDecimal

Is that intentional or an error?

In thing configuration the port is declared as integer: <parameter name="openHabPort" groupName="openhab" type="integer" ...


(Jim) #31

I just wanted to follow up about using the REST API as a backup/restore mechanism. I played around with it quite a bit tonight, and while it can be done, it is rather piecemeal. For example, I exported all the things into a file, and all the items into a file. I thought maybe if I first created the things by importing them, even one at a time, it would create the things in the new jsonDB, and it does. Sort of. The POST to create the thing will not take the json exported by the API as is. There is some error of redundant channels. I looked but did not see anything that looked redundant, so maybe this is a bug. I removed the channels from the json thing and then was able to POST to create the thing. I then did a PUT to update the thing to add the channels back in and strangely, this worked, using the same JSON that failed to POST. I was able to recover my zwave things and their configuration values this way.

Next I started to play with groups. A group is a type of item. Once again the json produced by the GET of items will not import into the new instance. This could be operator error though, as I have nested groups and that started to get confusing as well.

Only 1 item can be imported (POST/PUT) at a time, so to take this further would take some coding as opposed to dropping a large json file into the API. Same goes for things, and just about every other API endpoint it seems. This makes using the API for backup/re-import to a new instance a non-trivial task. This is probably not how it was intended to be used, but would this be a logical feature request?


(Greg Rapp) #32

FYI, I was able to use the REST API to backup and restore my items and links. I dumped my items and links out of my mapdb OH install via REST and stuck them in files (items.json & links.json).
I then whipped up a quick Node.js script to read the JSON dumps and then re-add my items and links to my jsondb OH install via REST.

Items:

var fs = require('fs');
var unirest = require('unirest');

var obj = JSON.parse(fs.readFileSync('items.json', 'utf8'));

obj.forEach(function(value){
  unirest.put(`http://openhab.local/rest/items/${value['name']}`)
  .auth({user: '', pass: '', sendImmediately: true})
  .type('json')
  .send(value)
  .end(function (response) {
    console.log(`${value.name} ${response.status}`);
  })
});

Links:

var fs = require('fs');
var unirest = require('unirest');

var obj = JSON.parse(fs.readFileSync('links.json', 'utf8'));

obj.forEach(function(value){
  unirest.put(`http://openhab.local/rest/links/${value['itemName']}/${value['channelUID']}`)
  .auth({user: '', pass: '', sendImmediately: true})
  .end(function (response) {
    console.log(`${value.itemName} ${response.status}`);
  })
});

(Mark Herwege) #33

Did you find any further info on this? I have exactly the same problem in a binding I am developing. It is also a parameter for a port defined as an integer. I now get around it in my code by dropping everything after the decimal dot, but it should not return this way in the first place. The value in the jsondb seems to be correct at first. I did however notice it getting the .0 appended when an update was written to the jsondb.


(Patrick) #34

I’ve changed the configuration readout. I don’t read all the parameters individually, but map the configuration completely to a class: ZWayBridgeConfiguration config = getConfigAs(ZWayBridgeConfiguration.class);. This works.

getConfigAs() is defined in BaseThingHandler.


(davorf) #35

Hello!

I’m going to update from older nightly to a current one (with JsonDB replacement), and I was wondering if I should delete old database before updating? Until now, I was deleting contents of userdata/cache userdata/tmp and runtime folders, and then extracting a new nightly. Where is an old MapDB database stored? Should I delete Z-Wave XML nodes and rediscover them, in order to get them saved to JsonDB?

Best regards,
Davor


(SiHui) #36

I did that, caused no problems (because it is not used anymore after the update)

/userdata/mapdb

Don’t delete /userdata/persistence/mapdb if you are using mapdb as a persistence service!

No, not necessary.


(davorf) #37

Hello!

Thank you for your help. I’m using MapDB as primary persistence service, so, your insight about folder locations is really helpful.

Best regards,
Davor


(Matthew Valeri) #38

I didn’t delete the database, no issues either.


(Wouter Born) #39

@Mherwege I don’t think the issue has to do with JSON DB. Although I have the same issue, and thought it was resolved with a newer version of OH, it hasn’t. I’m also seeing the issue for the LIFX fadetime configuration property (on restarts) regardless of whether .0 is appended in org.eclipse.smarthome.core.thing.Thing.json.

In PR #2633 a similar issue is addressed. I’ve done some analysis on this yesterday, see the last post of PR #2633. Perhaps @Kai knows who is familiar with the Configuration logic and when the properties of a Configuration should be normalized to integers?


(John Chiasson) #40

The only caveat I have with json is it doesn’t allow comments.