Migration from 2.5 to 3.x, why so difficult?

Setup, Configuration and Use Migration
1

We had a card need to be reloaded (2.5 was notorious for eating itself and needing to be reloaded on OpenHABian). So, loaded the latest image (from today), and of course it goes in with OH3. Restored the existing 2.5 backup, and although it claims it completes the restore, when it comes up, there is no config (no THINGS - all lost). Tried this a couple times with the same result.

So, I then re-imaged, and on boot (once initial load completed), I selected the option to downgrade to 2.5 (this time, thereā€™s no config, itā€™s just the OH3>OH2 software only).

Once on 2.5, I restore the 2.5 backup, and all my things reappear. So, I then tell it to upgrade to 3 (which is really 3.1), and once it completes, again, theyā€™re all gone.

I donā€™t understand why this was specifically built to be so complicated. In an ideal world I should be able to restore my 2.5 onto 3. But, OK if I canā€™t do that, I should AT LEAST be able to restore a 2.5 backup ON 2.5, then in-place upgrade to 3 and have the config carry over.

This is a very basic config, as OpenHAB is merely being used as a HomeKit bridge for a device not HomeKit enabled (HAI OmniLink). Thatā€™s it, nothing special, yet being made so complicated.

Does anyone have any idea why the in-place upgrade from 2.5 to 3 is choosing to wipe out everything each time, and how that can be corrected?

OpenHabian image: v1.6.5 (downloaded today)

2

No. Probably no software in the world allows for that when there have been major changes to the software and its config between versions.

It isnā€™t, itā€™s straightforward in fact. But you overcomplicated it yourself by not following the upgrade instructions.
I donā€™t understand why you did not upgrade in place ? Option 03 and you would have been set.

Of course, itā€™s a new install so you get the current version.
You can also have openHABian install OH 2.5 right away, then restore, then upgrade in place.
Why didnā€™t you properly read the docs before installing ?

Not with you giving only that little information.
If I was to guess, you didnā€™t understand the difference of up- or downgrading the openHABian environment vs. upgrading openHAB itself. Did you even notice the directories have changed ?
So youā€™re possibly just looking in the wrong place.
Thereā€™s of course many more potential reasons but noone can answer that if you donā€™t provide at least a log of the upgrade process and openhab.log of a startup attempt.

Image again, this time edit openhabian.conf to use the ā€˜stableā€™ branch, import your config, upgrade using menu 03.

1 Like
3

Iā€™m not sure you really processed my concerns properly, so please allow me to address each of yours.

There are many software applications that allow restore of a previous release (within reason - not like 10 releases back) and up-convert when reloaded (in the Voice world, I work on them every day). It happens all the time. However, I can accept that maybe wasnā€™t a priority for the devs and since an alternative could be to downgrade the loaded image back to 2.5, wasnā€™t the end of the world.

I didnā€™t ā€œnot follow the upgrade instructionsā€, I specifically led off with the fact the SD card ate itself and a re-imaging was required (by default, that would mean an in-place upgrade was NOT an option). Sadly, life created that particular complication, but we persevere. From there, I merely attempted a few different logical methods of getting to the latest release, one of which ultimately simulated your very suggestion (that I had already thought of). Every one of those methods erased most data (I did just discover the HomeKit config was still intact, so the real concern was all the Things were gone).

So, forced to reimage the SD, I got it back on 2.5 and successfully restored the database with all components intact. I then merely tell it to in-place upgrade (as you suggest) to 3.1 - that also blew away all the previously-built things. And if that had actually worked, I feel confident in saying I wouldnā€™t have created this thread.

Whether the image loaded with 3 and downgraded, or led off with 2.5 is completely and utterly irrelevant to this entire thread. Why? Because, I got the data restored while it was on 2.5. The pertinent question is, why (with an intact config on 2.5) did that in-place upgrade from 2.5>3 wipe out all the Things that were previously configuredā€¦

I appreciate you instantly assuming ā€œstupid userā€, and I further applaud your attempt to publicly ā€œe-flogā€ me (as well as celebrate your right to do so). Unfortunately, itā€™s mis-placed and ill-advised, as I do understand up and down (both directionally and in the form of software vs. platform x-grades), and in the detail I did know to provide, kinda already showed that actually wasnā€™t the issue. I did also notice the directories had changed, and thought that might have been obvious to not call out since I also initially stated I restored my 2.x file into the 3.x environment (which would have meant placing the restore file in the new directory path). (sigh)

I must say, I never claimed to be an expert on this platform, but I am a pretty smart and logical guy (despite your perceived assumption and attempted public degradation of my intelligence). When something seemingly simple didnā€™t work as expected, I tried multiple logical methods in an effort to collect additional data points (all yielding the exact same results). At that point, I knew enough to then reach out to others who are far smarter on this platform for their guidance. You took that ā€œvulnerabilityā€ as your shot to feel superior. Ok, cool - youā€™re superior - do you feel better now (rhetorical - donā€™t really need an answer)?

Your second-to-last sentence was the first useful and non-nasty line in your response. Cool, now I know a file name/location that might be needed so others might be more helpful and lessā€¦ wellā€¦ youā€¦

Thanks for your thought and effort in your responseā€¦ Itā€™s greatly appreciated, if not merely for itā€™s comedic value. :slight_smile:

3 Likes
4

What does that actually mean? Itā€™s gone round and deleted your xxx.things files, or what? Where are you looking? And you have no other Things before at all, just Homekit? Do you think maybe with a new version, there might be re-paring involved?

5

In the web portal (Administration>Settings>Things).
"No things yetā€¦ Things are the devices and serā€¦".

My existing Things were built with the OmniLink binding, which normally would just show all the Things with a handler error until the binding was reinstalled. So, I installed the OmniLink binding, but of course, since the Things donā€™t show up, thereā€™s nothing to provide a binding for.

Iā€™ll have to get up to speed on the new structure a bit and see if the files are gone as well. Checking now.

6

Well, you would only have xxx.things files if you originally created them. If you donā€™t know what thatā€™s about then you probably originally used PaperUI instead. Thereā€™s a different UI for this task in OH3 of course.

This one?

itā€™s a version 1 binding, and simply not available in OH3, so no config for that will ever work here.

Or this one?

7

No, the one I had was a 2.5 binding at the time (as I was able to configure it in the web portal). Looks to be the top one you posted, but Iā€™ll go grab that info as well to confirm. But I did confirm, OH3 shows my config path as etc/openhab/ and the things directory is empty (minus the readme, of course).

I had thought maybe they moved to a database instead of files in 3, so didnā€™t think too much of it at first, but when it got to the point of being on 2 with everything there, and moving to 3 and them being gone, admittedly I didnā€™t look to see if the files were still there.

Incidentally, my old OH2 directory, that things folder is also empty now.

8

As Markus said, directories have probably changed. You would need to restore your OH2 files to different places in OH3.

9

I could see that, but if I have them (my Things) all appearing in 2.5 and do an in-place upgrade, shouldnā€™t that copy/move them?

10

No idea, probably. Were they there? What does the upgrade log say, it probably records doing this kind of thing.

11

On one hand my understanding is that you wrote you used the UI to configure the things on the other hand you write

things folder is not filled by the UI.

My suggestion would be:

  • do a 2.5 install
  • shutdown OH
  • restore from backup
  • document what you have in which files and folders and UI
  • do an upgrade from OH2 to OH3
  • again clearly document what you have now including OH3 log files
    This should save time and nerves on both ends.
1 Like
12

I remembered after the fact theyā€™re JSON files now, so of course they were empty. Doh.

13

according to my logic it cannot be ā€˜of courseā€™. Then all others would have the same problem.
To analyze the problem more detailed input is required.
Bringing a car to the garage in case of the problem you need to describe the symptoms of the problem. The garage has the advantage to look into the on board unit and gets diagnostics, logs from there.
Without it itā€™s guessing.

Btw. guessing: is the disk full ? This could be root cause that files are created with size of 0 bytes.

14

Fresh load - 32gb SD - largest partition usage is 20% (most are 1%).
Filesystem Size Used Avail Use% Mounted on
/dev/root 30G 4.7G 24G 17% /
devtmpfs 800M 0 800M 0% /dev
tmpfs 961M 0 961M 0% /dev/shm
tmpfs 961M 1.4M 960M 1% /run
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 961M 0 961M 0% /sys/fs/cgroup
/dev/mmcblk0p1 253M 49M 204M 20% /boot
/dev/zram1 324M 432K 299M 1% /opt/zram/zram1
overlay1 324M 432K 299M 1% /var/lib/openhab/persistence
/dev/zram2 420M 27M 363M 7% /opt/zram/zram2
overlay2 420M 27M 363M 7% /var/log
tmpfs 193M 0 193M 0% /run/user/1000

Wondering if not just the file locations changed, but also the names, I did try on OH3 to build something fresh so a new THINGS file was generated. Indeed, the file names had changed as well, which then also means all the entries would require updating (as theyā€™d reference the old ā€œclassā€). I tried dropping the old files in, but renaming them to what appeared the new convention - That one Iā€™m sure I didnā€™t do correctly, so it seemed to have no effect.

I havenā€™t gotten a chance yet to re-image and grab logs from the ground up, but will attempt to do that today.

15

Of what? Are you still trying to restore OH2 backup to OH3?

16

Ignoring the snide/condecensing parts of mstormiā€™s message, the one useful part was the recommendation to edit the config file before first boot and force 2.x install. When done that way, restoring the 2.x config, then upgrade to 3.1 did indeed save the Things. So that method is what allowed a fresh load of the card to work.

It seems odd to me that if 3.x defaults and before even logging in you install 2.x instead, why it would seem to hold onto the blank 3.x config rather than migrate the 2.x data. But such is life.

The one real bummer is that doing that upgrade ended up wiping homekit, as I guess the old tags have been replaced by the metadata fields - so for a period of time that bridge has no homekit eligibile items in it (which removes them, and all their associated automation from Apple Homekit). So, I have to go put all the new MetaData tags back in and re-assign every HomeKit accessory to their proper rooms, then go rebuild all the automation again, but at least OH is on the new version and future loads will hopefully not be this painful. I like the OH3 tries to reconnect to Omni if an anomaly occurs (far better than rebooting).

Iā€™ve got 2 more boxes to do, each with almost 150 devices on them (in one home, had to split the lighting and contact sensors onto independent boxes, so the 150 device limit wasnā€™t reached). So, thatā€™ll be fun.

At this point, Iā€™m just happy itā€™s linked back into the mix, even if I do have to reassign all the accessories back to rooms, rebuild HomeKit scenes, and automation.

Thanks.

17

I hope I do not offend anyone, that is not my intention.

Sorry but it is definitively not true that ā€œā€¦probably no software in the world allows for that when there have been major changes to the software and its config between versionsā€¦ā€ bla bla

I used to have a colleague who was on the same pageā€¦ good sake he changed the company.
Good sake there are only a few that are on this trip.

Most software I know allow upgrading to knew major versions, some do it automatically, some with a few simple steps while doing the upgrade.
This depends on the capabilities of the developer.
But it seems to be a challenge for some developers to provide a complex upgrade functionality, it seem that they like it more only to focus on new stuff.

After reading the documentation (months ago) I tried both ways upgrading 2.5>3 and installing 3 and restored 2.5, both failed (on multiple points).
Ever done it exactly it is written in the documentation?
I mean, have you ever upgraded a fully configured openhab 2.5 with a huge set of things/items lots of bindings, rules etc. and with multiple connections to different databases etc.?

Every time I went through the openhab forum and other resources I only see problems and problems and problems and there is some info and there is some info, but I see nowhere a precise full upgrade guide.
It always comes to the point where some is saying to the other ā€œyou did not tell everythingā€ including ā€œyou did not read the documentationā€ and the other one says ā€œI have this bla and that blaā€ and then it comes to a detail discussion, some links are provided and in the end no one really helpedā€¦
No precise and complete upgrade guide, nowhere.
Always this piece by piece documenting, answering and tinkering.

I am watching this drama for some long time now, and to be true, I guess I will / must stick to 2.5 as long as possible, hoping the there will be someone of the developer of openhab that provides an upgrade functionality and precise documentation to upgrade 2.X to 3.X.

I hope I didnā€™t offend anyone, that wasnā€™t my intention.

Cheers and stay healthy

2 Likes
18

No-one will have ever followed the exact same path as you. There are more than a hundred different bindings, which users may configure in a variety of ways, with built-in databases or external, etc etcā€¦ Itā€™s one of the challenges, every installation is quite unique.

Thatā€™s receding into the distance. Volunteer developers will be looking forward, developing OH3 and have an eye on OH4 probably. That ship has sailed.

My primary system is still happily running OH2. When I do come to update it, Iā€™ll be referring to other peoples experiences on the forum, making my own blunders, finding my unique issues.

1 Like
19

openHAB and openHABian are OSS projects.
That is you get them for free, that is we developers spend our little spare time to provide you with a comfortable experience. Yes at times we fail, but weā€™re trying our best.
We even spend our time trying to help people here in the forum.
Also for free.

Then we are rewarded with posts from people like the OP who didnā€™t even bother reading the docs that exist (what I suggested and what he finally used is even part of those docs), that on their first post in 2 yrs(!) start right away by offending us (developers) in a number of ways, then to proceed (unprovokedly) and quickly escalate, complaining to moderators multiple times and finally even insulting me, comparing me to a d*ck. (Yes he did. And no, the star did not replace the character ā€˜uā€™).

I wonā€™t comment on that. But itā€™s what the OP has set as the scene and tone of this thread.

On upgrading and docs - well.
Sure my initial statement on software upgrades was exaggerated but you should mainly understand
it as a direct response to the OPā€™s ā€˜openerā€™.
Yours, however, doesnā€™t match the situation either.

Both of you apparently are not aware that for a OH2->3 upgrade on openHABian weā€™re actually
talking about less of an application but more of an Operating System upgrade.
At least the described issues arenā€™t with/due to the application part.
That is largely widening the event horizon and space of possibilities what exists prior to and what
can go wrong on any upgrade: every openHABian instance has a different starting point.
openHABian scripts are constantly being updated on every run (unless you select not to).
Every OS instance is different because of individualized system config and different package update
points in time. Most people modify their OS (we cannot know how), sometimes for good but often for
bad so they effectively break the upgrade procedure.

Do I need to emphasize that OS upgrade impact testing is a particularly nasty, laborious challenge ?
As is comprehensive documentation to cover all eventualities. It is something even major companies fail on.
openHABian is a 2 man show. Do you expect us to provide that ?
Let alone testing is something we cannot accomplish on our own (i.e. without proper user feedback).

Itā€™s a complex OSS project. Developer are free to focus on whatever they want to
invest their spare time into (reminder: itā€™s theirs). But even that claim by itself is misleading:
it takes a lot more than the pure willingness alone to go one direction or the other or some middle path.
Do you really believe any single person has full knowledge of everything the software will do under countless unknown operating conditions let alone the time it would take to write down truely comprehensive docs ?

No, there NEVER can be a complete guide to this as you want it, not even anything close.
Dislike that but face and accept it, please.

By the way, openHABian is just scripts and there even is a debug guide explaining
at least the basics so anyone with some Linux knowledge like the OP (well, and willing)
can not only debug his problem instance but also help identifying problem causes so we
(developers) can fix it for them and for the benefit of all other users.
Even anyone without this can contribute by providing qualified feedback with useful
descriptions and a logfile (BTW: none of you did).
For a start, you could also consolidate the information you have found on the forum.

So rather than to waste all of our time complaining in lengthy posts, why donā€™t you put your
energy up there and help with testing and documentation ?

Again: Weā€™re working for you. openHAB and openHABian are OSS and totally free, at your service.

20

@rossko57 and @mstormi
Holy moly, really, unfortunately exactly as feared, thanks for letting me know, finally.
You really should write (warn) this on you website, its just one short sentence: NO MAJOR VERSION UPGRADE SUPPORTED!

If I had had known this beforeā€¦ damn.
Thank you so much, finally.
So at least this journey ends here.
I appreciated your honest answer, again, thank you so much.