This is a complicated question with a complicated answer.
I’m going to limit this to just openHAB.
When should you upgrade? It depends.
Is there some new feature that you need?
OH has a rolling release. Once a version is created (e.g. 2.4 release, 2.5 M1) that version remains fixed. Theoretically important bug fixes can be back ported but that has never occurred. So 2.4 release has none of the changes that have been made since last December.
Obviously there are new features added and bugs fixed constantly. So if there is a new feature you need or bug fix you need then you have to upgrade to get it.
What is your willingness to accept bugs/untested code?
OH has three release tiers.
The Release tier has been the most thoroughly tested and it contains no known bugs at the time of release, or the bugs that are known are well documented with some sort of work around. Releases are usually made once every six months, though right now there are massive changes that have delayed the release.
The Testing tier, sometimes called Milestone, doesn’t receive and directed testing prior to release but there are no known bugs in it. Milestone releases are usually made every month but that hasn’t happened recently because of the previously mentioned massive changes. A milestone release could be considered a Beta release.
The Snapshot tier, is the bleeding edge of openHAB. It essentially contains all the changes that have been merged within the last 24 hours. This is the first time that most of these changes are seeing wide adoption (lots of OH users run the snapshot version). This is the riskiest tier and only recommended if there is a bug fix or feature your really really need or you are willing to be a tester and report and file issues on any problems you encounter.
Choose the release tier that matches your appetite for risk. Usually, the snapshot releases are reasonably safe to run, though they have been problematic for the past six months.
How much time are you willing to spend upgrading?
The flip side of the coin is that the longer you wait to upgrade, the more work it will require to move from your current version to a new version. So if you are on OH 2.3, it will take a whole lot more work to upgrade to OH 2.5 M1 than it will take to upgrade from Oh 2.5 M1 to M2. So the longer you wait, the more work you will have to do under pressure to get everything migrated to the new version.
How do I know when to upgrade?
It depends on how you installed and how much risk you want to assume. If you want only reasonably well tested versions, you should stick to the release version of OH and you will upgrade about once every six months. Watch the Announcements category on this forum and there will be an announcement made when a new release is made. There should also be a blog posting. Both will list or link to a list of all the breaking changes. Breaking changes mean you will have to do something to get your current version running on the new version.
If you want something stable, you should stick to the Testing branch and upgrade once a month. As with the releases, there should be an announcement in the forum, though I don’t think there will be a blog post.
If you want to help test OH then you should use the Snapshot branch and you should be upgrading at least once a week, perhaps more often. If you only installed the Snapshot to get a bug fix or new feature, then stick with that version until at least the next milestone release if you don’t want to help test.
If you installed via apt or are using openHABian, openHAB will get upgraded every time you run apt-get upgrade
.