Mythical soon-to-be-attracted-user

Development Misc
1

Last days there were many discussions on the forum about the future of openHAB. Very interesting and necessary discussions, and many times quite technical reads. One topic that comes back over time is the argument that a certain technology or development direction needs to be chosen to attract more users, not to scare away power-users, attract novice users, ‘grow’ the base etc.

I’ve been using openHAB since three years now, and the development over time has been impressive. My gut feel says that the community of users and the adoption of openHAB in home automation setups is growing. If we could have more data on this we can use this to give us confidence that we are on the right path and that our future improvements will satisfy our current base and newcomers alike. and be able to make arguments based on what current users want (new, noobs and powerusers and everything in between) instead of always referring to this mythical soon-to-be-attracted-user.

I started a discussion on this a while ago:

It would be great if at least we could get some numbers from maintainers on:
1) Social engagement:

  • Number of active forum users (min 1 visit / month)
  • Number of posts / month (new topics & replies)
  • Number of active myopenHAB users (local openHAB interacts at least once in that month with the cloudserver)

2) ‘Hard’ data

  • Number of downloads (openHABian, github, etc.)
  • Website visitors

And this data does not tell us a lot unless we can see the trends (monthly or quarterly / from 2015 - 2019).

Who can support this? I would be willing to collect all data and make a simple overview in Google Spreadsheets.

5 Likes
2

I can see some of this information but keep in mind this is just a snapshot in time.

At the time of this posting:

  • More than 100 active users on the forum within the last minute (the list will only show the top 100 but all have been seen within the last minute)
  • 36 new users created in the last 24 hours, more than 100 new users created in the last three days
  • 11 moderators/administrators
  • The top ten searched terms are mqtt, xiaomi, knx, alexa, sonoff, homekit, rules, habpanel, google home, zigbee with mqtt having nearly twice the searches (4137) as xiaomi (2353). I don’t know if this is searches of all time or only a certain period of time. The numbers seem low so I’m guessing they are only a certain period of time.

I can’t get the dashboard to come up right now for some reason. It’s generating a 502 error. Unfortunately that is where most of the answers related to the forum can be generated.

2 Likes
3

I don’t really want to be someone who deny a current way, but I want to throw some perspective on your question. The soon to be attracted user is an abstract term and no one can answer that question in accurate way.

In order to answer this precisely we would need to enable some kind of telemetry gathering on users (with their consent of course!) and see:

  • How big is setup in terms of number of items and things
  • What things are integrated
  • How active are devices
  • How often UI is being checked
  • How long whole thing runs

Without these answers we can just speculate what the attracted user is. We had numerous rants on forums on why openhab didn’t satisfy someone needs. Some were more accurate, some a bit less. The real point is that OpenHab is not for everyone and with all good things which happened over past years we need to acknowledge that amount of tinkering with it might be to high for some people.

Having said that - OpenHab is an open source project and not a product. Some people look for things which solve their troubles without checking a software license and documentation. If problem is not solved with few clicks then the tool is wrong, but not the user. In the end what counts is satisfaction of end users and not source code which leads to that.

We didn’t have a community survey so far, and even if we would organize such, till great extent, it will cover only these who visit forums and not people who installed it for a single reason and might be (un) happy with it.
In order to start improving that you need to do a follow up conversation with these who download OpenHab rather than expect that they will come and say that in forums.

Such commercial-a-like behavior might not be a nicest one for OSS project, I acknowledge that. But if you think about moving things forward for end users, you need to talk with them as potential customers, or let some company with such approach to step in. Attracting developers is way easier than endusers!

3 Likes
4

what…???

5

Exactly, so my point is that we might need to stop referring to it.

You overcomplicate things. I very simply asked to make public that data that we as a community already have, see my point 1) and 2).

I’m amazed by these numbers, but as you said, it is just a snapshot. Any other maintainers that can share info on the cloud service for instance, or the forum which shows the trend through time?

6

I just checked again so at least you will have two data points.

  • 97 active users in the last minute
  • 33 new users created in the last 24 hours

Signs point to a pretty steady growth rate but obviously more data points are necessary. The dashboard still won’t come up with a 503 error. That would need to be fixed before we can get anything more than hand counts.

7

I am impressed by those numbers as well, that is a lot of forum users. I’ve noted that the threads in the forum at times move very fast. As Maurits says, it is my ‘gut’ instinct that openHAB is growing as is home automation in general. I read a lot of mentions of openHAB in the reviews on Amazon ect. People are talking about it. A lot of older reviews and tutorials seem to give the software a reputation for having a steep learning curve.

8

Don‘t have actual numbers, but in May last year, we had about 23.000 (twentythreethousand!) registered forum users.

1 Like
9

We might talk about similar, yet completely different things. I point that biggest user base is built from these who are not tempted to write a configuration file or even read documentation - there are people who expect things to work even if they are free. This is a regular “consumer” group which usual companies making smart home products fight for, because that’s biggest part of pie. Beside that we have DIY fans and also people with some dose of will to understand and build things - these who run openhab, who are not scarred by failures and necessity to go over docs or some forum posts. Second group is obviously smaller than first one.

What you measure now, by throwing in forum numbers might be completely inaccurate.
What does “97 active users in last minute” tell to us? Well, just that we have 97 users online - mainly usual suspects who follow forums and do outstanding job answering other users posts. It is measure of community activity. Is it important? Yes! Does it tell us anything about users from group 1? Nothing at all. About group 2? A bit.
What insight does “33 new users created in the last 24 hours” provide? Nothing unless we check if these are spam bots, if these people actually posted anything or read anything after creating account. We can assume that some of 97 active users were some of the new ones, but will it lead us to proper decisions? I doubt.

Personally I believe that what should be measured in first place is amount of installations, their versions and their uptime. This is an measure which tells us how many software instances are running to host people needs. It will also tell us how often program was run once and terminated (a refusal rate).

Following forums numbers will tell us a lot about our community, yet it says nothing about people who downloaded software and didn’t come to discuss. Keep in mind that running openhab does not imply registering user account here or at myopenhab.org. We can use rising number of downloads as kind of pointer, but we don’t really know how many of these downloads ended up as working instances.

My point is fairly simple - hardest data you can get is amount of unique installations. Until we know that it is difficult to say how attractive project is for end users. I stand with you to publish stats which are already available, they will most likely proof that project is in continuous rising trend since several years at least, but I would not jump into too many assumptions from that.
As I pointed above we can not take forum as only one measure because it answers completely different question than topic author asked. After all rising amount of topics in certain forum categories might be understood as growing number of troubles. It can be also understood as indication of increasing user base.

Cheers :slight_smile:

10

Except as one of those users myself, the vast majority of the list are users I don’t recognize. I’d say no more than 10 either time I checked after the usual suspects. It would be interesting to compare the new user list to the active list but since it’s all manual counting I’m not going to sign up for that.

Except for the very new users, reach of those 33 new users spent at least 5 minutes reading posts, however the forum measures that. And if these users are bots, they are pretty silent bots as I can say as a moderator, they’re is not a lot of spam to deal with. Maybe one or two a month.

I agree that ee have to be careful to draw too much of a conclusion from the data, but I don’t think it’s meaningless. I think it shows we have an active and growing forum. From that I think we can surmise that OH user itself is growing. By how much is hard to say without myopenhab.org statistics.

This doesn’t exist and I think adding it to an open source project would be problematic on a number of fronts. How do you think the “no cloud services” folks would react to us suddenly connecting telemetry. And even if we did have this, it wouldn’t be an exact number. I run in Docket. Every time I restart OH I’m running a new install. That would mess up the statistics as much as anything else.

Ultimately we have the data we have which isn’t much and from that we can make some assumptions and estimates and come up with numbers that are close enough to make some decisions about.

We will never have perfect data. We will never even have good data. But for some decisions, a thumb in the air is sometimes all you need.

As long as we don’t try to draw too many conclusions from the data we have I don’t think it is impossible to draw any conclusions what do ever.

Think of it like the Drake Equation. We may not know all the variables but we can make educated guesses and come to some conclusions, plus or minus a huge error range.

But to play devil’s advocate, let’s say we do have all the info. Then what? This is still an open source project. Much if not all of any progress made will be because a developer decided it was something they wanted to implement. Kai is going to create a governence model which may help guide development but OH well never have to the sort of disciplined road map that a commercial project would have. So even if we can have perfect data and make decisions, would that actually lead to actually changing the direction of development?

2 Likes
11

Nowhere in my posts you will read that I draw conclusions based on the numbers that Rich and Hans-Jorg post. I think it is a good start to understand our userbase, so without the numbers I mentioned in my first post, and the trends among those we will not be able to draw any conclusions.

Agree, it is a combination of the available data that will lead us to our first insights.

12

To add a bit more, last May we had 35.000 registered myopenHAB users and 12.000 active instances. Fo foundation members, these numbers can be found in last years protocol of the general assembly.

3 Likes
13

12k myopenHAB instances ? Can you elaborate on that, what does it take for an instance to count ?
Is it unique IP addresses only so does everyone count who ever tried and maybe just fired it up once or do these instances have to persist for some time before they get counted ?

14

I guess it’s actively connected UIDs within a short (day, maybe week) range of time.

15

That sort of pure guessing will not get us anywhere, that’s why I asked …

1 Like
16

Exactly why i introduced it with

:wink:

17

I am not quite shure, got those numbers from our meeting protocol.
But as @Kai wrote „active“, I think those 12k are really using myopenhab, no „ghosts“. This will also explain why we needed to add more servers to myopenhab by end of 2018, running now at 2 NGINX, 2 App and 1(or 2, not quite sure) MongoDB servers.