Best approach for git and openhab config

Hi,

I’m new to OpenHAB. I recently switched from Domoticz because I needed more flexibility.

I’m trying to figure out what is the best approach to put my OpenHAB under version control.
I already set up VS Code, mounted both openhab conf and user datas with Samba as said in the tutorial.
It works nicely and it’s much more convenient than SSHing to my RPi and using vi to edit files …

I however don’t know how can I use git with this setup. How do you guys do this?

I was thinking about keeping the git repository on my PC and use a git hook or ansible to push the modification, but I would loose the items state in VSCode, wouldn’t I?

Question is simple: what is your workflow?

Thanks.

One night I couldn’t sleep, but I was neither in a mood to deal with it the good way, so I did the most dirty way you can imagine. I have config folder “mapped” as a drive on my windows pc, right clicked on the config folder and clicked on “Create git repo here” (I do have some git client installed on pc).
And it did create a git repo, and I did a initial commit, and I do still use it like that for 6 months now… The worst possible setup in my opinion, but… noone is complaining, yet :slight_smile:
I do regular automated backup of files by other “proper” methods.

1 Like

Thanks for your answer.

:slight_smile: yes I see, that’s exactly what I did and I actually don’t like it.

My first problem is that, during lunch time at work, I’d like to improve my openhab setup. And as you can imagine, I can’t mount my samba share at work.
The supposed workflow is to :

  1. pull the git repo on my workstation,
  2. modify what I want
  3. git push
  4. ssh to my openhab
  5. pull the repo.

The problem is that I don’t use the openhab user to connect with ssh.
From this point, headaches appears … Should I change the openhab homedir ? Should I add my ssh user to the openhab group and change permissions ? Should use a git hook ? a web-hook ?
Damn it! :smiley:

I really don’t know how to do it nicely, with minimal effort and security impact…

My OH runs on a VM and I edit either from a virtual desktop VM or my laptop when I’m home.

I run OH as a Docker container so my conf and userdata folders are in the same folder and mapped into the container. So for me I have /opt/openhab2/conf and /opt/openhab2/userdata. The import of this will be apparent in a moment.

I share my /opt/openhab2 folder as a samba shared folder which gets mounted to my laptop as a Network Drive (Windows) or to /opt/openhab2 on my virtual desktop.

I run Gogs on yet another VM as my git server. I’m lazy and didn’t want to set up encrypted files or redact my config so I could post them to something like Github of bitbucket (I don’t want to have my location or usernames and host names or passwords on a public repo).

When I’m remote, if it is a small change, I just ssh to my home server and make the changes using vim. If it is a big change, I connect VNC through an ssh tunnel to my virtual desktop and use VSCode. I edit the live rules, usually with a tail running in a terminal to watch for errors.

After making and verifying the changes, I’ll check in and push my changes to Gogs.

I set up my repo to include conf and userdata/etc and userdata/jsondb. I should probably include some of the other folders too but it hasn’t caused problems for me yet. I have .gitignores set up to ignore everything else in userdata.

I too face the permission problem but that gets addressed on my Windows machine and mount on the virtual desktop by supplying the user and group for the mount point. No matter what user I happen to be it gets converted to openhab/openhab. If I’m on my actual server, I make sure to do all the git operations using sudo -u openhab. I did have to set up the openhab user with my ssh certs. I’m sure there are better ways to handle this but it works for me.

2 Likes

@rlkoshak thanks for your detailed answer. I’m new to openhab, but I already know that you’re a great contributor. You always give nice, detail and concise answers.

You setup is more complicated than mine, but I think I’ll go with the sudo -s way. I don’t use this remote very often.
For the same reasons than yours, I also use Gogs, but it’s on a remote dedicated server (shared for several projects), with ssh keys protection. But pushing a public key for the openhab user will certainly do the trick.

I have one more question though. How did you configure the userdata/etc and userdata/jsondb in your git repo ? as a module ?
Actually, I just did a git init in the /etc/openhab2 folder.

Thanks in advance.

2 Likes

git repository is cool and stuff - but as there are some sensitive data in openHAB, how do you approach these?

  • not only internal IP-adresses
  • passwords for mqtt and stuff
  • internal credentials for all kinds of home devices

For my personal use, my git repository is private. I use gogs, with a password protection.
This way, nobody can access any sensitive data.

If you need a real collaboration for your Openhab config, and need to share to a public area like github, you’ll to do thinks differently.

One solution would be to use ansible to deploy your config, and encrypt sensitive data with ansible vault. In your files, you’ll only have references to ansible variables.
These ones will be set when deploying.

But with this, you have to deploy your configuration after each modification, it’s not like live editing your configuration through VS Code.

I don’t think internal IP address is a sensitive data, on the other hand, passwords are, indeed.

2 Likes

my openhab setup is within a VM on a proxmox host and i share my configuration via samba as well. I like the possibility of “live editing” with a running log:tail in a ssh/karaf terminal as well as @rlkoshak does.
When i do have to edit anything from my windows pc at work, i pull my current setup via nextcloud from within a LXC-container on the proxmox host - then edit what i want with vs code and upload the edited file via nextcloud again.

i also thought about setting up a gitlab in another LXC to manage the configurations, but i think that’s a serious overkill - but would be cool anyway…

@spy0r ok but this way you configuration is not version controlled, isn’t it? Well, at least, Nextcloud has its versioning system, but it’s not git.

Gitlab is for sure overkill, Gogs is the way to go : lightweight, simple and stable (wiki, ticket, pull-request, ssh etc. all that your need for a proper ssh remote)

Thanks. gogs would be a solution (didn’t know this one yet). I don’t need collaboration and stuff - just for me.
I do handle IP addresses also as sensitive data, not crucial ones, but if someone manages to plug in in my network, I don’t like the idea, everything is laid out already! :wink:

yes, you are right, the complete approach is not version controlled continuously, nextcloud has an own versioning, but i don’t use it very often. I do complete backups of the vm every 12h and of the configuration every 24h to a nas with ZFS snapshots and a rsync -a backup disc. So if there is really a situation, where i did a mistake and have to roll back to a previous version, i could do it like that as if i have the configuration from every night from the last 2 years lying there.

that’s not the intension of versioning, i know, but i see no need to know what configuration i had on a specific date, as i proof my changes very well and recover to the previous version if anything went wrong.

edit: i’m pretty excited to see, what solutions you all use and if i like them more than my own approach…
edit2: i see that you are much more professional than me, maybe i try out gogs and what i can do with it within another LXC.

@binderth I can understand you want to keep things a little secret. But if someones succeeds in accessing your private network, he/she would be skilled enough to perform a simple command (tcmdump or whatever) to discover all your network. Plus, private network ranges are known by anyone, 3 ip classes, not so many ranges, easy to discover :wink:
The best you can do is protect the entrance: strong wifi key, minimum routing, firewall, change the default ssh port, use SSL, use a ban system (fail2ban) etc.
Sorry, this is a little off topic, I just wanted to share :slight_smile:

1 Like

That’s a fairly proper approach :wink:
I like your use of ZFS + rsync, clever enough and does the job.

I’m actually a big fan of git, this is so useful. And I’m a Java developer for years now, professionaly, so I’m really use to it. It helps me a lot to organise my work. Any little projects, even my dotfiles are under git.
Gogs is really lightweight (written in Go) and has enough functionalities for small to medium projects/organisations. I used to have my git repo only locally. But sometimes I need to access them from somewhere else, a good example is to retrieve my home directory config (zsh, gitconfig etc.)

I hope others will share their solutions!

Great to hear, that you are a professional in git/projects. So if i also had a Gogs installation in another Container, how would i use it properly? Here are some questions:

  • i would have a “project” with my openhab config files which i could push/pull with any git client?
  • could i change the files directly from within the web frontend (if needed)?
  • to “activate” the changes on my openhab instance i have to pull the configuration from the openhab instance, right?

What I currently have:

  1. A git repository for my /etc/openhab
  2. The remote is configured to push to my Gogs instance (with a particularity, see below)

If I’m at home, I just use the samba shared folder (/etc/openhab) to live edit the config. I can commit directly, use the VS Code plugin etc. I push to remote when I’m done working. That allows me to pull the latest modifications from work the following day.

If I’m at work, I pull my repo, do some stuff, commit and push when I’m done editing or when I want to test. Then I connect to my openhab’s host, and do a git pull.

To go further and simplify the access, my gogs is configured as follows:

  • by default a repository is private
  • I have defined an “application” access, it’s in the user settings, which provides an http access token. This avoids to generate a ssh key pair for the openhab user.

To make it easy for you, create your LXC with your gogs instance, create a user, and a repo. When the repository is created, Gogs will give your instructions for cloning. Just be sure to change the clone url with the http access like this : git remote add origin http(s)://your_access_token@yourgithost.com/yourUser/yourRepo.git
Then do the initial commit, and git push -u origin master.

could i change the files directly from within the web frontend (if needed)?

I’m not sure to understand. What frontend are you talking about?

The same as for conf. I initially did a git init, copied over the conf and userdata folders to get started (also ~openhab/.java as that is where the oauth token stuff is stored for Nest connections). So I really just have the one repo which contains both my conf and userdata. I don’t do anything different for the two.

About the only complaint that I have with this setup is my cloned repo OH runs on is almost always “dirty” because I have code that copies around icons for weather and the jsondb is constantly being backed up so at any given time I almost always have something to check in. Therefore I can’t quickly tell whether I have changes I’ve made that need to be checked in. This has forced me to be a lot more diligent about making small concise changes and checking them in frequently.

So if you look at my repo you would see:

As you can see, I have .java, conf, and userdata just as folders in the repo. Then, because I’m using Docker I map those folders into the container to the appropriate locations. Before I started using Docker, I would use symbolic links to so map from where ever I cloned the repo to to /etc/openhab2 and /var/lib/openhab2.

Does that anwer your question?

I host my own git server (Gogs). Though there are other approaches like git-crypt which lets you encrypt sensitive files so only those with a password can read those with sensitive info in it.

It is sensitive info only in that it reveals quite a bit about your internal network topology, though the impact of that information being released is relatively low. But it does make someone who is specifically targeting you’s job easier. And it becomes much more sensitive when combined with information like lat/long and usernames, both of which would be in a typical OH config. Whether the aggregate of information in your repo makes internal ip addresses a risk for you is something you have to decide.

That is why I chose Gogs, it is super light weight and has pretty much all of the features of Gitlab that I care about. It is also really easy to set up. Here is my ansible role I use to deploy it:

---

- name: Create a git user
  user:
    comment: 'Gogs'
    createhome: no
    name: git
    shell: /bin/bash
    state: present
    system: yes
  become: yes

- name: Mount gogs working folder
  include_role:
    name: mount-cifs
  vars:
    mount_mode: '0660'
    cifs_user: "{{ share_user }}"
    cifs_pass: "{{ share_pass }}"
    cifs_domain: "{{ workgroup }}"
    mount_user: git
    mount_path: "{{ gogs_data }}"
    mount_src: "{{ gogs_mount }}"

- name: Start gogs
  docker_container:
    detach: True
    exposed_ports:
      - 22
      - 3000
    hostname: chimera.koshak.net
    image: gogs/gogs
    log_driver: syslog
    name: gogs
    published_ports:
      - "10022:22"
      - "3000:3000"
    pull: True
    recreate: True
    restart: True
    restart_policy: always
    volumes:
      - "{{ gogs_data }}:/data"
      - /etc/passwd:/etc/passwd:ro
      - /usr/share/zoneinfo:/usr/share/zoneinfo:ro

Note, it has bee a very long time since I’ve had to run this. It depends on there being a preconfigured app.ini file and variables defined. The data folders used by Gogs is servered by my NAS so I have a task to mount them. I no longer need to get to these folders from Windows but I’ve not converted them to NFS from CIFS yet.

For me it is incredibly useful to be able to go back and see the tiny little differences in Rules between versions as I help people on the forum. If I were smart, I would have cut a tag each time I moved to a new version. Gonna have to start doing that going forward.

Yes. You would work with it the same as you would with Github or Bitbucket or any other Git server. From your client’s perspective it is just like any other git server. And it has a nice web UI akin to what you are used to with services like Github as well.

It’s a little awkward but certainly. Browse to the file, click the pencil icon in the upper right, and you can edit away.

Yes. From the command line the workflow would look something like

// from your editing machine
vim myitems.item
/// and save make edits
 git add myitems.item
git commit -M "Added a new super cool Item"
git push

// from the OH server
git pull

Of course, you can edit the file through the browser or from the same OH machine.

I never looked that deeply into that option. I need to give it a second look. That is pretty cool.

I think he is asking if one can edit files from the Gogs web app directly or if you need to clone the repo and edit the file locally and push the changes.

3 Likes

@rlkoshak & @bebR : Thank you very much for your extensively answers. Like Rich mentioned, i asked to change some tiny little detail in a file directly in the Gogs web app, but that seems to work.

i think i will set up a Gogs container for testing, so far i just knew gitlab and as i tried it in the past it was way an overkill for my non professional home use.

great, new playground! Thanks

@rlkoshak thanks again for the detailed answer and the screenshot of your repo, that’s clearer to me now.

@spy0r glad to hear that, you got a new toy to play with for this weekend :wink:

i think i managed it… even if i deleted my complete config by accident, i got thrown into cold water but i could pull it back again :smiley:

let’s see if i can get more professional with this here…

Great!

Just two tips for your new pro workflow :wink:

  1. If you use the ticket feature (I use it to note what I plan to do), you can reference a ticket in a commit message using #ticketNumber. Just don’t use it at the beginning of a line or it will be considered as a comment. Doing this, you’ll have some nice references in both ticket and commit log.
  2. Gogs treats readme.txt and readme.md like github, which allows to document your repository without using the wiki (browse the source, you’ll see that the default readme.txt in the openhab folders are displayed below the file list)
1 Like