Best approach for git and openhab config

my openhab setup is within a VM on a proxmox host and i share my configuration via samba as well. I like the possibility of “live editing” with a running log:tail in a ssh/karaf terminal as well as @rlkoshak does.
When i do have to edit anything from my windows pc at work, i pull my current setup via nextcloud from within a LXC-container on the proxmox host - then edit what i want with vs code and upload the edited file via nextcloud again.

i also thought about setting up a gitlab in another LXC to manage the configurations, but i think that’s a serious overkill - but would be cool anyway…

@spy0r ok but this way you configuration is not version controlled, isn’t it? Well, at least, Nextcloud has its versioning system, but it’s not git.

Gitlab is for sure overkill, Gogs is the way to go : lightweight, simple and stable (wiki, ticket, pull-request, ssh etc. all that your need for a proper ssh remote)

Thanks. gogs would be a solution (didn’t know this one yet). I don’t need collaboration and stuff - just for me.
I do handle IP addresses also as sensitive data, not crucial ones, but if someone manages to plug in in my network, I don’t like the idea, everything is laid out already! :wink:

yes, you are right, the complete approach is not version controlled continuously, nextcloud has an own versioning, but i don’t use it very often. I do complete backups of the vm every 12h and of the configuration every 24h to a nas with ZFS snapshots and a rsync -a backup disc. So if there is really a situation, where i did a mistake and have to roll back to a previous version, i could do it like that as if i have the configuration from every night from the last 2 years lying there.

that’s not the intension of versioning, i know, but i see no need to know what configuration i had on a specific date, as i proof my changes very well and recover to the previous version if anything went wrong.

edit: i’m pretty excited to see, what solutions you all use and if i like them more than my own approach…
edit2: i see that you are much more professional than me, maybe i try out gogs and what i can do with it within another LXC.

@binderth I can understand you want to keep things a little secret. But if someones succeeds in accessing your private network, he/she would be skilled enough to perform a simple command (tcmdump or whatever) to discover all your network. Plus, private network ranges are known by anyone, 3 ip classes, not so many ranges, easy to discover :wink:
The best you can do is protect the entrance: strong wifi key, minimum routing, firewall, change the default ssh port, use SSL, use a ban system (fail2ban) etc.
Sorry, this is a little off topic, I just wanted to share :slight_smile:

1 Like

That’s a fairly proper approach :wink:
I like your use of ZFS + rsync, clever enough and does the job.

I’m actually a big fan of git, this is so useful. And I’m a Java developer for years now, professionaly, so I’m really use to it. It helps me a lot to organise my work. Any little projects, even my dotfiles are under git.
Gogs is really lightweight (written in Go) and has enough functionalities for small to medium projects/organisations. I used to have my git repo only locally. But sometimes I need to access them from somewhere else, a good example is to retrieve my home directory config (zsh, gitconfig etc.)

I hope others will share their solutions!

Great to hear, that you are a professional in git/projects. So if i also had a Gogs installation in another Container, how would i use it properly? Here are some questions:

  • i would have a “project” with my openhab config files which i could push/pull with any git client?
  • could i change the files directly from within the web frontend (if needed)?
  • to “activate” the changes on my openhab instance i have to pull the configuration from the openhab instance, right?

What I currently have:

  1. A git repository for my /etc/openhab
  2. The remote is configured to push to my Gogs instance (with a particularity, see below)

If I’m at home, I just use the samba shared folder (/etc/openhab) to live edit the config. I can commit directly, use the VS Code plugin etc. I push to remote when I’m done working. That allows me to pull the latest modifications from work the following day.

If I’m at work, I pull my repo, do some stuff, commit and push when I’m done editing or when I want to test. Then I connect to my openhab’s host, and do a git pull.

To go further and simplify the access, my gogs is configured as follows:

  • by default a repository is private
  • I have defined an “application” access, it’s in the user settings, which provides an http access token. This avoids to generate a ssh key pair for the openhab user.

To make it easy for you, create your LXC with your gogs instance, create a user, and a repo. When the repository is created, Gogs will give your instructions for cloning. Just be sure to change the clone url with the http access like this : git remote add origin http(s)://your_access_token@yourgithost.com/yourUser/yourRepo.git
Then do the initial commit, and git push -u origin master.

could i change the files directly from within the web frontend (if needed)?

I’m not sure to understand. What frontend are you talking about?

The same as for conf. I initially did a git init, copied over the conf and userdata folders to get started (also ~openhab/.java as that is where the oauth token stuff is stored for Nest connections). So I really just have the one repo which contains both my conf and userdata. I don’t do anything different for the two.

About the only complaint that I have with this setup is my cloned repo OH runs on is almost always “dirty” because I have code that copies around icons for weather and the jsondb is constantly being backed up so at any given time I almost always have something to check in. Therefore I can’t quickly tell whether I have changes I’ve made that need to be checked in. This has forced me to be a lot more diligent about making small concise changes and checking them in frequently.

So if you look at my repo you would see:

As you can see, I have .java, conf, and userdata just as folders in the repo. Then, because I’m using Docker I map those folders into the container to the appropriate locations. Before I started using Docker, I would use symbolic links to so map from where ever I cloned the repo to to /etc/openhab2 and /var/lib/openhab2.

Does that anwer your question?

I host my own git server (Gogs). Though there are other approaches like git-crypt which lets you encrypt sensitive files so only those with a password can read those with sensitive info in it.

It is sensitive info only in that it reveals quite a bit about your internal network topology, though the impact of that information being released is relatively low. But it does make someone who is specifically targeting you’s job easier. And it becomes much more sensitive when combined with information like lat/long and usernames, both of which would be in a typical OH config. Whether the aggregate of information in your repo makes internal ip addresses a risk for you is something you have to decide.

That is why I chose Gogs, it is super light weight and has pretty much all of the features of Gitlab that I care about. It is also really easy to set up. Here is my ansible role I use to deploy it:

---

- name: Create a git user
  user:
    comment: 'Gogs'
    createhome: no
    name: git
    shell: /bin/bash
    state: present
    system: yes
  become: yes

- name: Mount gogs working folder
  include_role:
    name: mount-cifs
  vars:
    mount_mode: '0660'
    cifs_user: "{{ share_user }}"
    cifs_pass: "{{ share_pass }}"
    cifs_domain: "{{ workgroup }}"
    mount_user: git
    mount_path: "{{ gogs_data }}"
    mount_src: "{{ gogs_mount }}"

- name: Start gogs
  docker_container:
    detach: True
    exposed_ports:
      - 22
      - 3000
    hostname: chimera.koshak.net
    image: gogs/gogs
    log_driver: syslog
    name: gogs
    published_ports:
      - "10022:22"
      - "3000:3000"
    pull: True
    recreate: True
    restart: True
    restart_policy: always
    volumes:
      - "{{ gogs_data }}:/data"
      - /etc/passwd:/etc/passwd:ro
      - /usr/share/zoneinfo:/usr/share/zoneinfo:ro

Note, it has bee a very long time since I’ve had to run this. It depends on there being a preconfigured app.ini file and variables defined. The data folders used by Gogs is servered by my NAS so I have a task to mount them. I no longer need to get to these folders from Windows but I’ve not converted them to NFS from CIFS yet.

For me it is incredibly useful to be able to go back and see the tiny little differences in Rules between versions as I help people on the forum. If I were smart, I would have cut a tag each time I moved to a new version. Gonna have to start doing that going forward.

Yes. You would work with it the same as you would with Github or Bitbucket or any other Git server. From your client’s perspective it is just like any other git server. And it has a nice web UI akin to what you are used to with services like Github as well.

It’s a little awkward but certainly. Browse to the file, click the pencil icon in the upper right, and you can edit away.

Yes. From the command line the workflow would look something like

// from your editing machine
vim myitems.item
/// and save make edits
 git add myitems.item
git commit -M "Added a new super cool Item"
git push

// from the OH server
git pull

Of course, you can edit the file through the browser or from the same OH machine.

I never looked that deeply into that option. I need to give it a second look. That is pretty cool.

I think he is asking if one can edit files from the Gogs web app directly or if you need to clone the repo and edit the file locally and push the changes.

3 Likes

@rlkoshak & @bebR : Thank you very much for your extensively answers. Like Rich mentioned, i asked to change some tiny little detail in a file directly in the Gogs web app, but that seems to work.

i think i will set up a Gogs container for testing, so far i just knew gitlab and as i tried it in the past it was way an overkill for my non professional home use.

great, new playground! Thanks

@rlkoshak thanks again for the detailed answer and the screenshot of your repo, that’s clearer to me now.

@spy0r glad to hear that, you got a new toy to play with for this weekend :wink:

i think i managed it… even if i deleted my complete config by accident, i got thrown into cold water but i could pull it back again :smiley:

let’s see if i can get more professional with this here…

Great!

Just two tips for your new pro workflow :wink:

  1. If you use the ticket feature (I use it to note what I plan to do), you can reference a ticket in a commit message using #ticketNumber. Just don’t use it at the beginning of a line or it will be considered as a comment. Doing this, you’ll have some nice references in both ticket and commit log.
  2. Gogs treats readme.txt and readme.md like github, which allows to document your repository without using the wiki (browse the source, you’ll see that the default readme.txt in the openhab folders are displayed below the file list)
1 Like

Thanks, i tried the thing with the readmes and issues… i kind of like my new pro setup :smiley: for sure my installation could be handled otherwise, too, but i think i can use it at home for my different projects as well…

1 Like

one small question is left for now:

if you are at home and edit within vscode on your samba share - How can you use the builtin git plugin to commit changes?

i tried that as well, but my samba user was not able to change things in the .git folder… Damn permissions, I should go deeper into this, this approach would be really cool…

edit:
On the openhab server i add “sudo -u openhab” before the git promts, as @rlkoshak does. But i think i need to allow the samba user to edit the .git folder as well, that should work out.

It really depends on how you’ve mounted the samba share and how is configured the share point in samba on the openhab server.

I personnaly don’t use the git’s vscode plugin. I always use the command line. So under my windows laptop, I ssh and sudo -u openhab for every git command. On my linux workstation, I don’t have the problem, I can either use the VS Code plugin or the command line.

But I’ve noticed on my laptop that the VS Code plugin detects a lot of changes, whereas there is none… This is probably a permission or a git configuration issue. I didn’t investigate yet…

that’s what i also saw… There are changes shown in some icon files which definitely are not there…

Okay, then i understood how you guys work with it and i’m nearly there. Thanks again.

For me I usually have a terminal open to the OH server open anyway, often in the terminal window in VSCode, so I’ve not used the VSCode git integration that much.

But it seems to me that if you shared the folder using openhab as the share user that should work.

I was tired of always typing sudo -u openhab before every git command.
So I just added the following alias in the .bashrc of the pi user: alias git='sudo -u openhab git'

My Raspberry Pi is only used for Openhab, there is no impact on anything else.

If I need to bypass the alias, I just have to escape it with a backslash: \git <command>

See you next time :wink:

that’s a freakin’ easy and good idea… i’ll copy that :smiley: