Slow downloads of openhab and openhab add ons from openhab.jfrog.io

Setup, Configuration and Use Installation
1
  • Platform information:
    • Hardware: Raspberry Pi 4 Model B Rev 1.2
    • OS: Raspbian GNU/Linux 11 (bullseye)
    • openHAB version: 4.1.1.
  • Issue of the topic: please be detailed explaining your issue

I was trying to upgrade my development raspberry to OH 4.1.1 with openhabian. The download speed was pretty low (less than 100kB/s) . Downloading the openhab and openhab add on files zip from the openhab repository is much faster and it is limited by my network connection bandwidth.

In order to start in a reproducible condition, I reinstalled a fresh openhabian image but the download of openhab and openhab addons is still taking an unusually long time.
I found a thread (Openhab.Jfrog.io always runs slow for me) mentioning a similar slow behavior but the proposed fix does not apply to my case, because I have not changed MTU in my network (I replaced my fritzbox 7590 last month, but I restored previous settings).

Is there any one else experiencing this issue?

Thank you for your attention,
Lionello

2

Downloading snapshot zip files from ci.openhab.org is super slow from Australia. I got random speed around 7 kB/s up to 200 kB/s.

Downloading from github gives me about 2 MB/s

3

Thanks for confirming that.
Actually, I am experiencing some download errors and installation fails
Here’s an excerpt from the first-boot.log file

+ echo -n '2024-01-14_09:46:57_CET [openHABian] Beginning install of latest openhab release (stable repo)... '
2024-01-14_09:46:57_CET [openHABian] Beginning install of latest openhab release (stable repo)... + [[ -n '' ]]
+ echo OK
OK
++ date +%y%m%d
+ [[ 240114 -lt 230801 ]]
+ running_in_docker
+ [[ -n '' ]]
+ grep -qs 'docker\|lxc' /proc/1/cgroup
+ [[ -f /.dockerenv ]]
+ return 1
+ [[ -z '' ]]
+ add_keys https://openhab.jfrog.io/artifactory/api/gpg/key/public openhab
+ local repoKey=/usr/share/keyrings/openhab.gpg
++ timestamp
++ printf '%(%F_%T_%Z)T\n' -1
+ echo -n '2024-01-14_09:46:57_CET [openHABian] Adding required keys to apt... '
2024-01-14_09:46:57_CET [openHABian] Adding required keys to apt... + curl -fsL https://openhab.jfrog.io/artifactory/api/gpg/key/public
+ gpg --dearmor
+ echo OK
OK
+ rm -f /etc/apt/sources.list.d/openhab.list
+ echo 'deb [signed-by=/usr/share/keyrings/openhab.gpg] https://openhab.jfrog.io/artifactory/openhab-linuxpkg stable main'
+ dpkg --configure -a
++ timestamp
++ printf '%(%F_%T_%Z)T\n' -1
+ echo -n '2024-01-14_09:46:58_CET [openHABian] Installing selected openHAB version... '
2024-01-14_09:46:58_CET [openHABian] Installing selected openHAB version... + apt-get clean --yes -o DPkg::Lock::Timeout=60
+ cond_redirect apt-get update
+ [[ -n '' ]]
+ echo -e '\n\033[90;01m$ apt-get update \033[39;49;00m'

e[90;01m$ apt-get update e[39;49;00m
+ apt-get update
Hit:1 http://davesteele.github.io/comitup/repo comitup InRelease
Hit:2 http://raspbian.raspberrypi.org/raspbian bullseye InRelease
Hit:3 http://archive.raspberrypi.org/debian bullseye InRelease
Get:4 https://deb.nodesource.com/node_18.x bullseye InRelease [4,586 B]
Hit:5 https://openhab.jfrog.io/artifactory/openhab-linuxpkg stable InRelease
Fetched 4,586 B in 2s (2,538 B/s)
Reading package lists...
+ return 0
++ apt-cache madison openhab
++ cut '-d|' -f2
++ head -n 1
++ xargs
+ openhabVersion=4.1.1-1
+ [[ -n 4.1.1-1 ]]
+ installVersion='openhab=4.1.1-1 openhab-addons=4.1.1-1'
+ cond_redirect apt-get install --allow-downgrades --yes -o DPkg::Lock::Timeout=60 --option Dpkg::Options::=--force-confnew openhab=4.1.1-1 openhab-addons=4.1.1-1
+ [[ -n '' ]]
+ echo -e '\n\033[90;01m$ apt-get install --allow-downgrades --yes -o DPkg::Lock::Timeout=60 --option Dpkg::Options::=--force-confnew openhab=4.1.1-1 openhab-addons=4.1.1-1 \033[39;49;00m'

e[90;01m$ apt-get install --allow-downgrades --yes -o DPkg::Lock::Timeout=60 --option Dpkg::Options::=--force-confnew openhab=4.1.1-1 openhab-addons=4.1.1-1 e[39;49;00m
+ apt-get install --allow-downgrades --yes -o DPkg::Lock::Timeout=60 --option Dpkg::Options::=--force-confnew openhab=4.1.1-1 openhab-addons=4.1.1-1
Reading package lists...
Building dependency tree...
Reading state information...
The following NEW packages will be installed:
  openhab openhab-addons
0 upgraded, 2 newly installed, 0 to remove and 57 not upgraded.
Need to get 506 MB of archives.
After this operation, 518 MB of additional disk space will be used.
Get:1 https://openhab.jfrog.io/artifactory/openhab-linuxpkg stable/main armhf openhab all 4.1.1-1 [110 MB]
Get:2 https://openhab.jfrog.io/artifactory/openhab-linuxpkg stable/main armhf openhab-addons all 4.1.1-1 [396 MB]
Err:2 https://openhab.jfrog.io/artifactory/openhab-linuxpkg stable/main armhf openhab-addons all 4.1.1-1
  Error reading from server - read (5: Input/output error) [IP: 34.74.126.177 443]
Fetched 110 MB in 1h 23min 3s (22.1 kB/s)
E: Failed to fetch https://openhab.jfrog.io/artifactory/openhab-linuxpkg/pool/main/4.1.1/openhab-addons_4.1.1-1_all.deb  Error reading from server - read (5: Input/output error) [IP: 34.74.126.177 443]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
+ return 100
+ echo FAILED
FAILED
4

ci.openhab.org is a different server to openhab.jfrog.io. I’m not sure if the speed issues are related, but I don’t know anything about openhab’s infrastructure.

5

if you’re on a PPPoE type Internet service, you may need to turn on “TCP-MSS Adjust”. My ISP MTU is 1492 - 8 bytes short of a cable modem service. It’s a 1 gig symmetrical fiber service, but with PPPoE, you have to account for header bytes or the fragmentation may be what’s causing your issue.

6

Hi Guys,
Do you any updates on this issue? I have exactly the same problem… I’ve decided to move from OH3 to OH4 by doing fresh install from SD card, but when the installation finished, there is no OH service installed. I logs, I found that downloading from jFrog repo failed. I try use installation from openhabian-config, but I got speed about 20-30 kB/s and after few minutes it was interrupted with error:

Err:1 https://openhab.jfrog.io/artifactory/openhab-linuxpkg stable/main armhf openhab all 4.1.1-1
  Error reading from server - read (5: Input/output error) [IP: 34.74.126.177 443]

I try burn image on SD card a few times, run at the different times and the results is always the same.
My home internet connections works correctly. on RPi MTU is 1500.

7

I run installation via WiFi and it worked! After switched back to eth and seems that everything is ok.

8

In my case the relatively slow download persists, but the download error of the add-ons did not happen anymore.
Anyway, even when add-ons fail to install it is simply matter of downloading the add-on file (and this always occur at full speed) and copy it in the add-on folder.

9

AFAIK the ci servers (Jenkins) are paid by the openHAB foundation and hosted in a single data center of some hosting company, I guess either located in Germany or the US.
In contrast to this, I expect JFrog to have servers around the world as their business is to supply software around the world.
But to really get to know if that’s correct, we have to ask @Kai or @digitaldan.

10

Correct, our CI servers run on a service provider in the Amsterdam region actually, but Jfrog hosts our artifacts for download.

2 Likes
11

Actually this slow download persists. On my FritzBox 7590 router there is no such possibility to adjust the TCP MSS parameter. I simply tried to modify the MTU on the raspberry to 1492, with the command

sudo ifconfig eth0 mtu 1492 up

but there is no difference in download speed of openhab. Updating a OpenHAB snapshot occurs a 100KB/s and therefore it takes a couple of hours.

12

I’m also seeing jfrog downloads to be slow at the moment.